On The Unix Philosophy, Taleb & Foucault

What does the french thinker Michel Foucault have in common with the Unix philosophy? Not much. But Foucault and the Unix Philosophy have a similar view on power interpreted in a very wide sense, as well as a comparable view on the nature of distributed power. Two very different contexts, but similar ideas.

The Unix Philosophy is outlined, for instance, in The Unix Programming Environment by Kernighan and Pike, and by Eric S. Raymond in The Art of Unix programming. There is no lengthy proclamation of intent, only short explanations.

Kernighan and Pike summarize the philosophy in a preface (viii, 1984),

Even though the UNIX system introduces a number of innovative programs and techniques, no single program or idea makes it work well. Instead, what makes it effective is the approach to programming, a philosophy of using the computer. Although that philosophy can't be written down in a single sentence, at its heart is the idea that the power of a system comes more from the relationships among programs than from the programs themselves. Many UNIX programs do quite trivial things in isolation, but, combined with other programs, become general and useful tools.

Eric S. Raymond, the historian of hacker culture, attempts to make the philosophy explicit in The Art of Unix Programming. Raymond draws from different sources and collects 17 meanings.

Central in my interpretation of the Unix Philosophy, is what Raymond labels 'the rule of modularity' as a way to handle complexity by use of simple building blocks and 'the rule of composition', principles for how these blocks are connected.

These principles are ordered by 'the rule of simplicity', a variant of Ockham's razor, saying we should stay with simplicity and only add complexity when needed. That is, we should only centralize our tools when necessary - so to speak.

The main point is, freely interpreted, on why we should avoid one single commander with centralized power. While perhaps effective, perhaps not – with all kinds of largescale administrative systems comes risks and liabilities.

And what happens when the commander is not fit for the task? …or when tools are not fit for a certain situation?

The cost of comfort in any elite administrative system is always the loss of flexibility.

In modernity, we have learned to put equal signs between safety and largescale administration. We believe this produces robustness.

This is what the Unix Philosophy questions if we draw its principles to their end-points. According to the Unix Philosophy, robustness should not come from a single application attempting to solve all possible problems within a certain scope.

Having one super-application attempting this is, with Raymond, 'cleverness'. Instead, the Unix Philosophy propagates having numerous less grand, small, and dumb programs that we can combine.

In this view, the multitude of small programs and the power to combine handles more situations, including unpredictable ones.

The main issue with large-scale applications trying to solve many problems within a scope is that while we never will be able to think of all possible things we want to do, we will be trapped in smart solutions.

With small programs having small scopes, this is easier. It is more important in a Unix environment to empower the user and let the user connect a multitude of small programs in creative, flexible, and robust ways.

This is also one of many nodes linking how the Unix culture aims at creating software with larger intellectual trends in our century.

As a side-note, the robustness The Unix Philosophy question in software design is close to the critique Nassim Nicholas Taleb has on large-scale systems in general in books like Anti-Fragility.

Taleb is very skeptical of the grandiose plans of the administrators, blueprints produced by the Architects of modernity - just as the Unix Philosophy is suspicious of unnecessarily up-scaled applications.

In this picture, the Unix crafter becomes an anti-architect.

Small problems have a small scope. If true, this means such software has fewer problems and risks and can be handled with greater ease. The bazaar vs. The Cathedral.

Again, while a single program is supposed to be dumb according to Unix philosophy, clarity arises from connectivity as opposed to the cleverness of the schemes and plots of large-scale systems.

The main point of Taleb's The Black Swan, a construct from the theory of science, but slightly modified and used more metaphorically, is that a large scale system is build to handle risks, while at the same time large-scale systems cannot fully predict all risks and when faced with catastrophe have no means to quickly solve the issues.

The essence of uncertainty and unpredictability is that the inner functioning is unknown to us.

If they weren't unknown or impossible to grasp, we would not speak about uncertainty and unpredictability. And for very similar reasons the Unix Philosophy advocates an anti-system build of a multitude of small programs, each program stupid but when combined better can meet specific and unpredictable cases.

Foucault spent most of his career to investigating 'power' in a wide sense. Although he had philosophical ambitions, I think his greatness is not as a philosopher but comes through how he uses his view on power and related terms in concrete studies on subject matters such as crime and punishment, madness and clinical facilities, and sexuality.

What separated Foucault from others of his age is his break with marxism, at the time the dominant theoretical foundation for understanding culture, society, and history. It's fair though to claim that the break is deeper.

Marxism was only one of several modernist attempts to find a superstructure for explaining all kinds of phenomena. What unites all modernist attempts is the ambition to find one or a very limited set of principles that through a hierarchical model is used to go from the assumed core structure of reality to its leaves.

Abstract notions, history, culture, class are higher in the hierarchy and more everyday experiences are to be perceived as leaves. Communism and fascism and other totalitarian modernist projects of 'utopia' have common roots in the history of ideas.

The main attempt of all modernist projects is to control the 'core' structures of society. If you control the core, you control the rest.

Foucault doesn't deny that power can be directed, 'administrated'. But he believes this is a limited, quite false metaphor, a mixing of notions. Foucault, therefore, turned this logic upsidedown.

Instead of asking how power is constructed on a macro level (hierarchically), Foucault sought to investigate the microlevel. Instead of seeing power as a consciously produced structure of hierarchical fashion, Foucault attempted to explain power in terms of relations – as a web. A web constituted only partly by deliberate attempts of ideology and intents, partly by much more subtle workings.

Power rises, power occurs when bodies meet and react to each other, according to Foucault, much like material bodies.

On an ideological level, this has great implications. While the heart of all communism is to have a new 'elite' and with the logic of Lenin during a phase terrorize all other groups, and then let the state wither away and let a "free" society emerge – a utopia – the reasoning of Foucault have more in common with anarchism (or, based on late lectures, liberalism).

The similarities between Foucault and his view on power and society and the Unix philosophy are somewhat far-fetched. But with some imagination, we can see similarities in how power is perceived but also connections in terms of ideology.

Consider the implicit philosophy of a modern user interface. While modern user interfaces most of the time are great the comforts come with a prize – generalizations made by an 'elite'. Someone says this feature is important for most users of the application. And most of the time such generalizations are great.

But what if our task is impossible to handle in the application? What if the task is out of scope?

This is one reason some argue that React is more popular than Google's Angular, for better or worse. In React you have a larger degree of freedom. The main reason though is hopefully because they view React as better in relation to some measures.

Although we have no reason to question the knowledge of the 'elite' (most of the time their generalizations are great), they are human. But the thing with generalizations is that they always have a cost, hidden or not. In its essence, to generalize is to forsake. No matter how smart a generalization is, something is forsaken.

Unix separates between programmers and ordinary users. A solution fit for programmers is not necessarily fit for ordinary users. There is nothing strange or elitist about this view. This is a result of the fact that programming is a craft that takes time to learn.

If I would build something at home, I would not use the tools of real crafters, I would use other tools - aligned for my skill set. In The Unix programming environment, Unix is described as a work-bench for a reason.

Just as Foucault views power, the Unix philosophy implicates that power rise from a multitude of sources put in connection. This is how power rise. But also how power spread and grow.

In Unix, this is shown by the use of pipes and filters but also through the command line interpreter in general (BASH scripting, but arguable also Python and Perl scripting).

Often we don't need much power to perform tasks. And as a beginner, I realize that the power I can yield is limited. But if we take an ordinary task, such as finding all files using a label or a function we can write

grep -rl someLabel directory/ | xargs nvim

…and open all files including the term someLabel. We use grep to find the pattern, xargs to parametrize the result so Vim can open the list of files.

The more specialized our task is, the less the likelihood that the modern UI covers the case. Opening all files including the pattern someLabel included two small programs and a third, the editor (NeoVim).

How about reading this post with only capital letters? A strange thing to do indeed. But by using a small number of very dumb Unix applications this task is made easy.

curl -s https://herebeseaswines.net | html2text | tr '[:lower:]' '[:upper:]' | more

The power of the multitude of small programs performing one task well, is present when solving a small problem. But it's power is more clearly demonstrated in larger tasks, solving large problems by simple means of combination.

A Unix program is supposed to be simple, and if you don't know how to use it or have forgotten how to use it you can either duckduckgo the Web or use the man page.

As a person, Unix fits me well. I am also very fond of the Unix philosophy as it is close to my liberal views in general. I am not sure it is always better to do things the Unix way. Actually, my guess this way will hinder you more than you're being enabled the first years. That is if you end-goal is to create software.

Until you reach a certain level, you're like a monkey trying to use a hammer. Although it's possible to produce some effect, most of the time you will use the hammer as a club. The sad part is that there is no way to get around this. You have to walk the walk, I presume.

Even though I am a beginner and will consider myself so for the years to come, it's possible to make a very general conclusion: if you never learn Unix, you can only reach so far - your capacity will always only reach as far as the applications you use let you.

This stands against a multitude of small tools we can combine as we use a language and thus adapt to the specific problem at hand.

What is a tool and what's not has many striations. To me, the Unix system is a so-called hacker tool. I am a programmer. But on the other hand, what is programming? Programming is a set of formal languages meant to solve concrete problems. Also mere tools but of another order. But this is true also for the applications we develop - the applications we develop are tools made to solve problems.

This chain has no clear end if we're not to end up with abstractions such as the Maslow hierarchy of needs. Shouldn't any tool, in the end, relate to such needs?