Programs are made of ideas

quietfanatic The fundamental unit of a computer program is not a function; it’s an action. CPUs don’t do lambda calculus; they follow procedures.

I saw this on twitter a couple days ago and responded with a jerk of my knee, only to commence a discussion1 in which I failed to accurately express my objection. So here is a full treatment.

First I would like to disclaim that I am not attacking quietfanatic in any way. His quote merely provides a convenient means to discuss a widespread pattern of thinking.

Before we start talking about fundamental units of X, it’s probably a good idea to get clear about what we mean by X (where X = computer program). Here are some of the possible definitions I can think of:

  1. A computer program is a pattern of memory intended to be read by a CPU — machine code.
  2. A computer program is an abstract specification for the behavior of the CPU when fed (to-be-determined) machine code.
  3. A computer program is the idea of its behavior. To illuminate what I mean by this, think of Firefox. What idea did you have just then? Was it of an abstract specification? A series of bits? I doubt it. You probably pictured a window browsing the internet or something like that. I’m talking about that kind of idea.
  4. A computer program is a state of a physical system. If it is not in a block of hard disk or ram somewhere, it cannot be considered to exist.

The difference between these is the level of abstraction at which they are defined. But importantly, they are all defined at some level of abstraction — which means that they are all just ideas. Perhaps (4) is not, depending on how concretely you see the world.

Here are some of the ways to think of how a program is run:

  1. The specification of the program is interpreted by some rules, resulting in some changes to physical things.
  2. The CPU deconstructs memory into a series of discrete instructions, which it then “performs” one after another.
  3. The computer is a system in which the states of some physical devices are coupled to a system of interacting voltages, some of which encode a program.

Again, they differ at the level of abstraction at which they are defined.

The way I read quietfanatic’s quote is talking about level (2) of the program and level (2) of the CPU. To me it says “CPUs perform instructions, so our abstract specifications of computer programs should be (ultimately) in terms of instructions”. With that phrasing there seems to be a wide logical gap between the two clauses. Why does the language we use for abstract specification have to correspond to one particular abstract conception of how a CPU works?

By permuting my list of interpretations, I can construct many plausible quotes that follow the same logic. “The fundamental unit of a computer program is a rule, since the program is ultimately interpreted as rules” or “the fundamental unit of a computer program is a voltage, since voltages are ultimately what drive the devices.”

But my objection is larger than this fallacy. To me, and I presume to most software developers, programs are made of ideas. When thinking about programming languages, we are looking for a way to express our ideas in a way that the computer can approximate them. Our ideas form the central theme, not the mechanisms that approximate them.

What is the fundamental unit of a number? How about of a web page? A shape? A set? A cursor?

All these things are ideas, each of which has many different lights under which it can be viewed, each of which may give a different answer about a “fundamental unit”. But ultimately ideas are not defined in terms of anything — we use definitions to grasp how our ideas connect to each other. Some might say that the fundamental unit of mathematics is the set, but when I think about a triangle, I’m sure not thinking about sets.

I am unable to see how computer programs are any different. Why constrain ourselves by forcing a fundamental idea? Must there really be a one true way which forms a basis for everything else to be defined? In the historical development of mathematics, our ideas have had a shifting fluid foundation, each generation bringing a new way to think about the same things. Currently the evolution of programming languages is serving as our shifting foundation. Let’s use our full vocabulary of ideas to express programs, and try not to limit our concepts’ true breadth and beauty by telescoping in on what they mean in terms of a fixed basis.

This article is reflecting a change in thought that I am going through right now. After a lot of time being committed to the lambda calculus, finding ways to construct the naturals out of some pretty-looking set of primitives, devising ways to abolish imperative thinking and embrace functional thinking, I realized that I was lost in a world of details. Everything I did and thought was coupled to the fundamental system I was creating.

It is dawning on me that the fundamental system is merely a means to express my ideas, and the true beauty is in the ideas themselves. My denotationalism fades as I perceive that most of my mental structures are intrinsic. They are not defined in terms of anything and they have no denotation other than themselves. There are ways to model those ideas, but I am suddenly being careful not to confuse the model with the idea itself.

1 To the extent that a series of tweets can be considered a discussion.

Did you like this post? Show, don’t tell Flattr this

About these ads

14 thoughts on “Programs are made of ideas

  1. > I realized that I was lost in a world of details. … It is dawning on me that the fundamental system is merely a means to express my ideas, and the true beauty is in the ideas themselves. My denotationalism fades as I perceive that most of my mental structures are intrinsic.

    A version of the Sapir-Whorf tarpit? Who could’ve known.

    Insofar as Luke Palmer is one person and not a multitude (pace Whitman) there exists a fundamental system underlying all intrinsic mental structures. What could it be?

  2. In other words, something like this: “The fundamental unit of a computer program is not a function; it’s whatever abstraction that I think solves my problem most elegantly. I only have to make sure that it can be translated (preferably automatically) into a language that the repetitive CPU brain can understand.”

  3. Interesting post. Reminds me of the book “Where Mathematics Come from: How the Embodied Mind Brings Mathematics Into Being”.

  4. Luke said: “I am unable to see how computer programs and CPUs are any different.”

    I suggest you look up the term “computer program” in the Wikipedia. You’ll see that a computer program is a collection of instructions in a form that can be executed by a CPU.

    You can think of ideas at any level of abstraction you want. But if you want to embody that idea, you must map it to something executable by the CPU.

    And, of course, there are limits to the types of ideas that a computer (Von Neumann architecture) can embody. (For example, the mathematical continuum cannot be embodied, only finitely approximated. The Navier-Stokes equations cannot be precisely expressed on a computer.)

    So I consider quitefanatic’s point well taken.

  5. Computer program can be considered as a set of particles in computer executing this program – all of them changing in accordance to some wave functions, then your basic abstraction will be function, albeit more complex than usual representation, but it’s valid interpretation, and more “basic” than CPU instructions level (also more realistic – it accounts for cosmic rays etc).

    We could go as low or as high in the tower of abstraction as we want, no level is more valid than any other, some are just more convenient, when it comes to reasoning about certain computer programs.

    Why not use functional abstractions, when it lets programs be described easier?

    Also – there are analogue computers that have no CPU, and we can program them.

  6. The problem is you don’t have a definition of what an ‘idea’ is, or “imaginary” or “mathematical” or “function” or “input” or “output” or “fundamental unit” or the other terms.

    I recommend the book andreas mentioned.

    Our embodied actions are at the root of our abstractions and ideas. They both enable abstraction and constrain it. There’s a reason why for example parallel (concurrent) processing is harder for us to comprehend than regular, serial computing processes. All abstractions are embodied, and the more embodied you make an abstraction, generally speaking, the easier it will be for people to grok it (like the ‘Actors’ approach to tackle concurrent programming).

  7. This is indeed an issue where the first step must be to agree upon the meanings of various words.

    However, from my POV, basing an argument on the CPU is flawed to begin with: The CPU (together with some other components) is an interpreter of the machine-code generated through the compilation process. Not only does it never see the program it self, but it dubious whether characteristics of the interpreter can be used to classify the message.

    (Note: I use the word “interpreter” in the everyday sense above—not in the “alternative to a compiler” sense.)

  8. I see a lot of diversity in the comments here, and there are some very interesting perspectives. Thanks, keep it up!

  9. There are many different problems that computer programmers work on, and there are a variety of methods we can use to reason about them. I’m a bit of a logical pluralist. Use whatever gets the job done. If reasoning through lambda calculus solves your problem, do that. If reasoning in the von Neumann paradigm works for another problem, use that. If reasoning using abstract algebra or calculus solves your problem, use that instead.

    Reasoning about real numbers can also map directly to propositions about computer programs so it’s a little weird to say the continuum can’t be embodied.

  10. A computer program is a state of a physical system.

    ===============

    So nothing about how it goes from its current state to a future state?

    Nick

  11. LP: Why constrain ourselves by forcing a fundamental idea? Must there really be a one true way which forms a basis for everything else to be defined?

    DP: I’m a bit of a logical pluralist. Use whatever gets the job done.

    These seem to be saying the same thing, but how much is DP really saying? Suppose there isn’t any job to get done. What is the natural basis to think of programs then?

    LP: Let’s use our full vocabulary of ideas to express programs, and try not to limit our concepts’ true breadth and beauty by telescoping in on what they mean in terms of a fixed basis.

    The problem is that there is no one vocabulary of ideas for otherwise you’d have your fixed basis already. There are vocabularies (in fact entire unmediated and easily mutually hostile cults) of ideas.

    Keep speaking in half a dozen tongues and see if you don’t get hauled away in a straitjacket. (Digression: Ivan Illich pulls this off but he does so in a company of other conlingual neo-indo-aryan-wannabes. You see he really speaks in only one tongue. BWAHAHAHA)

    This just won’t do. Computers are tools of staggeringly inhuman precision. Cue denotationalism’s “fixed basis.”

    Inasmuch as I am Luke and not Legion Palmer, there must and can only be One True Way. Anything else is to get “lost in the details.”

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s