At work I’ve had the pleasure of programming a preprocessor for our code base. It just takes an XML specification of structures and generates Lua accessors, serializers, etc. I did it in Python, mostly because our developers already have to have Python installed to use the test suite. This has given me a chance to get more familiar with Python as a programming language for doing real things (previously I knew enough to make minor modifications to code; basic block syntax, variable model).
First, I love the “usual” syntax. That is, as long as you’re doing fairly straightforward things, the syntax is beautiful, readable, and clean. I don’t mind the common complaint that variable declaration is implicit with assignment (I also don’t mind having a short declarator, either; as long as it catches name typos, I’m happy). I was happy to learn that Python had real closures via nested function definitions. Previously I believed that it did not support closures, which is why I did not port Glop to Python.
I think Python excels where other languages are mediocre is in communicating to the compiler and the user simultaneously, clearly, what is going on. The language layout essentially enforces that with statement-only assignment, indentation interpretation, and an overall choice of function names over syntactic constructs; it’s difficult to write code that looks like it’s doing something different than it is.
However, that strength of Python is also a weakness. After enough abstraction and indirection, you want to be communicating different things to the computer and to the user. If you communicated to the user what you communicate to the computer, you will just confuse the heck out of the user. At this level, I want a language which allows me to communicate precisely to the compiler, while communicating a good way to think about the code to the user. I would never be able to write a Language::AttributeGrammar for Python, simply because the level of abstraction in that module is too high. It takes a lot of my brainpower to think about its control flow path (driven by thunk evaluation), and if I were forced to communicate that path to the user, my module would be useless.
I think one of the biggest reasons why it is not possible to express code at this level is the lack of a multi-statement lambda. It seems like whether lambda is single-statement or multi-statement is irrelevant, since you can do the equivalent of a multi-statement lambda by using a nested function and naming the codeblock. However, it scatters around the control flow when it should be linear (or when it should look like it’s linear when it’s actually not). It also forces the user to think about functions and the concept of passing functions (which many programmers are not comfortable with), when it could just look like another block construct.
For example, I wrote a function which atomizes output; i.e. do a whole bunch of output operations in sequence, and if an exception is thrown at any time during the process, then nothing is output. This was important because I was using exceptions to communicate when an accessor couldn’t be generated, and I didn’t want my program to write the accessor declaration and opening brace if I couldn’t put something inside.
My use case code could look like this (using Ruby-style parenless block passing):
atomicio(handle) lambda handle: handle.write("foo") if blah: raise Exception, "baz" handle.write("bar")
The idea is that if blah ended up being true, then neither "foo" nor "bar" would be written. This abstraction has been important for the clarity of my script. Instead, it looks like this:
def writeFooBar(handle): handle.write("foo") if blah: raise Exception, "baz" handle.write("bar") atomicio(handle, writeFooBar)
Okay, that's not that bad, but your eyes jump around a bit more as you're reading this. First you skip past the def because it's a def, then you read the atomicio line and notice that it's passing the writeFooBar function, so you scan for the function and find it above you. It doesn't read like a book, it reads like an academic paper :-).
It gets worse when you want composability, though. Say I wanted to do atomic I/O on two handles at once:
atomicio(fh1) lambda fh1: atomicio(fh2) lambda fh2: ...
As opposed to:
def writeFooBar(fh1): def writeFooBarHelper(fh2): ... atomicio(fh2, writeFooBarHelper) atomicio(fh1, writeFooBar)
Now do you see the difference? The structure of the first example is pretty obvious, but the second example is significantly more awkward. writeFooBar logically takes two arguments, but that clarity has been lost by the explicit currying. Additionally, the order ends up appearing backwards in the source file (fh2 before fh1).
Of course, I don't think this is something that should be fixed in Python. If Python decided to abandon its very explicit vision of being clear about what you're telling the compiler, it would fail at being a more expressive language. The languages that really excel at this kind of thing are Perl, because you can play syntactic tricks on the Programmer's eyes, and Haskell, because the entire language is designed around functional abstraction (a Monad, for example, is just an extension of the idea I presented here). Python would have to change too much to compete with them, and it would almost certainly lose every value it has that puts it in front of other languages.