Tag Archives: codecatalog

The Plan

Last September, I decided that it was time to get a programming job again. After two months of trying to find paid work (of any kind, $10 would have been great!) as a composer, I realized that it’s really hard. There are a lot of people willing to work for free, and without much of a scoring portfolio (as opposed to the “pure music” I do) I have no way to distinguish myself to the studios that have a budget. Also, a lot of games want orchestral scores, and I don’t have the hardware and software I need to make convincing-sounding synthetic orchestral scores. Also, I’m sure once I get the necessary hardware and software, I will need time to practice with it. In short, I needed money and time. I am extremely fortunate to have, in my free-flowing way, stumbled onto a skill that is valued by the economy, and so I decided it was once again time to utilize that skill to achieve my other goals. I planned to live reasonably cheaply, save up money so that I can buy equipment and support myself for enough time to build up a portfolio by doing free projects.

Now I have been programming for Clozure for almost six months. As far as jobs go, it’s great. I get to work in my favorite language, Haskell, and they give me enough freedom to experiment with designs and come up with solutions that not only work, but that I would even consider good. My fear of programming jobs was based on having jobs where I constantly have to compromise my values, either by working in crappy languages or on startup-style timelines where there is no time to lose. With this job, I feel reunited with my love of software, and my inspirations for developer support tools have been once again ignited.

And so I have amended the plan: after I have saved enough money to support myself for several years, I will not only attempt to bootstrap a career composing, but dedicate my current work week to making a reality the software ideas which have been floating around in my head for half a decade. This prospect really excites me — the reason I have not been able to make my ideas is mostly the time pressure: there’s was always something else I should be doing, and so I always felt guilty working on my pet projects. I wonder, what am I capable of if my pet projects are the main thing?

I want to revive CodeCatalog. Max and I lost steam on that project for a number of reasons.

  1. Due to family pressure, I returned to school.
  2. I fell in love with a girl and got my heart all broken. That can be kind of a downer.
  3. The priorities of the project compromised my vision. We were attempting to use modern wisdom to make the project successful: first impressions and intuitive usability came first. Our focus was on making it pretty and satisfying to use (which took a long time since neither of us were experienced web front-end developers), and that required me to strip off the most interesting parts of the project because noobs wouldn’t immediately understand it.

So I want to re-orient (3) to make it more satisfying for me. I want to allow myself to make the large strides that I envisage rather than baby-stepping toward success — to encourage myself to use my own talents in design and abstraction rather than trying to be a front-end person, to emphasize the exciting parts (what Audrey Tang calles -Ofun). By funding myself, I will not feel the guilt that comes with working on a project at the same time as (1). I can do no more than hope that something like (2) doesn’t happen. (I have a wonderful, stable and supportive relationship right now, so if that continues, that’d cover it :-)

I have many ideas; the reason I want to return to CodeCatalog in particular is mainly because I have identified most of my ideas as aspects of this project. My specific fancies change frequently (usually to things I have thought about before but never implemented), and so by focusing on this project in a researchy rather than producty way, I can entertain them while still working toward a larger goal and eventually benefitting the community.

Here is a summary of some ideas that fit in the CodeCatalog umbrella (just because I’m excited and want to remember):

  • Inter-project version control — I have always been frustrated by the inability of git and hg to merge two projects while still allowing interoperation with where they came from. The “project” quantum seems arbitrary, and I want to globalize it.
  • Package adapters — evolving the interface of a package without breaking users of the old interface by rewriting the old package in terms of the new one. There is a great deal that can be done automatically in this area with sufficient knowledge about the meaning of changes. I talked with Michael Sloan about this some, and some of the resulting ideas are contained in this writeup.
  • Informal checked documentation — documenting the assumptions of code in a machine-readable semi-formal language, to get the computer to pair-program with you (e.g. you write a division x/y and you have no y /= 0 assumption in scope, you’d get a “documentation obligation” to explain in english why y can’t be 0).
  • Structural editing — coding by transforming valid syntax trees. Yes it’d be cool, but the main reason it’s compelling to me is in its synergy with other features. Once you have the notion of focusing on expressions, holes with contextual information (a la Agda), semi-automatic creation of package and data-type adapters, smarter version control (e.g. a change might rename all references to an identifier, even the ones that weren’t there when the change was made) all come as natural extensions to the idea.

I think the challenge for me will be to focus on one of these for long enough to make it cool before getting distracted by another. My plan for that is to set short-term goals here on my blog and use it to keep myself in check. I am considering involving other people in my project as a way to keep myself focused (i.e. maybe I can make a little mini-kickstarter in which my devotees can pledge small amounts in exchange for me completing a specific goal on time).

This is all two years away or more, which feels like a long time, but in the grand scheme is not that long in exchange for what I see as the potential of this endeavor. I’m just excited and couldn’t help but to think about it and get pumped up. Thanks for reading!

Oh, despite the date, this is totally not an April Fools joke (as far as I know ;-).

New Languages

It has been some time since I learned a new programming language. Perhaps Haskell has me so firmly entrenched in “desirable properties” that any new language I look at either looks like another Javascript (a decent language sullied by terrible syntax) or Smalltalk (a decent language sullied by terrible engineering). I completely admit that I have had opportunities to learn new languages that I have turned down because I didn’t find the language’s ideas aesthetically pleasing after reading about them for ten minutes. But all in all I have been unimpressed by new languages and haven’t felt inspired to learn them to the degree I would need to use them.

It has also been some time since I have been excited about programming. My readers are surely aware of this as the topics of this blog meander this way and that away from my expertise. I used to love seeking the elegant design, simmering 200 lines down into 40, hypothesizing about language features and module systems, imagining worlds in which all modern software was built upon my paradigm.

I think these two things are related. I am not sure which way the causation goes, or even if there is a causal relationship. But thinking back on my time as a programmer, the times when I was most productive I was learning: working in a new language or working in a new domain. I still think CodeCatalog is a great idea, in total a few hard weeks’ work, and I can’t convince myself to write another line of the damned thing. That’s because I know how it’s gonna go; I know what I have to do; there is no mystery to it.

What if, instead of twisting my arm and trying to force myself into “good behavior”, I embraced this aspect of myself? There has to be some beautiful experimental kernel to that project; there has to be some beautiful way to express it. And it is certainly possible, even likely, that the result won’t end up looking and feeling like StackOverflow or Google Whatever (beta).

So what?

Have I been so brainwashed by the business of software that I will abandon a project because I cannot inspire myself to bring it to business standards? I think it’s because we wanted to make money. It would be nice not to have to worry about paying rent, I admit, but not worth exchanging for the beauty of an inspired work of code.

Someone invent a beautiful web/database language. I mean beautiful and unique — not practical or productive or real-world or familiar or interoperable or scalable. I don’t care about those adjectives, those are for engineers.

Flattr this

Watering the blog

In order to keep this blog from shriveling up and dying, here’s an entry, even though I don’t know what it’s about.

My thoughts have been consumed, after a brief detour following my viewing of the excellent Feynman quantum electrodynamics lectures, by CodeCatalog and its consequences. Even though there have not been visible changes on the site, we have stuff in the oven. The biggest change in direction came when we decided to support the process of coding, rather than just the result, on the site. This has a lot of consequences to our model so it is taking a while to follow the ripples, but we also get to support wicked stuff like this:

  • Automatically searching for snippets when you use an undeclared identifier (building an immediate vocabulary)
  • Providing a codepad-style sample evaluation, being able to extract sample evaluation into a test with a click.
  • Showing documentation for functions you are using and suggesting useful snippets based on correlations to other users (distant future).

I got sidetracked for a little while developing jsexp, an in-browser structural (i.e. working with code as syntax tree rather than text) editor. I had some bullshit reasons that I convinced myself it was a good idea, but really I just wanted to make a structural editor. After that proved a greater challenge than my justification-utopia had laid out, I had to abandon ship and refocus on what we’re really trying to do. Thanks to Jude for kicking me in the butt and making me step back — er, I guess it would be forward. :-)

So we’re aiming to make it a nice place to edit code — not because it’s the world’s greatest editor (although we are using CodeMirror which is decently spiffy), but because the world’s greatest standard library will be loyally awaiting your command with no hassle. That’s valuable to me, at least — every time I have to integrate a new library it breaks my flowstate.

My thoughts have taken me wandering around this general area, returning to my catch-phrase and the opportunity of software organization it presents. How shall we organize an infinitely large (in the sense that it is open to expansion in all directions) codebase? This codebase can’t turn to mush as it gets big because its goal depends on it being big; how do we finagle that?

Anyway, up to my usual antics of long walks talking to myself, whiteboarding, staring at the ceiling, and occasionally not forgetting to water my blog.

Flattr this

Announcing CodeCatalog

I’d like to share what I’ve been working on instead of homework for the past month: codecatalog.net. From the FAQ:

CodeCatalog is a lot like a wiki for source code. We aim to socially build a database of high-quality, well-documented, instantly-usable code in a variety of languages.

It is the fruit of my thoughts’ recent focus on reusability. I noticed that because of purity, Haskell was very close to being able to have a “database of all reusable code ever”, and I kept focusing on what language features were missing from languages that prevented that. After sharing the idea with Max (cofounder of Hubris Arts), we determined that the main missing feature was, er, a database of all reusable code ever. CodeCatalog is the first buddings of an attempt to create one.

Check out the FAQ for a glimpse of our philosophy. Haskell sophisticates might be able to see beyond my simple wording into the underlying vision — that was my intent at least. I’ll write a post soon describing our deeper plans in more detail. We’ll be working on this all summer, and if we meet our goals, by the end of the summer it should be pretty freaking cool. :-)

Oh, the code that we have on the site so far is mostly Python and Javascript, mostly because that’s what we wrote the site in and we were eating our own dogfood while developing it. But Haskell is also supported.

Anyway, fellow Haskell community, I encourage you to check out the site. We would appreciate any feedback you have about it. There’s a UserVoice tab on the right if you’re into that kind of thing. Thanks. :-)

And if you would like to support us financially, you can always Flattr this.