Designing with Interfaces

In this post I will share a personal discovery about object-oriented design. It is so simple and obvious that it may not be news to experienced OO programmers. However, it should be noted that before I became a Haskell nut, I was a dedicated OO programmer for 7 years. I occasionally empolyed the pattern in my designs without realizing its full generality. So it may yet be news to you vets.

I came across it when exploring the consequences of my recent post, Encapsulation Considered Harmful. I was trying to design a concept language around the idea, and realized that the language design I kept coming back to could be mostly encoded in existing languages.

Here’s the rule: never reference a class directly — always go through an interface. This includes creating new objects — go through a factory interface or take it as a parameter instead.

Obviously this rule cannot be followed 100% because you’ve got to give a concrete instantiation at some point. But push that as far “up” in your program as possible. Also, doing this 100% in contemporary languages would probably involve too much boilerplate. That’s okay, it can be done piecewise, and many of its advantages can still be reaped if it is done partially.

We have recently done this refactor on our game, and I have to say, it is gorgeous! I really like the clean, uniform way it separates concerns. More importantly, it is an abstraction instead of an encapsulation mechanism. That means that when code doesn’t depend on a particular detail, you can always substitute something else for that detail. Keep that property in mind, it will guide you in applying this pattern correctly.

Here is a simplified example from our game. We used to have a class FileSystem to access the filesystem. ProgressResult<T> was a class that monitored the progress of loading a file from disk and yielded a T when it was finished. Unit is the trivial type: struct Unit {}.

class FileSystem
{
    ProgressResult<List<string>> List();
    ProgressResult<byte[]> Load(string filename);
    ProgressResult<Unit> Save(string filename, byte[] bytes);
}

And in our World class, a class for the main game interface, we used it like so:

class World
{
    FileSystem filesystem;
    public World(...) {
        filesystem = new FileSystem();
        ...
    }
    // using methods on filesystem here
}

After applying this pattern, it becomes:

interface IProgressResult<T>
{
    ...
}

interface IFileSystem
{
    IProgressResult<List<string>> List();
    IProgressResult<byte[]> Load(string filename);
    IProgressResult<Unit> Save(string filename);
}

class FileSystem : IFileSystem { /* as before */ }

class World
{
    IFileSystem filesystem;
    public World(IFileSystem filesystem) {
        this.filesystem = filesystem;
    }
}

See how we avoided creating a filesystem in World, and instead took it as a parameter? World is now more flexible than it used to be: we can (and did!) instantiate it with a disk filesystem and an Amazon S3 filesystem. They have different sorts of ProgressResults too, thus that is also an interface.

Now, if you go to apply this pattern in your project, you might find that it breaks down for more complex designs. It’ll do that if you aren’t precise about your phrasing. For example, in another part of our project we had the following combinator library for our UI:

class UI 
{
    public static UI Over(UI over, UI under) { ... }
    public static UI VerticalGroup(params UI[] uis) { ... }
    public static UI Window(string title, UI contents) { ... }
    public static UI Button(string text, Action onClick) { ... }
    // some non-static members for inspection
}

To abstract over these implementations, we need to promote these static functions into methods on a factory-like interface. It may seem that this is the way forward:

interface IUI { /* the non-static members */ }
interface IUIModule 
{
    IUI Over(IUI over, IUI under);
    IUI VerticalGroup(params IUI[] uis);
    IUI Window(string title, IUI contents);
    IUI Button(string text, Action onClick);
}

But this is not the way. Imagine if you were to construct some IUIs with a SilverlightUIModule and then try to combine them (with, say, VerticalGroup) with a UnityUIModule. What is the unity module supposed to do with silverlight UIs? This design flaw will show its head much earlier than this predicament, however: you will find in UnityUIModule that you need to downcast the IUIs you get into a specific type. Downcasting is an indication that you are doing it wrong. The correct way is to make IUIModule parametric in the UI type:

interface IUI { /* the non-static members */ }
interface IUIModule<TUI> where TUI : IUI 
{
    TUI Over(TUI over, TUI under);
    TUI VerticalGroup(params TUI[] uis);
    TUI Window(string title, TUI contents);
    TUI Button(string text, Action onClick);
}

Now UnityUIModule implements IUIModule<UnityUI> and SilverlightUIModule implements IUIModule<SilverlightUI>, so it is a compile-time type error to pass a silverlight UI to a unity combinator. In addition, you do not need to downcast: the UnityUIModule now knows statically that it will only be passed SilverlightUIs.

One thing remains… how are you supposed to use that damn module? Surely you don’t make every class that uses a IUIModule parametric in the TUI parameter like this:

class World<TUI> where TUI : IUI
{
    IUIModule<TUI> uiModule;
    public World(IUIModule<TUI> uiModule) {...}
}

Type parameters would accumulate in classes, and every time you used that class you would have to write all those fucking type parameters. That can’t be the way to go!

Well… actually… remember the rule? You never reference a class concretely, you only go through interfaces. And the interfaces don’t accumulate type parameters from usages in the implementation like classes do, they only accumulate type parameters from the cleaner and scarcer usage in interfaces. So the worry of unsightly verbose code is unfounded. It’s fine, let the type parameters accumulate, they are a way of stating your assumptions. They tell you exactly how a class may be reused. So to defeat this notational burden in this example, we must continue the pattern: create an IWorld!

As you do this, all your assumptions will bubble up to the top of your program. Each more complex thing is parametric in all the simpler things it is made of. When you go to write code that can actually be executed, you will find an ocean of flexibility: you basically have created a rich combinator library for your domain, allowing it to target multiple underlying frameworks easily, getting multiple different (reasonable, even desirable!) behaviors just by passing a different parameter to some object you are building at the top level. Mmmm, parametricity.

(For dynamically typed languages, duck typing will mostly take care of this pattern for you. But the design motivation remains, make your code parameteric: take parameters saying where it should get the objects it is creating and the functions it is calling rather than referencing them directly!)

Did you enjoy this article? Let me know and Flattr this. I appreciate it.

14 thoughts on “Designing with Interfaces

  1. Isn’t what you describe the Dependency Injection paradigm that you get for example in Spring (http://static.springsource.org/spring/docs/3.0.x/spring-framework-reference/html/overview.html#overview-dependency-injection), for Java in that case of course?
    One issue I have with this approach (that we use a lot in our own code) is that debugging requires a … debugger, since you sometimes need to find out at runtime what actual implementation of an interface is being used. So static analysis is more difficult.

  2. @Jonathan Allen, thanks for the book link. Can you explain “we tried it and it sucked” a bit more deeply? For example… why?

    @Harry M, if I were to take the pattern to its limit, then I would factor out any duplicated code into a separate “utilities” interface. I am sure that is quite cumbersome in C#, but it is a goal for the concept language I am still exploring.

    @JP Moresmau, that is basically what I’m trying to get at, but without all the enterprise overengineering smell. I haven’t yet arrived, but I think there is a simple and beautiful principle behind all this.

    @gasche, thanks for the link. I agree wholeheartedly :)

  3. I cannot do the book justice, but I can give you a summary from my own experiences.

    First of all, terms:

    Public Interface: This is the methods, events, and properties exposed on a class via the “public” keyword in C#.

    Private/Internal Interface: This is a superset of the public interface that includes private/internal methods, events, and properties.

    Abstract Interface: This is something defined by the “interface” keyword in C#.

    Programming to an Interface: This expression comes from the C/C++ world where one has to choose between directly manipulating fields and manipulating them via functions or methods.

    Now the history:

    In the world of COM, classes don’t have public interfaces. The only way to access a class outside your own library is via the abstract interfaces implemented by the class. Since every COM interface is abstract, any class can implement any other class’s interface. Essentially you end up with an extreme form of polymorphism.

    The downside of this is that you cannot change interfaces without breaking a lot of code. Even something minor like adding a new method requires creating a new matching interface. This is why you often see names like “ISomeClassEx2” in the Microsoft libraries.

    This leads to a lot of casting operations. If you have a IFoo and need to call Baz, chances are you have to cast your IFoo into an IFoo2. This is normally OK, as you know that the Foo class implements both IFoo and IFoo2. But as soon as someone tries to have the Bar class implement IFoo, your cast fails and your code breaks. So then you change all of your methods to only accept IFoo2 parameters, which causes ripples throughout your code base and possible into other people’s code base.

    An option available to C# programmers is to have a one-to-one mapping between your abstract interface and the class’s public interface. As new public methods are added, the interface is updated accordingly. This destroys your ability to have other classes implement the interface. Every time you add a feature to a class you have to add that same feature to every other class that shares the same interface.

    In the long run you are better off by starting with no interfaces at all. Only create interfaces when you can point to a specific set of classes and say “These MUST all implement the same interface because I NEED to perform the same operation on all of them.” At that point you know exactly which methods are actually important to the interface and you won’t be as tempted to grab everything under the sun.

  4. @stephen, yeah :-(. The completeness of Unity just couldn’t be argued with. It took about the same amount of time to write the COLLADA model loader for haskell as it did to make the first prototype of the game in Unity. It was a hopeful ideal, but just not practical for doing it for real.

  5. @Jonathan, thanks for your reply. This is valuable to me since I am a fierce proponent of increasing software reuse, and it sounds like COM tried to do that and ran into the can of worms on the other side. Though it is hard to extract the full experience from your description, it almost sounds like they ran into the problem of “too much reuse” — suddenly interfaces had so many disparate implementors that the graph of usage became hard to manage.

    I don’t see the inability to change interfaces as a problem in and of itself. ISomeClassEx2 is lame though, and a result of a poor naming mechanism.

    Unsafe casting is a manifestation of hidden assumptions, which always cause modularity problems in large codebases. I would not hesitate to completely outlaw casts if I were designing such a system. Then… you have an IFoo and you need an IFoo2? Well then you need a function IFoo -> IFoo2. My guess is that writing such functions would require creating an adapter class with an IFoo as a member and a bunch of trivial methods. Even I would use a cast as an alternative to such a boilerplateful approach.

    I have been pondering the idea of integrating version control with the language itself. When you “change” IFoo(version 1), you actually get a new interface IFoo(version 2) together with an automatically generated function IFoo(version 1) -> IFoo(version 2). This function takes the place of the cast, but does so in a safe and modular way.

    So I guess I am theorizing that COM had the right idea, but got enough of the details wrong that it became hell. That happens a lot, and unfortunately it is hard to tell in advance which details will contain devils.

    Anyway thank you again for sharing your experience with COM. It is a big help on my search for the greatest language nobody will ever use. :-P

  6. One reason COM failed is poor compositional performance. Essentially, interfaces work best and remain simple when you can combine a bunch of simple operations to perform a complex operation. But with the per-operation round-trip delay introduced by COM RPC (e.g. to obtain intermediate results) makes using a bunch of small messages impractical. Therefore, developers are under pressure to extend the interface with whatever complex operations have the most immediate requirement.

    Including ‘complex’ operations in the interface is a step down a path that, if taken to its end, leads ultimately to adding a whole scripting language to the interface. Unfortunately, all the refactoring and rework on that path will be quite painful, and the people walking the path are probably not programming language designers. While performance compositionality is one cause, there are other causes for the same phenomenon. For example, concurrency control could become an issue that encourages complex messages to avoid unpredictable ‘interleaving’ of smaller ones from multiple clients. Transactions or eventual consistency are proposed to help with concurrency woes. For the latency issue, batching and promise pipeling are both effective and quite simple.

    If you run enough user stories, it isn’t very difficult to find out what is likely to go wrong with a system like COM… but ‘Worse is Better’ tends to win the race every time. People with plenty of RPC experience predicted the hell that COM became well in advance, but it doesn’t mean they could have provided a better solution. The people who actually think about such problems are also the people most subject to analysis paralysis! Besides latency and concurrency, we might also consider compositional properties of our programs in terms of parallel scalability, progress, partitioning tolerance, resilience, graceful degradation, plug-in extensibility, security… at some point you’ll discover the constraint that breaks the human’s brain.

    Getting back onto the subject of interfaces, for example: much more compositional than a coarse-grained interface types is a simple record of function objects. Using such records, wrapping or replacing or sharing just one method, or mashing multiple objects together, would be simple and efficient. Unfortunately, your language probably does not optimize this case or make it convenient to express.

  7. COM had many flaws: it used reference counting, which doesn’t work in a component system (if you need to know where cyclic dependencies can occur, you know too much). Furthermore the problem with interfaces is that they don’t describe allowed side effects of an implementation, so replacing one implementation by another usually doesn’t work at all. It would be interesting to see how this works in a functional setting, where side effects do not occur (however, space time complexity can differ vastly between implementations…)

    Its funny to see that the software engineering world runs in cycles, they always keep re-introducing ideas that were flawed or did not catch on. On the other hand, sometimes re-inventing something can result in a good think and change the world. Look at Apple’s iPhone: tried many times before by others (Psion, PocketPC, etc) and that failed miserably…

    Anyway, I still use interfaces (in C#) a lot. When designing a new program, I always start by writing down all possible interfaces, even if those interfaces won’t be used in the final implementation (because of performance or whatever reasons). So I use interfaces as a “semantical design language” or something like that :)

  8. Anonymous :
    Furthermore the problem with interfaces is that they don’t describe allowed side effects of an implementation, so replacing one implementation by another usually doesn’t work at all.

    There is something wrong here. Of course in most languages the linguistic interface construct is not expressive enough to express side-effects and various invariants of the expected implementations. This is even the case in strong-typed languages such as Haskell, where for example the monad laws are expressed outside the language. In those cases you must distribute a description of the behavior described by the interface, including invariants and side effects. Every does (or should do) that and it’s widely recognized, for example as a basis for Liskov’s Substitution Principle.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s