Composable Input for Fruit

The other day I had an idea for a game which required a traditionalish user interface (text boxes, a grid of checkboxes, …). But I’m addicted to Haskell at the moment, so I was not okay with doing it in C#, my usual GUI fallback. Upon scouring hackage for a GUI widget library I could use, I realized that they all suck—either they are too imperative or too inflexible.

So I set out to write yet another one, in hopes that it wouldn’t suck. The plan is to write it on top of graphics-drawingcombinators, my lightweight declarative OpenGL wrapper, and reactive, the latest (in progress) implementation of FRP. In researching the design, Conal pointed me to a paper on “Fruit”, which is a very simple design for GUIs in FRP. It’s nice (because it is little more than FRP itself), and it’s approximately what I’m going to do. But before I do, I had to address a big grotesque wart in the design:

data Mouse = Mouse { mpos :: Point,
                     lbDown :: Bool,
                     rbDown :: Bool }
data Kbd = Kbd { keyDown :: [Char] }
type GUIInput = (Maybe Kbd, Maybe Mouse)
type GUI a b = SF (GUIInput,a) (Picture,b)

So every GUI transformer takes a GUIInput as a parameter. The first thing that caught my eye was the Maybes in the type of GUIInput, which are meant to encode the idea of focus. This is an example of the inflexibility I noticed in the existing libraries: it is a very limited, not extensible, not customizable idea of focus. But there is something yet more pressing: this input type is not composable. The type of input is always the same, and there is no way to build complex input handling from simple input handling.

I took a walk, and came up with the following:

Scrap GUI. Our interface will be nothing more than pure FRP. But that doesn’t solve the input problem, it just gives it to the users to solve. So to solve that, we build up composable input types, and then access them using normal FRP methods.

We will start with Kbd and Mouse as above. The problem to solve is that when we pass input to a subwidget, its local coordinate system needs to be transformed. So the only cabability input types need to have is that they need to be transformable.

-- A class for invertable transformations.  We restrict to affine transformations
-- because we have to work with OpenGL, which does not support arbitrary
-- transformation.
class Transformable a where
    translate :: Point -> a -> a
    rotate    :: Double -> a -> a
    scale     :: Double -> Double -> a -> a

instance Transformable Point where
    -- .. typical affine transformations on points

-- Keyboard input does not transform at all
instance Transformable Kbd where
    translate _ = id
    rotate _ = id
    scale _ _ = id

-- The mouse position transforms
instance Transformable Mouse where
    translate p m  = m { mpos = translate p (mpos m) }
    rotate theta m = m { mpos = rotate r (mpos m) }
    scale sx sy m  = m { mpos = scale sx sy (mpos m) }

-- Behaviors transform pointwise.  In fact, this is the instance
-- for Transformable on any Functor, but we have no way of telling
-- Haskell that.
instance (Transformable a) => Transformable (Behavior a) where
    translate = fmap . translate
    rotate = fmap . rotate
    scale sx sy = fmap (scale sx sy)

Widgets that accept input will have types like: Behavior i -> Behavior o where both i and o are transformable (o is usually a Drawing, or a Drawing paired with some other output). So we can transform a whole widget at once by defining a transformable instance for functions.

instance (Transformable i, Transformable o) => Transformable (i -> o) where
    translate p f = translate p . f . translate (-p)
    rotate r f = rotate r . f . rotate (-r)
    scale sx sy = scale sx sy . f . scale (recip sx) (recip sy)

The way we transform a function is to inversely transform the input, do the function, then transform the output. This is called the conjugate of the transformation.

And that’s it for composable input: just a class for affine transformations. A typical GUI might look like:

-- Takes a mouse position, returns the "pressed" state and its picture.
button :: Behavior Mouse -> Behavior (Bool,Drawing)

And if we want two buttons:

twoButtons = (liftA2.liftA2) (second over) button (translate (1,0) button)

That is, just transform each subGUI as a whole (rather than separating input and output) and combine appropriately. That’s the theory, at least. For this to actualy work correctly, we would need one of the following:

instance Transformable b => Transformable (a,b)
instance Transformable Bool    -- do nothing

Neither of these rubs me the right way. That seems like the wrong instance of transformable (a,b) to me (however, (a,) is a functor, so it’s consistent with what I said earlier). I don’t like having transformable instances for things that don’t actually transform. I’m thinking about maybe a type like this:

newtype WithDrawing a = WithDrawing (a,Drawing)
instance Transformable (WithDrawing a)

(Or the appropriate WithTransformable generalization)



3 thoughts on “Composable Input for Fruit

  1. I used this Transformable approach in Pajama (an unreleased Pan successor), with arbitrary invertible functions in place of affines. My choice was

    instance (Transformable a, Transformable b) => Transformable (a,b)
    instance Transformable Bool    -- identity
  2. Two more comments:

    I don’t think you’d be happy with the WithDrawing solution, as it’s very specific and so doesn’t handle many other transformable things. Also because it prevents transforming more than a single thing in a nested pair structure.

    Also, I suggest simplifying & generalizing your Transformable class by introducing a type of invertible transforms (restricted to affines, if you like). You can form them with translate, rotate, scale, and compose. All Transformable instances then become very simple. I’ll happily share the relevant Pajama code with you.

  3. Yeah, the Transformable class here was just for demonstration; I was planning on making an AffineTransform type as you suggested.

    You’re right, I’m not happy with WithDrawing (or WithTransformable). I’m also not happy with instance Transformable Bool. First off, Bool is not transformable, so why would we say it is? (Of course it actually is, by the trivial transformation) But it’s more that the solution does not scale; we go around saying a bunch of things are only transformable trivially, and then someday someone finds that one of them has a nontrivial transformation that we hadn’t thought of, and not only were we wrong, everyone’s code is wrong if they assumed we were right.

    What feels like the right solution to me is:

    newtype Trivial a = Trivial { unTrivial :: a }
    instance Transformable (Trivial a)

    But it’s a pain in the ass to use. I’m considering doing something like:

    type GUI i o = Behavior i -> Behavior (Drawing, Trivial o)
    type TGUI i o = Behavior i -> Behavior o

    (That was rough off the top of my head, could use some refinement) and provide combinators for composing and transforming GUIs and TGUIs, to localize the newtype bullshit.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s