“Common tests” framework

I’ve started on a system that will generate the same test cases for all languages, and also test them against the denotational semantics. For instance I can write this Haskell:

stream_orelse :: SemanticTest
stream_orelse = SemanticTest "Stream" "orElse1" $
    LetStream (MkStream [([0], 0 :: Int), ([2], 2)]) $
    LetStream (MkStream [([1], 10 :: Int), ([2], 20), ([3], 30)]) $
    LetStream (OrElse (StreamRef prev2) (StreamRef prev1)) $
    AssertEquals prev1 [([0],0),([1],10),([2],2),([3],30)] $
    End

The times are given “semantically”, so [0] and [1] represent different transactions. LetStream is a variable assignment, and prev1 and prev2 refer to previously defined variables.

This will get translated into the following ugly Java and so for all languages:

  public void test_Stream_orElse1() {
    StreamSink<Integer> a = Transaction.run(() -> {
      StreamSink<Integer> a_ = new StreamSink();
      return a_;
    });
    StreamSink<Integer> b = Transaction.run(() -> {
      StreamSink<Integer> b_ = new StreamSink();
      return b_;
    });
    Stream<Integer> c = Transaction.run(() -> {
      Stream<Integer> c_ = a.orElse(b);
      return c_;
    });
    List<Integer> c_0 = new ArrayList<>();
    Listener c_0_l = Transaction.run(() -> {
      Listener c_0_l_ = c.listen((Integer val) -> {
        c_0.add(val);
      });
      return c_0_l_;
    });
    Transaction.runVoid(() -> {
      a.send(0);
    });
    c_0_l.unlisten();
    assertEquals(Arrays.<Integer>asList(0),c_0);
    List<Integer> c_1 = new ArrayList<>();
    Listener c_1_l = Transaction.run(() -> {
      Listener c_1_l_ = c.listen((Integer val) -> {
        c_1.add(val);
      });
      return c_1_l_;
    });
    Transaction.runVoid(() -> {
      b.send(10);
    });
    c_1_l.unlisten();
    assertEquals(Arrays.<Integer>asList(10),c_1);
    List<Integer> c_2 = new ArrayList<>();
    Listener c_2_l = Transaction.run(() -> {
      Listener c_2_l_ = c.listen((Integer val) -> {
        c_2.add(val);
      });
      return c_2_l_;
    });
    Transaction.runVoid(() -> {
      a.send(2);
      b.send(20);
    });
    c_2_l.unlisten();
    assertEquals(Arrays.<Integer>asList(2),c_2);
    List<Integer> c_3 = new ArrayList<>();
    Listener c_3_l = Transaction.run(() -> {
      Listener c_3_l_ = c.listen((Integer val) -> {
        c_3.add(val);
      });
      return c_3_l_;
    });
    Transaction.runVoid(() -> {
      b.send(30);
    });
    c_3_l.unlisten();
    assertEquals(Arrays.<Integer>asList(30),c_3);
  }

Other notable news:

  • The Haskell version is broken on ghc-7.10. I’m working on that. It needs a lot of love all round right now, as do all languages other than Java. This is why I developed “common tests”.
  • A stream can now only have one event per transaction. s1.merge(s2) with no combining function is renamed to s1.orElse(s2) to make it clear that it can drop events in the simultaneous case.
  • The book “Functional Reactive Programming” is nearly finished! Reviewing and tidying up now.

 

Java version is now really solid

All my book writing, has forced me to go over the Java version of Sodium in detail. The Java version is now complete in a way that hasn’t been achieved to date. All the nagging little details are now defined, and the Java version won’t be changing much now. It now builds for Java 1.5 so should work on Android.

I will need to bring the other language implementations up to scratch, meaning that they should look like the Java version. If anyone wants to do that on a particular language, do it any time now and the Java version won’t change annoyingly.

One thing I still want to do is to generate the test cases from a single source, so that I can write them once, and they’ll get magically translated into all target languages. This shouldn’t be too hard. Since there is an executable version of the formal spec, the test cases will themselves be verified, and this effectively proves mathematical compositionality for all the systems.

The C++ version has some memory issues, which makes it leak memory if you use anything reasonably complex that includes a switch. This is because of cyclic smart pointers. I am working on a v2 version that includes a clever (if slightly hacked) cycle detector but it will take me a little while to get to finishing it.

The main text of the book is now finished, and I’ll be doing revisions, so some time this year I will have spare time. I’m looking forward to that!

Denotational semantics for Sodium

I’ve written a denotational semantics document for Sodium which constitutes the formal definition. Here you’ll find a PDF file and some code:

https://github.com/SodiumFRP/sodium/tree/master/denotational

When I’ve finished the book manuscript I’ll do some work on bringing everything up-to-date, as there have been some changes to the code. The Java version is currently the most up-to-date version and will act as reference for the updates to the other versions. Have fun!

Five things I want to do to Sodium

I’m busy writing my FRP book, but when I have some time again, there are FIVE issues that I want to address in Sodium:

  • [Already done in the Java version.] Get rid of the sampleNow infrastructure and replace it with something much simpler. The way it was done was problematic, and we only actually need something very simple.
  • Finish the renaming of Behavior -> Cell and Event -> Stream. This was urged by the publisher because of utterly different OOP meanings that ‘behavior’ and ‘event’ have.
  • We need to be able to say (Java syntax):
    LazyValue<Integer> x = bx.sampleLazy();
    LazyValue<Integer> newX = x.map(x_ => x_ + 1);
    // Also applicative lift operations
    Cell<Integer> by = ey.holdLazy(newX);

    This gives us the complete ability to use sample() with looped values where the loop hasn’t been closed yet.

  • [EDIT: I probably won't do this. See below.] I would like to make it possible to have a Cell type that has no updates() or value() method, so we can make it so changes in Cells are not observable. This makes it possible to simulate continuous time without any possibility of leaks in the abstraction. Here’s what I’m thinking:
    • Make it so hold() returns a subclass of Cell, called StreamCell.
    • updates() and value() are defined on StreamCell only.
    • We’ll also need StreamCellSink and StreamCellLoop, liftSC and mapSC methods.
    • Is there a better name?

    Then we get the best of both worlds: Semantic purity and convenience.

  • I want to make a multi-language test case system, so that each test case is written in each language next to each other so they are easy to maintain. Run it and you’ll get a big table of language vs. test case, with a tick/cross in each square. It should be possible to use it on systems with only some of the languages installed.

If I can make these changes, then I’ll be happy that Sodium is finished. Then people can write code on a solid foundation, and in parallel people can work on performance and parallelism. We should always keep the reference implementations in each language separate from the performance/parallel implementations. Then it’s possible to easily test whether bugs are in the performance/parallel Sodium or in the application.

EDIT: updates() is needed in certain practical circumstances for normal work in non-lazy languages. An example of this is the Zombicus example described in chapter 7 of the book. The problem is that without doing this, others is a hard cyclic reference during start-up and this breaks it. StreamCell complicates the interface somewhat. Is it worth it for the benefit (waterproof continuous cells/behaviors)? I am trying to “sell” Sodium on the simplicity and practicality of its interface. I’d appreciate your thoughts.

cover

Announcing: Book “Functional Reactive Programming” from Manning Publications

coverAnnouncing Functional Reactive Programming the book by Stephen Blackheath and Anthony Jones from Manning Publications

The book is in production and the first six chapters are now available through MEAP (Manning Early Access Program). The first chapter is free. The book will be Deal of the Day August 17th. 50% off with code dotd081715au

We’ve dealt with event handling frustrations over the years, and this led us to FRP and to writing Sodium. We think FRP is a game changer for user interfaces, video games, control applications and anywhere else an event-based programming model is found. So we decided to write a book that brings FRP to the coalface for a general programming audience. Underlying functional programming concepts are introduced along the way. We show you how to think in FRP using a range of FRP systems and languages.

Announcing: Scala implementation of Sodium

The Sodium team is delighted to announce that Mark H. Butler has ported Sodium to Scala! Thank you so much, Mark.

@Test
def testAccum() {
  val ea = new StreamSink[Int]()
  val out = new ArrayList[Int]()
  val sum = ea.accum[Int](100, (a, s) => a + s)
  val l = sum.updates().listen(out.add(_))
  List(5, 7, 1, 2, 3).foreach(ea.send(_))
  l.unlisten()
  assertEquals(Arrays.asList(105, 112, 113, 115, 118), out)
}

Announcing: C# implementation of Sodium

The Sodium Project is proud to announce a C# implementation of Sodium, by Michael Lund. C# has been on our wish list for a long time, so thank you Michael for your hard work!

    [Test]
    public void TestAccum()
    {
      var ea = new EventSink<Integer>();
      var @out = new List<Integer>();
      Behavior<Integer> sum = ea.Accum<Integer>(100, (a, s) => a + s);
      Listener l = sum.Value().Listen(x => { @out.Add(x); });
      ea.Send(5);
      ea.Send(7);
      ea.Send(1);
      ea.Send(2);
      ea.Send(3);
      l.Unlisten();
      CollectionAssert.AreEqual(new[] { 100, 105, 112, 113, 115, 118 }, @out.Select(x => (int)x));
    }

Should ‘sample’ exist? Should ‘updates’ exist?

I posted this in response to a bug report: My musings about the philosophy behind ‘sample’ and ‘updates’.

Yes – execute + sample should be equivalent to snapshot. I have wrestled with the question of whether I should have done sample, and Anthony has questioned me on it, because in most cases you should use snapshot instead. I think it is correct that I have done it, though, for three reasons:

  1. collect can’t be defined without it (see below). Collect is a little strange, because it treats a behavior as an event. This is a different philosophical question – whether ‘updates’ and ‘value’ should exist, but I have decided that the usefulness of it outweighs the loss of FRP purity. It violates the principle that changes in behaviors should not be observable. I think Heinrich Apfelmus of Reactive Banana disagrees with ‘updates’ for this reason. If you are concerned about FRP purity, it is possible to scan the code to search for uses of ‘updates’. Some systems solve this with three types, Event, Behavior and a sort of Bevent where changes are observable, but the type awkwardness of this could be severe. One use case of ‘updates’ springs to mind: An animation clock as a Behavior of Double. It’s useful to treat it as a behavior and to use ‘updates’ to allow you to generate alarms from it. So in Sodium I have decided that FRP is hard enough as it is and we need all the help we can get for practical work, so this is a compromise, yes, but it seems tolerable to me. ‘value’ in Sodium – which is a bit magical in the way it works – seems to me to be so useful (especially in combination with switch) that it justifies all this agony. So, I am 98% happy with it.
  2. It seems reasonable to me that you should be able to poll a behavior with sample from your imperative code, asking for its current value.
  3. sample runs within a transactional context, so there is nothing actually directly wrong with it that I can see.
-- | Transform a behavior with a generalized state loop (a mealy machine). The function
-- is passed the input and the old state and returns the new state and output value.
collect :: Context r => (a -> s -> (b, s)) -> s -> Behavior r a -> Reactive r (Behavior r b)
collect f zs bea = do
    let ea = coalesce (flip const) (updates bea)
    za <- sample bea
    let (zb, zs') = f za zs
    rec
        bs <- hold (zb, zs') ebs
        let ebs = snapshot f ea (snd <$> bs)
    return (fst <$> bs)

There is one practical problem with sample in strict languages. Sample won’t work too well with a value that has been looped but not defined yet. With Haskell it’s fine because the za value doesn’t get evaluated until the end of the ‘sync’ transaction.

Perhaps the solution in strict languages is to throw an exception and tell the user to be careful about this case. This is a compromise that I’m not entirely happy with.

Sodium and GHCJS

geraldus has written a tutorial about his experiences with Sodium and GHCJS, the Haskell to Javascript compiler for writing client-side web apps in Haskell. With the new GHC release, GHCJS should get much easier to install very soon.

  • how to use Sodium’s merge and switch primitives;
  • how to listen “external” javascript events using GHCJS;
  • how to combine GHCJS and Sodium to create function reactive ui in browser.

Here’s the link… Tutorial. GHCJS and Sodium.