ØMQ Extension for Stream

Jon McBee

I've been wanting to write a network extension for Stream for about a year now, adding the ability to extend the mediator to the network and allow pub/sub between actors on different machines.  I finally got the motivation to do it after starting to collaborate with Jarobit Peña Saez at Bits 2 Byte on the Stream project.  This extension is a result of our collaboration efforts and the sharing of ideas.

​​I decided to use ØMQ (pronounced zero-em-queue) as it is a free messaging library for use in distributed applications that handles the TCP socket layer for you.  The base implementation is done in C++ but there are bindings for most popular languages including this binding for LabVIEW, which I used.  A cool aspect of the many bindings available is that ØMQ makes it easy to setup messaging not only between machines on network but between machines running different operating systems and using different programming languages.  As you saw in the demo above it was pretty trivial to connect Stream in LabVIEW to a simple Python script using ØMQ.  ØMQ has great documentation, I recommend that you read more here.

​Let's establish a little vocabulary so we can discuss the design.  The ØMQ Network Extension for Stream is meant to allow Stream actors to communicate across a network without the need to write any TCP/IP level code.  Let's refer to a system of actors on a machine as a microsystem, and the system comprised of many machines as a system.  We can now say that our system is comprised of many microsystems where each microsystem is comprised of many actors.  In the current implementation of the ØMQ Network Extension for Stream there must only be one server per system, and it is up to the developer to make this so.  I don't love the fact that we need a server and think that the design can be improved, but as it stands today there must be one server to rule them all, one server to find them, one server to bring them all and in the darkness bind them (as the saying goes).

​Within each microsystem there can be one or more mediators, this is really a design decision left to the architect.  I typically only have one mediator per microsystem but I know others like to spin up one global mediator per microsystem and one private mediator for each actor in the microsystem.  One thing to keep in mind with this implementation is that every mediator creates its own ØMQ context, and then using the context to create sockets.  The context is the container for all sockets in a single process and the ØMQ guide recommends "Do one create context at the start of your main line code, and one destroy context at the end."

​So how does the ØMQ Network Extension for Stream work?  At a high level the extension is a message broker that uses the ØMQ Publish/Subscribe pattern as the transport, and serialized JSON data packets as the data message.  The system level server maintains a list clients and spins up LabVIEW Topic Actor for each unique topic in the system.  The Topic Actors are assigned an available port and the server handles distributing the updated ports to all known clients as they come in.  The Topic Actors then each subscribe to the appropriate ØMQ endpoint and register as publishers on the appropriate local Stream topics.  For this to work each message object must override the Pack.vi and Unpack.vi methods, these methods handle serialization and deserialization of the message objects.

One nice thing about using ØMQ is that it handles a lot of the hard stuff for you.  ØMQ doesn't care if the server or the clients spin up first, it handles reconnecting after network dropouts, and makes it easy to bring the clients or the server down and back up again without the entire system going down.  I did add a client heartbeat message to the server so that when clients go down the server recognizes it and broadcasts the change to the other clients.  Likewise, if I bring the server down it will alert the clients, allowing each microsystem to run isolated until a new server spins up, at which point the mediator stitches the topics back together.

As it stands right now the ØMQ Network Extension for Stream works as designed, but there are a handful of design decisions that I would like to change.  If you would like to contribute or collaborate on this project please let us know!  The code for this project has been open sourced under an MIT license and can be found at this Composed Systems Bitbucket page.

---

Jon McBee is the Founder and Managing Partner at Composed Systems and is a Certified LabVIEW Architect, Certified LabVIEW Embedded Developer, Certified TestStand Developer, an NI Certified Professional Instructor, a LabVIEW Champion, and a Certified ScrumMaster