Tutorial

In this tutorial, we'll build a simple Thing and deploy it on the Internet. The Thing blinks an LED attached to a Raspberry Pi.

Previous experience with Go, HTML, and JavaScript is not required but recommended.

Step 1: Blink the LED

In this step, we'll build, compile and run our Thing on the Raspberry Pi, blinking the LED. (Blinking an LED is the "Hello, world" of the embedded world). In the next step, we'll put a user interface on our Thing.

Hardware Setup

We'll need a few parts to build our Thing.

Parts List:

  • Rapsberry Pi (any model except Pico)

  • An LED

  • A 120ohm resistor

Wire the LED and resistor to GPIO pin 17 and ground as shown below.

Software Setup

  1. Install Raspberry Pi OS Lite

  2. Install the Go programming language

  3. Create a user "merle"

  4. Install the Merle framework

Let's create a Go file called blink.go. This is our first Thing. Compiling and running blink.go on the Raspberry Pi will blink the LED.

We'll break down the code line-by-line in a bit, but if you're familiar with Arduino sketches, then you'll recognize the init() function is same as the Arduino's start() function. Likewise, run() is same as Arduino's loop() function.

BTW, this example uses the excellent Gobot.io to do the heavy lifting to blink the LED. In fact, Merle probably would not be possible without Gobot.io, so a big Thank You to the developers on that project. Nice job. πŸ™

[The files for this tutorial are located in the merle/examples/tutorial directory. Each step in this tutorial has its own sub-directory, blinkv1, blinkv2, etc.]


// file: examples/tutorial/blinkv1/blink.go


package main


import (

"log"

"time"


"github.com/merliot/merle"

"gobot.io/x/gobot/drivers/gpio"

"gobot.io/x/gobot/platforms/raspi"

)


type blink struct {

adaptor *raspi.Adaptor

led *gpio.LedDriver

}


func (b *blink) init(p *merle.Packet) {

b.adaptor = raspi.NewAdaptor()

b.adaptor.Connect()

b.led = gpio.NewLedDriver(b.adaptor, "11")

b.led.Start()

}


func (b *blink) run(p *merle.Packet) {

for {

b.led.Toggle()

time.Sleep(time.Second)

}

}


func (b *blink) Subscribers() merle.Subscribers {

return merle.Subscribers{

merle.CmdInit: b.init,

merle.CmdRun: b.run,

}

}


func (b *blink) Assets() *merle.ThingAssets {

return &merle.ThingAssets{}

}


func main() {

thing := merle.NewThing(&blink{})

log.Fatalln(thing.Run())

}


Build Thing


$ cd merle

$ ./build examples/tutorial/blinkv1


Run Thing


$ cd merle

$ ~/go/bin/blinkv1

[dc_a6_32_7a_a6_d0] Model: "Thing", Name: "Thingy"

[dc_a6_32_7a_a6_d0] Received [SYSTEM]: {"Msg":"_CmdInit"}

[dc_a6_32_7a_a6_d0] Skipping public HTTP server; port is zero

[dc_a6_32_7a_a6_d0] Skipping private HTTP server; port is zero

[dc_a6_32_7a_a6_d0] Skipping tunnel to mother; missing host

[dc_a6_32_7a_a6_d0] Received [SYSTEM]: {"Msg":"_CmdRun"}


The LED should be toggling on/off once a second. If not, recheck the wiring. I get the LED polarity backwards 50% of the time :)

Line-by-Line Discussion

Let's look at Thing code line-by-line (well, lines-by-lines):


package main


import (

"log"

"time"


"github.com/merliot/merle"

"gobot.io/x/gobot/drivers/gpio"

"gobot.io/x/gobot/platforms/raspi"

)

Things are just regular Go programs, with a package main. There are some standard import packages, and some third-party import packages. All Things must include "github.com/merliot/merle". The gobot.io packages are for accessing the Raspberry Pi hardware to blink the LED.


type blink struct {

adaptor *raspi.Adaptor

led *gpio.LedDriver

}

This is our Thing type struct which implements the Thinger interface. Without getting into Go too deep, that just means the Thing has to implement a structure with two methods. The two Thinger methods are Subscribers() and Assets(). We'll see below the two methods implemented.

In our blink structure, we'll save pointers to the GoBot Raspberry Pi adaptor and LED. In general, the Thing type struct will hold Thing state.


func (b *blink) init(p *merle.Packet) {

b.adaptor = raspi.NewAdaptor()

b.adaptor.Connect()

b.led = gpio.NewLedDriver(b.adaptor, "11")

b.led.Start()

}

This is our CmdInit message handler. We'll initialize the hardware resources. See the GoBot documentation on Raspberry Pi platform and LED drivers for more information.


func (b *blink) run(p *merle.Packet) {

for {

b.led.Toggle()

time.Sleep(time.Second)

}

}

This is our CmdRun message handler. It is Thing's main loop. It toggles the LED every second, forever. We'll at least as long as the Thing doesn't die.

CmdRun handler should not exit (unless Thing must quit running, for some reason, like a restart).


func (b *blink) Subscribers() merle.Subscribers {

return merle.Subscribers{

merle.CmdInit: b.init,

merle.CmdRun: b.run,

}

}

This is the first of two methods a Thinger must implement: Subscribers(). Subscribers() is a list of message types Thing subscribes to. For each message type, there is an associated message handler. Thing receives messages on its message bus and for each received message, does a lookup against the subscribers list for the message's type. If there is a match, the associated handler is called, passing in the message (JSON-encoded inside a packet) to the handler.

In our case, we subscribe to two message types: CmdInit and CmdRun. As mentioned earlier, these are the moral equivalent of Arduino sketch's start() and loop(), respective.

In subsequent steps, we'll subscribe to more messages.


func (b *blink) Assets() *merle.ThingAssets {

return &merle.ThingAssets{}

}

This is the second of two methods a Thinger must implement: Assets(). Assets() returns a ThingAssests structure. Assets are Thing's user interface assets, things like HTML, JavaScript, CSS, images, etc.

In our example so far, we have no user interface elements so Assets is empty. We'll add a user interface in a subsequent step.


func main() {

thing := merle.NewThing(&blink{})

log.Fatalln(thing.Run())

}

Every Thing is a Go program with a main() function. Here, we create a new Thing and then run the Thing. The Thinger passed into NewThing() is an instance of our Thing structure blink.

Since thing.Run should run forever; we'll log a fatal error if thing.Run exits.

Log Output Discussion

Every Thing logs output to stdout. (See Running for more info).

Let's go over the output of our Thing:

$ ~/go/bin/blinkv1

[dc_a6_32_7a_a6_d0] Model: "Thing", Name: "Thingy"

[dc_a6_32_7a_a6_d0] Received [SYSTEM]: {"Msg":"_CmdInit"}

[dc_a6_32_7a_a6_d0] Skipping public HTTP server; port is zero

[dc_a6_32_7a_a6_d0] Skipping private HTTP server; port is zero

[dc_a6_32_7a_a6_d0] Skipping tunnel to mother; missing host

[dc_a6_32_7a_a6_d0] Received [SYSTEM]: {"Msg":"_CmdRun"}

Thing's ID is the prefix for each log line. In this case, Thing was assigned an ID of dc_a6_32_7a_a6_d0. If that looks like a MAC address, you're right. Every Thing has an ID and since one wasn't given in the configuration, a default is assigned, made up from a MAC address of one of the network interfaces on the platform. (In this case, it was the MAC address of eth0 on the Raspberry Pi).

You can override the default ID assignment by setting thing.Cfg.ID = "myID" before running.

Thing received a CmdInit message and later a CmdRun message. Every Thing will receive those two messages.

The "Skipping ..." messages are Thing interfaces that didn't get enabled. We'll talk about those interfaces later in the tutorial, so we'll "skip" them for now.

Step 2: Add User Interface

In this step, we'll add a user interface (UI) to Thing. In following steps, we'll run our Thing on the Internet.

Code for this step: merle/examples/tutorial/blinkv2.

Putting a UI on our Thing lets the user view and modify Thing's state. Thing state is a central concept in Merle. We'll talk about state hand-in-hand when talking about UI.

Thing is a web server and it serves up its UI as a single web page, known as a Single-page application (SPA). (SPA are a great way to have a native mobile application feel without having to write a native mobile application...just need a browser). Thing's UI is stateful and interactive. If Thing's state changes, the UI will reflect the change, immediately (ignoring network latencies). Likewise, if the user makes some state change at the UI, Thing's state will be updated immediately.

To build our SPA, let's add some HTML with embedded JavaScript, and flesh out Assets():


const html = `

<!DOCTYPE html>

<html lang="en">

<body>

<img id="LED" style="width: 400px">


<script>

image = document.getElementById("LED")


conn = new WebSocket("{{.WebSocket}}")


conn.onopen = function(evt) {

conn.send(JSON.stringify({Msg: "_GetState"}))

}


conn.onmessage = function(evt) {

msg = JSON.parse(evt.data)

console.log('blink', msg)


switch(msg.Msg) {

case "_ReplyState":

case "Update":

image.src = "/{{.AssetsDir}}/images/led-" +

msg.State + ".png"

break

}

}

</script>

</body>

</html>`


func (b *blink) Assets() *merle.ThingAssets {

return &merle.ThingAssets{

AssetsDir: "examples/tutorial/assets",

HtmlTemplateText: html,

}

}

There is a single <img> element in the HTML used to display the LED image (on or off).

The JavaScript opens a WebSocket back to Thing and, on opening, sends a GetState message to Thing. Thing will reply with a ReplyState message containing Thing state. Incoming message (from Thing) are handled, in particular "_ReplyState" and "Update" messages. Both messages have the same msg.State field, so they're handled the same. The handler updates the <img> based on msg.State. We add a couple of images to assets/images, and set AssetsDir to the root of the assets directory.


merle/examples/tutorial/

β”œβ”€β”€ assets

β”‚ └── images

β”‚ β”œβ”€β”€ led-false.png

β”‚ └── led-true.png

β”œβ”€β”€ blinkv1

β”‚ └── blink.go

└── blinkv2

└── blink.go


We're not done. Thing must handle the GetState request from JavaScript and reply back with a ReplyState. To do that, we need to define Thing's state. For our simple tutorial, the only hardware state we're tracking is LED state.


type blink struct {

adaptor *raspi.Adaptor

led *gpio.LedDriver

Msg string

State bool

}

We'll mirror hardware LED state with software LED state using the State field of our Thing type struct.


func (b *blink) init(p *merle.Packet) {

b.adaptor = raspi.NewAdaptor()

b.adaptor.Connect()

b.led = gpio.NewLedDriver(b.adaptor, "11")

b.led.Start()

b.State = b.led.State()

}

And initialize the state field.


func (b *blink) run(p *merle.Packet) {

for {

b.led.Toggle()

b.State = b.led.State()

b.Msg = "Update"

p.Marshal(b)

p.Broadcast()

time.Sleep(time.Second)

}

}

We'll modify our main loop to track LED state and broadcast "Update" messages once a second. The real hardware LED state is synchronized with Thing software state, and connected WebSocket clients (SPA above) will get periodic "Update"s when state changes.


func (b *blink) getState(p *merle.Packet) {

b.Msg = merle.ReplyState

p.Marshal(b)

p.Reply()

}


func (b *blink) Subscribers() merle.Subscribers {

return merle.Subscribers{

merle.CmdInit: b.init,

merle.CmdRun: b.run,

merle.GetState: b.getState,

}

}

Add handler for GetState message. getState() replies back the requester with a ReplyState message. Thing type struct has a Msg field, which we use to set message type before replying. The call to p.Marshal(b) JSON encodes our Thing type struct into a ReplyState message. Only exported fields of Thing type struct are included in the message.


func main() {

thing := merle.NewThing(&blink{})


thing.Cfg.Model = "blink"

thing.Cfg.Name = "blinky"

thing.Cfg.PortPublic = 80


log.Fatalln(thing.Run())

}

Thing returned from NewThing() will have a default configuration. Before running Thing, we can modify the configuration by accessing the thing.Cfg structure (see ThingConfig).

Let's give our Thing a model and a name. And, lastly, let's configure the web server to run on port :80.


Build Thing


$ cd merle

$ ./build examples/tutorial/blinkv2


Run Thing


$ cd merle

$ ~/go/bin/blinkv2

[dc_a6_32_7a_a6_d0] Model: "blink", Name: "blinky"

[dc_a6_32_7a_a6_d0] Received [SYSTEM]: {"Msg":"_CmdInit"}

[dc_a6_32_7a_a6_d0] Public HTTP server listening on port :80

[dc_a6_32_7a_a6_d0] Skipping public HTTPS server; port is zero

[dc_a6_32_7a_a6_d0] Skipping private HTTP server; port is zero

[dc_a6_32_7a_a6_d0] Skipping tunnel to mother; missing host

[dc_a6_32_7a_a6_d0] Received [SYSTEM]: {"Msg":"_CmdRun"}

[dc_a6_32_7a_a6_d0] Would Broadcast: {"Msg":"Update","State":true}

[dc_a6_32_7a_a6_d0] Would Broadcast: {"Msg":"Update","State":false}

[dc_a6_32_7a_a6_d0] Would Broadcast: {"Msg":"Update","State":true}

[dc_a6_32_7a_a6_d0] Would Broadcast: {"Msg":"Update","State":false}

[dc_a6_32_7a_a6_d0] Would Broadcast: {"Msg":"Update","State":true}

[dc_a6_32_7a_a6_d0] Websocket opened [ws:192.168.1.60:41790/ws/dc_a6_32_7a_a6_d0]

[dc_a6_32_7a_a6_d0] Received [dc_a6_32_7a_a6_d0]: {"Msg":"_GetState"}

[dc_a6_32_7a_a6_d0] Reply: {"Msg":"_ReplyState","State":true}

[dc_a6_32_7a_a6_d0] Broadcast: {"Msg":"Update","State":false}

[dc_a6_32_7a_a6_d0] Broadcast: {"Msg":"Update","State":true}

[dc_a6_32_7a_a6_d0] Broadcast: {"Msg":"Update","State":false}


The LED should still be toggling on/off once a second, as in the last step.

"Would broadcast" log messages mean no one is listening. Opening a browser window on Thing's IP address will create a listener. Let's do that and open a web browser on http://localhost. The LED in the browser should be in-sync with the real LED. Open multiple browser windows to http://localhost and see that all are in-sync with each other and the real LED.

The "Would broadcast" log messages turn into "Broadcast" messages once there are one or more listeners. In the log output above, a browser opened a WebSocket shortly after Thing started running and requested GetState. Thing replied back with ReplyState with State=true. The next message is an "Update" message with State=false.

We can get a snapshot of Thing's state by adding /state to the path. This works for any Thing. This is the instantaneous state; re-run the command to get the current state.


$ curl localhost/state

{

"Msg": "_ReplyState",

"State": false

}

Step 3: Protecting State

In this step, we'll tighten up state accesses. In the next step, we'll run our Thing on the Internet.

Code for this step: merle/examples/tutorial/blinkv3.

In a Thing, there are several Go functions trying to read or write to Thing's state. CmdRun main loop runs in its own Go function. Other subscribed message handlers run in separate Go functions. If we aren't careful, these concurrent Go functions may race with each other in accessing Thing state, so we must protect those accesses with synchronization.


type blink struct {

sync.Mutex

adaptor *raspi.Adaptor

led *gpio.LedDriver

Msg string

State bool

}

Let's add a mutex to our Thing type struct:


func (b *blink) run(p *merle.Packet) {

for {

b.led.Toggle()

b.Lock()

b.State = b.led.State() // writing

b.Msg = "Update" // writing

p.Marshal(b) // reading

b.Unlock()

p.Broadcast()

time.Sleep(time.Second)

}

}

Hold the mutex lock while writing to state and then reading state for JSON-encoding of packet message. Release the lock before broadcasting the packet.


func (b *blink) getState(p *merle.Packet) {

b.Lock()

b.Msg = merle.ReplyState // writing

p.Marshal(b) // reading

b.Unlock()

p.Reply()

}

Hold the mutex lock while reading or writing state. Release the lock before replying packet back to requester.

Compiling and running step 3 gives basically the same results as step 2, so we'll skip those details.

Step 4: Thing on the Internet

In this step, we'll run our Thing on the Internet.

Code for this step: merle/examples/tutorial/blinkv4.

In this step we'll put Thing on the Internet by running a second copy of Thing called Thing Prime on a VM on the Internet. There are a few code additions need to turn Thing into Thing Prime. We'll do those first, and then "run our Thing on the Internet".


func (b *blink) saveState(p *merle.Packet) {

b.Lock()

p.Unmarshal(b) // writing

b.Unlock()

}


func (b *blink) update(p *merle.Packet) {

b.saveState(p)

p.Broadcast()

}


func (b *blink) Subscribers() merle.Subscribers {

return merle.Subscribers{

merle.CmdInit: b.init,

merle.CmdRun: b.run,

merle.GetState: b.getState,

merle.ReplyState: b.saveState,

"Update": b.update,

}

}

Thing Prime connects to Thing over a WebSocket, the same way JavaScript in the browser opens a WebSocket back to Thing. Thing Prime uses the same code base as Thing so let's update Thing with additional message handlers to support Thing Prime. In particular, Thing Prime will listen to ReplyState and "Update" messages to save state. Now Thing and Thing Prime are state-synchronized.

Command Line Options

To make our code work as Thing and Thing Prime, we can use Go's flags package to give our Thing program command line options.


func main() {

thing := merle.NewThing(&blink{})


thing.Cfg.Model = "blink"

thing.Cfg.Name = "blinky"

thing.Cfg.PortPublic = 80

thing.Cfg.PortPrivate = 6000


flag.StringVar(&thing.Cfg.MotherHost, "rhost", "", "Remote host")

flag.StringVar(&thing.Cfg.MotherUser, "ruser", "merle", "Remote user")

flag.BoolVar(&thing.Cfg.IsPrime, "prime", false, "Run as Thing Prime")

flag.UintVar(&thing.Cfg.PortPublicTLS, "TLS", 0, "TLS port")

flag.Parse()


log.Fatalln(thing.Run())

}


Gives:


$ ~/go/bin/blinkv4 -h

Usage of /home/merle/go/bin/blinkv4:

-TLS uint

TLS port

-prime

Run as Thing Prime

-rhost string

Remote host

-ruser string

Remote user (default "merle")

Hardware Setup

For Thing, we'll using the Raspberry Pi + LED setup as in step 3, so no hardware changes.

For Thing Prime, any system that has a public routable IP address on the Internet can be used, and can run Merle, will work. It could be your server at home running dynamic DNS or a VM running in the cloud, doesn't really matter as long as it's addressable.

I'm using Linode cloud hosting service in this step to run a basic Linode VM that cost $5/month.

Thing Prime Software Setup

  1. Install Ubuntu on VM

  2. Install the Go programming language

  3. Create a user "merle"

  4. Install the Merle framework

  5. Copy SSH key from Thing to VM

Build Thing and Thing Prime

The same steps are used to build Thing and Thing Prime. Thing will be build on the Raspberry Pi. Thing Prime will be built on the VM. Same source code for both, just different compile targets.


$ cd merle

$ ./build examples/tutorial/blinkv4

πŸ”‘ SSH Password-less Setup

If we haven't already, we need to create and install an SSH key from Thing system to Thing Prime system. This is a one-time operation, but needs to be done before Thing connects to Thing Prime. See SSH Security for more information. There is a helper script to create and push SSH key from Thing to Thing Prime called key-push. On the Raspberry Pi, let's create and push a SSH key to our remote host, linode.merliot.org. (Substitute you're own user and remote host name).

$ ./scripts/key-push merle@linode.merliot.org

This will create (if not already there) two keys id_rsa and id_rsa.pub in the user's .ssh directory. The script will also copy and install the id_rsa.pub key on the remote host.

Run Thing

We'll run Thing on the Rapsberry Pi as before, but this time we'll specify the remote host for Thing Prime command line option -rhost. Our remote host in this example is linode.merliot.org. (Substitute you're own remote host name).


$ cd merle

$ ~/go/bin/blinkv4 -rhost linode.merliot.org

[dc_a6_32_7a_a6_d0] Model: "blink", Name: "blinky"

[dc_a6_32_7a_a6_d0] Received [SYSTEM]: {"Msg":"_CmdInit"}

[dc_a6_32_7a_a6_d0] Public HTTP server listening on port :80

[dc_a6_32_7a_a6_d0] Skipping public HTTPS server; port is zero

[dc_a6_32_7a_a6_d0] Private HTTP server listening on port :6000

[dc_a6_32_7a_a6_d0] Received [SYSTEM]: {"Msg":"_CmdRun"}

[dc_a6_32_7a_a6_d0] Would Broadcast: {"Msg":"Update","State":true}

[dc_a6_32_7a_a6_d0] Tunnel getting port [ssh [merle@linode.merliot.org curl -s localhost:6000/port/dc_a6_32_7a_a6_d0]]

[dc_a6_32_7a_a6_d0] Would Broadcast: {"Msg":"Update","State":false}

[dc_a6_32_7a_a6_d0] Tunnel got port 6001

[dc_a6_32_7a_a6_d0] Creating tunnel [ssh [-CNT -o ExitOnForwardFailure=yes -R 6001:localhost:6000 merle@linode.merliot.org]]

[dc_a6_32_7a_a6_d0] Would Broadcast: {"Msg":"Update","State":true}

[dc_a6_32_7a_a6_d0] Would Broadcast: {"Msg":"Update","State":false}

[dc_a6_32_7a_a6_d0] Websocket opened [ws:[::1]:37346/ws]

[dc_a6_32_7a_a6_d0] Received [dc_a6_32_7a_a6_d0]: {"Msg":"_GetIdentity"}

[dc_a6_32_7a_a6_d0] Reply: {"Msg":"_ReplyIdentity","Id":"dc_a6_32_7a_a6_d0","Model":"blink","Name":"blinky"

[dc_a6_32_7a_a6_d0] Received [dc_a6_32_7a_a6_d0]: {"Msg":"_GetState"}

[dc_a6_32_7a_a6_d0] Reply: {"Msg":"_ReplyState","State":false}

[dc_a6_32_7a_a6_d0] Broadcast: {"Msg":"Update","State":true}

[dc_a6_32_7a_a6_d0] Broadcast: {"Msg":"Update","State":false}

[dc_a6_32_7a_a6_d0] Broadcast: {"Msg":"Update","State":true}

Run Thing Prime

We'll run Thing Prime on the VM, specifying -prime command line option. We'll also specify -TLS 443 to set the HTTPS server port to :443.


$ cd merle

$ ~/go/bin/blinkv4 -prime -TLS 443

[] [Thing Prime] Model: "blink", Name: "blinky"

[] Private HTTP server listening on port :6000

[] Tunnel connected on Port[6001]

[] Sending: {_GetIdentity}

[] Received: {_ReplyIdentity dc_a6_32_7a_a6_d0 blink blinky true 2022-08-07 07:15:21.103778433 +0100 +0100}

[dc_a6_32_7a_a6_d0] Websocket opened [port:6001]

[dc_a6_32_7a_a6_d0] Received [dc_a6_32_7a_a6_d0]: {"Msg":"_ReplyState","State":false}

[dc_a6_32_7a_a6_d0] Public HTTP server listening on port :80

[dc_a6_32_7a_a6_d0] Public HTTPS server listening on port :443

[dc_a6_32_7a_a6_d0] Would Broadcast: {"Msg":"_EventStatus","Id":"dc_a6_32_7a_a6_d0","Online":true}

[dc_a6_32_7a_a6_d0] Received [dc_a6_32_7a_a6_d0]: {"Msg":"Update","State":true}

[dc_a6_32_7a_a6_d0] Would Broadcast: {"Msg":"Update","State":true}

[dc_a6_32_7a_a6_d0] Received [dc_a6_32_7a_a6_d0]: {"Msg":"Update","State":false}

[dc_a6_32_7a_a6_d0] Would Broadcast: {"Msg":"Update","State":false}

[dc_a6_32_7a_a6_d0] Received [dc_a6_32_7a_a6_d0]: {"Msg":"Update","State":true}

[dc_a6_32_7a_a6_d0] Would Broadcast: {"Msg":"Update","State":true}

View Results

Open a web browser on the Raspberry Pi's address to view Thing locally.

Open a web browser on the VM's address to view Thing over the Internet.

Both views show the LED synchronized with the real LED.

Local view of Thing

Remote view of Thing Prime

Note that Thing continues to run regardless of whether or not it is connected to Thing Prime. You can play around with starting and stopping Thing and/or Thing Prime to see how each react.

It is important for Thing to continue to function, running the main loop, keeping the device happy.

That concludes the tutorial.