Adding the REST API to IoT Gateway

This is the second milestone of the Organic Gardner IoT project. The first milestones we implemented MQTT subscriptions to gather sensor data. This milestone we are going to add HTTP RESTful API to provide access to that data. The same HTTP library will later serve up our IoT dashboard.

Previously we have added MQTT to the IoT Gateway as part of the Organic Gardner Project. This gave the gateway the ability to collect and store a lot of environmental data from sensors all over the place.

Now we need to provide a way for others programs, like the IoT Dashboard access that data via a REST API.

Implementing a RESTful API

The RESTful API convention is exteremely popular, well supported and is perfectly suited to providing access to the sensor data that has been collected. REST is ubiquitous these days, most people involved in frontend or backend development most likely have quite a bit of experience with REST.

For those not so familiar may find it interesting to know that REST is neither a protocol nor a standard but a rather brilliant conventional usage of the existing HTTP protocol aimed primarily at machine to machine communication.


HTTP is used to serve up a traditional webpage by accepting a client request to a URL along with some optional arguments. The response returns a file with HTML to be displayed in a browser capable of rendering HTML for human consumption.

HTTP when used for automation or machine to machine communication can take the same style URL and optional arguments to produce a file of another format typically JSON these days, making it easy for another program including a web application to easily access and use this data.

JSON is the most widely used formay for RESTful APIs, however other formats like: XML, SOAP, binary or csv can also be used.

Just like MQTT, HTTP is also built on top of TCP which passes along all the goodness TCP provides in particular reliability and efficient use of network bandwidth.

Go HTTP Package

Go has a small, robust and very efficient implementation of the HTTP protocol for both clients and servers. The IoT Gateway will import this package giving it an embedded web server allowing the gateway to serve up the sensor data and eventually the webapp without the overhead of an additional HTTP server like nginx or apache.

The following code is a minimal but full working HTTP server capable of handling heavy loads for a sustained period of time.

import (

// Msg holds each datapoint as it arrives 
type Msg struct {
    StationID   int         `json:"station-id"`
    SensorID    int         `json:"sensor-id"`
    Timestamp   time.Time   `json:"time"`
    Value       interface{} `json:"value"`

// DataCache is an array (queue) of Msg's that are pushed to
// the rear of the message.
type DataCache struct {
    Data    []Msg           `json:"data"`

var cache DataCache

// ServeHTTP satifies the http.Server interface requirement hence 
// allows our global data cache structure to act as the *receiver*
// function for the DataCache struct.
func (d DataCache) ServeHTTP(w *http.Writer, r http.Request) {
    // Only allow read requests
    if r.Method != "GET" {

func main() {

    http.RegisterFunc("/api/v01/data", d)
    // Start listening on port 8080 on any interface
	err := http.ListenAndServe(":8080", nil) 
    log.Println("HTTP Server has stopped ", err)

This example demonstrates a couple points: first it is very easy to add an embedded webserver to an application by following these three basic steps:

  1. Define the endpoints the application will support
  2. Write the callbacks that will handle incoming HTTP requests
  3. Start up the HTTP server and listen for incoming HTTP requests.

Let’s unravel these points a bit more.

Define Endpoints

When discussing APIs the term endpoint simply means the root of the URL that demarcs API calls effecting a particular group of data. In our example we have a single endpoint also known as a URI: /api/v01/data.

Endpoints are appended to a base URI that results in the complete URL that will be used by a client to retrieve the specific data a client is interested in.

For example: If the base URL is the complete URL for this application end points becomes:

The IoT Gateway is now able to allow any HTTP client to send a GET Request to the server and get a bundle of data back.

NOTE: we have not implemented any security preventing anybody access to the data. HTTP and API security will be addressed in a later writing.

Testing an Endpoint

We can use the incredible curl to test our endpoint with a single command.

% curl

We are going to be using curl a lot for testing and debugging as the project progresses. With 4 endpoints to be implemented there will be plenty of testing ahead!

Refering to the line

if r.Method != "GET" {

From this code snippet we can see only the GET HTTP request type is supported. In other words we are allowing READ ONLY requests, no POSTS, PATCHES, or DELETEs (only -R– of CRUD).

Do notice how easy it is to return a proper error to our mis-informed user.

Auto encoding JSON for HTTP Request

This single line of code is pretty amazing. We simply pass the DataCache variable we want to return to the user to the Encode() function to be magically encoded into JSON.


This causes our Msg above to look like this:

    "data": [
            "station-id": "station-01",
            "sensor-id": "tempf",
            "time": "2020-02-02T23:32.12Z",
            "value": "76.32"

One reason it is so easy for Go to encode a structure into JSON is due to reflection with the help of these string literal tags refering to the `json:"xxx"`.

Check out the encoding/json package for more detail.

Classes vs. Functions as Callbacks

Go has a very unique approach to Object Oriented (OO) programming. While it has types and methods it does not have a type hiearchy also known as inheritance. Rather Go approaches the solution by encouraging composition. An interesting nuance that will be handled in another article.

For those familiar with OO terminology the Go type struct is similar to a Class where a reciever is similar to an Object (or instantiation of the type).

That brings us to an interesting twist on providing the typical callback. In most langauges a callback is expected to be a function, in some cases the callback can be a method of a class by using a little bit of pointer trickery to use a method of object allowing the callback access to the objects private variables and methods.

Go can handle simple functions as callbacks by using the http.RegisterFunc() method. However, Go also allows one to add the method ServeHTTP(w http.Responsewriter, r *http.Request) to any type instantly making that type capable of accepting callbacks.

A type with ServeHTTP method

That was all a bit abstract and maybe confusing, a simple example will clarify things a bit.

The Data type and /data endpoint and callback

type Configuration struct {
    Addr        string      `json:"addr"`
    Broker      string      `json:"broker"`

var config Configuration

func (c Configuration) ServeHTTP(http.Request r, *http.Writer) {
    json.NewEncoder(w).Encode(c) // encode config

func main() {

By adding the ServeHTTP function to the type reciever allows these Go instatiations with the ability to handle HTTP requests! I don’t know about you, but I think that is pretty far out!


OK now we have something interesting to start working with! The IoT Gateway collects MQTT sensor data, stores it in RAM and a REST interface that can be used by an API client to fetch the cached data.

Let’s get a demo going…

Mocking MQTT publisher and a REST client

To demonstrate (test) the development we have done for the IoT Gateway implementation to this point we’ll need to fake sensor data using mosquitto_pub and likewise use curl to fetch the fake sensor data as an API client.

Starting the Demo

The iothub will subscribe to the MQTT Topics ss/data/+/+ and ss/net/announce for the network of stations. The HTTP package opens a TCP socket on port 8011 (by default) and waits for incoming HTTP requests.

The top part of the screen from the image below the top part of screen captured below show the output from iothub program.


The lower screen shows two commands: the first command is mosquitto_pub mocking a temperature value.

% mosquitto_pub -t ss/data/station-01/tempf -m 72.3

Followed by a the curl command mocking a REST client much like the dashboard will access historical data sensor data. the to the REST API data endpoint.

This example returns the single data point we have collected.

% curl http://localhost:8011/api/data

If you are playing along at home, feel free to send a lot of data points and mix in new stations and sensors. That will demostrate how the IoTHub caches data. Put mosquitto_pub in a loop and let it run for a while!

What Next? Web App or Real Sensors?

Now we have a very simple but working IoT Gateway! There are more cool features to be added to the gateway, however that let’s consider working on one of the other components of the project.

We are mocking both the Collector responsible for gathering and publishing sensor data as well as the Dashboard which is an API client that consumes the sensor data.

Development can begin on either the Collector or Dashboard at anytime, actually development could have taken place while we started building the hub by also mocking the hub before it was ready.

Gathering real data with real sensor hardware is exciting, I think it will become more meaningful to many people if we start developing the Webapp now since a visual representation of the data will resonate with most people vs. looking at raw data in JSON format.

Next Up the Web App UI

On to the next milestone: building the dashboard to display historical data. The Dashboard is going to be what is known as a Responsive Single Page Application (SPA) written with the Vue reactive framework.

Anyhow, get ready for GUI interfaces next!

Next IoT Dashboard in Vue