IoT Bridges as Microservices in DC/OS

23 August 2016
By Rafael Zubairov, Senior Architect

IoT technologies have become a natural part of our life. And therefore the number of IoT protocols has grown exponentially. Device manufacturers (e.g. wearables, sensors, or temperature / light / environment controllers) use their own protocols for communication and cloud enablement. The cloud infrastructure can also use custom protocols of a higher level to receive device state updates and management information.

Some platforms use low-level protocols (e.g. COAP) to interact with devices and MQTT to communicate with a central hub. Others use their own standards, built upon HTTP or WebSockets. Central hubs in turn can use a variety of protocols and methods for marshalling data. And imagine the mashup of all those protocols to connect to external services.

The chances of finding a solution combining all required protocols are close to zero. This solution would have to have implemented adapters gathering all the platforms together.

There are several approaches to solving the problem of combining different protocols together. One of them is based on using existing ESB (Enterprise Service Bus) products. It allows leveraging the existing connectors. And future development of the infrastructure is based on adding the new integration points. This architecture is not a universal solution and it’s limited by the ESB product’s constraints. For example it can have scalability, business logic, or languages and libraries limits.

The alternative approach is to combine resource management frameworks with a micro-service architecture. This solution design approach makes the system more universal, extendable, and scalable. And moreover it doesn’t require developing a custom framework covering all types of communication and business logic.

Also, this approach makes the micro-service architecture and bridges be seen as business logic units. The DC/OS framework is used as a solution basis. DC/OS allows deploying containerized micro-services in a manageable environment and it makes business solutions isolated. And as a result it allows overcoming the internal limitations of any bridge used.

image03

The adapters’ business logic is implemented and packed as a separate container, which is deployed to a DC/OS managed environment. Container state monitoring and adapter resource management are performed on the abstraction level.

Every new adapter will be isolated from all other functionality. The diagram below shows the abstraction levels. All lower levels can be changed without changing the business logic. It is abstract and it doesn’t depend on hardware, nor the deployment method.

image02

There’s a couple of service integration examples. We will consider integrating the Scriptr service with 2 platforms that allow connecting devices:

We will use JavaScript and NodeJS to describe our adapters’ logic.

To run the examples you will need to deploy DC/OS locally, on AWS, or on any other hosting service. The instructions and sets of scripts are available on the DC/OS site.

Our first example will connect the IBM platform with Scriptr. The diagram below shows the details.

image01

IBM Internet of the Things allows creating a virtual device. To make it simpler you can use the following link to get a device emulator. Scriptr will handle our new device data management. So let’s connect our device to it.

We will use Node-RED as a gateway for our device. The default data flow has a virtual device sending information about its state over the MQTT protocol.

Our adapter will consist of 2 components:

  • MQTT broker;
  • Business logic gathering data from MQTT and sending it to Scriptr.

Making the MQTT broker visible to the external environment will require installing a load balancer available as a packet. The detailed HAProxy deployment documentation can be found here. Also, pay attention to the information about the MQTT service descriptor for Marathon — we need to specify a special label “HAPROXY_GROUP”:”external”.

The second broker component uses the service discovery mechanist to search for addresses and open MQTT broker ports. Once they’re found, it established a connection between the broker and Scriptr. Messages receiver from the broker are transmitted to Scriptr. If changing the message format is required, it can be performed in the adapter decreasing the transmitted data volume.

The adapter source code is available in the GitHub repository, the link to which can be found at the end of the article below.

Using public docker containers will limit our task to just creating a Marathon service descriptor file. Private containers can be used with Amazon EC2 Container Service. In this case our adapter image pushes to Container Service, and then Marathon downloads it from the private repository.

The adapter configuration is stored in the service descriptor file. In our case it’s the Scriptr token and the channel to be used to transmit data. Once the required parameters are specified, we can deploy the container with the adapter. After connecting to Scriptr using the websocket protocol, we’ll start receiving messages:

wscat -c wss://api.scriptr.io/my_token_goes_here

And then we subscribe in the command line to the messages in the Scriptr channel:


{
    method: “Subscribe”, 
    “params”: {
        “channel”: “my_channel”
    }
}

You can unsubscribe from the channel messages by closing the console or typing the command:


{
    method: “Unsubscribe”, 
    “params”: {
        “channel”: “my_channel”
    }
}

Our second example connects IoT-X and Scriptr. IoT-X provides a WebSocket interface to read the transmitted device data. The diagram below shows the intercommunication scheme.

image00

The second example allows turning off the data message thread received by Scriptr. To see the thread follow the instructions below:

wscat -c wss://api.scriptr.io/my_token_goes_here

And then we subscribe in the command line to the messages in the Scriptr channel:


{
    method: “Subscribe”, 
    “params”: {
        “channel”: “my_channel”
    }
}

To allow transmitting data use the following command:


{
    "method": "Publish", 
    "params": {
        "channel": "scriptr_channel", 
        "message": "Subscribe"
    }
}

To stop data transmission:


{
    "method": "Publish", 
    "params": {
        "channel": "scriptr_channel", 
        "message": "Ubsubscribe"
    }
}

And to see the statistics:


{
    "method": "Publish", 
    "params": {
        "channel": "scriptr_channel", 
        "message": "Stats"
    }
}

You can unsubscribe from the channel messages by closing the console or typing the command:


{
    method: “Unsubscribe”, 
    “params”: {
        “channel”: “my_channel”
    }
}

The source code is available on GitHub.


Add Comment

Name Mail Website Comment