The MQTT protocol: Sparkplug B and practical implementation

The second of a two-part article explaining the fundamentals of the MQTT protocol, and how it can be adapted to industrial applications.

In Part 1 of this article, the general operation of the MQTT protocol was introduced and its features and benefits described. Because the default specification of MQTT lacks features necessary for practical application in an industrial IoT environment, the Sparkplug MQTT Topic Namespace and Payload Definition was introduced. In Part 2 we continue with more explanation of Sparkplug B, and how MQTT with Sparkplug B can be practically implemented.

Sparkplug B features

Topic namespace

Sparkplug B (SpB) defines a standard format for MQTT topic paths, creating a unified namespace for all SpB clients on the network:

spBv1.0/<Group ID>/<MESSAGE TYPE>/<Edge Node ID>/<Device ID>

The elements are defined in Table 1.

Table 1: Sparkplug B topic namespace definition. For a clearer image click here.

Payload definition

Sparkplug B defines a standard, structured, data-rich but efficient payload format.

The core payload contains a structured series of key-value pairs defining an array of metrics and associated metadata. The specification defines a variety of optional fields for each metric, such as name, description, datatype, checksum, historical data flag and many more.

Data type indicators accommodate complex types common in industrial applications, like matrices and user-defined types (UDTs). The payload specification also permits an array of custom properties for each metric, such as engineering units or scaling limits, that are published along with the primary process variable or data object.

The full payload is then timestamped, sequenced and encoded using Google protocol buffers (protobufs), an efficient, interoperable, binary representation of the structured data, which maintains a small on-the-wire footprint without sacrificing complexity. When decoded, the payload is typically represented in JavaScript object notation (JSON) but is compatible with many formats.

State management

Sparkplug introduces the concept of birth and death certificate messages to define and ensure the use of appropriate state monitoring mechanisms.

The death certificate simply formalises the use of MQTT’s last will and testament message. Triggered when a client’s keep-alive timer has expired, the death certificate indicates an unexpected disconnection. The topic path follows the standard Sparkplug format, using the message types NDEATH and DDEATH, indicating that the certificate pertains to a node or to an attached device, respectively.

The birth certificate, on the other hand, is a new addition of the Sparkplug spec. It is a special message, utilising message types NBIRTH or DBIRTH, that a client is required to publish for itself and each of its attached devices when first coming online. These messages contain all the published topics for that client or device and inform all subscribers that the client or device is online.

Primary application

While an MQTT network may contain any number of application clients, an industrial setting often has a single critical application used for operations management or process control, like an IIoT or SCADA host. This application is typically the only one permitted to send commands into the network. The Sparkplug specification adapts the MQTT framework to this reality by allowing one application node to be designated as the primary application.

Figure 1: The MQTT/Sparkplug B architecture defines clear roles and behaviours for MQTT clients and their data, adapting the underlying MQTT framework to better support typical SCADA/IIoT use cases.

An SpB-compliant primary application — like Cirrus Link’s MQTT Engine module for Ignition SCADA or Canary Labs’s MQTT Data Collector — publishes special birth and death certificates using the topic STATE/<client ID>. Unlike regular death certificates, these messages are published with QoS 1 and are retained by the broker to ensure that any MQTT client can identify the primary application’s state at any time. This mechanism enables two additional functions that are critical to building fault-tolerant industrial networks:

• Enhanced failover: If an MQTT client loses its current broker connection and other connections are available, it can switch automatically. In addition, a Sparkplug B-compatible client can also switch connections based on the value of the primary application’s STATE topic. When the primary application’s connection is interrupted, its death certificate is published by the broker, indicating that STATE has changed to OFFLINE. SpB clients will see this and begin searching through their alternate connections for a broker that reflects an ONLINE status for the application.
• Store-and-forward historisation: In combination with the flags built into the Sparkplug B payload, a client can indicate that a message contains historical, rather than real-time, data. If capable, a client can begin storing records any time it or the primary application disconnects, queuing them up until it is assured of safe delivery, then publishing them as historical records. Since these messages are distinguishable from real-time data, store-and-forward provides better continuity for time-series data and time-critical operations than MQTT’s standard quality of service levels.

Figure 2: The primary application mechanism enables smart failover and historisation.

Advantages for industry

Interoperability

The Sparkplug B specification addresses the potential for inconsistency in MQTT implementations by defining standard client roles and data interfaces designed around industrial applications. By standardising on Sparkplug B, MQTT clients from different vendors can identify, interpret and make use of published data without needing to know the details of the originating client. Even brownfield sites with legacy automation networks and devices can leverage Sparkplug because of its explicit support for gateway-attached devices.

Unified namespace

This level of interoperability satisfies a critical requirement for digital transformation by making possible an enterprise-spanning unified namespace, a common data source defining all business data. Instead of hunting for operations data spread throughout a hierarchy of applications (PLC > SCADA > MES > ERP, for example), each with its own schema and data structures, MQTT devices and applications all contribute to a single data endpoint: the MQTT broker.

Enterprise clients can consume data from the field via the broker, using Sparkplug to provide a common exchange format and uniform context. Their results can be published back into the network to be detected, interpreted and acted on by other clients. In this way, data can be shared seamlessly throughout the organisation, eliminating data silos, reducing the potential for discrepancies and allowing a connected system to scale up smoothly.

Figure 3: With Sparkplug, data from many domains can be shared interoperably across an organisation.

Enhanced data integrity

Sparkplug also improves data integrity and consistency with its combination of reliability features. By enforcing the last will and testament mechanism, all subscribers are notified when pertinent data becomes stale, and the birth certificate lets them know when data is fresh again. And for applications like those in regulated industries that require historical data for auditing or root cause analysis, store-and-forward historisation provides protection against critical data loss in the face of network or client instability.

Low administration

The combination of Sparkplug’s enhanced state management and interoperability unlocks an additional contributor to network scalability by reducing the administrative overhead required to integrate each node and device.

As mentioned, integrating traditional industrial data involves time-consuming tag/variable mapping between applications. But with Sparkplug’s birth certificate mechanism, this process is sped up by orders of magnitude. Subscribers, the primary application in particular, can use the information published in the birth certificate to map out available topics in a matter of moments. Since a birth certificate is required from each client whenever it connects to the network or updates its topic structure, the tag hierarchy can be discovered automatically. And thanks to Sparkplug B’s payload structure, it comes complete with all metadata and UDT definitions.

Building mission-critical MQTT networks

Sparkplug B fits MQTT to the needs of industrial applications by defining an open, interoperable standard for industrial data exchange that supports common use cases and the complex data they require, without compromising efficiency. By explicitly defining client roles and message types, Sparkplug B also enhances MQTT’s ability to reliably deliver data to the right people at the right time, all of which can give users confidence as they scale up the network.

However, MQTT/Sparkplug B is of little value without robust support from all network clients. Specific features that Sparkplug provides, like legacy device integration and store-and-forward historisation, can’t be fully realised without field device clients that implement the necessary gateway and storage functions.

And fundamental network reliability is undermined unless every network participant implements complementary cybersecurity and connectivity features. Even the sheer volume of data generated by a large Sparkplug B IIoT network can be overwhelming unless clients are able to filter and sanitise data locally before publishing. Unfortunately, traditional industrial devices were designed with a narrow scope of operation in mind and lack much in the way of general-purpose processing, communication and storage.

To address these general gaps in operations technology (OT), many industrial automation vendors are drawing inspiration from information technology (IT), turning to an approach known as edge computing — a form of distributed computing that addresses bandwidth constraints in global and regional networks by moving computing resources geographically closer to areas of high demand on the edge of the network.

Industrial edge computing

Industrial edge computing mimics this approach by embedding data processing, networking and storage at the local process level, rather than strictly in the network core. With more resources at the edge, data can be prepared locally before broad distribution, modern security measures can be layered onto legacy systems, and advanced functionality can be embedded in the process in support of a resilient network.

Industrial edge computing devices come in many varieties, including edge I/O modules and edge programmable industrial controllers.

• Many edge I/O modules can scale horizontally, integrating a variety of traditional field signals (V, mA, ICTD, TC, thermistor, discrete I/O, dry contacts, and so on) and making their data available directly to Ethernet networks.
• Edge controllers offer more computing and networking power alongside a traditional PLC/PAC control engine, providing a broader array of integration options and allowing disparate automation networks to be combined into a single system.
   

Using industrial edge devices like these in combination with appropriate server hardware, users can build up a secure MQTT/Sparkplug B network following a basic pattern:

First, instrument assets if needed, and connect the I/O to an edge device.

If existing controls are in place already, they can be connected to a segmented network interface on the edge controller, which can use OPC UA drivers to pull the data in.

Then, the edge controller can open an encrypted, authenticated connection to an MQTT broker, residing either in the cloud or somewhere on premises, ideally in the corporate network’s DMZ.

Now, applications can be plugged into the infrastructure. A gateway application with a Cirrus Link MQTT Engine, can subscribe to the entire system’s tags and make all that data available to other applications, including back into OPC, if needed.

With this foundation, the system can scale up with additional edge devices or additional brokers to build out a highly resilient, high-performance data exchange and command-and-control system that’s completely secure.

Conclusion

MQTT is popular, proven and well supported in enterprise and consumer applications. It makes organisation-spanning data exchange possible by decoupling data producers and consumers using a brokered publish-subscribe architecture and by defining a lightweight, data-agnostic communication format that supports millions of connections.

Sparkplug B defines an MQTT implementation standard that ensures client interoperability and enhances MQTT with features designed to support the demands of mission-critical industrial systems. The resulting infrastructure delivers data that is fit for use in operations, gracefully handles instability and helps organisations scale by reducing administrative overhead.

Industrial edge devices complete the architecture by providing complementary reliability in the physical layer, along with a variety of integration options and sufficient computing power to process and transmit field data efficiently and securely.

Top image: ©stock.adobe.com/au/Michael Traitov