Using Device Twins with Azure IoT Hub

In today’s article I’d like to discuss Device Twins and how you can use them with Azure IoT Hub. I will be discussing what Device Twins are, how they are used and then provide a code example of using them with Azure IoT Hub.

What are Device Twins?

Device Twins are JSON documents that store state information about IoT devices. This includes state information along with metadata about the device. Configurations and conditions on the device are also stored. Azure IoT Hub maintains a device twin for each registered device that is connected to that hub.

You can use Device Twins to store specific metadata about your devices in the cloud. For example, you can keep track of a device location by storing location metadata about that device in its Device Twin.

Device Twins can also be used to report state information about the device. This can be useful to determine if a device is connected to IoT Hub, or if it is connected to a WiFi network.

Additionally, the process of updating the firmware on one or more devices can benefit from Device Twins, as firmware installation progress can be reported to a back-end application.

Device Twin Structure

The JSON document that makes up a Device Twin has a few components that I will describe in this section. Here is an example of the overall JSON structure of a Device Twin:

Figure 1: Device Twin Structure

Tags

The Tags section is used exclusively by a back-end solution. Device applications cannot see or query tags at all, but a back-end solution does have the capability to read/write tag values to this section of the Device Twin. The information in this section can be whatever identifying data you need on the back-end solution side. Here is an example, which shows tags for the device location:

{
…
    "tags": {
        "deviceLocation": {
            "city": "Chicago",
            "building": "Willis Tower",
            "floor": "12",
            "room": "NE Conf 1"
        }
    },
…
}

Reported Properties

This section is used in coordination with the Desired Properties section to synchronize device configurations. The device application has the capability to read and write data to this section while a back-end solution can read information from this section as well as receive change notifications when data changes. Typically, this section is used by the device application to report its current state to the cloud.

For example, let us say that we want to report to a back-end solution the current WiFi information for the device. We would serialize the following JSON string into the proper format for Reported Properties and then report the state to the IoT Hub that the device is registered with.

{
…
    "properties": {
…
        "reported": {
            "wifi.wifiIP": “192.168.3.100”,
            "wifi.wifiMask": “255.255.255.0”
        }
    }
}
Figure 2: Sending Reported Properties (device app)

The IoT Hub that maintains the Device Twin for the registered device would update the information being received from the device. In our example, the WiFi IP and Mask values would be updated accordingly in the device’s device twin. A solution back-end can then query the Reported Properties to retrieve these values.

Figure 3: Reading Reported Properties (back-end solution)

Desired Properties

This section is used in coordination with the Reported Properties section to synchronize device configurations. A back-end solution has the capability to read and write data to this section, and the device application can read and receive change notifications for this section. Typically, this section is used by a back-end solution to communicate to the device of some change requested that the device application should handle.

For example, let us say that we want to change the interval time value on the device that is used for sending data to IoT Hub. A back-end solution could update the desired properties and that change would be communicated to the device. As shown in the JSON snippet below we can set the ‘sendDataInterval’ value in the Desired Properties section to ‘5000’ indicating we want to set the interval value on the device to 5 seconds.

{
…
    "properties": {
        "desired": {
            "sendDataInterval": "5000
        },
…
    }
}
Figure 4: Update Desired Properties (back-end solution)

When a change is made to the Desired Properties for a device, a notification event is sent to that device informing the device of the change. Code on the device would configure a Device Twin callback method that would be fired when the change notification is sent to the device.

Figure 5: Configure Device Twin Callback Method (device app)

The callback method would extract the JSON information which would contain the Desired Properties section of that device’s Device Twin record. The code would then be able to interpret what has changed and perform whatever necessary actions are required.

Figure 6: Device Twin Callback Method (device app)

A device application should perform whatever actions necessary to process the change in the Desired Properties and then send back an update (via the Reported Properties) to the cloud.

Device Identity Properties

This is the root of the JSON document and contains the read-only properties that specify the device identity. This information can be used to retrieve the device identity along with connection state and authentication type.

{
    "deviceId": "devA",
    "etag": "AAAAAAAAAAc=", 
    "status": "enabled",
    "statusReason": "provisioned",
    "statusUpdateTime": "0001-01-01T00:00:00",
    "connectionState": "connected",
    "lastActivityTime": "2015-02-30T16:24:48.789Z",
    "cloudToDeviceMessageCount": 0, 
    "authenticationType": "sas",
    "x509Thumbprint": {     
        "primaryThumbprint": null, 
        "secondaryThumbprint": null 
    }, 
    "version": 2, 
…
}

I hope this article was helpful in explaining the details of Device Twins. For more detailed information please visit https://docs.microsoft.com/en-us/azure/iot-hub/iot-hub-devguide-device-twins.

If you would like to download the full source code for this example, please visit my github repository here.

Thanks for reading!

Reading Sensors on the MXChip AZ3166 IoT Device

Recently I have been playing around with the MXChip IoT DevKit device (AZ3166, you can purchase one here) and I’ve noticed that while there are great code samples designed to get you started with the MXChip DevKit, there aren’t any basic examples showing you how to use the different sensors built into the device. This article aims to go through each of the available on-board sensors (temperature & humidity, pressure, accelerometer & gyroscope, and magnetometer) and give examples on how to use them.

I am assuming you already went through the steps to update to the latest firmware and connect to your WiFi network (steps are here; ignore the sections regarding connecting to Azure IoT Hub as we do not need that for this article). Also make sure your VSCode environment has the Azure IoT Tools extension installed.

There are 4 on-board sensors that come with the MXChip IoT DevKit device:

  • Temperature & Humidity
  • Pressure
  • Accelerometer & Gyroscope
  • Magnetometer

The MXChip IoT DevKit device is Arduino-compatible, and there is support for this hardware via the Microsoft Azure IoT Developer Kit in VSCode (via Arduino Board Manager). Once you’ve connected your MXChip device to your WiFi you can start creating Arduino sketches to upload the code for.

Setup

First off, you will want to create a new project in VSCode. Open VSCode and press F1 and type in Azure IoT Device Workbench in the textbox that appears.

Figure 1: Creating a New Project

Find the entry labeled Azure IoT Device Workbench: Create Project… and click on it. Enter a descriptive name for your project.

Figure 2: Project Name

Once you press Enter you will see another selection dropdown asking for the platform for this project. Choose Arduino as our project will be based on the Arduino platform.

Figure 3: Platform Selection

Pressing Enter will bring you to the next selection choice: the project template. This is where the different boards and service functionality are listed for the project being created. Since we are using the MXChip we want to select the choices that are available for the MXChip. You will notice that there are a couple of entries that have support for Azure IoT Hub or Azure Functions. For this article, we only need the base device support so select MXChip IoT DevKit selection.

Figure 4: Project Templates

At this point VSCode will create a new workspace with the project name you provided and create a device.ino file for your Arduino sketch.

If you look under the .vscode directory in your project (under Device) you will see 4 JSON files that were automatically created as part of this project. These files define what board the project is using along with other information such as the sketch name and the serial port being used.

Figure 5: Project Configuration

General Information

Since we are coding an Arduino sketch for our device, I thought I’d briefly go through some of the general code and setup that is needed before we delve into the sensor code specifics.

First off, there are the setup() and loop() methods, which need to be coded first. The setup() method is for code that is executed once when the sketch is run, while the loop() method runs on a continuous loop (after setup is complete).

There are a few initializations we need to perform when the code first runs. These are as follows:

  • Initialize the OLED display
  • Start the serial support
  • Initialize WiFi

First thing is to initialize the OLED display on the board so we can display text. Fortunately, the Microsoft Azure IoT Developer Kit library provides a Screen object that we can use (this is defined in the Arduino.cpp file):

Figure 6: Initializing Screen object

Also shown is the Screen.print() method which allows us to display text on the OLED display.

Next is to enable the serial port support for the board. This is done by another class in the Microsoft Azure IoT Developer Kit library called Serial. We want to utilize 115200 baud so we pass that number as the parameter to the Serial.begin() method to start our serial port.

Figure 7: Serial Support

We also want to make sure our WiFi connectivity is working properly. Even though our example in this article doesn’t explicitly use WiFi, it is a good practice to enable this and verify connectivity since we would need this for any communication with the cloud or over-the-air (OTA) updates, such as firmware updates.

Again, the Microsoft Azure IoT Developer Kit library files have support for WiFi for the AZ3166 chipset (see the AZ3166WiFi.cpp file) so all we have to do is utilize the WiFi class in our InitWiFi() method. This method enables WiFi for the board and checks if it connects successfully. If it does, it sets the hasWiFi flag to true and prints the IP address the device is using to the screen. The setup() method will return if the WiFi is not connected.

Figure 8: WiFi Initialization

The final initialization we need to perform is the individual sensors we are utilizing. We need to initialize the temperature & humidity sensor, the pressure sensor and finally the magnetometer sensor. In order to do this, we need to instantiate a helper class of type DevI2C which will handle the details of I2C peripherals. We also need to pass 2 parameters to the instantiation of this helper class which correspond to GPIO pins 18 and 17 respectively (D14=GPIO_18, D15=GPIO_17).

Figure 9: I2C Helper Class Instantiation

There is also the main loop() method which runs on a continuous basis (after setup() finishes executing). This method will repeat indefinitely and here is where we want the code to do our work. Please note that I have a line of code that calls a delay so the loop doesn’t continuously run over and over again without a pause. We use this delay to give some time for the device to perform any background tasks it hasn’t yet finished.

Figure 10: Main Program Loop

Temperature & Humidity Sensor

In order to read from the temperature and humidity sensor we need to initialize this sensor in the setup() method. This is done by instantiating a new class of type HTS221Sensor (which is part of the Microsoft Azure IoT Developer Kit library). This class takes a parameter of type DevI2C so we pass the pointer to the DevI2C variable we created earlier. We then call the init() method to initialize the sensor.

Figure 11: Initializing the Temperature & Humidity Sensor

Once we have initialized this sensor we can now use it in the main loop() method. There are a number of supported methods for this sensor (located here). We are reading the temperature and humidity sensor readings by calling getTemperature() and getHumidity(), respectively. These methods read the sensor and place the results into a float variable which we pass as a pointer to the methods. Please also note that we enable the sensor when using it and disable the sensor when finished.

Figure 12: Reading Temperature, Humidity Sensor

Pressure Sensor

In order to read from the pressure sensor we need to initialize this sensor in the setup() method. This is done by instantiating a new class of type LPS22HBSensor (which is part of the Microsoft Azure IoT Developer Kit library). This class takes a parameter of type DevI2C so we pass the pointer to the DevI2C variable we created earlier. We then call the init() method to initialize the sensor.

Figure 13: Initializing the Pressure Sensor

Once we have initialized this sensor we can now use it in the main loop() method. There are a number of supported methods for this sensor (located here). We are reading the pressure sensor reading by calling getPressure(). This method reads the sensor and places the results into a float variable which we pass as a pointer to the method.

Figure 14: Reading Pressure Sensor

Accelerometer & Gyroscope Sensor

In order to read from this sensor we need to initialize it in the setup() method. This is done by instantiating a new class of type LSM6DSLSensor (which is part of the Microsoft Azure IoT Developer Kit library). This class takes a parameter of type DevI2C so we pass the pointer to the DevI2C variable we created earlier. We then call the init() method to initialize the sensor. Please also note that we can enable the accelerometer, gyroscope, and pedometer individually if we want.

Figure 15: Initializing the Accelerometer & Gyroscope Sensor

Once we have initialized this sensor we can now use it in the main loop() method. There are a number of supported methods for this sensor (located here). Note that we read the positional axes (x, y, and z) for the accelerometer and gyroscope, while we read a step count for the pedometer.

Figure 16: Reading Accelerometer & Gyroscope Sensor

Magnetometer Sensor

In order to read from this sensor we need to initialize it in the setup() method. This is done by instantiating a new class of type LIS2MDLSensor (which is part of the Microsoft Azure IoT Developer Kit library). This class takes a parameter of type DevI2C so we pass the pointer to the DevI2C variable we created earlier. We then call the init() method to initialize the sensor.

Figure 17: Initializing the Magnetometer Sensor

Once we have initialized this sensor we can now use it in the main loop() method. There are a number of supported methods for this sensor (located here). Note that we read the positional axes (x, y, and z) for this sensor.

Figure 18: Reading Magnetometer Sensor

Code Execution

In order to run this code on our MXChip device, we first need to compile the code successfully. In VSCode, press F1 and enter Azure IoT Device Workbench: Compile Device Code and select the entry. This will compile the code for your Arduino sketch file.

Figure 19: Compiling Arduino Sketch File

Once the file compiles successfully, you are now ready to upload the code to the device. Again, press F1 and type Azure IoT Device Workbench: Upload Device Code and select the entry. You can alternatively click the Upload button in the upper right.

Figure 20: Uploading Code to Device

Once the code uploads successfully, it begins executing immediately. If you open the Serial Monitor in VSCode you can see the output, which should look something like this:

Figure 21: Code Execution Output

I hope this article was helpful in explaining the details of utilizing the different on-board sensors in the MXChip. If you would like to download the full source code for this example, please visit my github repository here.

Thanks for reading!

Using Azure Stream Analytics with IoT Devices

Azure Stream Analytics is Microsoft’s PaaS (platform-as-a-service) event-processing engine that allows you to analyze and process large volumes of streaming data from multiple incoming sources. You can configure different input sources including IoT devices, sensors or business applications for data ingestion. Delivery outputs can also be configured to send the processed data to those destinations for performing actions or further analytics.

Overview

Azure Stream Analytics works on the concept of jobs. These jobs consist of one or more inputs, a query and an output. Stream Analytics ingests incoming data from one or more of these configured inputs and then a query is applied to this data to filter, sort, aggregate or join other reference data over a period of time. This transformed data is then sent to the configured output for storage or further downstream processing.

In this article, we will be using an example where I have a physical IoT Device (an MXChip dev kit) sending temperature data for processing.

Resource Creation

You can create an instance of Stream Analytics in the Azure Portal. Login to your portal at https://portal.azure.com. Once authenticated, click on Create a Resource to start the creation process.

streamanalytics1
Figure 1: Azure Services

Enter Stream Analytics Job in the search box and press Enter or click the magnifying glass.

streamanalytics2
Figure 2: Creating Stream Analytics Job Resource

This will return a list of results that match your entry. Select Stream Analytics Job to create a new instance of Stream Analytics. A description page will appear describing what Stream Analytics is, along with available pricing plans and documentation resource links. Click Create to begin the creation process.

streamanalytics3
Figure 3: Stream Analytics Job Overview

You need to provide some details when creating a Stream Analytics job in Azure. First, you will need to specify a name for your Stream Analytics instance. This name only needs to be unique to your subscription.

There is also a dropdown for selecting the Azure subscription to create the Stream Analytics instance under. It is auto populated with your current subscription but if you have multiple subscriptions you can select from one in the list.

You also need to specify a resource group that the Stream Analytics instance is hosted under. You can either select from a list of existing resource groups or click Create New to create a new resource group immediately and use that for your Stream Analytics instance.

Location specifies where you will be creating the Stream Analytics instance. Choose the location nearest to your physical location for best performance.
Hosting environment specifies where the Stream Analytics job will be deployed. Cloud specifies that the job will be hosted in the cloud (aka – the Azure Portal) while Edge specifies an on-premise IoT Edge gateway device.

Streaming Units determine the number of computational resources the Stream Analytics job will use when processing a query. Since the jobs perform all processing in memory, increasing this value will increase the amount of CPU and memory resources to process queries. This can be useful for scaling up and providing low latency stream processing.

streamanalytics4
Figure 4: Create Stream Analytics Job Details

Once created, your Stream Analytics job is stopped by default. In order to configure your job for processing you need to create three (3) things:

  • Inputs
  • Queries
  • Outputs

Let’s review each of these in detail.

Inputs

The first thing to configure are inputs. Inputs define a connection to an existing data source, and Stream Analytics accepts incoming data from these inputs. You can choose from three different input types: Azure Event Hubs, Azure IoT Hub, or Azure Blob Storage.

You can navigate to the Inputs page by clicking Inputs under the Job Topology section (on the left). This page will display a list of configured inputs while also allowing you to create a new input.

streamanalytics5
Figure 5: Stream Analytics Inputs

There are also two types of inputs: Stream and Reference. Stream inputs define an unbounded sequence of data events over time. This is the stream of input data coming from an input source and Stream Analytics requires at least one of these input types to be defined.

Reference inputs define a static set of data or data that slowly changes. This input type is typically used for data lookups or data correlation and is optional depending on your needs. An example would be having a lookup table that you would join with data from your stream input. Currently Azure Blob Storage and Azure SQL Database are supported as input sources for reference data.

For our example, we will be connecting a stream input of type IoT Hub to an existing Azure IoT Hub instance. You are prompted for an input alias name for this input source, along with selection of the subscription and IoT Hub you are specifying. There are also entries to specify IoT Hub routing endpoint to use as well as event serialization format and encoding.

streamanalytics6
Figure 6: Adding IoT Hub Input

Clicking Save will create the input and run a connection test to verify that Stream Analytics can connect to the input properly.

Outputs

Outputs must also be configured in order for the Stream Analytics job to function properly. Stream Analytics needs to send the transformed/processed data to a destination. Configuring one or more output satisfies this requirement.
There are a number of different outputs you can choose from, and are listed here:

  • Event Hub
  • SQL Database
  • Blob Storage/Data Lake Storage Gen2
  • Table Storage
  • Service Bus Topic
  • Service Bus Queue
  • Cosmos DB
  • Power BI
  • Data Lake Storage Gen1
  • Azure Function

Additional information is available for each of these types here.

streamanalytics7
Figure 7: Stream Analytics Outputs

For our example I will be configuring an output to Cosmos DB. Adding an output of this type displays a blade that allows entry of the details of the output being created. You need to enter an alias name for the output, along with the choice of entering the Cosmos DB setting manually or selecting from your Azure subscription. We are selecting an existing Cosmos DB resource that was created previously. Please note that you must have previously created a container in Cosmos DB before specifying the Container Name and Document Id values.

streamanalytics8
Figure 8: Adding CosmosDB Output

Queries

Once you have one or more inputs and outputs configured, you can now create a query to process the incoming data. Queries are the mechanism used by Stream Analytics to transform the incoming data in real time. These queries are written using Stream Analytics Query Language which are similar to SQL statements.

Structurally the query has a SELECT statement which selects data from one or more configured inputs and uses an INTO clause to specify an output to emit the transformed data to. There are also FROM, JOIN and WHERE clauses which help specify and filter the incoming data and perform processing against it.

In our example, when you click on the Query section under Job Topology, a page is displayed that shows the current query for this Stream Analytics job. You are able to edit in the window to specify your select query statements. Notice that we are using SELECT * wildcard to specify all fields from the input source and saving those to the specified output.

streamanalytics9
Figure 9: Stream Analytics Queries

You can see preview data displayed from our input source (IoT Hub device):

streamanalytics9-1
Figure 10: Input Preview Data

This querying editor also has the ability to perform discrete field selection and filtering. So, for example if we wanted to specify certain fields we could modify our query to look like this:

streamanalytics10
Figure 11: Discrete Field Selection in Queries

Additionally, if we wanted to filter the incoming data we have that ability as well. For example, if we wanted to exclude any incoming temperature data that did not exceed 100 we could specify a filter like this:

streamanalytics11
Figure 12: Query Filtering

Once we have the inputs, outputs and queries configured we are ready to turn on our Stream Analytics job. Looking at the Overview page we can see at a glance the configured inputs, outputs and what the query is specified as.

When we start the Stream Analytics job and wait a few minutes, we should begin to see data coming in and being processed. The Monitoring and Resource Utilization graphs would show some data coming through as it is being processed.

streamanalytics12
Figure 13: Stream Analytics Graphs

We can also see our data appearing in our CosmosDB database, as shown here:

streamanalytics13
Figure 14: CosmosDB Data Records

Conclusion

In conclusion, Azure Stream Analytics provides the ability to process complex events in real time, with the purpose of providing real time analytics to incoming data sources. We also showed an example on how to create and configure Azure Stream Analytics to accept incoming IoT device telemetry data and process and save it into a CosmosDB database. I hope this article has been helpful and please click here for more information.

Cheers!

Azure IoT Hub Messaging

The Azure IoT Hub service provides the ability for bi-directional communication with your devices. You can use Azure IoT Hub to send messages from your devices to your IoT cloud solutions, as well as send commands from your IoT cloud solutions to your devices. This article will describe the different types of communications possible with Azure IoT Hub and how you can use them.

Prerequisites

Before you can send/receive messages between Azure IoT Hub and your devices you need the following:

  • An Azure account. Sign up for a free trial here
  • Provision an instance of IoT Hub
  • Create at least one device with your IoT Hub

Bi-directional communication means just that – in both directions. What that means for Azure and IoT is that you either send data from your devices to IoT Hub, or data or commands from IoT Hub to your devices. I will describe the different options to accomplish this, as seen below:

  • Device-to-Cloud messages
  • Cloud-to-Device messages
  • Device Twins
  • File Upload
  • Direct Methods

Device-to-Cloud Messages

Sending information from your IoT devices to the cloud is supported by Azure IoT Hub with device-to-cloud (D2C) messages. Your devices can send telemetry time series and alert data to IoT Hub to be routed to other services for processing.

Azure IoT Hub comes with a built-in endpoint that back-end services can use to read telemetry data being sent from your devices. This endpoint is compatible with Event Hubs and you can also use standard IoT Hub SDKs to read from this endpoint.

When D2C messages are sent from your devices to IoT Hub, by default they are routed to this built-in service endpoint (messages/events). This endpoint is exposed to your devices with MQTT, AMQP and HTTPS protocols. You also have the ability to route these messages to custom endpoints that you create.

Messages can be routed to multiple endpoints, so if you create custom endpoints and setup the routing queries you can have a message be routed to more than one endpoint. IoT Hub currently supports the following custom endpoints:

  • Azure Storage (Blob or Data Lake Gen2)
  • Service Bus Queues
  • Service Bus Topics
  • Event Hubs

Here is an example route that filters out messages that have a temperature alert flag specified and routes the messages to the default events endpoint.

Figure 1: IoT Hub Routing Example

Please note that the built-in Event Hubs endpoint (messages/events) is the fallback route, so if incoming device messages do not satisfy query conditions for any custom endpoints they will be sent to this fallback route. You must enable the fallback route capability when you turn on message routing or your incoming messages will not be routed properly.

D2C messages received are stored temporarily by IoT Hub up to 7 days. They are lost if not processed by a configured back-end service. Messages can also have a maximum size of 256KB for each message. The throughput of what an IoT Hub can handle depends on the pricing tier. Details are here.

Additional details on D2C communications can be found here.

Cloud-to-Device Messages

Sending one-way communications from a cloud solution to one or more devices is possible with cloud-to-device (C2D) messages. Your cloud solutions can send notifications to devices, or commands that can alter the state of devices (device twins).

Azure IoT Hub will send messages to devices through a service-facing endpoint (messages/devicebound), and the device(s) then receive the messages through a device-specific endpoint (devices/{deviceid}/messages/devicebound).

Azure IoT Hub provides a guarantee that messages will be sent to devices at-least-once. In order to do this, IoT Hub persists C2D messages in per-device queues. These messages persist in the queues until the devices explicitly acknowledge completion (aka – they received the message successfully). Once acknowledged, IoT Hub removes the message from the appropriate per-device queue. This approach guarantees resiliency in the case of connectivity and/or device failures.

Additional details on C2D communications can be found here.

Device Twins

JSON documents that contain device state information are what is known as device twins. These documents store the metadata, configurations and conditions for a device. Azure IoT Hub maintains a device twin for each device you connect to IoT Hub. There are multiple uses that device twins provide:

  • Store device-specific metadata in the cloud
  • Report current state information about the device
  • Synchronize the state of long-running workflows between the device and cloud solution
  • Querying device metadata, configuration or state

The structure of a device twin is shown in Figure 2:

IoTHubMessaging2
Figure 2: Device Twins

Tags are a section of the JSON document that the cloud solution can read/write to. Devices cannot see these tags or their values.

Desired Properties are a section of the JSON document used to synchronize device configurations or conditions. The cloud solution can set these properties and the device can read them. The device can also receive notifications that the desired properties have changed.

Reported Properties are a section of the JSON document also used to synchronize device configurations or conditions. Devices can set these properties and the cloud solution(s) can read them.

Root object properties define the device identity and also provide container objects for tags and the reported, desired properties sections.

Here is an example of a device twin:

{
    "deviceId": "testDevice1",
    "etag": "AAJWKSOAAAAAc=", 
    "status": "enabled",
    "statusReason": "provisioned",
    "statusUpdateTime": "0001-01-01T00:00:00",
    "connectionState": "connected",
    "lastActivityTime": "2020-02-28T12:44:48.976Z",
    "cloudToDeviceMessageCount": 0, 
    "authenticationType": "sas",
    "x509Thumbprint": {     
        "primaryThumbprint": null, 
        "secondaryThumbprint": null 
    }, 
    "version": 1, 
    "tags": {
        "$etag": "123",
        "deploymentLocation": {
            "building": "A",
            "floor": "1"
        }
    },
    "properties": {
        "desired": {
            "telemetryConfig": {
                "sendFrequency": "5m"
            },
            "$metadata" : {...},
            "$version": 1
        },
        "reported": {
            "telemetryConfig": {
                "sendFrequency": "5m",
                "status": "success"
            },
            "batteryLevel": 55,
            "$metadata" : {...},
            "$version": 4
        }
    }
}

The cloud solution backend can operate on the device twin using these operations (through HTTPS):

  • Retrieve Device Twin: Returns the JSON document for the device twin
  • Update Device Twin: Enables the partial update of the JSON document. Tags and Desired Properties can only be updated. Cloud solution can also add or remove properties in these sections. Setting a property to null removes it from the section. There is also the ability to completely overwrite the Tags and Existing Properties with new JSON.
  • Receive Device Twin Notifications: Allows the cloud solution to be notified when a change is made to the device twin. A route in the IoT cloud solution called twinChangeEvents must be created and configured as a route for this operation to succeed.

The device can operate on its device twin using these operations:

  • Retrieve Device Twin: Returns the JSON document for the device twin. Tags are not visible.
  • Update Device Twin: Enables the partial update of the JSON document. Reported Properties can only be updated. This operation works in the same way as the cloud solution updates a device twin.
  • Observe Desired Properties: The device can choose to be notified of updates to the Desired Properties when they occur.

There are size limitations on the different sections in the device twin, as seen here:

  • Tags: 8KB max size on values
  • Desired Properties: 32KB max size on each value
  • Reported Properties: 32KB max size on each value

As an example, if the cloud solution wants to change the sendFrequency value for the device it modifies the device twin value for sendFrequency, as seen here:

"desired": {
   "telemetryConfig": {
   "sendFrequency": "5m"
},
…

The device(s) would be notified of this change immediately or when they become connected (if offline). The device would then report back to the cloud solution the updated configuration on the device:

"reported": {
   "telemetryConfig": {
   "sendFrequency": "5m",
   "status": "success"
},
…

File Upload

IoT devices can start file uploads through a device-facing endpoint in IoT Hub. You can have your devices request file uploads to an associated Azure storage account in the cloud, with IoT Hub acting as a dispatcher for the uploaded file (requested by a device) to the Azure storage account. File uploads can be used for large files such as media files or large telemetry batches.

In order for file uploads to work, you must first associate an Azure storage account to IoT Hub. This can be done via the Azure Portal or programmatically through the IoT Hub REST APIs.

IoTHubMessaging3
Figure 3: File Upload Section

Figure 4: File Upload Configuration in IoT Hub

Once you have made this association you can write code in your device to upload files to the cloud. IoT Hub will handle the upload to its associated blob storage account and provide notifications that you can monitor. Here is a great tutorial that reviews the steps needed to perform a file upload from your device.

Direct Methods

Direct methods provide an alternative way of communication with devices that work independently of the other communication options. Direct methods allow us to call a method on the device as a local service, which means we can get immediate feedback from the results of that call.

Direct methods are implemented on the device and use a request-response pattern when communicating. You can invoke a direct method through a service-facing URI and are a synchronous call. The invocation will either succeed or fail (after the timeout period), returning a result with an optional payload. The benefits/drawbacks of direct methods are as follows:

Benefits

  • Define multiple named direct methods
  • Pass an optional payload
    Returns a status code and optional payload
  • High throughput

Drawbacks

  • Only supports AMQP or MQTT
  • Doesn’t work if device is offline

As an example, here is how you would write code on your device to receive calls to a direct method:

await device.SetMethodHandlerAsync(“showMessage”, ShowMessage, null);
…


private static Task ShowMessage(MethodRequest methodRequest, object userContext)
{
…
}

Calling the SetMethodHandlerAsync class registers a named method handler with the device, which in this example the method name is ShowMessage. Parameters are the name of the message handler method as well as the reference to that handler method and an optional context object.
The definition of the ShowMessage handler method is also defined, which takes a MethodRequest object parameter and optional context object and returns a Task object.

Now that our devices are able to handle direct messages from the cloud, the only other thing to do is write code in our IoT cloud solution to call the direct method on our device. We can do this by defining a method to invoke the direct method call on our device, as seen in this console application example:

static async Task Main(string[] args)
{
   …
   var serviceClient = ServiceClient.CreateFromConnectionString(svcConnString);
   await CallDirectMethod(serviceClient, deviceId);
   …
}


private static async Task CallDirectMethod(ServiceClient sClient, string deviceId)
{
   var method = new CloudToDeviceMethod(“showMessage”);
   method.SetPayloadJson(“’This is a test direct method’”);

   var response = await sClient.InvokeDeviceMethodAsync(deviceId, method);

   Console.WriteLine($”Response status: {response.Status}, payload:{response.GetPayloadAsJson()}”);
}

As you can see, the Main method creates an instance to our IoT hub using the ServiceClient class. We then create a private method named CallDirectMethod(), which takes the instance of our ServiceClient class along with the Device ID value. This method instantiates the direct method via the CloudToDevice() class, which we specify the name of the direct message when instantiating (showMessage in our example). We can then set the payload JSON and asynchronously invoke the direct method on the device. A response is returned to let us know if the method ran successfully or not.

Conclusion

In conclusion, we described the different types of communications possible with Azure IoT Hub and how you can use them. I hope this article has helped you understand the different types of Azure IoT Hub messaging.

Cheers!

Azure IoT Central

Azure IoT Central is Microsoft’s SaaS (software-as-a-service) offering that allows you to quickly provision an IoT solution. This service reduces the burden and cost of developing, managing and maintaining enterprise-level IoT solutions, allowing you to focus more on your IoT data and how it can transform your business. I will be going through the steps to create an Azure IoT Central application and how easy it is to create and maintain an IoT solution.

You can use IoT Central to create a custom, cloud-based IoT solution without having to worry about the details of the underlying infrastructure. Customization of your solution can all be done from the browser in the application UI, and includes capabilities to:

  • Define the layout of properties and settings on a device template
  • Configure custom dashboards
  • Configure custom analytics to explore device data

 

Prerequisites

The only prerequisites needed are a Microsoft account (personal, work or school) as the creation process will ask you to login to your account, and an Azure subscription.

 

Getting Started

First you need to navigate to the Azure IoT Central Build site. You will notice a left-hand navigation menu which consists of several sections:

IoTCentral1

Figure 1: Azure IoT Central Menu

 

Clicking on the Build menu selection will take you to a page that displays a number of options to build your IoT application. You can start with a number of pre-generated templates that are geared towards specific industries or start with a blank application template.

Once you select either an industry-specific or custom template, a page displays that allows you to specify information about your application.

IoTCentral2

Figure 2: Creating an IoT Central Solution

The Application Name is pre-populated for you, as well as the URL. Please note that the application name and URL must be globally unique. The Application Template dropdown is only visible on custom applications, as you’re already selecting a specific template if you choose one of the industry-specific templates.

You also need to choose the pricing plan – you can try the solution for free for 7 days or select one of the other pricing tiers. These other tiers are based on how often you want to send messages from your devices. The Billing Info allows you to select the Directory (Azure Active Directory) and Azure Subscription you wish to have this solution hosted from. Location is also required as the system will provision the IoT Central solution resources in the closest region.

 

Web Application

Once the system provisions your IoT Central application you are taken to the Dashboard page, which displays some starting links such as adding devices, quick start demos, tutorials and documentation.

IoTCentral3

Figure 3: Dashboard page

 

Devices

Adding devices is easy in IoT Central – just click on the Devices tab and you are taken to a page that displays all the devices currently in the application. Adding a device is done by clicking on the +New button. A dialog window is displayed allowing you to configure the details about the new device being added:

IoTCentral4

Figure 4: Adding a new device

Device ID is a unique identifier that is used by the device to connect, while Device Name is a friendly name for the device. You can also toggle the Simulated option which specifies whether your device is simulated or real.

 

Device Templates

You can also create one or more device templates, which can then be used to create new devices that have the template characteristics. Click on the +New button on the Device Templates tab (under App Settings) and you can select the template type:

IoTCentral5

Figure 5: Device Templates

You have the ability to create a template for an IoT device, Azure IoT Edge device, or one of the pre-configured device templates listed. The list of certified devices (with IoT Plug and Play) are shown here and more are being added all the time.

Once you publish your device template, you can add devices based on that template. On the Devices tab you will notice a selection with the name of your device template that you can click on. This will display a list of the current devices that are based on this device template. Clicking the +New button will allow you to create a device based on this template.

IoTCentral6

Figure 6: Creating a Device using Device Template

 

Device Groups

Device Groups are a logical grouping of devices in the system based on specific attributes. You can create one or more Device Groups to filter one or more devices in your system. This is useful if you have hundreds (or thousands) of devices and you need to look at specific devices quickly.

Navigate to the Device Groups tab and click on the +New button to create a new device group. You will see a page that allows you to enter information for your new device group such as the Name and Scope. Adding one or more filters in the Scope area will define the filters needed to only display the device(s) you want to see.

IoTCentral7

Figure 7: Device Groups

 

Rules

You can create telemetry-based rules in IoT Central. This is done under the Rules section in the application. Clicking on the Rules tab will display a list of the current rules in the system. Create a new rule by clicking on the +New button. This will display a page that allows you to create a new rule.

IoTCentral8

Figure 8: Rules

Give the rule a name and then specify a target device template, which will define the devices to which the rule applies. If you need to specify less than all the devices in the device template you select, create one or more filters to filter the list of devices further.

Conditions are created to specify when the rule is triggered. You can specify one or more conditions the rule must meet to be triggered.

There is also a section for Actions. These are the things that happen once the rule has been triggered. You notice you can either send an email or call a web hook when the rule is triggered. You can create one or more actions if you choose.

IoTCentral9

Figure 9: Email Action

 

IoTCentral10

Figure 10: Webhook Action

 

Analytics

Azure IoT Central also has the capability to analyze device data including time series information. You can specify the telemetry values to display along with splitting the data collections by attributes (such as Device ID). Select the Device Group and Telemetry value(s) to analyze and click the Analyze button when you are finished.

IoTCentral11

Figure 11: Analytics

 

Jobs

You can use Azure IoT Central to manage your connected devices using jobs. Jobs are useful for performing bulk updates to your device properties or for running commands. Clicking on the Jobs tab will display a list of the existing jobs in the system, job status and when they were last run. Creating a new job will allow you to specify details of the job such as Name, what device group the job targets as well as the Job Type and detailed information.

IoTCentral12

Figure 12: Creating a Job

Running a job is done by clicking on the Run button. This will execute the job and return results of the operation.

IoTCentral13

Figure 13: Running a Job

 

Administration

The IoT Central application also allows administration of the website, including Users in the system, Roles, Pricing and the Device Connections. You also have the ability to permanently delete the solution.

IoTCentral14

Figure 14: Administration

 

Underlying Infrastructure

Since IoT Central is a SaaS (software-as-a-solution) offering, you do not have access to the underlying architectural components in Azure. What this means is that Microsoft fully hosts and supports the IoT Central application but they also manage the resources themselves. You do not have the ability to configure/customize/remove any of the individual Azure resources used in your IoT Central application. If you need to customize the IoT Central application beyond what is provided to you through the SaaS offering, then you need to consider Azure IoT Solution Accelerators which do provide the ability to manage and modify the underlying Azure components.

 

In conclusion, Azure IoT Central is a nice offering from Microsoft if you want to get your IoT solutions up and running quickly! Device provisioning and customizations are part of this service, as well as Rules, Jobs, and analytics for the device data. I hope this brief tour of Azure IoT Central has been helpful to you.

Cheers!

Message Routing with Azure IoT Hub

IoT solutions can be complex, especially if you have multiple devices that you need to send to more than one destination in your cloud solution. Azure IoT Hub provides the capability to send your device data to multiple destinations through message routing. This article will describe the available options with message routing and how they are used.

IoT solutions can be complex, especially if you have multiple devices that you need to send to more than one destination in your cloud solution. Azure IoT Hub provides the capability to send your device data to multiple destinations through message routing. This article will describe the available options with message routing and how they are used.

Prerequisites

Before you can use message routing with your devices you will need the following:

  • An Azure account. Sign up for a free trial here
  • Provision an instance of IoT Hub
  • Create at least one device with your IoT Hub

Incoming Messages

The devices in your IoT solution will be sending telemetry data into Azure IoT Hub. Conceptually, you normally think of these messages all being sent to one place and being handled/processed by the hub. However, what if you want these messages sent to different cloud services that may be part of your solution? In the past you might have had custom code to handle these messages and route them appropriately. That made your solution overly complex when you were required to handle all the details about these incoming messages, such as handling transient faults, how to deal with lost messages, and maintaining high reliability.
Fortunately, Microsoft has made this easier for us by providing message routing as part of Azure IoT Hub. IoT Hub will now take care of all those complex details for routing messages so we can focus on our custom IoT solution code. IoT Hub provides scalability for incoming messages, handling transient faults, and how to handle lost messages. The details of message routing are abstracted from us, and we only need to know how to configure the routing to specify where we want our messages to go.
MessageRouting1

Figure 1: Message Routing

 

Endpoints

There are a number of places you can route your device messages to by configuring IoT Hub. These are the final destinations of your data, allowing backend processing services to execute actions against the data you route to it. The different types of endpoints are as follows:

  • Event Hub
  • Service Bus Queue
  • Service Bus Topic
  • Storage

Also note that paid SKU hubs can have up to 10 additional endpoints while free SKU hubs can only have 1 additional endpoint.

Event Hub

Creating an event hub allows you to route your device data to a specific event hub resource to allow further/dedicated processing. Note that IoT Hub comes with a built-in endpoint that is compatible with this type. Here is an example of adding a new event hub endpoint to IoT Hub:

MessageRouting2

Figure 2: Event Hub Endpoint

 

Service Bus Queue/Topic

You can create a custom endpoint to a Service Bus Queue/Topic, which would then create a record that can be processed by an Azure function for example. Here is an example of adding a new Service Bus Queue endpoint to IoT Hub:

MessageRouting3

Figure 3: Service Bus Endpoint

Creating a custom endpoint to a Service Bus Topic works in the same way you would create one for a Service Bus Queue, but please note that your Service Bus Namespace must be the Standard (or higher) pricing tier for topics to be available. Also be aware that you cannot create a custom endpoint to a Service Bus Queue/Topic if they have sessions or duplicate detection enabled.

Storage

You can also create a custom endpoint that uses Azure blob storage, allowing the saving of incoming messages to an existing container (as blobs only). Here is an example of adding a new Azure Storage endpoint to IoT Hub:

MessageRouting4

Figure 4: Storage Endpoint

Batch Frequency specifies (in seconds) the maximum amount of time before the system will batch and write records to Azure Storage. The Chunk Size Window specifies the maximum size of blobs (in MB). Apache AVRO is the default encoding, although you can specify JSON.

Routes

Routes are created/configured to direct the appropriate data source to an endpoint for processing. When you create an endpoint you must specify the Name, Endpoint (built-in or custom), Data Source, and filter query.

MessageRouting5

Figure 5: Adding a Route

The Data Source field specifies where the data is coming from and must be one of the following selections:

MessageRouting6

 

You also have the ability to filter the incoming message data, which can be done using queries in the Routes. IoT Hub defines a common format for all device-to-cloud messaging, and filtering is done by writing a query against this common format. Here is an example:

{ 
  "message": { 
    "systemProperties": { 
      "contentType": "application/json", 
      "contentEncoding": "UTF-8", 
      "iothub-message-source": "deviceMessages", 
      "iothub-enqueuedtime": "2017-05-08T18:55:31.8514657Z" 
    }, 
    "appProperties": { 
      "processingPath": "{cold | warm | hot}", 
      "verbose": "{true, false}", 
      "severity": 1-5, 
      "testDevice": "{true | false}" 
    }, 
    "body": "{\"Weather\":{\"Temperature\":50}}" 
  } 
}

Querying on message system properties should prefix a ‘$’ symbol on the query. Queries on application properties do not need this symbol prefixed. For example, querying on the system property ‘contentEncoding’ would look like this:

$contentEncoding = 'UTF-8'

Querying on an application property would look like this:

processingPath = 'hot'

Please note that device-to-cloud messaging is not case-sensitive, so using unique property names is recommended. Here is an example of a query that filters records that have the application property ‘temperatureAlert’ = true.

MessageRouting7

Figure 6: Message Routing Queries

A fallback route is present so that any messages that do not satisfy any specified query conditions will be sent to this route. The default fallback route is the ‘messages/events’ route that is used by the built-in endpoint (compatible with Event Hub). You can turn on message routing in IoT Hub, which will allow you to create custom routes. Once a custom route is created, data stops flowing to the built-in endpoint unless you specifically create a route to that endpoint. Please be aware that if you delete all custom routes and you disable the fallback route, data will not be sent to any destination.

 

In addition to telemetry data, routes can be created for other events such as Device Twin changes, Device Lifecycle events and Digital Twin changes (currently in public preview). If you create a route with the data source set to one of these other events, IoT Hub will send messages to the appropriate endpoint for that event (Device Twin, Digital Twin, Device Lifecycle).

 

Conclusion

In conclusion, using message routing with Azure IoT Hub increases the flexibility, reliability and scalability of your IoT solution. Routing also enables you to keep track of specific events in your solution, such as device twins and device lifecycle. I hope this article has helped you understand the different options for message routing with Azure IoT Hub.

Cheers!

Creating and Configuring Azure IoT Hub

Microsoft provides the Azure IoT Hub service to assist in implementing Internet of Things (IoT) solutions. This fully managed Azure service enables reliable and secure bidirectional communication between your IoT devices and a back-end solution hosted in Azure. I am going to review the steps to create an IoT hub in Azure, along with how to register a device and configure tier and scaling options.

Prerequisites

The only prerequisite you need to follow along is an Azure account. You can sign up for a free trial here.

Creating an IoT Hub Instance

Azure Portal

Probably the easiest way to create an instance of IoT Hub is to use the Azure portal. You do this by first logging into the Azure portal with your Azure credentials, and then clicking on Create a Resource in the upper left. This displays a new blade with a number of choices under Azure Marketplace. Click on Internet of Things and you should see the IoT Hub selection, as seen here:

CfgIoTHub2

Clicking on IoT Hub will display a new blade that allows you to enter in details of your new IoT Hub, such as the Subscription, Resource Group, Region and IoT Hub name. The Subscription is auto filled with the subscription you are currently logged into. You can select a Resource Group to place the new IoT Hub in or create a new one. Region specifies the location that your IoT Hub will be created in, and the IoT Hub Name is a globally unique name you give to your IoT Hub.

CfgIoTHub3

Clicking Next: Size and Scale takes you to the next screen which allows you to select a pricing tier for your IoT Hub, along with the number of IoT Hub units allocated. The pricing tiers include a Free Tier as well as 3 levels of either Basic or Standard Tiers (B1-3, S1-3 respectively). The choice you make here determine a number of options relating mainly to data throughput and scaling. Details are here.

At the time of this writing, the highest pricing tier selection is S3: Standard Tier as shown below:

CfgIoTHub4

Clicking Review + create will take you to a summary page listing out the selection choices you made. Clicking Create here will instruct the Azure Portal to create the IoT Hub with the choices you selected.

CfgIoTHub5

Azure CLI

Another way to create an instance of IoT Hub is by using the command-line interface (CLI). Azure hosts an interactive shell environment called Azure Cloud Shell that you can use to execute CLI commands. You can launch Azure Cloud Shell by either going to https://shell.azure.com in your browser, or by clicking on the Cloud Shell button in the upper right in the Azure Portal.

CfgIoTHub9

Once you are in your Cloud Shell environment, you can execute the following command to create an instance of IoT Hub:

az iot hub create --name {your iot hub name} --resource-group {your resource group name} --sku S1

The only required parameters are the name of your IoT Hub along with the resource group it will be associated with. Insert a globally unique name for your IoT Hub along with the name of an existing resource group. The optional –sku parameter designates what pricing tier to create the IoT Hub with. Pricing details are located here.

There are additional optional parameters when creating an IoT Hub and are listed here.

Configuring IoT Hub Tiers

There are two tiers for Azure IoT Hub: Basic and Standard. You should consider what your IoT solution is designed to do before you select a tier. The difference between Basic and Standard tiers are the number of features they support. A Free tier is also provided, which has all the functionality of the Standard tier but has limited messaging allowances.

If your solution collects data and you are analyzing it centrally, then the Basic tier is probably the best choice. If you want to use advanced configurations to control your IoT devices remotely, or you want to distribute some of your workloads to the devices themselves, then the Standard tier is probably the best choice.

Each tier has three levels, listed in numerical order (1, 2, 3). The difference between these three levels is how much data they allow to be send from devices per day:

  • B1, S1 tiers: 400,000 messages/day/unit
  • B2, S2 tiers: 6 million messages/day/unit
  • B3/S3 tiers: 300 million messages/day/unit

The following features are available in all IoT Hub tiers:

  • Device-to-cloud telemetry
  • Per-device identity
  • Message routing, enrichments and Event Grid integration
  • HTTP, AMQP, and MQTT protocols
  • Device Provisioning Service
  • Monitoring and diagnostics

The following features are only available in the Free/Standard IoT Hub tier:

  • Cloud-to-device messaging
  • Device twins, Module twins, and Device Management
  • Device streams (preview at the time of this blog)
  • Azure IoT Edge
  • IoT Plug and Play (preview at the time of this blog)

Registering Devices

Once you have your IoT Hub instance created, you can register one or more devices identities that you intend to use with your IoT Hub.

Azure Portal

After you log in to the Azure Portal, navigate to your IoT Hub and click on the IoT Devices selection (under Explorers section):

CfgIoTHub6

This will display a list of the current devices created for the IoT Hub. You can click on the + New link towards the top of the page and a new blade will be displayed, allowing you to enter the following information about your device:

  • Device ID – enter a name for the device. This is used for authentication and access control
  • Authentication Type – select how the device will authenticate to IoT Hub. Symmetric Key is the default and allows entry of the primary/secondary keys (see below). X.509 Self-Signed and X.509 CA Signed allow you to select security certificates for the device (certificate created prior)
  • Primary/Secondary Keys – allows you to set a unique value for accessing the device. Disabled if auto-generate keys is checked (valid for Symmetric Key authentication type only)
  • Auto-generate keys – this will instruct Azure to generate unique values automatically for the primary/secondary keys (valid for Symmetric Key authentication type only)
  • Connect this device to an IoT Hub – specifies if this device will interact with this IoT Hub
  • Parent Device – specify if this device has a parent device. Used for IoT Edge

CfgIoTHub7

Clicking Save will instruct the Azure Portal to create the device with your selections. Once completed, you can access the device details by clicking on its name in the list of devices:

CfgIoTHub8

Azure CLI

You can also register device identities with your IoT Hub using Azure CLI. Execute the following command in your Cloud Shell instance, replacing {iothub_name} with your IoT Hub name and {device_id} with the name you wish to use for your device:

az iot hub device-identity create -n {iothub_name} -d {device_id}

The console will emit a JSON string upon success which give information about the device you just registered, such as primary/secondary key values, its connection state and whether it has IoT Edge capabilities:

CfgIoTHub10

You can verify in the Azure portal that the device is registered by navigating to the IoT Devices section in your IoT Hub. Your newly-registered device should be displayed in the list:

CfgIoTHub11

Additional parameters can be found here.

I hope this article has helped you understand what Azure IoT Hub is and how to create an instance and register your IoT devices to it, so you can begin sending data to Azure.

Cheers!