Azure IoT Hub Streaming Analytics Simulator

Azure IoT Hub Streaming Analytics Simulator is an application written by Manny Grewal. The purpose of this blog is to explain What, Why and How of this application.

 

What?

Streaming analytics is a growing trend that deals with analysing data in real-time. Real-time data streams have a short life span, their relevance decreases with time, so they demand quick analysis and rapid action.

Some areas where such applications are highly useful include data streams emitted by

  • Data centres to detect intrusions, outages
  • Factory production line to detect wear and tear of the machinery
  • Transactions and phone calls to detect fraud
  • Time-series analysis
  • Analogy Detection

 

Data used by streaming analytics applications is temporal in nature i.e. it is based on short intervals of time. What is happening at the interval TX can be influenced by what happened 2 minutes ago i.e. at the interval TX-2

So the relationships between various events are time-based rather than entity based (e.g. as in general Entity Relational Database based systems)

Take the scenario of a Data Centre which has two sensors that emit a couple of data streams – Fan Speed of the server hardware and its temperature.

If temperature reading of server hardware is going high, it could be related to the dwindling Fan Speed reading. We need to look at both the readings over an interval of time to establish a hypotheses on their correlation.

 

 

Why?

In order to model and work with streaming analytics it is important to have an event generator that can generate the data streams in a time-series fashion.

Some example of such generators can be vehicle sensors, IoT devices, medical devices, transactions, etc. that generate data quickly.

The purpose of this application is to simulate the data generated by those devices, it just helps you setup quickly and start modelling some data for your IoT experiments.

 

 

Main benefits of this app

1. Integrated with Azure IoT Hub i.e. the messages emitted by this application are sent to the Azure IoT Hub and can be leveraged by the Intelligence and Big Data ecosystem of Azure.

2. This app comes with 4 preset sensors

a. Temperature/Humidity

b. Air Quality

c. Water Pollution

d. Phone call simulator

3. Configure > Ready. App can be easily pointed to your Azure instance and can start sending messages to your Azure IoT Hub

4. Can be extended, if you are handy with .NET development. I have designed the app on S.O.L.I.D framework so it can be extended and customised the link to source code is below

 

How?

 

App and source code can be downloaded from my Github

 

A quick tour of the app is below

IoT Hub

 

 

Configure

The app needs to be configured with details of your Azure IoT Hub account.

The following files need to be configured

1. App.Config

2. If you are registering Devices in the Hub, then keys for the devices need to be stored in the SensorBuilder.cs

3. You may need to restore the Nuget Packages to build the application

 

Once the above three steps have been completed, you can build the application and the EXE of the application will be generated.

 

Sensor Tuning

Sensors can be tuned from the classes inheriting IDataPoint e.g. in the FloatDataPoint.cs

The following properties can be used to tune the sensors

Property Name Tuning
MinValue The minimum value of the sensor reading e.g. for climatic temperature it can be -40C
MaxValue The maximum value of the sensor reading e.g. for climatic  temperature it can be 55C
CommonValue This is the average value of the sensor e.g. for warmer months it can be 30C
FluctuationPercentage How much variance you want in the generated data
AlertThresholdPercentage When should an alert be generated if the reading passes a certain threshold e.g. 80% of the maximum value

 

Azure IoT Hub

The messages sent by the sensor simulator can be accessed in the Azure IoT Hub. Once you have configured your hub and related streaming jobs. The messages can be seen in the dashboard as below

image

 

The messages are sent in the JSON format and below is a structure of one of the messages emitted by a sensor located at Berwick, VIC


{
"IncludeSensorHeader": 1,

"MessageId": "949a3618-c4a4-42bc-9c2a-39da86aa9191",

"EmittedOn": "2017-06-30T11:13:45.3543200",

"SensorDataHeader": {
"Sensor":
"Berwick",
"DeviceId":
"G543",
"Lat":
-38.0309,
"Long":
145.3461
},
"SpecialMessage":
null,
"Readings": [

{
"ReadingValue":
27.9943523,
"MetaData":
{
"Name":
"Temperature",
"Unit":
"C"

},
"Level":
"Normal"
},

{
"ReadingValue":
49.6043358,
"MetaData":
{
"Name":
"Humidity",
"Unit":
"RH"

},
"Level":
"Normal"
}

],
"EventProcessedUtcTime":
"2017-07-01T11:26:53.1434112Z",
"PartitionId":
0,
"EventEnqueuedUtcTime":
"2017-06-30T11:13:48.4340000Z",
"IoTHub":
{
"MessageId":
null,
"CorrelationId":
null,
"ConnectionDeviceId":
"G543",
"ConnectionDeviceGenerationId":
"636297589019910475",
"EnqueuedTime":
"2017-06-30T11:13:47.5760000Z",
"StreamId":
null
}
}