Stream Data Filters

Built On: Streams, Devices, Variables, and Projects

For a full introduction to data management in IOTile Cloud please see the article on The Four Building Blocks.

1. Stream - A unique time series of data.  
2. Device - A unique source of data. 
3. Variables - A set of data streams that share a common index and physical units.   
4. Projects - A set of devices grouped together to accomplish useful work.  

Each of these fundamental objects are represented with globally unique ID's of the following form:

Project: p--<project> (e.g. p--0000-0008)
Device: d--<device> (e.g. d--0000-0000-0000-0212)
Variable: v--<project>--<variable> (e.g. v--0000-0008--5001)
Stream: s--<project>--<device>--<variable> (e.g. s--0000-0008--0000-0000-0000-0212--5001).

The Device ID is equivalent to its serial number and is always printed on the label for physical IOTile Devices.  The Stream ID, corresponding to an individual time series of data, will be the most fundamental here for creating Filters.

Stream Data Filters

A mechanism by which action is automatically taken if data values meet a pre-defined condition.

A common initial action taken on sensor or machine data is to set off an alarm.  For a temperature sensor, such a Stream Data Filter may be used to set off such an alarm if the value of a temperature Stream moves out of a pre-defined "safe" range.  For a refrigerator, this could mean temperature rising to a level that may rapidly spoil food, or for a motor, temperature rising to a level that may cause the motor to overheat and seize. Such an alarm may be used to spur a human to action (closing the refrigerator door or turning off the motor) or in compound systems to spur another automated action (increasing the chiller power to a new set point or decreasing the speed of the motor).

To create such Stream Data Filters, IOTile Cloud utilizes the concept of a State Machine with the following concepts: 

• States - A static and defined representation of the system.  For a State Machine to function, at least two states must be defined.  For a temperature sensor, this could be a COLD state and a HOT state.  
• Transitions - The change from one State to another.  The key function of a State Machine is encoded in the Transition between one State and another.  These Transitions are caused by Triggers and result in Actions.  For example a Transition from the COLD state to the HOT state could be caused by a Trigger if temperature rises above 0°C resulting in the Action of an alarm. 
• Triggers - Pre-defined conditions which if met cause a Transition. Triggers are typically made of simple mathematical comparisons such as  "greater than", "greater or equal than", and "equal to." But Triggers can also be compound, where a Transition is only caused by a set of Triggers all being met simultaneously or complex, where more complicated mathematical functions are run to calculate whether a Trigger is met. 
• Actions - Work done as the result of a Transition. Actions can be taken on entry to or on exit from a given State.  Typical alarm-based actions are: "Send an email", "Send an SMS text", or "Post to a Slack channel." Other kinds of actions generate data, such as "create a data entry on a derived data stream." 

In the refrigerator temperature example, upon Transition from the COLD to HOT State, we may want an Action to "set the value of the chiller to 1" which may in turn cycle on the motor.  If the temperature then falls Triggering a Transition back to the COLD state, we may want an Action that sends an email letting the user know that the system is again operating normally.

The following image shows an example of a simple, binary State Machine which defines two States: "Alarm" and "Normal." The Transition between States is Triggered by the data value being 'less than' or 'greater than or equal to' 10 in this case. 

Creating a Stream Data Filter

Stream Data Filters are currently created by using  Classic IOTile Cloud on any Project Detail page. The following step-by-step instructions show how to create the binary filter shown in the image above:

NOTE: ONLY ORGANIZATION ADMINS CAN CREATE FILTERS

1. From the Classic IOTile Cloud, login into your account, select the desired Organization and the Project you want to set a filter for.
2. From within the Project Detail Page, go to the bottom of the page where you will find a list of existing filters. If you are an admin, you will be able to click "Create a new filter."
3. Enter a filter name, and select the variable and device (thus defining the data stream) to which this filter will apply. Note that if you leave the device ID blank, then the filter will apply to all streams with the defined variable within the project (i.e. to all devices of the same type within a project).  Finally, you can enter one or more comma separated names for the states here (note that this field can also be left blank and the states created later).
   
   
4. From the filter page, you will see information about your filter, a list of states, and a list of transitions. Click on "Create a new transition", and create a transition from "normal" to "alarm" when data is greater than or equal to 10.

   

5. Repeat step 4 for the inverse transition, from the "alarm" to "normal" state, only with "less than 10" as the trigger. It is important to ensure that the trigger conditions used are consistent and cover all possible changes in value of the streams selected.  This will ensure that the current state matches your model.  You should end up with the following transition table:

   

6. Finally, create an action to send a text upon the transitions into the "alarm" state. On the list of states section, go to the "Action on entry" column and click "Set" to create an Action. 

   

7. Select "Send Email Notification" as the action type.

   

8. On the Email Action form, you can select one or multiple members of your organization, or enter a list of emails in the "Other emails" section.
   
   
9. At this point, you can either leave the "Custom Body" section empty, in which case, a default email will be sent when the action is triggered, or you can enter a custom email body. You can use a set of predefined text variables which will be replaced before the email is sent. For example, "Your system has transitioned {on} {state} state at time: {ts}" would generate something like: "Your system has transitioned into ALARM state at time: 2017-12-12T10:00:00Z".
   
   
10. After clicking "Submit", you will be allowed to create additional actions, delete or modify existing ones. Once done, click on the filter ID to go back to the filter detail view
    
    

That's it!  You've automated your temperature checking by use of your own filter. Now, whenever new data is uploaded from the selected device(s), the Trigger condition will be checked.  If the temperature exceeds the value of 10 we used here, you will receive the notification email.