Editing Raw Data and Applying a Transformation

Raw data from sensor readings is collected by the device in its “native” form, prior to scaling, filtering, and so forth. For example, the device's firmware reports readings from a 4-20mA analog sensor in microamps, not milliamps. The sensor readings must be converted from these raw values to meaningful engineering units, known as “final values”.  

Transformation is the process of converting raw data to engineering values. The Configuration & Threshold tab of a data stream enables operations performed on the raw data during the process of converting it to an engineering value.

You can change the formula used to convert raw data into engineering units if the formula being used is linear. You can define how to round up and round down engineering values for display. You can apply the formula to the stream’s previous raw values.

You can set the Raw Data Filter's minimum and maximum values to hide samples that you do not want to display. Raw data that is outside this range will be hidden and not used in Visualization, API, Agents, and Reports. Account Owners can see the data displayed in the Samples tab as a Hidden value, otherwise the Hidden values are not displayed. Setting these values is useful in cases where you know that values outside this range are not real values. For example, the values might be caused by the sensor being disconnected from the Wavelet device.

If a data stream applies a transformation to the raw data and an alert is configured, the alert will be triggered based on the Raw Values (not the Final Values). Final Values are processed only after Raw Values reach the Ayyeka Cloud or your on-premise server, so there might be some delay. In contrast, the alerting system originates in the device that initiates an alert and an immediate transmission as soon as the Raw Value reaches the threshold that you set.

 

Prerequisites

  • Minimum and maximum Final Value Limits of a data stream must be set before a threshold can be defined for that stream.
  • To see Hidden values for Raw Values that are outside the minimum and maximum range, you must have Account Owner role. 

 

To edit or apply transformation to raw data:

  1. In the left pane, click Devices.
  2. In the Sites Tree pane, click the stream name.
  3. In the right pane, select the Configuration & Threshold tab. The Data Processing pane opens.
  4. To exclude bad raw values in the sample data, click the pencil icon in the Raw Value Filter area. The Update Raw Value Filter window opens.
    1. Type in the lower and upper limits for valid values. 
    2. You can apply this filtering to historical samples up to four weeks back. If you need to update samples older than four weeks back, contact support@ayyeka.com. 
      5.   To change the formula for converting raw data to engineering units, click the pencil icon in the Convert to Engineering Units area. The Update Linear Transformation window opens.

     

      1. Define the formula for converting raw data to engineering units in either of the following ways:
        • Click the pencil icon  to edit the existing formula, as described above.
        • Click the beaker icon  to define the formula by entering a pair of (Raw Value, Final Value) values. The system will compute the actual formula. If you know any two points (current or voltage with known engineering units), you can more accurately convert Raw Values to Final Values.

             b.   Check the Update old samples check box to retroactively update old samples by applying the new conversion formula. You can update samples up to four weeks back. If you need to update samples older than four weeks, contact support@ayyeka.com.
              c.   Click Submit.


       6.   To limit the display of engineering values, click the pencil icon in Final Value Limit area. The Update Final Value Limit window opens.

      1. Type in the lowest and the highest numbers that the engineer values will display. For example, if you type in 0 and 145, then any Final Value less than 0 will be displayed as 0. Any Final Value greater than 145 will be displayed as 145.
      2. Click Update old samples to retroactively update old samples according to the new range of values. You can update samples up to four weeks back. If you need to update samples older than four weeks back, contact support@ayyeka.com.
      3. Click Submit.
    7.   To update old samples without changing the transformation, click Reapply . Use the date picker to update samples up to four weeks back. If you need to update samples older than four weeks, contact support@ayyeka.com.