Dual Data Path - MQTT data directly from gateways

Introduction

The Neuron Ethernet and Cellular gateway enable the connection and transmission of sensor data to a customer-defined MQTT broker or custom cloud. This feature is beneficial for sending data directly to a local MQTT broker or a personalized cloud solution.

Neuron Cloud

The neuronsensors system is hosted on Google Cloud Platform servers located in Frankfurt, Germany. Currently, every component in the Neuron system interacts with or retrieves data from these servers. The gateway delivers data from sensors to the Neuron Cloud MQTT broker, data calculations are performed in containers, and data is stored in BigQuery and Firestore.

Glossary

• Sensor value: Measured value of the sensor, such as temperature in °C or pressure in Bar.
• Metadata: Internal data from the sensor, including configuration parameters, name, tags, and serial number.
• Configuration: Editable parameters determining sensor behavior.
• Neuron application: Web-hosted software for data visualization, system configuration, and alarm handling.
• Data storage: Location where historical data is stored.
• Measurement processing: Conversion of directly measured sensor values to other values, such as adding offsets or averaging values over time.

Differences between Cloud and Dual Path Delivery Methods

Sensor Values

• Cloud: Sent to Neuron Cloud.
• Dual Path: Sent to Neuron Cloud and/or custom MQTT broker. Values sent to Neuron Cloud can be masked, making them available only through the custom MQTT broker.

Metadata

• Cloud: Sent to Neuron Cloud for device status.
• Dual Path: Sent to Neuron Cloud for device status.

Connectivity Requirements

• Cloud: Requires a constant connection to Neuron Cloud.
• Dual Path: Requires a connection to Neuron Cloud when changing device settings or adding new sensors.

Neuron Application

• Cloud: Full functionality available, including visualization, alarms, sensor configuration, and system administration.
• Dual Path: Functionality limited to sensor configuration and administration if data to Neuron Cloud is masked; otherwise, all functions are available.

Data Storage

• Cloud: Sensor data, metadata, and configurations stored and backed up on Neuron Cloud.
• Dual Path: Metadata and configurations stored on Neuron Cloud. Sensor data must be stored on client servers if data to Neuron Cloud is masked.

Measurement Processing

• Cloud: Sensor data processed in Neuron Cloud.
• Dual Path: Sensor data processed on the gateway. Changes to processing logic are administered in Neuron Cloud and later uploaded to the gateway when connected.

Neuron Gateway to Customer Server

Premium feature

The Dual Data Path is a paid feature and needs to be enabled to show in the app. Please contact sales@el-watch.com to get an offer.

The cost of the feature is decided out from number of gateways

Configuration of Server

To utilize this feature, customers need to set up a broker in their own systems, which will only receive sensor samples. Additional metadata can be obtained from the Neuron REST API for alias, description, tags, images, and configurations.

Neuron Cloud checks for duplicates on incoming data, and if there are multiple gateways in the system, customers need to implement duplicate checking on their own cloud.

Duplication Check Process in Neuron Cloud

1. Time Check:
   The code examines the time on the sample and calculates the closest time "snapshots" (floor and ceiling) at every 4 seconds.

2. Create IDs:
   Unique IDs are generated using details like a serial number (sn) and status, along with the previously calculated time snapshots. Example: 20004321_15_1702629259.

3. Check for Matches:
   The code checks if the current data has been seen before by looking in the cache.

4. If Seen Before:
   If a match is found, the sample has been seen before and is considered a duplicate.

5. If Not Seen Before:
   If the data is new, it receives a new ID based on the time and is remembered for future reference.

Based on this information, customers can decide on actions to take with duplicated samples.

Configuration of Client

Requirements:

• Gateway must be connected to Neuron Cloud to receive setup information.
• The gateway must be connected to Neuron MQTT-Broker to ensure correctly formatted configurations.

To add a gateway to the Neuronsensors system and set up a Secondary MQTT broker, follow these steps:

1. After logging in, go to gateway settings and open "Secondary broker settings."
2. Configure server details such as URI/IP address, port, TLS usage, connection status, username, password, data masking, topic format, payload format, and selected sensors.

Adding Sensors

While the gateway is online, the sensor list is automatically synchronized with the Neuronsensors system when adding or deleting a sensor. If data masking is enabled, a sample without data is sent from the gateway to the Neuron Cloud to confirm the correct serial number addition.

MQTT Data

When configured, the Neuron gateway connects to the MQTT broker and starts publishing sensor data. The topic is adjustable in the config, but the gateway serial number must be included.

Publish Topics

Topics are adjustable in the config, including the gateway serial number.

MQTT Payload

Sensor data is sent as JSON and is similar to other APIs. An example payload:

[
  {
    "sn": 20001234,
    "time": 1626351076861,
    "status": 1,
    "rssi": -52,
    "node": "OD1",
    "gatewaySn": "VP919600940910",
    "rawValues": 44,
    "values": 4.4,
    "si": "C",
    "valueIndex": 0
  },
  {....}
]

• sn: Sensor serial number
• time: Timestamp from the gateway when the sample was received (milliseconds since Unix Epoch)
• status: Status of the sample (usually countdown from 9 to 1 and loop around). 100 - Button pressed, 105 - Critical data transmission
• rssi: Sensor to gateway signal strength in dBm. A more positive value indicates a better connection.
• node: Antenna or node of the gateway that retrieved this sample.
• gatewaySn: Serial number of the gateway that sent the sample
• rawValues: Raw sensor value before calculation
• values: Calculated sensor value
• si: SI unit of the sensor value
• valueIndex: Index of this value. Useful as some sensors have several values.