Six Rules of our Architecture for Integration of Scientific Data Types

updated March 2023

Introduction

This document discusses Networked Robotics' architecture for network data collection. The Networked Robotics approach relies on modular and mutable electronic hardware to accomplish integrated real-time data collection from diverse scientific instruments from multiple vendors. The discussion section compares our approach to that of current standards in networking and automation.

From Networked Robotics' point-of-view enabling common real-time data collection from diverse scientific instruments and sensors is primarily an act of complexity reduction. We reduce the problem into two main classes of complexity 1) Physical, the different kinds of electronics and connectors that are used to talk to other devices and 2) Logical, the computer languages that each machine uses to talk to the outside world. In our approach, physical complexity is reduced according to the diagram below - via a modular hardware implementation, the hardware elements of the system are 1) the network data collection server, a computer anywhere on the network ( not shown) 2) specialized network hardware (green) 3) An interface (yellow) and 4) the "monitored device", which by our definition are either scientific instruments or sensors, (purple), but could also be manufacturing machines, toasters, or any other entity that produces data.

Networked Robotics' Physical Data Collection and Integration Architecture

Examples of "purple" might include scientific incubators, liquid nitrogen cryofreezers, radiation sensors, gas sensors, analog probes, switches, ovens, stability cabinets, PH meters and many more. Examples of their physical data communication methods might include voltage outputs, RS-232, RS-485, TTL, I2C, ethernet, switches, binary outputs, 4-20 milliamp, etc.

The logical complexity-reduction in the process is handled by our network hardware's ability to 1) understand and unify data collection protocols for multiple machine languages, and 2 ) to "learn" new machine languages for new "purple" devices via network downloadable firmware (.NRF or Networked Robotics Firmware files).

In our approach there are 3 possible levels of message-passing, 1) from a network data collection server (e.g. a Tempurity Server) to the Networked Robotics NTMS network hardware 2) from NTMS network hardware to a Networked Robotics interface, and 3) from the interface to the "monitored device", or the sensor or instrument that produces data. The levels are increasingly complex. Level 3 reflects the full diversity of how engineers choose to implement the ability for their device to talk to the "outside world".

The six "rules" of how this system is implemented by Networked Robotics are specified below..The rules limit capability in some respects (precision, the delinking of metadata) but provide the ability to collect data from any source in a common way across the entire pantheon of instrumentation and sensors, and to provide a common interface that allows any analyzing process to use it.

The rules are based on control, the command of scientific data streams in the device's own language. Here they are:

The 6 Rules of Networked Robotics' Simplified Method of Integrated Real-time Network Data Collection:

1) Network Master:

The network controls data collection. Monitored devices are queried through the network via the NTMS network hardware. They must be able to respond to a request for data. Data is always requested by the network data collection service. The monitored device is always a service to the Networked Robotics' NTMS hardware and thus to the data collection server on the network.

Networked Robotics' NTMS hardware units poll monitored devices for data even in the absence of network commands requesting data. Network-side commands return the last-read value of the monitored device stored in the NTMS. This principle is designed to reduce data collection latency and to work with widely varying data collection intervals for scientific instruments and sensors. In our approach there is no guarantee that a network-commanded reading will result in a new value.

2) Network Immutability:

Stored data values at the data collection server are the same as the values that are sent by Networked Robotics' NTMS network hardware. The datum that is stored is in the same form that it is when it leaves the NTMS, but is probably not in the same form that it was in when it leaves the instrument because the NTMS may convert the data format, including its precision. In many cases the NTMS commands data acquisition but drops all irrelevant (not requested by the network data collections service) data, making available only the requested information of interest to the data collection server.

3) Network Metadata Assignment:

The monitored device type (e.g. humidity, co2 concentration, id, temperature) is determined by the network command/response sent to the NTMS by the data collection service. The "tagging" of parameters by a device's own machine language is thus irrelevant to the network data collection service. (It is relevant for NTMS firmware performance) A voltage request by the data collection server always returns a voltage or an error. A temperature request always returns a temperature or an error. The network data collection service thus relies on the NTMS network hardware to accurately relay the correct data type based on its understanding of the target machine connected and its machine language.

4) Network Command Single Response:

Only one monitored device type ( e.g. humidity, co2, id, temperature) response value is returned for each query by the network data collection server. As described above, in most cases the NTMS drops all irrelevant data, taking only the requested information of interest all the way to the data collection server. Responses by many instruments are thus "filtered" by the network hardware or interface modules.

5) Network Command Equivalence:

The network commands issued to read data from a single monitored device type (temperature, humidity, carbon dioxide concentration) from any of very many diverse monitored devices is always equivalent on the network side regardless of the monitored device (instrument, brand, or model) from which data is to be collected. Although "local" commands (the monitored device's logical protocol) are diverse and specific to the monitored device, the network commands for acquiring these same parameters from any monitored device are standard.

6) Universal Time:

Each network-collected datum is tagged with a universal time at the time of acquisition.

Discussion:

Several powerful standards exist for industrial control and automation, building control and automation, scientific automation, and network control and automation. In automation Modbus is common, and many scientific instruments support versions of Modbus protocols. Bacnet, with origins in the building control world provides an excellent protocol for collecting data.SNMP has effectively unified the world of network devices.

Modbus and Bacnet in our opinion suffer from their roots in industrial automation. Bacnet unique IDs are superfluous in a network world where MAC addresses are already unique and well-controlled and ubiquitous. Modbus suffers from a lack of simplicity such that there are so many potential implementations that the standard becomes a non-standard.

Rule 5 above for command single-response is a feature of Modbus and of SNMP.

Of all the standards Networked Robotics agrees with the philosophy of SNMP the most because of the word "simple" in Simple Network Management Protocol. SNMP reduces the complexity of message-passing to 5 simple types that are able to obtain virtually any form of data and metadata with fast response times.

All existing standards fail to address a key new ability, that is for devices to "learn" through net-upgradeable firmware. Some will say that this is true of SNMP MIBs for example, but MIBs are essentially tree-based data structures that do not include the possibility of instrument-wide reprogramming of functions and methods.

Scientific instruments and sensors are engineered by people with their own idea of which standard or implementation to follow. The Networked Robotics approach takes the integration out of the hands of these engineers such that any implementation that they follow is capable of being addressed and integrated via our architecture which acts as an intermediate level of complexity-reduction.

This approach currently applies only to constantly-collected real-time data such as environmental data and not to "batch" collections such as those in chromatographic or mass spectographic runs. Data collection speeds are expected to be slow compared to bandwidth limits. What is slow? In the past the network was criticized as a vehicle for real-time data. The internet is non-deterministic. It is difficult to tell when competing packets will come in that will delay data collection. It would not be appropriate to base millisecond decisions on network-collected data. The kind of environmental data that we collect from is much slower. The latency of data collection for a single packet across the planet now is generally the speed of light across oceans. Internet (TCP/IP) Messages from anywhere on the planet arrive at 500 milliseconds or faster - only 7.5 times the theoretical maximum of 67 milliseconds. That means that from anywhere on the planet one-way latencies less than a second are now a safe bet, which makes internet data collection pretty deterministic for many applications across the globe.

Perhaps the most important rule is number 6. Universal time is independent of time zone but can be easily converted to a local time. Although storage of local time with time zone would be equivalent, most applications ignore time zone, assuming that the time zone is irrelevant because the data will be used locally. Networked Robotics assumes the opposite, that all data will be used globally.

back to Support / Tutorials