Mastering Time-Series Data with an RTOS

Mastering Time-Series Data with an RTOS

In modern application programming, particularly for “IoT edge” devices, handling the ever-increasing data load is a significant challenge. Whether it’s sensor data from industrial equipment, telemetry from transportation systems, or usage metrics from consumer electronics, the volume and velocity of this information is growing at an exponential rate. This data must be processed in real-time, stored for future analysis, and sometimes even discarded selectively. Navigating this complex data landscape is crucial for the success of these applications.

The inevitable increase in application complexity has driven developers to offload elements that are “standard” for any application to third-party packages. For example, whereas 10 years ago it was usual to harness various open-source elements to create a graphics subsystem and GUI, these days most project use commercially available tools for this purpose. Equally we no longer worry about hardware layers, expecting everything we need to be exposed to the API via a provided BSP.

Looking at it another way, we can add most value by focusing on our unique competencies, and the unique features of the product we are developing.

In this vein, many RTOS (Real-Time Operating System) projects are already incorporating off-the-shelf high-performance time-series data management systems, such as ITTIA DB, to address this challenge. These solutions are designed specifically for capturing, processing, and storing time-series data in embedded applications, allowing developers to focus on the unique value proposition of their product.

Designed specifically to deal with real-time data acquisition, ITTIA DB offers a powerful heuristic approach to data collection, rationalization, and dynamic processing. It provides “live queries” – a set of rules that determine how the data is downsampled, stored, or acted upon. These rules can be adjusted throughout the application’s development, testing, and even the product’s lifecycle, enabling adaptive data management to best suit the evolving needs of the application.

The value of time-series data management extends beyond mere storage and processing. Databases, in general, allow us to extract meaningful insights from complex, multi-dimensional datasets, which is becoming increasingly relevant as the sheer quantity of data continues to grow. As an example, an F1 car is said to generate over a million data points per second – a clear indicator that the Internet of Things (IoT) is not far behind in the data deluge.

By leveraging specialized time-series data management solutions like ITTIA DB, application developers can focus on their core competencies and unique value proposition, rather than being burdened by the increasing complexity of data ingestion and processing. This approach enables a more efficient and effective development process, ultimately leading to better, more innovative products.

However, it’s important to note that not all time-series data is created equal. Some of this information must form the basis of the application’s business logic – to be acted upon in real-time, some must simply be stored, as part of the “big data” associated with modern embedded devices, and maybe some should be actively discarded. Striking the right balance between these competing demands is a crucial part of the engineering process.

The adoption of a commercial RTOS in the first place is often driven by mission-critical, security, and compliance requirements. In this environment, the integration of a high-performance time-series data management system like ITTIA DB is a natural fit, as it aligns with the overarching goals of the project – ensuring deterministic processing, secure data handling, and adaptability to changing requirements.

By embracing specialized time-series data management solutions, application developers can shift their focus towards the unique value proposition of their product, rather than being bogged down by the ever-increasing complexity of data ingestion and processing. This strategic approach ultimately leads to more innovative, efficient, and successful embedded applications.

Find out more

A good way to quickly grasp the power of this class of solution is to watch a video of a brief demo in which real-time sensor data is evaluated and downsampled. The video uses an automotive example, but applies to all real-time data management scenarios.

Alternatively, if you have a project and would like to quickly assess whether this technology can help you, during Sept/Oct ’24, ITTIA are offering a FREE Teams meeting with a data scientist in their HQ.