Data harmonisation

The Data Harmonisation service supports the process of standardising and integrating data from different and heterogenous sources by transforming it into a common format (e.g., AIM). This includes the processing of data, ensuring compatibility with standard-based data sources, and providing an integrated data view and access layer.

Interested in this service? Contact us at kontakt@agrifoodtef.pl
 

Overview

Support for data harmonisation and integration, i.e., processing and transforming data from different sources into a standard common format. This process requires the specification and implementation of mappings between a data source model and a target (standard-based) model, the integration with other standard-based data sources (establishing links between data elements), and the provision of interfaces to access the harmonised data. The harmonisation can be performed as a one-time activity, e.g., with (semi-) static data sources, but more importantly in a continuous manner, whenever new data is becoming available. For the latter, a pipeline is defined with the specified mapping once, and then it can be re-executed (via CLI or API) as frequently as needed just by changing the input data.

More about the service

Data harmonisation will enable customer
-enhanced interoperability between different systems, machines, and software platforms by making data consistent, standardised, and easily exchangeable across various technologies.

Harmonised data:
- ensures that different systems can communicate by using a common data format
- Support the connection of legacy systems with modern platforms, allowing businesses to adopt new technologies without discarding existing infrastructure.
-enables IoT devices, agro-robots, and other smart systems to share data in real time, improving automation and efficiency.
- facilitates interoperability between different sectors, such as agriculture and manufacturing, by ensuring data can be used across multiple domains.
- helps businesses meet industry standards and legal requirements, reducing risks and improving data governance.
- enables an integrated view and access of data coming from different sources, for instance, by data analytics components and decision support systems to leverage the full value of all available data Data harmonisation acts as a "universal translator" that enables different systems and machines to work together smoothly, making processes more efficient, automated, and future-proof.

Remotely as a cloud service, for instance, in a scenario requiring a service that transforms agricultural data from different IoT devices into a standard format (e.g., AIM). It can also be provided as an on-premise deployment where a customer integrates the service into its internal system to standardise machine data with the support of a data engineer, or as a CLI tool (deployed via Dockerfile), which can be easily integrated into internal systems. Additionally, the customer will receive mappings of the dataset(s) identified and target data model (e.g., AIM) and the pipeline definitions to carry out the harmonisation.

The service can be customised to fulfil particular customers’s needs. The processing of the service typically involves the specification of custom mappings for specific data sources, which would also be provided. Additionally, in many cases, the service will need to be customised to integrate it within a data processing module in the customer’s own systems.
Location
At user's premises
Poland
Remote
Type of Sector
Arable farming
Food processing
Greenhouse
Horticulture
Livestock farming
Type of service
Conformity assessment
Data augmentation
Provision of datasets
Accepted type of products
Data
Design / Documentation
Software or AI model