Catalogue of Services

The Benchmarking & Testing Suite for Edge Hardware Systems delivers a set of tests designed to evaluate the performance, reliability, and functionality of edge hardware and its components under various operational conditions.

The tests can be compared to specific industry standards or the performance of other solutions available on the market. Examples of tests offered as part of the service:

- Environmental tests: Assessing devices' resistance to extreme conditions, such as temperature, humidity, and vibrations.

- Signal tests: Evaluating devices using GNSS signal generators, testing their resilience to interference or false signals.

- Network tests: Evaluate device performance within a prototype 5G network infrastructure.

- Functional tests: Assessing the capability of devices, such as remote PTZ (pan, tilt, zoom) cameras, to perform operational tasks in field conditions.

- Integration tests: Examining the cooperation of edge devices with sensors, AI systems, and their responses to data input failures. - Accuracy tests: Measuring the precision of sensors and control systems. Test results can be compared to specified standards or the performance of competitive solutions, enabling customers to better understand their devices' capabilities.

Conformity assessment and compliance tests

As part of this service, we test and measure, among other things, mechanical, physical, acoustic, radio, electromagnetic compatibility and electrical parameters for the purpose of assessing the overall user safety of agricultural, horticultural, forestry and food machinery, equipment and components. Verification is based on the requirements of standards and directives declared by the manufacturer. We carry out an initial assessment based on the design documentation provided or/and on measurements taken on a prototype based on harmonised and non-harmonised standards with relevant EU and sectoral legislation. This is to ensure that for instance essential requirements coming from for instance New Legislative Framework directives and other EU law are fulfilled to better protect both consumers and professionals from unsafe products to be placed on the European internal market. One of the aims is to help manufacturers in legal placing agrifood products on EU single market and in CE-marking process. The results obtained can be used in the further process of product labelling, declaration of conformity to affix the CE mark to the device within the scope of EMC, LVD, RED, MD, MR on other NLF directives and regulations.

Certification

Conformity assessment

Data validation and processing services

The service will provide validation and processing of data sets, in particular video data, coming from AI-based systems testing and/or digital measurement and data collection in cloud technologies. The scope of the proposed data validation will include, among other things, the service provider's activities of checking the presence of all data streams in the analysed image or video files, detecting any breaks in transmission, and checking the focus or exposure of the camera images. The next stage of the service will be the preparation of the material for evaluating the performance of AI-based technical systems under test, e.g., by formatting, identifying images containing selected visual markers, or linking images from a given system under test to data of other modalities.

Evaluation of AI solutions for crops, livestock, and food products analysis

The service provides the testing, validation, and training of machine learning algorithms specialised in the detection of weeds, crops, livestock, and food products. The model data for the training implementation can come from different sources. They will come from services run by the Agrifood TEF service network. They may come from the service provider's implementation of other similar services, or they may be provided by the client of the service as collections resulting from their research. In this case, the datasets provided will be validated to assess their quality and suitability for machine training of artificial intelligence algorithms. The envisaged scope of the service, the algorithm training, relates to a wide range of fields and includes farming and plant breeding, animal husbandry, and food processing.

AI model training

Performance and sustainability testing of robotic or AI-driven solutions

This service provides a comprehensive assessment of the performance of systems, focusing on sustainability indicators such as energy consumption and input usage per hectare. It evaluates the impact of artificial intelligence (AI) or robotic solutions on the performance and sustainability of the associated equipment. The service includes tests such as functional testing, accelerated durability testing, and assessments of the system’s resilience to environmental factors and electromagnetic interference (EMI). By identifying potential design flaws or manufacturing defects, the service supports the development and optimisation of tested solutions, contributing to sustainable technological advancements.

Performance Testing of Robotic Systems and AI Solutions

We provide physical testing services, under both laboratory and field conditions, to assess the proper performance of the object under test, the level of quality, and the performance achieved. These tests focus on evaluating the performance of an object under laboratory and operational (field) conditions while maintaining its proper functionality. Our testing process helps identify potential weaknesses in the design of the device, offering insights that can inform adjustments to algorithms or hardware. This service aims to ensure that systems perform reliably under a variety of conditions, helping to prevent failures and maintain optimal performance in real-world applications.

Physical Testing of Endurance & Reliability

We provide physical testing services, both in laboratory and field environments, to evaluate the endurance and reliability of devices and their components. These tests focus on the ability of components to withstand external conditions while maintaining proper functionality. Our testing process helps identify potential weaknesses in device design, offering insights that can inform adjustments to algorithms or hardware. The service is designed to ensure your systems can perform reliably under a variety of conditions, helping to prevent failures and maintain optimal performance in real-world applications.

People training

Provision of datasets

Providing demonstration testing on the test field of robotic or AI-driven solutions

As part of the service, a testbed will be provided for product demonstration of robotic machines along with AI systems. The polygon will be configured and prepared to meet the customer's individual needs in terms of field operations, technological processes, and communication. The service provider will provide polygons with a variety of surfaces according to the customer's specifications, e.g., meadow, ploughed field. The service will include soil moisture control, continuous monitoring of the field, and all activities carried out on it. The service providers will provide the necessary assistance and infrastructure needed to support the fieldwork carried out by the surveyed sites. The service may also involve demonstration of AI systems used in apiaries.

People training

Provision of datasets

Provision of real-time data streams

The service provides access to data streams collected from equipment tested by other services or using specially installed sensor or camera arrays in specific test environments. It includes the provision of data streams from sensors and cameras installed to meet specific customer needs in agreed areas and tailored to work in these specific locations. Such areas are, for example, fields, orchards, vineyards, forests, or specific locations such as farms, barns, or warehouses. Furthermore, the service includes access to corrections from local RTK stations and meteorological station data. In this way, it provides comprehensive and reliable data access support for a variety of customer requirements.

Provision of datasets

Testbed preparation

Testbed preparation will enable costumers to execute physical tests of their technology/machines/algorithms in line with their expectations.

Within this service, costumers might get a well prepared flat/slopped arable lands or permanent grasslands.

We can also prepare a land with "controlled rain conditions" or a dump ground. There is also a possibility of providing and preparation of apiary or a sprinkler for software testing. We can help costumers run physical experiments by preparing a cultivated/non-cultivated fields or run tests on test plots, specially designed for testing. Within the latter, we will take care of such elements like: soil preparation, seed/plant procurement, seeding/planting, plant growth management, preparation of technical infrastructure. All this to meet your needs to ensure optimal conditions for conducting and monitoring tests.

This service can be executed in fields with various area with system of weather stations, phenological stations and ICT infrastructure. We can also prepare costumers sets of data collected for these areas like archive cultivated plants, treatments, crops, etc.

Testing of error handling, failure monitoring, and safety related to cyber threats

As part of the service, its provider offers to grant access to the possibility to perform and carry out specialised tests for artificial intelligence systems and robots, aimed at detecting system faults, monitoring failures, and providing reliable assessments of cybersecurity risks present in the operation of IT systems, computer systems, data transmission, etc. These are now necessary measures to ensure the safe, reliable, and secure operation of artificial intelligence systems, autonomous vehicles, and robots. They require continuous testing by their user, the owner, which goes beyond basic functionality. In addition, continuous monitoring of failures through sensor data, system logs, and performance indicators is crucial to identify potential problems before they escalate. These activities are also part of a continuous cybersecurity threat assessment of owned ICT systems, databases, etc.

Cybersecurity

Testing of plant health assessment tools for high-value crops