Catalogue of Services

Through this service, we offer customers a complete feasibility study of their digital systems and/or data, evaluating their suitability for the test use cases identified by the customer. The study involves both technical and agronomic aspects. The goal of this service is to provide the customer with a comparison between the features of the system or data submitted for review and the requirements of the use cases chosen for (possible) subsequent testing. Examples of systems and data requiring digital testing include (but are not limited to) software modules for data processing (e.g., machine vision modules), AI models, software architectures (e.g., control systems of robots), simulations, datasets collected in the field, synthetic datasets, designs of software systems, and so on. This assessment is crucial to provide customers with key information on elements of their system that need to be further developed to become “fit for testing.” It can also identify changes to apply to either the system or AgrifoodTEF’s testing facilities to meet the requirements for experimentation.

Desk assessment

Desk assessment activities for physical systems

Through this assessment, we offer customers a complete feasibility study of the use cases and systems that customers would like to test, comprising both technical and agronomic aspects. Examples of systems requiring physical testing include (but are not limited to) precision sprayers, weeding machines, robotised fruit picking systems, and other implements; examples of subsystems include (but are not limited to) perception systems (e.g., based on vision), autonomous driving modules, localisation systems, and mapping and navigation subsystems. This assessment is crucial to provide customers with key information on elements of the system that need to be further developed to become “fit for testing”. It can also identify changes to apply to either the system or AgrifoodTEF’s testing facilities to meet the test requirements.

Desk assessment

Evaluation of results of physical testing

This service performs the crucial step that follows the execution of physical experimentation, i.e., evaluation of the performance of the system under test. This activity requires the application of suitable performance metrics to specific test data collected during the tests and then interpreting the results in view of the features of the system and the experimental setting. Depending on the specific use case, performance metrics can involve pure computer processing, human expertise by agronomists, or a combination of both. Application of performance metrics may include the development of custom software to extract the necessary information from experimental data and/or to apply suitable processing to the information. If the performance metrics require the data to be subjected to some form of pre-processing, Service S00115 may be leveraged to prepare the data. Besides the object of this service (i.e., evaluation of results), AgrifoodTEF can, on request, support the customer along any other aspect and phase of the physical experimentation pipeline. For instance, for the design of the environment and protocol for the test, the customer can leverage services S00106 and S00107. Preparation of the test environment can be done either by the customer or by AgrifoodTEF via service S00110; in both cases, assistance in interconnecting the system under test with AgrifoodTEF’s test infrastructure is available via service S00111. Finally, data collection during the test can, if needed, be provided by AgrifoodTEF via service S00113.

Data analysis

Execution of digital testing

This service assists customers in executing digital tests using agrifoodTEF’s digital infrastructure. Tests can either be defined by the customer or designed through agrifoodTEF services (e.g., S00176, S00177, S00178). If specific activities are required to properly interface the system under test with the digital infrastructure, these can be handled by either the customer or agrifoodTEF (via Service S00180).

Support provided by this service includes general assistance during the test execution phase, such as monitoring to ensure compliance with the testing protocol or offering immediate help in identifying and resolving issues. Additionally, the service provides technical support during the test execution of the system under test, including ensuring that all necessary processes are correctly configured and running, as well as helping the customer address any unforeseen challenges that may arise when complex digital systems interact.

Finally, through this service, agrifoodTEF manages the technical infrastructure for test execution, ensuring smooth data exchange between the system under test and the agrifoodTEF infrastructure, and verifying that all testing requirements—potentially identified during the desk assessment phase (Service S00179)—are met during the actual test runs.

Lukasiewicz - Poznanski Instytut Technologiczny

Functional safety testing of machines

Within the framework of this service, we offer the possibility of carrying out functional safety tests of objects in the form of, for example, autonomous vehicles, robots, machines, agricultural equipment and components, etc. The service provides opportunities to test these devices with measurements of their mechanical, physical, acoustic, electrical parameters and electromagnetic compatibility to assess the overall safety of the user of the tested vehicles, machines, equipment and components for agriculture, horticulture, forestry and food. Verification of the tested parameters is based on the requirements of standards and directives declared by the manufacturer as compatible with its product and general normative safety regulations. Initial evaluation of an object is carried out based on its provided design documentation and/or measurements made on a prototype based on harmonized and non-harmonized standards with relevant EU and sector regulations. In the case of autonomous vehicles, we also offer tests of the correctness of response and operation of safety devices built into their structures in standardized cases of hazardous situations. The results obtained can be used in the further process of product labelling, declaration of conformity to place the CE mark on the device.

Certification

Conformity assessment

ELSA assessment

LCA assessment

People training services

The AgrifoodTEF consortium comprises a set of testing and experimentation facilities and targeted services aimed at supporting customers in the testing and validation of systems and devices based on advanced technologies. Given the technical complexity of these facilities and services and the associated complexity of many activities making use of them, many customers can benefit from a preliminary training activity. The goal of such training is to make best use of the facilities and services and fully exploit the allotted time when actual testing/experimentation occurs. Customer training – involving both technical and procedural aspects – can either be sought by the customer autonomously leveraging the documentation provided by AgrifoodTEF or directly supported by AgrifoodTEF via this service; a combination of both approaches is also possible. This service does not limit its applicability to customers that already know what they require from AgrifoodTEF. The service also provides an opportunity for customers to discuss their specific needs and requirements with our team and be directed towards the set of services in the catalogue that best suit their needs. Especially where the customer envisages an articulate testing campaign, exploring their idea together with AgrifoodTEF via this service can be very beneficial in planning subsequent activities. This service can, if needed, be integrated with services S00109 (desk assessment activities for physical systems) or S00179 (desk assessment activities for digital systems or data) to anchor its operation and possible further testing activities more closely to the specific features of the system(s) developed by the customer.

People training

Performance evaluation of an AI model

Our service provides a thorough evaluation of AI models by utilising a wide range of benchmark datasets—from images and sensor data to remote sensing and multimodal data. This comprehensive testing assesses the model's performance across standard metrics, specifically chosen based on the model’s intended tasks. The results offer insights into the model’s effectiveness and accuracy, guiding developers in refining their AI solutions to achieve optimal performance in real-world applications.

Preparation of computational test environment

To successfully conduct a digital testing campaign, preparatory activities are usually required to set up the computational environment used for testing. This service performs activities such as: - setting up the hardware resources needed to run the tests - configuring and initialising the virtual environments used for testing (e.g., Docker images, private “data rooms” for safe data sharing) - installation and configuration of required software packages and dependencies - Setting up authentication layers, user roles and credentials as needed- Migration and/or exchange of required data and ground truth annotations - installation and configuration of a simulator - importing and configuring a previously defined simulated environment - executing dry runs to check that all elements of the test environment operate as required Environment preparation is done according to an environment design provided by the customer. If needed, such design can be done by AgrifoodTEF for the customer via Service S00176. Interested customers can get support from AgrifoodTEF for the entire pipeline involved in digital testing, from the design of its other elements beside the environment (namely, the testing protocol via service S00177 and the evaluation metrics via service S00178) to test execution (service S00182) and data collection (service S00183) and evaluation (S00184). Support in interconnecting the systems under test to the digital testing environment, if needed, is available via service S00181.

Provision of datasets

During the development of systems based on AI and/or robotics, tailored data are needed to support the realisation, validation and optimisation of systems and subsystems. This service aims at providing such data. AI models are a noteworthy example of systems that require data to be trained; more generally, any system which processes data produced via artificial perception (such as images, videos, LiDAR, ultrasound, radar...) requires suitable data to be built, tested and fine-tuned. This service allows a customer to define together with AgrifoodTEF the specific features of the data that the customer’s activities require and to receive datasets precisely matching such features. Among the features to be defined are the type, number and specifications of the data streams comprised in the datasets (e.g., when multi-sensor data are required) and the range of variations encompassed by the data (e.g., variety of environmental conditions represented in a sensor stream); lower-level features to be defined concern formatting and packaging of the data.

Provision of datasets

Testing of AI-based sensor performance

Our service thoroughly evaluates AI-based sensors by measuring key metrics like accuracy, mean squared error, and other tailored parameters. Testing is conducted in real-world environments where the sensor will be used, with reference data recorded or labelled for analysis. For certain sensors, we also utilise benchmark datasets to validate performance. We assess consistency under identical conditions, adaptability to environmental changes (e.g., light, temperature, or weather), and improvements in accuracy over time. Additionally, we measure power consumption, processing efficiency, and memory usage to ensure optimal resource utilisation. Response time between detection and action is evaluated for real-time applications, while extreme condition testing (e.g., bright light, rainy conditions, compacted soil) ensures robustness in challenging environments. We also verify data security and resilience against adversarial attacks. This comprehensive testing is essential, ensuring sensors meet high standards of precision, reliability, and efficiency. Benchmark datasets, standardised data collections used for validation, provide a reliable baseline for performance comparison. By addressing these aspects, our service ensures your sensor technology is ready for real-world deployment, delivering the performance and adaptability your application demands.

Collection of test data

Testing of biomass and yield estimation

Our testing service assesses the accuracy of your system's biomass estimation capabilities. You can opt to evaluate your model virtually using existing datasets, such as satellite or image data, or we can gather new field data with our field sampling equipment specifically for your needs. The main targeted crops of the biomass estimation are grassland as well as clover and alfalfa, but also grain crops are possible. As yield and biomass are highly correlated in grassland, both can be determined with this test. We provide results using metrics like Pearson correlation/R-squared and MSE/MAE to give you a clear picture of your model's performance.

Collection of test data

Testing of edge intelligence and IoT architectures

Fondazione Bruno Kessler

Testing of AI and robotics solutions leveraging edge computing and communication protocols in the IoT cloud-edge continuum. Can be a digital test (run on virtualised machines) or a physical test deployed either at the customer's premises or using the wide coverage wireless network installed in Trento valleys (Val d'Adige and Val di Non) for fruit crops (including but not limited to apples and grapes). FBK can support test design, proposing a benchmarking architecture to stress and monitor key components, helping to understand bottlenecks, allowing for better performance, lower bandwidth use, more reliability, and lower operating costs of tested solutions.