Catalogue of Services

AI model training

Politecnico di Milano

Location

Italy

Remote

Arable farming

Food processing

Greenhouse

Horticulture

Livestock farming

Tree Crops

Viticulture

This service concerns training AI models on behalf of the customer for a specific task and optimisation objective, e.g., improving accuracy on crop classification from image data. The target model is the solution provided by the customer that needs to be enhanced with respect to a set of pre-determined features to reach the desired performance level. However, if required, the training can also be applied to additional state-of-the-art models available in the market for benchmarking purposes. If not defined by the customer, some features of the training process can be identified via service S00179 (desk assessment activities for digital systems and/or data): for instance, model features to improve, reference model baselines to include in the performance comparison, as well as benchmark datasets. The data used for training the model can be either provided by the customer or annotated ad hoc as a preparatory activity to model training (via service S00290 - Data Labelling); another possibility is that data are retrieved among reference benchmark datasets that are openly available. We will also agree with customers on the level of hardware acceleration required, based on the considered AI models: e.g., GPU acceleration via connection to a remote server vs. on-device training.

Assessment of AI performance and software optimisation on edge computing devices

Fondazione Bruno Kessler

Location

At user's premises

Italy

Remote

Arable farming

Food processing

Greenhouse

Horticulture

Livestock farming

Tree Crops

Viticulture

The tests in this service are aimed at products that run complex apps (based on a multitude of microservices, including heavy AI algorithms) in the fields, with limited to no connectivity to the cloud. In these cases, we propose a benchmarking designed to understand what the bottlenecks are in the implemented solution architecture to help TEF customers improve the efficiency and reliability of such products/solutions. The service may also encompass virtual facility elements (enriched datasets, virtual machines hosting containerised modules) blended with physical sensors and infrastructure in our test field or directly installed at the client's premises if need be for the duration of the experimentation.

Collection of test data during physical testing

Politecnico di Milano

Location

Italy

Arable farming

Food processing

Greenhouse

Horticulture

Livestock farming

Tree Crops

Viticulture

Through this service, we provide specialised technical systems (both hardware and software) and expertise for collecting relevant data during testing.

The scope of the data collected can vary, encompassing information generated by the agrifoodTEF technical infrastructure (e.g., through the installation of IoT sensors), data from the system under test, and ground truth data for system evaluation. If needed, this service can facilitate the interconnection between agrifoodTEF's data collection systems and the customer's systems.

In addition to the raw data collected, we will supply customers with comprehensive documentation detailing the logged features, conditions of the testing environment at the time of testing, and any parameter values, variation ranges, and specifics necessary for reproducibility.

When required, data collection and documentation activities may involve field experts such as engineers and agronomists.

Please note that annotation and labeling of collected data, if necessary, can be provided through service S00115. 

Data labelling

Politecnico di Milano

Location

Italy

Remote

Arable farming

Food processing

Greenhouse

Horticulture

Livestock farming

Tree Crops

Viticulture

This service concerns the curation of ground truth annotations for testing and experimentation data. More specifically, data labelling concerns the association of portions of data (e.g., image regions) to descriptions that provide information about the significance of those portions. In particular, labelling is required whenever the data are meant to be used for model training. We also provide labelling services for domain-specific data attributes, such as disease indicators and their confounding factors, by experts in agronomy. The labelled data will be accompanied by a report synthesising quality metrics and statistics like the number of data points annotated, the percentage of the full set covered for the annotations, the number of annotators and the inter-annotator agreement.

Design of computational environments for digital testing

Politecnico di Milano

Location

Italy

Remote

Arable farming

Food processing

Greenhouse

Horticulture

Livestock farming

Tree Crops

Viticulture

Test activities involve three key components: the environment (where the tests are conducted), the protocol (defining what tests are performed and how), and evaluation metrics (used to assess the test results). Through this service, we assist customers in designing a computational environment tailored to running the digital tests required for their software components. Specifically, this service:

• Defines the data (and metadata) requirements needed to support the test
• Identifies the necessary software environment (e.g., operating system, libraries)
• Selects an appropriate simulator and defines simulated environments, if required
• Chooses remote access tools, if applicable.

As a result, we deliver a comprehensive design of the testing environment, including specifications such as memory requirements, GPU/TPU acceleration, and necessary libraries. This environment can be deployed on either the agrifoodTEF digital infrastructure or the customer's computing infrastructure (i.e., on premises).

Design of evaluation metrics for digital testing

Politecnico di Milano

Location

Italy

Remote

Arable farming

Food processing

Greenhouse

Horticulture

Livestock farming

Tree Crops

Viticulture

Any test activity involves three main components, i.e., environment (where the tests take place), protocol (defining what activities are executed and how), and evaluation metrics (used to assess the results of the tests). This service concerns the last element; its goal is to design the best metrics to evaluate the performance of digital systems such as (for instance) AI models or computer vision software. The digital environment and the testing protocol metrics can be designed—if required—via services S00176 and S00177. Our team will identify and define with customers the most adequate set of quantitative metrics to assess the outcome of the digital testing activities. In order to ensure the relevance of the metrics with respect to the real-world use cases, the team will involve engineers and agronomists. The metrics will be adapted not only to the task that the digital system under test (e.g., a piece of software) is designed to perform, but also to the features of the data used for the tests. For instance, a customer that has developed a machine incorporating an AI model will be interested in testing the model on data generated by their own machine: the performance metrics will therefore need to be adapted to the specific features of that data.

Design of evaluation metrics for physical testing

Politecnico di Milano

Location

Italy

Remote

Arable farming

Food processing

Greenhouse

Horticulture

Livestock farming

Tree Crops

Viticulture

Any physical test activity involves three main components: environment (where the tests take place), protocol (defining what tests are executed and how) and evaluation metrics (used to assess the results of the tests). This service concerns the last element; its goal is to design the best metrics to evaluate the performance of a customer solution taking into consideration the use cases specified by the customer and the environment and protocol chosen for the tests (which, if needed, can be designed via services S00106 and S00107). Our team will identify and define with customers the most adequate set of quantitative (i.e., based on instrumental measurements) and/or qualitative (i.e., relying on expert human judgement) metrics to assess the system functionalities of interest. This phase will involve, in particular, agronomists and experts in agricultural machinery. Based on the defined evaluation metrics, a set of requirements for the collection of required data and ground truth annotations will also be defined accordingly. For instance, the service may lay out the specifications for dedicated data collection campaigns (possibly executed via service S00113). This phase will involve engineers and experts in AI and robotics. On request, the output of the service will include analyses on additional environmental factors than those directly tracked through the designed metrics (e.g., seasonal effects, impact of test distribution over time on results).

Design of testing protocols for physical testing

Politecnico di Milano

Location

Italy

Remote

Arable farming

Food processing

Greenhouse

Horticulture

Livestock farming

Tree Crops

Viticulture

Any physical testing activity comprises three main components:

• the environment (where the tests are conducted),
• the protocol (which defines the tests to be executed and their methodology),
• and the evaluation metrics (used to assess the test results).

This service focuses on the protocol, while the environment and metrics can be designed as needed through services S00106 and S00108.

In the context of testing customer solutions within physical facilities, this service aims to create a suitable testing protocol based on the use cases specified by the customer. The components of the testing protocol defined in this phase may include:

• Defining the different phases of the protocol and their duration
• Outlining the operations to be executed in each phase
• Quantifying the effort required for each phase, including the number and qualifications of personnel involved
• Distributing testing operations over time, considering seasonal and daily variations, as well as weather conditions
• Establishing acceptable and desired variation ranges for each configurable element in the testing environment
• Determining the number of test repetitions required and their distribution across the variation ranges

To ensure a comprehensive approach to protocol design, this service involves a collaborative team of both engineers and agronomists.

Design of testing protocols for digital testing

Politecnico di Milano

Location

Italy

Remote

Arable farming

Food processing

Greenhouse

Horticulture

Livestock farming

Tree Crops

Viticulture

Any test activity involves three main components, i.e., environment (where the tests take place), protocol (defining what activities are executed and how), and evaluation metrics (used to assess the results of the tests). This service concerns the second element, i.e., the design of the testing procedure for digital systems such as (for instance) AI models or computer vision software. The digital environment and the evaluation metrics can be designed—if required—via services S00176 and S00178. In the context of testing customers’ solutions within digital environments, this service is targeted at designing a suitable protocol for digital testing based on the use cases specified by the customer. The components of the testing protocol can include: - Selecting the datasets to be used for testing - Selecting reference AI models to be used for testing (if needed) - Choosing data formats and metadata standards. - Defining data pre-processing and preparation steps - Defining values and ranges of test parameters- Defining the different phases of the protocol- Outlining the operations to be executed in each phase- Thoroughly describing the protocol specifics to ensure reproducibility Datasets to be used for testing can be provided by AgrifoodTEF and/or by the customer; if nothing suitable is available, other AgrifoodTEF services can be leveraged to collect and/or generate tailored data. The technical team executing this service comprises expert engineers but can also involve agronomists when this is necessary to ensure the relevance of the tests for the use case, e.g., to determine the distribution of test repetitions across the variation ranges.

Desk assessment activities for digital systems or data

Politecnico di Milano

Location

Italy

Remote

Arable farming

Food processing

Greenhouse

Horticulture

Livestock farming

Tree Crops

Viticulture

Through this service, we offer customers a complete feasibility study of their digital systems and/or data, evaluating their suitability for the test use cases identified by the customer. The study involves both technical and agronomic aspects. The goal of this service is to provide the customer with a comparison between the features of the system or data submitted for review and the requirements of the use cases chosen for (possible) subsequent testing. Examples of systems and data requiring digital testing include (but are not limited to) software modules for data processing (e.g., machine vision modules), AI models, software architectures (e.g., control systems of robots), simulations, datasets collected in the field, synthetic datasets, designs of software systems, and so on. This assessment is crucial to provide customers with key information on elements of their system that need to be further developed to become “fit for testing.” It can also identify changes to apply to either the system or AgrifoodTEF’s testing facilities to meet the requirements for experimentation.

Evaluation of the energy efficiency of the embedded hardware associated with the AI functionality

Laboratoire National de Métrologie et d’Essais - LNE

Location

France

Arable farming

Food processing

Greenhouse

Horticulture

Livestock farming

Tree Crops

Viticulture

By having your AI or robotic system tested by LNE, you ensure it meets the highest standards of safety and performance, boosting your product’s reliability and trustworthiness in the market. Partnering with LNE for rigorous testing of your robotic and AI technologies gives you a competitive edge and trusted third-party assessment that enhances your product credibility and opens doors to new market opportunities both locally and globally. This service provides testing of devices with AI-integrated control systems, focusing on energy efficiency and consumption during the product life cycle. By simulating a wide range of scenarios, we provide detailed insights into the energy performance of these AI systems with regard to their expected operational environment. These tests can be conducted under different conditions, either in a completely simulated way, through a hybrid test bench where the actual physical device is assessed within a simulated environment, or using physical infrastructures based on the device's operational environment to test it in a 'real' setting. Tests are conducted under controlled laboratory conditions, ensuring optimal reproducibility and repeatability, providing reliable insights into the system's efficiency and safety-critical functionalities. Typical assessments may cover the energy efficiency of SLAM, various types of end effectors, and other relevant components such as sensors, motors or AI software. This rigorous testing process helps identify potential areas for improvement, enabling companies to enhance the efficiency and sustainability of their products, ultimately leading to cost savings and reduced environmental impact.

Execution of digital testing

Politecnico di Milano

Location

Italy

Remote

Arable farming

Food processing

Greenhouse

Horticulture

Livestock farming

Tree Crops

Viticulture

This service assists customers in executing digital tests using agrifoodTEF’s digital infrastructure. Tests can either be defined by the customer or designed through agrifoodTEF services (e.g., S00176, S00177, S00178). If specific activities are required to properly interface the system under test with the digital infrastructure, these can be handled by either the customer or agrifoodTEF (via Service S00180).

Support provided by this service includes general assistance during the test execution phase, such as monitoring to ensure compliance with the testing protocol or offering immediate help in identifying and resolving issues. Additionally, the service provides technical support during the test execution of the system under test, including ensuring that all necessary processes are correctly configured and running, as well as helping the customer address any unforeseen challenges that may arise when complex digital systems interact.

Finally, through this service, agrifoodTEF manages the technical infrastructure for test execution, ensuring smooth data exchange between the system under test and the agrifoodTEF infrastructure, and verifying that all testing requirements—potentially identified during the desk assessment phase (Service S00179)—are met during the actual test runs.