Catalogue of Services

Design of AI algorithm test scenario

GRADIANT

Location

Remote

Arable farming

Food processing

Greenhouse

Horticulture

Livestock farming

Tree Crops

Viticulture

This service helps organisations evaluate and validate their AI solutions and datasets in a systematic way. We work with you to design comprehensive test scenarios that match your specific needs and objectives. Whether you want to test your existing AI model's performance, validate a dataset's quality, or compare different machine learning approaches, we follow standard ML development practices to create a structured testing process. This includes designing the testing environment, defining test protocols, establishing evaluation metrics, and setting key parameters. Using TEF's infrastructure, we implement these elements to create test conditions that reflect real-world usage. We can work with both your own AI solutions and datasets or help you select appropriate ones from TEF's resources. This systematic approach ensures you get clear, actionable insights about your AI solution's performance, reliability, and potential areas for improvement. The testing scenarios are designed to be transparent, repeatable, and aligned with industry best practices.

Design of computational environments for digital testing

Politecnico di Milano

Location

Italy

Remote

Arable farming

Food processing

Greenhouse

Horticulture

Livestock farming

Tree Crops

Viticulture

Test activities involve three key components: the environment (where the tests are conducted), the protocol (defining what tests are performed and how), and evaluation metrics (used to assess the test results). Through this service, we assist customers in designing a computational environment tailored to running the digital tests required for their software components. Specifically, this service:

• Defines the data (and metadata) requirements needed to support the test
• Identifies the necessary software environment (e.g., operating system, libraries)
• Selects an appropriate simulator and defines simulated environments, if required
• Chooses remote access tools, if applicable.

As a result, we deliver a comprehensive design of the testing environment, including specifications such as memory requirements, GPU/TPU acceleration, and necessary libraries. This environment can be deployed on either the agrifoodTEF digital infrastructure or the customer's computing infrastructure (i.e., on premises).

Design of evaluation metrics for digital testing

Politecnico di Milano

Location

Italy

Remote

Arable farming

Food processing

Greenhouse

Horticulture

Livestock farming

Tree Crops

Viticulture

Any test activity involves three main components, i.e., environment (where the tests take place), protocol (defining what activities are executed and how), and evaluation metrics (used to assess the results of the tests). This service concerns the last element; its goal is to design the best metrics to evaluate the performance of digital systems such as (for instance) AI models or computer vision software. The digital environment and the testing protocol metrics can be designed—if required—via services S00176 and S00177. Our team will identify and define with customers the most adequate set of quantitative metrics to assess the outcome of the digital testing activities. In order to ensure the relevance of the metrics with respect to the real-world use cases, the team will involve engineers and agronomists. The metrics will be adapted not only to the task that the digital system under test (e.g., a piece of software) is designed to perform, but also to the features of the data used for the tests. For instance, a customer that has developed a machine incorporating an AI model will be interested in testing the model on data generated by their own machine: the performance metrics will therefore need to be adapted to the specific features of that data.

Design of evaluation metrics for physical testing

Politecnico di Milano

Location

Italy

Remote

Arable farming

Food processing

Greenhouse

Horticulture

Livestock farming

Tree Crops

Viticulture

Any physical test activity involves three main components: environment (where the tests take place), protocol (defining what tests are executed and how) and evaluation metrics (used to assess the results of the tests). This service concerns the last element; its goal is to design the best metrics to evaluate the performance of a customer solution taking into consideration the use cases specified by the customer and the environment and protocol chosen for the tests (which, if needed, can be designed via services S00106 and S00107). Our team will identify and define with customers the most adequate set of quantitative (i.e., based on instrumental measurements) and/or qualitative (i.e., relying on expert human judgement) metrics to assess the system functionalities of interest. This phase will involve, in particular, agronomists and experts in agricultural machinery. Based on the defined evaluation metrics, a set of requirements for the collection of required data and ground truth annotations will also be defined accordingly. For instance, the service may lay out the specifications for dedicated data collection campaigns (possibly executed via service S00113). This phase will involve engineers and experts in AI and robotics. On request, the output of the service will include analyses on additional environmental factors than those directly tracked through the designed metrics (e.g., seasonal effects, impact of test distribution over time on results).

Design of test environments for physical testing

Politecnico di Milano

Location

Italy

Remote

Arable farming

Greenhouse

Horticulture

Tree Crops

Viticulture

Any physical test activity involves three main components: environment (where the tests take place), protocol (defining what tests are executed and how) and evaluation metrics (used to assess the results of the tests). This service concerns the first element, i.e., the design of a physical testing environment for use cases such as (for instance) weeding, plant phenotyping, and precision spraying solutions. The protocol and the evaluation metrics can – if required – be designed via services S00107 and S00108. Depending on your requirements and reference system/solution to be tested, our team will design an ad hoc setup equipped with all the required features for testing. Environmental features include, for example, the crop and weed species to be prepared and their growth stage. - the plant layout and intra-row configuration, - seasonal weather and climate-related conditions (e.g., lighting conditions, wind, rain), - the type of soil, moisture level, and terrain conditions (e.g., uneven terrain, presence of any slopes, and so forth), - the technical infrastructure supporting the tests (e.g., electrical layout, network infrastructure, environmental sensors, data acquisition systems…). In order to consider all aspects of the environment, this service involves a team comprising both engineers and agronomists.

Design of testing protocols for digital testing

Politecnico di Milano

Location

Italy

Remote

Arable farming

Food processing

Greenhouse

Horticulture

Livestock farming

Tree Crops

Viticulture

Any test activity involves three main components, i.e., environment (where the tests take place), protocol (defining what activities are executed and how), and evaluation metrics (used to assess the results of the tests). This service concerns the second element, i.e., the design of the testing procedure for digital systems such as (for instance) AI models or computer vision software. The digital environment and the evaluation metrics can be designed—if required—via services S00176 and S00178. In the context of testing customers’ solutions within digital environments, this service is targeted at designing a suitable protocol for digital testing based on the use cases specified by the customer. The components of the testing protocol can include: - Selecting the datasets to be used for testing - Selecting reference AI models to be used for testing (if needed) - Choosing data formats and metadata standards. - Defining data pre-processing and preparation steps - Defining values and ranges of test parameters- Defining the different phases of the protocol- Outlining the operations to be executed in each phase- Thoroughly describing the protocol specifics to ensure reproducibility Datasets to be used for testing can be provided by AgrifoodTEF and/or by the customer; if nothing suitable is available, other AgrifoodTEF services can be leveraged to collect and/or generate tailored data. The technical team executing this service comprises expert engineers but can also involve agronomists when this is necessary to ensure the relevance of the tests for the use case, e.g., to determine the distribution of test repetitions across the variation ranges.

Design of testing protocols for physical testing

Politecnico di Milano

Location

Italy

Remote

Arable farming

Food processing

Greenhouse

Horticulture

Livestock farming

Tree Crops

Viticulture

Any physical testing activity comprises three main components:

• the environment (where the tests are conducted),
• the protocol (which defines the tests to be executed and their methodology),
• and the evaluation metrics (used to assess the test results).

This service focuses on the protocol, while the environment and metrics can be designed as needed through services S00106 and S00108.

In the context of testing customer solutions within physical facilities, this service aims to create a suitable testing protocol based on the use cases specified by the customer. The components of the testing protocol defined in this phase may include:

• Defining the different phases of the protocol and their duration
• Outlining the operations to be executed in each phase
• Quantifying the effort required for each phase, including the number and qualifications of personnel involved
• Distributing testing operations over time, considering seasonal and daily variations, as well as weather conditions
• Establishing acceptable and desired variation ranges for each configurable element in the testing environment
• Determining the number of test repetitions required and their distribution across the variation ranges

To ensure a comprehensive approach to protocol design, this service involves a collaborative team of both engineers and agronomists.

Desk assessment activities for digital systems or data

Politecnico di Milano

Location

Italy

Remote

Arable farming

Food processing

Greenhouse

Horticulture

Livestock farming

Tree Crops

Viticulture

Through this service, we offer customers a complete feasibility study of their digital systems and/or data, evaluating their suitability for the test use cases identified by the customer. The study involves both technical and agronomic aspects. The goal of this service is to provide the customer with a comparison between the features of the system or data submitted for review and the requirements of the use cases chosen for (possible) subsequent testing. Examples of systems and data requiring digital testing include (but are not limited to) software modules for data processing (e.g., machine vision modules), AI models, software architectures (e.g., control systems of robots), simulations, datasets collected in the field, synthetic datasets, designs of software systems, and so on. This assessment is crucial to provide customers with key information on elements of their system that need to be further developed to become “fit for testing.” It can also identify changes to apply to either the system or AgrifoodTEF’s testing facilities to meet the requirements for experimentation.

Desk assessment activities for physical systems

Politecnico di Milano

Location

Italy

Remote

Arable farming

Greenhouse

Horticulture

Tree Crops

Viticulture

Through this assessment, we offer customers a complete feasibility study of the use cases and systems that customers would like to test, comprising both technical and agronomic aspects. Examples of systems requiring physical testing include (but are not limited to) precision sprayers, weeding machines, robotised fruit picking systems, and other implements; examples of subsystems include (but are not limited to) perception systems (e.g., based on vision), autonomous driving modules, localisation systems, and mapping and navigation subsystems. This assessment is crucial to provide customers with key information on elements of the system that need to be further developed to become “fit for testing”. It can also identify changes to apply to either the system or AgrifoodTEF’s testing facilities to meet the test requirements.

Desk assessment of the proposed solution

Fondazione Bruno Kessler

Location

Remote

Arable farming

Greenhouse

Horticulture

Livestock farming

Tree Crops

Viticulture

This service consists of a detailed analysis of the customer's system, software, or AI solution. It is a preliminary step for testing and validation services, providing important information on the solution and leading the following activities. As a consequence, the detailed mapping of the solution can help the customer understand bottlenecks and better prepare the solution for testing and validation activities.

Evaluation for Ethical, Legal, and Social Aspects (ELSA Scan) for responsible AI development

Wageningen Research - WR

Location

At user's premises

Netherlands

Remote

Arable farming

Food processing

Greenhouse

Horticulture

Livestock farming

Tree Crops

Viticulture

What are the ethical, legal, and social aspects (ELSA) of AI-driven technology in agri-food? Mitigation of potential risks and finding opportunities are easier by identifying ELSA aspects in early development stages, when the technology is still in the making. Examples of ELSA aspects are autonomy, transparency, and bias, but also data privacy and other legal aspects. How does your AI technology deal with ELSA aspects? A combined survey and interview help AI developers to identify opportunities and issues with ELSA in the agri-food context from the perspective of end users and more broadly, for society. For example, to better align with the sustainability objectives of the AI-driven technology. The outcomes of the ELSA scan are a list of identified key ELSA aspects and high-level recommendations to further improve the AI technology. 

Evaluation of AI model for agrifood applications

GRADIANT

Location

Remote

Arable farming

Food processing

Greenhouse

Horticulture

Livestock farming

Tree Crops

Viticulture

This service helps businesses and organisations evaluate how well their AI models perform in real-world conditions. We conduct thorough testing of your AI model using either your own dataset or our reference datasets, depending on your needs (see the related services). Our evaluation process goes beyond basic accuracy metrics to provide a comprehensive understanding of your model's strengths and limitations. We analyse various aspects of performance that matter for your specific use case, whether it's detecting diseases in crops, sorting produce, or monitoring livestock behaviour. We work with you to determine the most relevant performance metrics for your application, taking into consideration factors like accuracy, speed, reliability, and resource usage. This helps you understand if your AI solution is ready for deployment, needs improvement, or requires adjustments for specific conditions. The evaluation provides clear, actionable insights about your model's performance, helping you make informed decisions about its readiness for real-world agricultural applications.