“When robots learn from people, technology stops being complex and becomes really usable”
Robots that learn from people and understand instructions in noisy environments
Industry is at a turning point. It needs processes that are more flexible, adaptable and easier to reconfigure. The IKASprod initiative marks a turning point by introducing a new generation of intelligent systems that are capable of learning directly from people, thereby reducing programming times and improving plant efficiency.
Robots that are programmed by demonstrating, without the need for code
IKASprod is researching how workers can teach tasks to a robot in the same way they teach another person, by using spoken instructions, natural gestures and demonstrating what to do in an immersive virtual environment.
- It will no longer be necessary to shut down machines or spend hours on traditional programming.
- The system interprets what the user wants to do, automatically generates the control code required in the form of behaviour trees (BTs) and executes it in a safe manner.
- This is achieved using optimised technologies that can be run locally at an industrial plant, without the need for cloud infrastructure, and operate in noisy industrial environments.
Natural interaction even in extreme noise environments
One of the key innovations of this initiative is the improvement of human-machine interaction in real-world factory settings. IKASprod has developed a multimodal speech recognition system that combines:
- Voice, using automatic recognition that is resilient to industrial noise.
- Lip reading, with training to master the project.
- Intelligent fusion of both signals using a language model that selects the best interpretation in each situation.
This technology understands spoken instructions, even in very noisy environments, thereby improving safety and reducing errors.
Programming by demonstrating in XR environments
Extended reality (XR) technology plays a leading role. IKASprod integrates an immersive, demonstration-based programming environment, where workers can:
- Demonstrate movements or procedures by moving the robot in a digital twin.
- Receive assistance from a conversational assistant, who guides the session.
- Automatically generate the corresponding behaviour tree to run on the physical robot.
This approach validates processes before implementing them, thereby reducing start-up times and minimising errors during integration.
Three real-life cases that look to the future
The project’s results have been tested in representative real-world scenarios at the IMH Campus:
- Robot programming using natural language: this system is capable of converting spoken commands into behaviour trees (BTs) and executable code, while maintaining traceability and human supervision.
- Demonstration-based programming in an XR environment: this is a virtual laboratory where operators teach the system how to act, generating the control programme based on their movements.
- Multimodal speech recognition (voice + lip reading): a demonstrator designed for complex industrial environments, where noise makes other voice-based interaction systems unworkable.
A strategic project for Basque competitiveness
IKASprod has a direct impact on productivity and the quality of employment:
- It significantly reduces the time taken to configure and program robots and AGVs.
- It improves safety and ergonomics by eliminating repetitive or hazardous tasks.
- It increases flexibility on production lines with frequent changeovers.
- It empowers workers, who stop being programmers and become trainers of the system.
This vision puts the Basque Country at the forefront of the Industry 5.0, where technology adapts to people, rather than the other way round. Its application is particularly important in sectors such as machine tools, industrial robotics, logistics, flexible manufacturing and short-run production.
A consortium ready to lead the way
IKASprod is an ELKARTEK project run by the Basque Government, developed by TECNALIA, Tekniker, Orai and IMH Campus, who have pooled their expertise in AI, robotics, advanced interaction, natural language processing and flexible manufacturing to drive new solutions in smart manufacturing environments.
The consortium has spent two years working on developing these advanced technologies and speeding up their transfer to industry, particularly to SMEs, thereby ensuring their economic and social impact. To this end, it was supported by a Technical Review Committee, made up of organisations from the industrial, educational and technological sectors, which have contributed their expertise and insight to ensure the project addresses real-world needs.
Towards an intuitive, people-centred, flexible industry
IKASprod is not only a technological breakthrough, but also a new paradigm: teaching a robot will be as simple as explaining how to do it, and people will regain a central, creative role within the production process.
Industry that learns faster.
Industry that understands people.
A more humane industry.