Post-doctoral research visit f - m ai-powered control of exoskeletons with multimodal models h/f

A propos d'Inria

Inria est l'institut national de recherche dédié aux sciences et technologies du numérique. Il emploie 2600 personnes. Ses 215 équipes-projets agiles, en général communes avec des partenaires académiques, impliquent plus de 3900 scientifiques pour relever les défis du numérique, souvent à l'interface d'autres disciplines. L'institut fait appel à de nombreux talents dans plus d'une quarantaine de métiers différents. 900 personnels d'appui à la recherche et à l'innovation contribuent à faire émerger et grandir des projets scientifiques ou entrepreneuriaux qui impactent le monde. Inria travaille avec de nombreuses entreprises et a accompagné la création de plus de 200 start-up. L'institut s'eorce ainsi de répondre aux enjeux de la transformation numérique de la science, de la société et de l'économie. Post-Doctoral Research Visit F/M AI-powered control of exoskeletons with multimodal models
Le descriptif de l'offre ci-dessous est en Anglais
Type de contrat : CDD

Niveau de diplôme exigé : Thèse ou équivalent

Fonction : Post-Doctorant

Contexte et atouts du poste

Context and funding :

The position is funded by the EUROBIN project, an European Network of Excellence project that regroups several partners at the European Level.

In this project, the HUCEBOT team is involved in thepersonal robotics challenge to design AI-powered controllers forrobots for improved assistance. Foundation models and AI-based models that leverage human predictions, contextual visual informationand natural language instructions are at the core of the current team approaches. Previous work has been applied to humanoids and mobile robots, whereas in this position we want to explore the application of active exoskeletons.

About the team :

The candidate will join the Human Centered Robotics team (HUCEBOT) in the Inria Center of the University of Lorraine in Nancy, France.

The team HUCEBOT develops control, learning, and interaction skills of human-centered robots, such as humanoid, mobile manipulators and exoskeletons. The team develops learning and control algorithms for teleoperated / supervised / autonomous robots, involved in complex manipulation tasks in man-made environments. IT also develops prediction and control techniques for wearable exoskeletons designed to assist humans at work. The team has excellent robotics facilities, including several humanoid robots (Talos, iCub, G1), manipulators, drones, passive and active exoskeletons, wearable sensors, force plates etc. Its laboratory has a 3D printing facility and a mechatronic workshop for prototyping and maintenance.

The team consists of many research scientists, postdocs, PhD and has the support of 1 software and 1 mechatronics engineer. The team is international - English and French speaking. French is not required, although free French classes are available in the institute for non-French speakers.

About the laboratory and Nancy :

The Inria Center of the University of Lorraine, is co-located with the Loria laboratory, in the Science and Technology Campus of the University of Lorraine (Nancy, France), next to the Botanical Gardens, at 20 minutes by public transportation or bike from the Nancy train station and City Center. Several student residences and facilities are at walking distance. Nancy is a University town, with a high quality of life and a vibrant student and expat community.

Mission confiée

Theposition is about equippingan exoskeleton with microphones and cameras to enable vocal commands. Thepostdoc will work on an active exoskeleton prototype. He/she will select adequate hardware (sensors and electronics) to enable audio processing and audio-to-text processing embedded on the exoskeleton, as well as visual processing to retrieve contextual information, using AI systems. He/she will conduct experiments with the sensorized exoskeleton toprocess audio signal, detect commands in natural language, and retrieve contextual information.

The sensors will BE mounted on one of the team exoskeletons and tests will BE done in laboratory conditions. On the software side, the sensors will have to BE visible on a ROS2node. The candidate will collaborate with the team to re-use existing audio-to-text models (e.g., Whisper) and visual language models (VLMs).

Thepostdoc will BE in charge of sensor integration and conducting validating experiments.

The postdoc will develop and explore multimodal models that combine language and images with robot actions, targeted at assisting the human during their gestures.

A presentation of the ongoing work at the EUROBIN meetings is required.

Principales activités
- Review state of the art in audio and image sensing devices that can BE embedded on wearables
- Selecting, testing and mounting the sensors in one of the exoskeletons of the team
- Write software to read from the sensor
- Integrate sensor readings with audio-to-text and VLM
- Write hardware/software report
- Develop adequate multimodal AI models (VLM, LLM, VLA) that leverage the sensors and assist the exoskeleton controller, taking care of prediction time, computation resources, etc.
- Collaborate with the team to integrate the sensors in the current experiments and developments

Compétences
- Technical skills :
- Very good programming skills.
- Excellent skills with electronics / mechatronics, sensors.
- Love working with real robots.
- Experience with exoskeletons.
- Interest for foundation models for speech and image processing
- Soft skills :
- Excellent communication skills at work, and ability to report progress
- Proactivity.
- Not afraid of challenging projects.
- Rigour and intellectual honesty
- Curiosity and desire to learn
- Practical mindset and ability to develop robust and reliable solutions
- Autonomy and organizational skills
- Love working in a multi-cultural environment
- Team player

Avantages
- Subsidized meals
- Partial reimbursement of public transport costs
- Leave : 7 weeks of annual leave + 10 extra days off due to RTT (statutory reduction in working hours) + possibility of exceptional leave (sick children, moving home, etc.)
- Possibility of teleworking (after 6 months of employment) and flexible organization of working hours
- Professional equipment available (videoconferencing, loan of computer equipment, etc.)
- Social, cultural and sports events and activities
- Access to vocational training
- Social security coverage

Rémunération

2788 € gross/month