PhD Scholarship: Visual Action Models for In-Cabin and Robotic Applications
How do you turn video, audio and context into a model that not only recognizes human action but translates it into adaptive behavior? That is the focus of this PhD position at the Industrial Deep Learning (IDL) group at TMDT – at the intersection of computer vision, machine learning and human-machine interaction. In close collaboration with Aptiv's Interior Sensing Machine Learning team, you will work on multimodal visual action models for two closely related application domains: automotive interiors and robotics.
Funding: €2,000/month · Duration: 3 years · Start: at the earliest possible date · Application deadline: 15 August 2026
What it's about
Visual perception is the foundation for machines to understand human behavior and respond to it meaningfully. In this project, you will develop multimodal models that fuse video, audio and contextual information into actionable understanding – models that do not stop at classification but close the loop from observation to action.
The vehicle interior is an ideal research environment for this: a structured, data-rich setting in which such models can be trained and evaluated under realistic sensory conditions. The representations and insights gained here transfer naturally to robotics, where the same perception and reasoning capabilities are needed to interpret human behavior and translate it into physical action. In this way, the project contributes to current research topics such as robotic foundation models and vision-to-policy paradigms.
Research Focus
The project connects two application domains that build on shared multimodal representations.
In-Cabin: perception inside the vehicle
The goal is to improve safety, comfort and interaction in the vehicle through deeper visual action understanding. Topics include:
- Human-machine and human-human interaction: Detecting and interpreting gestures and behavior to enable intuitive interfaces and social context awareness.
- Safety-critical situations: Automatic detection of critical events to trigger appropriate response protocols.
- Social interaction & context: Understanding social situations inside the vehicle to enrich contextual awareness.
- Novel gesture recognition: Models that learn new, user-defined gestures for personalized interaction.
Robotics: from seeing to acting
Building on the representations developed in the in-cabin context, you will extend visual action understanding toward robotic perception and control. The goal is for robots to interpret and replicate complex human actions – based on the same multimodal embeddings:
- Visual-to-action transfer: Translating observed human action into executable robotic motion primitives.
- Imitation learning without motion capture: Learning from human demonstrations directly from video and context, rather than from traditional MoCap systems.
- Joint state & motion estimation: Estimating joint configurations and motion trajectories from visual input for adaptive manipulation.
Key Research Questions
Beyond the applications, the PhD addresses fundamental methodological challenges relevant for publication at leading conferences:
- Simulation & sim-to-real transfer: How can simulated environments be made realistic enough for trained models to transfer reliably to the real world?
- Action policy learning: How do models derive optimal action strategies from visual cues?
Why pursue your PhD at TMDT
At the TMDT, we combine rigorous methodological research with genuine industrial transfer. For you, this means working on a real, industrially relevant problem – with the ambition to publish results at leading AI conferences. Scientific depth and application relevance are not a contradiction here.
How closely in-cabin perception and cutting-edge research are connected is shown by recent work from our team on robust synthetic-to-real segmentation in automotive NIR imagery – accepted at ECML PKDD 2026. This exact methodology – models that transfer reliably from synthetic training data to real sensor data – is at the heart of this PhD.
Synthetic-to-real segmentation in NIR interior and exterior scenes. Source: Stillger et al., "Texture-Shape Bias Balancing for Robust Synthetic-to-Real Semantic Segmentation in Automotive NIR Imagery," accepted for ECML PKDD 2026.
What to expect:
- Research with a publication ambition: We treat the PhD as scientific work aimed at high-quality publications, not as pure contract development.
- Access to real-world data and deployment environments through the collaboration with Aptiv – an advantage purely academic projects rarely offer.
- Modern research infrastructure for training and evaluating large multimodal models.
- Structured supervision within a team that understands applied AI research and industrial transfer as a shared mission.
Collaboration with Aptiv
You will work closely with Aptiv's Interior Sensing Machine Learning team, contributing to real-world applications of visual action models in next-generation automotive systems. This setup connects academic research with industrial innovation, with access to real-world data, sensing expertise and deployment environments.
Your Profile
We are looking for a highly motivated candidate with a completed master's degree in computer science, electrical engineering, data science, computational engineering, robotics or related disciplines, and a strong background in one or more of the following areas:
- Computer vision
- Machine learning / deep learning
- Robotics
- Simulation and reinforcement learning
Solid skills in Python and PyTorch or TensorFlow are required. Experience with 3D simulation environments (e.g. Blender, Unity, Gazebo) is a plus. Strong analytical skills, creativity and a passion for applied research complete your profile.
Terms and Application
The scholarship amounts to €2,000/month, is limited to three years and serves to support a doctoral degree. The legal basis is the Framework Regulations for the Awarding of Scholarships for the Preparation of Doctoral Degrees at the University of Wuppertal. The scholarship does not establish an employment or service relationship and is not subject to social security contributions.
Documents to be submitted:
- Cover letter including the possible starting date
- Curriculum vitae, if applicable including employment references and certificates of further training
- Copy of proof of the university degree (certified or upon presentation of the original)
- Proof of any professional employment
- A certificate of enrollment must be submitted subsequently if the scholarship is awarded.
The selection committee, consisting of Dr. Timo Rehfeld (Aptiv) and Prof. Dr.-Ing. Tobias Meisen (University of Wuppertal), will decide on the award. Letters of recommendation do not need to be submitted.
Important – how to apply: Applications must be submitted by post only. Applications submitted electronically cannot be considered. Please address your documents to:
Bergische Universität Wuppertal Fakultät für Elektrotechnik, Informationstechnik und Medientechnik Lehrstuhl für Technologien und Management der Digitalen Transformation Herrn Univ.-Prof. Dr. Tobias Meisen 42097 Wuppertal
Application deadline: 15 August 2026
