← Back to projects
XR Startup and Applied Research
Building embodied XR systems from movement to emotion
Applied XR development spanning startup product engineering at HoloSuit and experimental social-cognition research at ENS Paris.
Technical Layer
Tools & Methods
- XR product development
- Full-body motion capture
- Wearable sensor data
- Real-time data pipelines
- Quaternion kinematics
- Personalised calibration
- Gesture recognition
- Dynamic Time Warping
- Deep learning
- 3D face reconstruction
- Unity
- Blender
My role across startup and research R&D
At HoloSuit, I worked in an early-stage XR startup environment on the software foundations of a real-time automated training platform. I began by modelling yoga-posture classification from Microsoft Kinect quaternion data, then moved into the wearable HoloSuit ecosystem. I built modules for full-body movement capture and replay, gesture-based skeletal calibration, finger-gesture control of remote data collection, and temporal comparison of trainer and learner movements using Dynamic Time Warping. The work required translating exploratory algorithms into components that could support a demonstrable user-facing product. At ENS Paris, I extended this embodied-computing perspective towards social cognition by reconstructing, processing, and validating three-dimensional emotional faces for immersive experiments.
One question beneath two projects
HoloSuit Yoga and Social Cognition VR initially appear to address different problems. One concerns learning physical movements, while the other concerns perceiving and responding to emotional expressions. But both depend on the same computational challenge: converting embodied human behaviour into structured, interpretable, and interactive digital representations.
Building inside an XR startup
HoloSuit was not a conventional academic software project. The work took place within an early-stage technology company developing a full-body motion-capture and haptic platform for training and immersive interaction. This required working with evolving wearable hardware, incomplete product requirements, experimental data, and demonstration-driven development. My role connected algorithmic exploration with the practical modules needed to capture users, personalise the system, inspect recordings, and compare performances.
HoloSuit Yoga: the body as a real-time interface
The HoloSuit Yoga project explored a real-time automated yoga-training platform in augmented and virtual reality. Instead of treating a posture as a static photograph, the system treated yoga as full-body movement unfolding through time. This required capturing joint orientations, preserving motion sequences, calibrating the system to an individual body, and comparing learner and trainer performances.
Learning posture from quaternion data
I first simulated yoga-pose classification using deep neural networks applied to quaternion joint angles obtained through Microsoft Kinect. Quaternion representations provided a structured way to describe three-dimensional joint orientations without reducing the posture to a flat two-dimensional image.
Building the motion-capture foundation
I subsequently developed a file-saving module for capturing, storing, and replaying full-body HoloSuit movement data. This provided a foundation for recording performances, inspecting temporal movement sequences, and comparing repeated training sessions.
Personalising the digital body
Two people do not share identical skeletal proportions, so a useful training system cannot assume one universal digital body. I developed a gesture-based calibration module for personalising bone lengths and added finger-gesture recognition for remotely controlling the data-capture process.
Comparing movement through time
I used Dynamic Time Warping to compare temporal sequences from learner and trainer yoga postures captured through the HoloSuit's three-dimensional sensors. This allowed similar movements to be aligned even when they were performed at different speeds, creating a foundation for personalised posture comparison rather than rigid frame-by-frame matching.
From raw movement to personalised feedback
The platform required more than recognising a yoga pose. It needed a complete motion-intelligence pipeline: capture full-body joint orientations, store and replay sessions, calibrate the virtual skeleton to the user, recognise control gestures, align trainer and learner movement sequences, and convert their differences into actionable feedback. I contributed modules across several stages of this path from wearable sensor data to an interactive training application.
A platform problem, not only a yoga problem
Yoga provided a demanding demonstration because correct performance depends on whole-body geometry, sequencing, balance, and individual anatomical variation. But the underlying architecture was broader: record expert movement, model an individual learner, align their performances, and provide personalised feedback. The same pattern is relevant to physical rehabilitation, sports coaching, ergonomics, occupational training, remote instruction, and simulation-based learning.
Foundational skills for embodied and physical AI
The HoloSuit project predates the current expansion of embodied and physical AI, but it addressed several of the field's foundational engineering problems: representing a human body computationally, calibrating a system to physical variation, modelling actions through time, connecting sensing to real-time interaction, and translating behaviour between a person and a digital agent. These capabilities now sit close to work in digital twins, adaptive training, teleoperation, rehabilitation technology, robotics, and multimodal human-machine systems.
Why this experience is industrially relevant
HoloSuit gave me experience beyond a conventional academic workflow. I worked with an emerging hardware platform, evolving specifications, real-time human data, personalisation requirements, and the need to turn technical experiments into demonstrable product components. Social Cognition VR added rigorous stimulus construction, validation, and behavioural experimentation. Together, they show that I can move between product development and scientific evaluation while keeping the human body at the centre of the system.
Social Cognition VR: emotion as a 3D signal
The Social Cognition VR project approached embodiment from another direction. To support the transition from conventional two-dimensional emotional-face stimuli to immersive experiments, I reconstructed emotional expressions from the Radboud Faces Database as three-dimensional facial models using DECA, a deep-learning model for detailed face reconstruction and animation.
Creating a 3D emotion database
The reconstructed faces were processed, aligned, and corrected using MeshLab and Blender to create a novel set of three-dimensional emotional stimuli. The work focused particularly on anger and fear, emotions closely connected to social threat, interpersonal distance, and approach-avoidance behaviour.
Validation before immersion
A three-dimensional stimulus is not scientifically useful merely because it appears realistic. I therefore created an online PsyToolkit experiment to evaluate whether observers perceived the intended anger and fear expressions in the reconstructed faces. The database was under validation phase by the time I completed my internship period.
From a face to an embodied social agent
I developed a Blender prototype combining an emotional three-dimensional face with a corresponding body pose. This extended the stimulus from an isolated facial expression towards a complete social agent whose emotional state could be communicated through both face and body.
Virtual approach and avoidance
The broader research direction involved collaborating on a Unity-based virtual-reality setup for studying how people approach or avoid emotional social agents. This creates a more embodied experimental context than conventional screen-based tasks because participants can respond through movement and interpersonal distance.
The computational social layer
During my work with the Social Cognition Group at the Laboratory of Cognitive and Computational Neuroscience, ENS Paris, I also explored reinforcement-learning models of behavioural data from socio-emotional experiments. This added a computational account of how actions and social outcomes may shape future decisions.
A bridge between product and research
The HoloSuit Yoga and Social Cognition VR projects were distinct in their goals, but they shared a common focus on embodied human behaviour. Both required capturing, modelling, and interpreting signals from the body, whether for training or social interaction. The work gave me experience across both startup product development and academic research, with a consistent emphasis on the human body as an interface.
Primary outputs
Platforms and technical foundations
HoloSuit Yoga Capture & ReplayFull-body āsana capture and replay for the Holosuit automated yoga-training platformView ↗︎Social Cognition - 3D Emotion AvatarsBlender combining 3D Emotional Faces with full-body Avatars for Social Cognition Experiments in Virtual RealityView ↗︎HoloSuit XR PlatformFull-body motion capture and haptic-feedback platform used for the automated yoga-training workExplore ↗︎
Featured in
Coverage
HoloSuit in GamesBeatIndependent coverage of the startup's full-body tracking, wearable sensors, haptic feedback, and HoloSuit Yoga demonstrationRead ↗︎HoloSuit in ForbesCoverage of HoloSuit within emerging virtual reality, augmented reality, and immersive technologyRead ↗︎HoloSuit on CatapoooltCrowdfunding presentation of the wearable full-body motion-capture platformView ↗︎ENS Social Cognition Team where I internedResearch context for emotion, adaptive social decisions, social behaviour, and their cognitive and neural mechanismsExplore ↗︎How Emotion Motivates Action - Bigger Project of which my internship was a partENS research context for emotional displays, value-based decisions, and approach-avoidance behaviourRead ↗︎
Continue Exploring
Explore the wider body of work.
Further projects span physiological computing, computational neuroscience, embodied systems, and interdisciplinary engineering.