Pathic MR
A mixed reality training product that lets people rehearse emotionally difficult conversations with an AI counterpart, then switch sides and live the conversation from the other person's position.
Built for managers, clinicians, and HR teams, it combines role reversal, AI-guided dialogue, and structured reflection.
The product isn't an interface. It's a repeatable way to change how people behave in conflict.
Project Snapshot
- • Problem: Conflict training is often abstract, emotionally flat, and easy to ignore
- • Solution: Mixed reality role-swapped simulations with AI-driven dialogue and guided reflection
- • Team: Sejal Shail (UX Research) + Blessing Mangwiro (Product Design / XR Development)
- • Outcomes: 14+ interviews, 2 conflict narratives, 10+ interaction flows
- • Learning Lens: Emotional awareness, reflection, transfer, and safe practice
"Almost all stakeholders identified the critical importance of stepping into the other's shoes."
- Research Finding
This was a collaborative project. I worked with Sejal Shail on the research foundation and learning direction; her work on workplace conflict, emotional awareness, and reflective practice is part of the story here.
- Project Type Mixed reality training product, two-person team
- Timeline Mar 2025 – May 2025 • 10 weeks
- Team 1 UX Researcher: Sejal Shail • 1 Product Designer: Blessing Mangwiro
- My Role Research planning, interviewing, storyboarding, user flow design, spatial design, mixed reality interaction mapping, Unity engineering (C#), and design implementation in Unity
- Tech Stack Unity, Meta Quest 3, XR Interaction Toolkit, Convai AI
- Research 14–15 stakeholder interviews across managers, HR staff, professors, doctors, and employees
- Status Working MR prototype complete
The hard calls, and how I made them
This had to read as learning experience design, not XR spectacle. The structure is built on verifiable learning principles: safe practice, active participation, reflection, feedback, transfer and repetition.
You get one attempt at a hard conversation. Managers, HR staff, doctors and professors face emotionally difficult conversations with no safe place to rehearse, and real relationships absorb the cost of getting it wrong.
How I solved itPathic creates a low-risk rehearsal space where learners can attempt the conversation, make mistakes and recover without damaging anything real.
Principle: Deliberate practice requires psychological safety.Empathy taught as instruction doesn't stick. Being told to consider the other person's perspective is abstract, and abstract instruction produces weak emotional encoding.
How I solved itRole-switching puts the learner on the receiving end of their own words, converting empathy from instruction into lived experience.
Principle: Perspective-taking through embodiment.Immersion alone isn't learning. A vivid headset experience can feel meaningful and still transfer nothing into the real conversation it was meant to prepare you for.
How I solved itReflection is built into the system rather than left to chance, debrief prompts help learners notice emotion, intent and communication patterns after the scene ends.
Principle: Reflection drives learning transfer.Interface mechanics compete with emotional signals. Every second spent working out how to operate the system is a second not spent reading the other person.
How I solved itVoice-first interaction, specific scenarios and guided prompts reduce interpretive overhead so attention stays on the conversation.
Principle: Reduce cognitive load, increase relevance.Design principles & psychology at work
What makes this a learning product rather than a demo:
- A loop, not an eventStructured as experience → debrief → retry → improve, using repetition instead of one-off spectacle.
- Teaching through actionThe system teaches by doing rather than explaining.
- Built for transferSuccess is measured by what carries into real conversations outside the headset.
- Psychological consistency over factual accuracyThe RAG architecture targets a character who stays emotionally coherent, because an inconsistent counterpart breaks the rehearsal.
01Learning Design, Not XR Spectacle
- • Designing AI systems for behavior change, not just interaction
- • Translating research into embodied learning environments
- • Building voice-first XR interfaces with emotional context
- • Structuring products as learning loops: practice → reflection → transfer
Learning Experience Design Principles
Principle 1: Safe Practice Before Real Stakes
A low-risk rehearsal space for deliberate practice: try the conversation, make mistakes, recover.
Principle 2: Perspective-Taking Through Embodiment
Role-switching turns empathy from instruction into lived experience, for stronger emotional encoding than lecture or scripted role-play.
Principle 3: Reflection Drives Learning Transfer
Debrief prompts help learners notice emotion, intent, and communication patterns, so the experience transfers beyond the headset.
Principle 4: Reduce Cognitive Load, Increase Relevance
Voice-first interaction and guided prompts keep attention on emotional signals, not interface mechanics.
What Makes This a Learning Product
- • It teaches through action, not only explanation
- • It uses reflection and repetition instead of one-off spectacle
- • It supports skill transfer into real conversations
- • It is structured as a loop: experience → debrief → retry → improve
02Collaboration with Sejal Shail
Sejal's research framing sharpened four things: workplace conflict as the first high-value use case, emphasis on emotional awareness and psychological safety, stronger reflection and progress tracking, and clearer participant segmentation.
I translated those insights into product structure, XR interaction flows, scenario design, and the spatial conversation model.
Sejal’s Added Value
- • Workplace-conflict framing instead of vague empathy positioning
- • Progress tracking and follow-up learning concepts
- • Stronger emphasis on emotional awareness and safe practice
- • Participant framing around managers, HR, employees, and psychologists
- • Clearer articulation of Pathic as an accessible training tool
What I Drove
- • Translating research into an XR learning experience
- • Role-shift mechanics and interaction flow architecture
- • Voice-first MR conversation structure
- • Scenario framing and embodied perspective-taking logic
- • Spatial design and Unity engineering (C#) implementation of the full experience
03Why Conflict Training Fails
Workplace conflict is more often avoided than addressed, and unresolved tension quietly erodes collaboration, morale, and trust.
People often struggle to see another person’s perspective when emotions escalate, exactly the moment when perspective-taking matters most.
The gap between knowing you should empathize and actually feeling another person's view is the barrier.
Why Current Training Fails
- • Role-playing exercises feel artificial
- • Theoretical discussions do not create emotional memory
- • Organizations lack real-time practice tools
- • Static scripted tools focus on outcomes, not awareness
- • Learners experience high cognitive load and poor guidance
- • Conflict conversations are avoided or delayed instead of practiced
"A lot of conflicts just come from people misunderstanding each other’s tone or intent. Emotions get lost in translation."
- Interviewee 1
"Training works only when it feels real, not theoretical."
- Stakeholder Research
Goal
Create an immersive way to train emotional perspective-taking in workplace conflicts so people can build empathy, confidence, and better conflict habits through interactive mixed reality experiences.
04Research, Personas and Scenarios
Research combined literature review, stakeholder interviews, scenario analysis, and iterative prototyping.
Research Methods
- • Literature review across HCI, spatial computing, empathy research, and learning
- • 30–60 minute semi-structured interviews with diverse stakeholders
- • Broad exploratory interviews before narrowing the Pathic focus
- • Competitive analysis across conflict-resolution and wellness tools
- • Mid-fidelity user flows for scenario entry, role-switching, and debrief
Key Findings
- • People often keep conflict to themselves because they fear judgment or escalation
- • Emotional cues are frequently lost or misread
- • Existing tools often create dependency on facilitators
- • Generic content increases frustration and inaccessibility
- • Role-switching makes complex conversations easier to navigate
- • MR scenarios engage users physically and emotionally, improving retention
Research Themes That Shaped Design
Pathic must support emotional awareness, not only problem-solving.
The product should encourage independent conflict-resolution skills rather than overreliance on external help.
Reflection and follow-up matter if learning is supposed to persist beyond the headset.
Personas & Scenarios
Stakeholder research revealed multiple user archetypes, each facing different conflict dynamics and learning needs.
The Overwhelmed Student
Struggles with academic pressure and power dynamics in advising situations. Needs empathetic communication that goes beyond policy and time pressure.
Goal: Build empathy and communication skills to guide students more holistically.
The Frustrated Employee
Navigates workplace tension but lacks safe ways to process emotion and rehearse responses before real conversations happen.
Goal: Gain confidence and improve communication through guided immersive practice.
The Reflection-Seeking Learner
Feels overwhelmed by complex theory and wants a clearer, more hands-on way to build conflict-resolution skill over time.
Goal: Track growth through guided MR sessions and simple progress feedback.
Core Scenario Types
- • Academic advising conflicts and power dynamics
- • Workplace feedback and collaboration tensions
- • Social misinterpretations involving tone and intent
- • Follow-up reflection and progress review
- • Safe role-swapped simulations before real-life conversations
- • Future expansion into team conflict and broader workplace contexts
Conversation Design
The hardest problem wasn't spatial, it was conversational. Sustaining emotionally coherent dialogue through an unpredictable role reversal took three layers: a branching narrative, grounded knowledge retrieval, and a prompt objective that kept the avatar in character without breaking psychological safety.
Layer 1: Narrative Design - Branching Conversation Trees
Tool: Convai Narrative Design Editor
Each scenario is a branching conversation tree in Convai's Narrative Design editor: an Objective opens each section, then Decision branches route by user intent.
In the academic advising scenario, the opening forks: accept a tour (Section 2) or refuse (Section 3, a recovery path that re-engages without pressure). The avatar always has a fitting response, avoiding the dead ends that kill role-play.
The Welcome Section branches into accept or refuse, each routing to a different section. No path dead-ends.
Objective Nodes
Each section opens with a prompt that sets the avatar’s emotional register and communicative intent. Not a rigid script, a behavioral anchor the LLM improvises within.
Decision Branches
Decision nodes capture the most likely user responses at each fork. The NLU layer interprets utterances against these labels, routing the tree even when the user’s exact words don’t match.
Recovery Sections
A refusal branch doesn’t terminate, it routes to a re-engagement section. Every dead end was redesigned as a psychologically safe re-entry point.
Layer 2: Knowledge Banks - RAG for Grounded, Hallucination-Free Dialogue
Architecture: Retrieval-Augmented Generation via Convai Knowledge Bank
The core risk is hallucination: an avatar inventing details that break the scenario or psychological safety. Convai's Knowledge Bank served as a retrieval layer, giving the avatar a curated foundation before it responds.
Each Knowledge Bank held persona background, institutional policies, the student's situation, and the conflict's emotional subtext. Asked "Why wasn't I told about this deadline?", the avatar retrieves context instead of confabulating: enterprise RAG, aimed at psychological consistency rather than just factual accuracy.
Persona Context
Character background, role, communication style, and emotional state entering the conversation.
Scenario Facts
Institutional policies, deadlines, prior interactions, and factual context that ground the conflict in reality.
Emotional Subtext
What the character feels but hasn’t said — unspoken frustration, fear, or need beneath the surface conflict.
Guardrails
What the avatar must never say, topics that break psychological safety or terminate the scenario prematurely.
Layer 3: The Runtime Loop - What the User Actually Experiences
Scene Cue
Spatial environment and avatar establish emotional register. The user knows what kind of conversation they’re entering before they speak a word.
User Utterance
The user speaks naturally. NLU interprets intent against the current decision branches and routes the narrative tree accordingly.
RAG-Grounded Response
Avatar retrieves from Knowledge Bank, generates within the objective anchor, delivers with synchronized expression and pacing. No hallucination.
Reflection Prompt
At key branch points, a coaching prompt surfaces - a short question that makes the user notice what just happened emotionally before continuing.
The Three Hardest Conversation Design Calls
How directive should the objective prompt be?
Too prescriptive and the avatar sounds scripted, breaking immersion. Too open and it hallucinates outside the scenario. The answer: behavioral anchors that define emotional posture and intent without scripting words.
When does a dead end become a recovery path?
Early builds let users reach conversational dead ends where the avatar exited character. Every dead end was redesigned as a re-engagement section, a psychologically safe way back into the scenario.
What belongs in the Knowledge Bank vs. the prompt?
Behavioral intent → objective prompt. Factual and emotional grounding → Knowledge Bank. Keeping these separate lets the LLM improvise naturally while still retrieving scenario-specific context on demand.
05System Architecture and Implementation
Input Layer
- • Voice input capture
- • Gaze raycasts
- • Spatial XR interactions
- • Proximity detection
Interpretation
- • Convai LLM processing
- • Emotion classifier
- • Narrative state logic
- • Branch management
Response Generation
- • Real-time avatar speech
- • Facial expressions
- • State-driven animations
- • Natural prosody
Spatial Layer
- • Room-scale MR layout
- • Passthrough grounding
- • Anchored placement
- • Proximity transitions
Technical Implementation
What Worked
- • Role-shift mechanic created genuine perspective shifts
- • Emotion-driven pacing improved realism
- • Voice-recognition stability supported natural interaction
- • Avatar gaze correction improved presence
- • Spatial anchor realignment solved drift
Voice UX
- • Latency handling for natural feel
- • Turn-taking and interruption management
- • Interruptible responses
- • Confidence fallback (“Let me rephrase…”)
Lessons Learned
- • Emotional UX is harder than technical UX
- • Timing and pacing shape emotional tone more than content alone
- • Conversation must adapt, not dominate
- • Safety is foundational, and control enables vulnerability
06Competitive Landscape: No One Combines All Three
Most platforms cover one dimension (guidance, wellness support, or scripted practice). None combine embodied role-switching, adaptive AI dialogue, and structured reflection without a facilitator.
Talespin and Mursion come closest on immersion, but neither supports true perspective-switching, and both need human facilitators, which limits scale.
07Interaction Flows & Learning Loop
Pathic is not just an immersive scene. It is a learning loop designed to move the user from orientation to practice to reflection to follow-up.
User Flow 1: Empathy Mirror Simulation
- • MR headset login
- • Dashboard navigation
- • Scenario setup
- • Immersive role-switching simulation
- • Reflective debriefing
- • Follow-up resources
User Flow 2: Follow-up & Progress Tracking
- • Open progress dashboard
- • Review session insights
- • Upload reflection notes
- • Submit progress
- • Receive recommendations
Why These Flows Matter
The debrief and follow-up make Pathic a repeatable skill-building product rather than a one-time XR demo.
08The Final Solution and Its Capabilities
A complete empathy-training experience: AI-driven MR conversations that respond to user input and emotional state, building conflict-resolution skill through practice rather than theory.
Role-Shifting Perspective
The same conflict from multiple viewpoints: understanding, not just intellectual agreement.
Emotional Mirroring
Avatars respond with nuance across extended sessions.
Adaptive Simulation
Difficulty adapts to responses and training goals.
Core Capabilities
Built on Meta Quest 3: spatially anchored AI characters, emotionally responsive dialogue, and role-shifting mechanics.
Spatially Anchored AI
AI characters in Unity and XR Interaction Toolkit, placed in meaningful spatial relationships to the user.
Emotional Dialogue Engine
Convai responses with custom emotion-scaffolding prompts.
Role-Shifting System
Experience conversations as yourself or as the other person, changing how users respond, not just how they think.
360° Scene-Based Scenarios
Scenarios grounded in real workplace, academic, and relationship conflict.
Gaze-Triggered Transitions
Narrative transitions driven by gaze, without constant controller use.
Voice-First UX
Hands-free, embodied interaction that keeps immersion intact.
09Learnings
Designing for emotion differs from designing for utility. Technical systems succeed when they work; emotional systems succeed when they feel authentic, and authenticity comes from restraint, timing, and psychological safety.
Conversational AI showed me how much communication lives in non-verbal cues. The work was aligning dialogue, avatar behavior, pacing, and reflection into one learning experience.
The real product isn't the headset or the AI. It's the learning system: safe practice, embodied perspective-taking, guided reflection, and transfer into real behavior.
Core Insight
Empathy can be trained, but training requires embodiment. Telling people to understand others does not work. Creating safe spaces where they can inhabit another person’s perspective, reflect, and try again creates learning that sticks.