A Virtual Reality Project developed for second-year medical students at UBC to enhance patient interaction through immersive simulations.

Role

Interaction Designer

UX/UI Designer

3D Animation

Design System

Team

Oliver Taw (UX,UI Designer)

EdTech at UBC Faculty of Medicine

Timeframe

12 Months

May 2023- Aug 2024

Tool

Figma

Unity

Immersive clinical expereince

This project involved designing the user interface, interactions, and animations for a clinical virtual reality (VR) experience for medical students. This experience guides learners through scenarios where they interact with virtual patients, ask questions, perform physical exams, and complete quizzes.

“The challenge was to create intuitive and engaging UI elements that boosts student engagement, confidence, and learning outcomes.”

Conceptualization

Discovery Phase

The project theme, set by the project manager, combines retro and modern elements, blending vintage styles with contemporary aesthetics and technologies to create a nostalgic yet fresh experience. As the lead on visual design, I began by creating a mood board, collecting and analyzing various retro games—both VR and non-VR—to study their design and effectiveness.

Compiled various screenshots into a mood board to showcase early design concepts and ideas.

Sketches and Low-fi Prototypes

Multiple design iterations of an element were sketched and refined based on feedback.

Initial concepts and wireframes were sketched out in 2D but quickly translated into low-fidelity design in Figma. This allowed me to test the spatial arrangement of UI elements and their usability in a 3D space. The goal was to make sure that these UI elements were easily visible and reachable without causing motion discomfort.

Concept ideas were first sketched out in paper and then transition into low-fi design in Figma to test and refine the design.

Iterative design

We conducted several rounds of early testing with both the internal team and students new to VR, observing how they interacted with the UI. These early and rapid playtests provided valuable insights into users' experiences with hand gestures, poke interactions, and head movements. Based on this feedback, I refined various UI elements to ensure they were not only functional but also intuitive and enjoyable to use.

Task List UI Iteration

Task List UI evolution to better reflect the theme of the project.

Final task list in the VR environment

Strength Meter UI Iteration

In the early iteration, the Strength Meter was labeled "Input Strength" and featured neon color bars. After several playtests, it became clear that "Input Strength" was confusing, as users were unsure whether it referred to the player’s input or the VR patient’s strength.

Patient’s Strength Meter UI iteration

In the next iteration, I updated the label to "Patient's Strength" to better represent the patient's condition, from weak to strong. The color scheme was also adjusted, moving away from the true neon retro aesthetic to align more with a contemporary theme.

Interaction + feedback

Since this VR project is based on clinical scenarios with complex interactions—such as branching quizzes, physical examinations like neck palpation, and monitor screens within the VR environment—quick sketches and low-fidelity wireframes in Figma were used to efficiently brainstorm and refine user flows and interaction designs.

Branching Quiz (Quiz Tree)

The branching quiz requires users to select top-level answers that lead to lower-level responses, all within a single screen. The initial design used shortened answer initials in circular gear-like icons. Clicking a circle revealed the full answer. However, feedback showed that students struggled to understand the short forms, making the interaction difficult to use.

First version of the quiz tree with gear like icons and overview of the whole branch.

In response to the feedback, the design evolved to a collapse/expand method, showing short forms that expand into full answers when clicked. With further user testing, the final design (image below) merged both ideas, displaying full answers upfront while allowing branching into lower-level responses.

Second version was designed into a collapsible branch to reduce cognitive overload of the user.

Final design of the quiz tree with updated interactions and button states.

Result

The final quiz design made it easier for students to navigate complex scenarios by presenting full answers upfront, reducing cognitive load and enhancing their ability to focus on clinical decision-making.

Final version of the quiz tree in the VR experience.

Nurse Station Monitor UI (Doctor’s Notes)

The monitor displays the doctor's notes, patient history, and imaging, etc, framed within a 3D object rather than as a floating UI. The challenge was to present various sections of the notes in an easily accessible way within the VR environment.

Final design of the monitor UI (Doctor’s Note, Lab results and Imaging)

The initial design drew inspiration from 80s computers, where users clicked folders to access information. However, feedback revealed the user flow was confusing and information wasn't readily accessible.

Initial design of the monitor UI with deep folder structure and hidden menus

In response, the layout was redesigned with topics listed on the left and main tabs at the top, allowing easier navigation. Arrows were added for forward/backward movement between sections.

2nd design iteration: contents were organized as tabs on the top with left and right direction buttons.

Result

By refining the layout of the nurse station monitor, students could quickly reference key patient data, allowing them to focus on diagnostic tasks without unnecessary distractions, improving both the flow of interaction and the learning experience.

Changes in the final version includes:

Moved the hard-to-notice top tabs to the left side, introducing better button hierarchy.
Adjusted arrow directions from left/right to top/bottom to align with the vertical content layout, enhancing affordance

This improved navigation significantly, allowing users to easily identify and access the main sections.

Final design of the monitor UI (Doctor’s Note, Lab results and Imaging)

Other UIs

Images below are just a few examples of the UI elements designed, with more iterations and features developed throughout the project.

final project walkthrough video

Building out the simple design system

Given the project's scale and the involvement of co-op students, it is essential to ensure that both the design and file structure have a clear purpose and organization. I led the creation of a small design system/UI kit which could be easily understood, used, and maintained by both current and future team members. This helps the design system meet the team’s needs effectively, focusing on essential elements and interactions to maintain consistency and scalability while minimizing unnecessary complexity.

Conducting a UI Audit

The first step in creating the design system involved conducting a thorough UI audit to identify existing patterns, inconsistencies, and areas for improvement across the initial design elements. This audit included a detailed analysis of colors, fonts, button states, floating UI panels, and monitor screen interfaces. Insights from this process laid the groundwork for a cohesive design system, ensuring consistent colors and typography, standardized opacity for background elements, and organized design elements throughout.

Establishing Core Foundations

The next step in building the design system focused on establishing core foundations, such as standardized colors, typography, background material, and interaction states. Each element was carefully defined to support a consistent and accessible experience, with color choices reflecting the retro-futuristic theme while maintaining readability and contrast requirements. Typography was selected to balance visual appeal with clarity, ensuring ease of reading in both floating and fixed UI elements. These core foundations serve as essential building blocks, ensuring consistency and scalability as the project evolves.

Crafting Consistent Interactions

Given the unique nature of this VR experience, where buttons and interactive elements are used sparingly—often only two or three times each—defining components focused less on large-scale reusability and more on maintaining consistency in style and behavior across different contexts. Each button and interaction was carefully created to align with the established colors, typography, and interaction states, ensuring a cohesive experience even with limited repetition.

When this project first began, I wasn’t involved until the team reached a stage where UI elements were essential. At that point, I joined to contribute my design skills and gaming background, not only in creating UI elements but also in developing the 3D character animations for the models used in the project.

One of the main challenges in a long-term project like this was coordinating with multiple contributors, including co-op students who had worked on some of the initial designs. As the lead designer on the project, I recognized the need to establish visual consistency, which drove the creation of a streamlined design system to ensure that all elements aligned with the project's guidelines and goals.

Overall, the project received very positive feedback and has now been successfully integrated into the second-year medical curriculum, enhancing students' learning experiences and engagement with clinical scenarios in VR.