Stepping Stones for Healthcare Metaverse – An Overview of AR and VR Applications

How can we start the journey of the medical Metaverse? Before clinical application and saving lives, one should expose themselves to this still-strange technology. This paper explores publicly available applications with potential. This study aims to list and describe Metaverse applications that are currently assessable and have potential usage in the medical field. Providers collaboration, medical visualisation, and wellness were considered when defining the medical usage scenarios. When determining multiple candidates, we chose the one with easy accessibility to give readers the best chance to experience the technology.

Key points

• The concept of Metaverse is seductive, but due to the nature of the technology involved, it is very hard to demonstrate with words, and the best thing for health professionals and managers is to start trying out metaverse-related technologies to better grasp its potential.
• There are some key concepts which are useful to understand Metaverse-related technologies (MRT): Augmented Reality, Virtual Reality, Mixed Reality, Avatars in videoconferencing solutions and Avatars in social media and for interactive platforms.
• MRTs are being used in health for multiple purposes. While such a review is off-scope, this paper aims to present easy-to-reach, experimental solutions that can better ground the discussions, perspectives, early projects, and pilots of their use in healthcare.
• A set of MRT applications, its review considering healthcare and medical education and how to easily find them is provided in this paper.
• The shorter route to achieving Metahealth or healthcare Metaverse is to understand the limitations and real potential of simple, easy-to-test solutions to help set in motion healthcare-specific testing and experimentation at scale.
• Health managers, educators and professionals have a role in making the healthcare Metaverse real, which starts by understanding its present reality and experimenting with the thrill of today’s real precursors of such future virtuality.

Introduction

“Healthcare will move to the Metaverse! Metaverse, digital patients, in silico clinical trials…” and similar statements were commonly heard in late 2021 and 2022. Now, the hype is fading. ChatGPT and its competitors took the front stage in the often-illusive hope of quick fixes to failing or underperforming health systems. The good news is that this leaves space for serious reflection and solid work on building this virtual world.

The term “Metaverse” is formed by combining the prefix “meta,” denoting transcendence, with the word “universe.”. It describes a massive virtual environment linked to the physical world, in which users interact through digital avatars (Lee et al. 2021). Two years ago, there was a notable surge in the discussion and exploration of the Metaverse concept, generating much interest among the scientific community and the public. Numerous research papers have been authored to explore the prospective applications of the Metaverse and its underlying technologies, such as AR, VR, and MR within the context of healthcare (Iwanaga et al. 2023; Moro 2023; Petrigna and Musemeci 2022; Kye et al. 2021).

The COVID-19 pandemic has also boosted the Metaverse hype because it prompted a rapid shift in digital usage patterns, particularly promoting two trends: telehealth and the value of individualisation (Vargo et al. 2021). These two by-products of the pandemic are here to stay in different ways despite some health systems’ resistance and some drawback movements observable in some countries. During the lockdown, there has been a notable shift towards virtual platforms for collaboration and interactions. This transition has been observed across various sectors, including business, education, and healthcare, leading to the implementation of novel workflow dynamics (Schumacher 2022). The contemporary landscape is witnessing the emergence and advancement of virtual productivity platforms, hybrid working and learning places, and public services. In recent times, there has been a growing trend towards integrating our daily lives and endeavours into the world of digital platforms.

The current deflation of optimistic projections regarding the Metaverse can be attributed to the unmet need for development in the areas that make the Metaverse a reality. A few examples are Displays, Haptics, Batteries, Computer Vision and required multipurpose advanced artificial intelligence (AI). The ongoing development of such technologies is necessary for creating such a virtual world. Furthermore, only when these technologies are barely noticeable extensions of human beings, ubiquitously available and, in some cases, sufficiently miniaturised, will a seamless transition to metalife be possible.

Hence, we ask: what is the current state of the Metaverse? And how is that in relation to hopes of its use and positive impacts on health? And finally, how should health professionals and managers prepare or work towards a Metahealth scenario?

Related Work and Motivation

Currently, the Metaverse is accessible through a limited range of platforms, predominantly utilising VR, AR, and MR technologies. However, it has not yet achieved a comprehensive and fully digitised replication of the physical world. The concept of the Metaverse requires an interaction between the two realities (Kye et al. 2021). Four types of Metaverse have been defined (Davis et al. 2009):

1. Augmented Reality– the creation of a smart environment using location-based technologies.
2. Lifelogging – collecting data from people (through wearables).
3. Mirror World – a digital world that is a “reflection” of the real world.
4. Virtual Reality – a digital world built with avatars that represent the user.

These types have been developing independently and at a different pace. The most advanced are AR and VR, which in combination are often referred to as Mixed Reality (MR). The innovation behind the Metaverse as a collective sociotechnical construct is the new and further advanced combinations of these.

We have compiled a comprehensive evaluation of currently accessible applications that could provide users with a glimpse into the nature and experience of the Metaverse. In the process of app selection for review, our attention was directed towards applications that possess potential for health professionals, managers, and educational institutions in the future. This study aims to identify and examine applications that exemplify the current condition and future prospects of the metaverse.

Moreover, it is worth noting that the Metaverse is a complex notion that is challenging to demonstrate without hands-on involvement with these technologies. Consequently, we have taken care to exclusively choose applications that are readily accessible and frequently utilised. Our objective is to ensure that individuals with the requisite equipment or platform have the opportunity to personally experience these applications.

The primary objective of this study is to provide an analysis of the current state of the Metaverse and its subjective elements. In an era characterised by escalating healthcare costs, there is a growing trend toward the use of preventive measures and digital health. Additionally, it is important to note that various lifestyle factors play a significant role in the development of numerous diseases, such as diabetes, obesity, and heart conditions. This realisation presents a novel opportunity to enhance public health education and promote healthier habits, ultimately reducing healthcare expenditures and alleviating the financial burden on global healthcare systems. This is what the current literature has envisioned for the Metaverse.

However, the Metaverse remains a theoretical construct that is yet to be realised in practice, as the necessary technologies for its implementation are still being developed. Several platforms and applications that integrate VR, AR, and MR technologies offer a first insight into the potential of the metaverse. Demonstrating AR, VR, and MR is challenging without first-hand experience. This paper will attempt to provide a comprehensive description of the aforementioned technologies. However, we highly recommend everyone to personally experience them.

The primary objective of this article is to provide tangible illustrations of the present-day manifestations of the Metaverse and give concrete examples of apps anyone equipped with the right gear can try.

Methods

As a first step, we established a set of selection criteria for choosing apps that could serve the purpose of our study. For app selection, we looked for the following criteria:

• Innovation and representatives: One understands that the app should be a new way to do something from the real world in VR or AR. Furthermore, we selected apps that could be representatives of a whole app category (such as fitness or education) since we did not want to include very similar apps. Lastly, these apps also include a functionality or option that distinguishes them from other apps with the same intended purpose.
• Connection to real-world practice: Another point is to ensure these apps enhance or make for a better experience than their out-of-metaverse competitors. In other words, these apps allow the user to do something he/she otherwise would do in the real world but with a more immersive, engaging, and fresh perspective.
• Customisation and avatars: A fundamental building block of the Metaverse is avatars and worlds that persist whenever you log in to the Metaverse (a concept known as lifelogging and digital twins). In the selection process, we ensured that these apps allow for at least one of these, differentiating them from other digital solutions.
• Potential impact on Healthcare: It was important for us that these apps have a real-world impact on patient engagement and education.

After selecting eight apps, we used an instrument of comparative analysis to review them. The selected apps were: i) Spatial; ii) Horizon Workrooms; iii) YouTube VR; iv) Tripp; v) Awake Heart; vi) Complete Anatomy; vii) Supernatural; viii) Holo Anatomy

Review of Applications

The eight applications are presented in detail as follows:

Spatial (demo available here - https://www.spatial.io/ )

Category - App Platform

Platform - PC, Mobile & Meta Quest

Description and Purpose

Spatial is an innovative Metaverse application designed to empower creators and businesses to build immersive virtual venues that facilitate sharing culture, experiences, and creative endeavours. One of Spatial’s standout features is its commitment to empowering users to unleash their creativity. It is a versatile canvas for users to share a wide array of content, develop a thriving community, and market creative works and products. By enabling such multifaceted interactions, Spatial aligns itself with the core principles of the Metaverse, where digital spaces serve as hubs for collaboration and expression.

Another noteworthy functionality is its ability to facilitate the development of 3D environments or objects that can be minted as NFTs (Non-Fungible Tokens). This feature is pivotal as it allows users to take their virtual creations to the next level by turning them into unique, tradable assets. By embracing the NFT trend, Spatial taps into the broader Metaverse trend of digital asset ownership and exchange. Furthermore, the app facilitates commerce by enabling users to sell or rent the 3D environments they create, opening economic opportunities within the Metaverse ecosystem.

Furthermore, Spatial distinguishes itself through its advanced world customisation options and avatar capabilities. Users can tailor their virtual environments to suit their vision, fostering a more immersive and personalised experience. Additionally, the app offers diverse avatar choices, enhancing user identity expression within the Metaverse.

Lastly, another strength of Spatial is its multi-platform accessibility, supporting PC, mobile devices, and Meta Quest head mount displays (HMD). This broad availability ensures that users can access and contribute to the Metaverse from various devices, making it more inclusive and adaptable to varying technological environments.

Key take-away and innovative features:

• Empowering Creativity
• 3D Environment Creation, NFT Minting and Commerce
• Advanced World Customisation and Avatars:
• Multi-Platform Accessibility:

Spatial is a Metaverse application that embodies the core principles of creativity, community, and commerce. Its 3D environment creation, NFT minting capabilities, advanced customisation options and multi-platform accessibility position it as a noteworthy player in the Metaverse landscape. As the Metaverse continues to evolve, Spatial is an important tool for health professionals to know and explore the future possibilities of the Metaverse landscape.

Horizon Workrooms (demo available here - https://workrooms.workplace.com/signup/

Category – Productivity

Platform - Meta Quest (PC compatible)

Description and Purpose

Horizon Workrooms, developed by Meta, is a virtual office platform offering an immersive collaborative work environment that aims to answer the evolving needs of modern work environments. At its core, Horizon Workrooms transforms the concept of a traditional office into an immersive virtual environment. It provides teams with a unique space for collaborative work, bridging the gap between physical and digital worlds. The app’s compatibility with the Meta Quest headset ensures that users have access to a high-quality, immersive experience. Additionally, its PC compatibility extends its reach to those who prefer 2D screens, making it inclusive and adaptable to various hardware configurations.

Moreover, the app facilitates various aspects of collaboration, including team meetings, brainstorming sessions, and idea generation. Through spatial audio, lifelike avatars, and interactive whiteboards, Horizon Workrooms enhances the quality of communication and engagement, fostering a sense of presence often missing in traditional video conferencing tools. Furthermore, users can seamlessly share documents, presentations, and 3D models within the virtual workspace. This capability allows for real-time collaboration and feedback.

Finally, the app also excels in overcoming geographical barriers, enabling seamless collaboration among teams regardless of their physical locations, which is especially valuable in the context of remote work and distributed teams. Moreover, it effectively addresses the needs of the evolving workplace landscape, accommodating hybrid office models and telework arrangements and offering a versatile solution for organisations navigating the transition between physical and virtual workspaces.

Key take-away and innovative features:

• Immersive Virtual Office
• Enhanced Collaboration
• Hybrid Office and Telework Capabilities

Horizon Workrooms by Meta reimagines the way teams collaborate and work together. Offering an immersive virtual office environment enhances collaboration, productivity, and engagement, regardless of geographical distance. Its compatibility with both Meta Quest headsets and 2D screens, along with its responsiveness to the changing dynamics of hybrid work, makes it a valuable tool for health organisations and health managers seeking to harness the potential of the Metaverse for their workspaces. As the workplace continues to evolve, Horizon Workrooms represents a significant step toward the future of work and collaboration.

YouTube VR (demo available here - https://vr.youtube.com )

Category – Entertainment and Education

Platform - Any VR Headset

Description and Purpose

YouTube is becoming a more prominent tool in the educational scene. YouTube VR is an application that brings the world of video education and online tutorials into the immersive realm of virtual reality. YouTube VR revolutionises the viewing experience by presenting content in an immersive 360-degree virtual reality format, enabling users to actively engage with videos. This departure from passive observation empowers users to fully explore and immerse themselves in digital experiences. Additionally, the app offers user-friendly 360 video creation tools, democratising content creation and enabling users to share their distinctive perspectives within the Metaverse, fostering creativity and individual expression.

The app becomes a virtual classroom, offering educational institutions and learners a platform to engage in 360-degree educational content. This interactive learning experience can enhance comprehension and retention, making it a valuable tool for immersive education.

Lastly, YouTube VR’s compatibility with any VR headset ensures accessibility for many users. This inclusivity expands the app’s reach, making immersive content and interactive experiences available to a broad audience.

Key take-away and innovative features:

• 360-degree Videos and User-Friendly Video Creation Tools:
• Immersive and Interactive Education:
• Multi-Platform Accessibility:

YouTube VR represents a remarkable evolution of entertainment within the Metaverse. Its ability to deliver immersive and interactive learning experiences sets it apart in the educational category. By offering user-friendly 360 video creation tools, the app empowers users to consume and actively shape their Metaverse experiences. Its compatibility with various VR headsets underscores its commitment to accessibility and inclusivity. As the Metaverse continues to evolve, YouTube VR plays a pivotal role in defining the future of immersive entertainment and education.

Tripp (demo available here - https://www.oculus.com/deeplink/?action=view&path=app/2173576192720129&ref=oculus_desktop )

Category - Lifestyle

Platform - Mobile, Meta Quest, & PlayStation VR

Description and Purpose

Tripp is a multifaceted wellness and meditation application that enhances traditional mindfulness practices. The app intends to enhance mental well-being and creativity within the Metaverse. Tripp is a comprehensive wellness companion with an extensive library of guided meditations and breathing exercises to guide users toward mental clarity, relaxation, and emotional balance. Notably, it integrates research-supported visuals meticulously designed to induce relaxation and enhance emotional well-being, elevating the meditation experience by providing an immersive and impactful visual dimension.

Furthermore, the app empowers users to tailor their meditation sessions to their specific requirements. With options to customise the virtual world, select a spiritual guide, choose a preferred voice, and adjust the duration of the experience. Tripp extends its reach to address critical issues in mental health and sleep disorders. The app’s guided meditations and visual experiences are valuable tools for individuals seeking relief from stress, anxiety, and sleep-related challenges. Its therapeutic potential in these areas underscores its relevance and applicability within the Metaverse.

The last of Tripp’s strengths is its availability on multiple platforms, including mobile devices, Meta Quest, and PlayStation VR. This versatility ensures users can access its wellness resources and meditation practices conveniently, regardless of their preferred device.

Key take-away and innovative features:

• Guided Meditations and Breathing Exercises in VR
• Customisable Meditation Sessions
• Mental Health and Sleep Disorders

Tripp is a wellness and meditation app within the Metaverse. By addressing mental health issues and sleep disorders, this application takes a significant step in addressing the concerns of loneliness and depression in the contemporary digital era. Furthermore, its cross-platform accessibility enhances inclusivity and convenience. As the Metaverse increasingly incorporates a holistic approach to well-being, Tripp presents a valuable avenue for individuals to seek relaxation, foster connections, and stimulate creativity within the continually evolving digital landscape.

Awake Heart (demo available here - https://www.microsoft.com/en-us/p/awake-heart/9nvm1kvsrrlt?activetab=pivot:overviewtab )

Category – Educational

Platform - Microsoft HoloLens (AR)

Description and Purpose

Awake Heart is an educational application that ushers users into the realm of immersive and spatial learning. Awake Heart leverages holographic technology to enhance the study of anatomy and physiology, offering a new dimension to education and training within the Metaverse.

Awake Heart offers users an immersive learning experience by providing detailed holographic anatomic models that allow hands-on exploration of human anatomy and physiology. This engaging approach fosters comprehension and retention, making it a valuable educational tool for learners and institutions, transcending traditional textbook methods.

Moreover, Awake Heart also offers users a way to assess their knowledge through comprehensive exams. This feature helps learners gauge their proficiency and reinforces the educational experience by providing feedback and insights into their progress. Awake Heart prepares healthcare professionals and students for real-world hospital procedures and situations by enabling users to participate in clinical simulations. This practical training component is invaluable for new professionals seeking to build their confidence and skills and for health institutions to get new professionals up to speed. Awake Heart’s choice of platform, Microsoft HoloLens, aligns with its commitment to mixed-reality educational experiences and offers an ideal environment for spatial learning.

Key take-away and innovative features:

• Engaging Learning Experience
• Comprehensive Exams
• Clinical Simulations Beyond Traditional Education

Awake Heart is an educational application that leverages holographic technology to revolutionise the study of anatomy and physiology. Offering clinical simulations in an immersive environment bridges the gap between education and real-world practice, particularly for healthcare professionals. As the Metaverse continues to evolve, Awake Heart sets a high standard for immersive and effective education within this digital landscape.

Complete Anatomy (demo available here - https://3d4medical.com/ )

Category - Educational

Platform – PC and Mobile (AR)

Description and Purpose

Complete Anatomy offers users an immersive and spatial learning experience through high-definition 3D virtual anatomic models that allow for in-depth exploration of human biology, surpassing the constraints of traditional textbooks. Capitalising on the three-dimensional nature of anatomy and physiology, the app provides an intuitive and comprehensive understanding of the subject matter, transcending traditional 2D representations. The app covers a wide spectrum of educational content, including anatomy, physiology, and pathology. Furthermore, Complete Anatomy offers alternative ways to visualise the body, such as X-rays, CT scans, and MRI scans, enhancing the user’s ability to grasp complex medical concepts.

Furthermore, among all the standout features of Complete Anatomy is its inclusion of step-by-step surgical procedures. Users can gain practical insights into surgical techniques. This hands-on approach to learning enhances comprehension and confidence. The app is available on both PC and mobile platforms, but its AR component is only available on mobile.

Key take-away and innovative features:

• Immersive 3D Anatomic Models
• Multifaceted Learning
• Step-by-Step Surgical Procedures

Complete Anatomy is an educational application for studying human anatomy and related fields within the Metaverse. Its immersive anatomic models, diverse learning content, and practical surgical procedures make it an interesting resource for learners and medical professionals. As the Metaverse continues to evolve, Complete Anatomy exemplifies the potential for immersive and comprehensive education within this digital frontier.

Supernatural (demo available here - https://www.getsupernatural.com/signup/ )

Category - Fitness

Platform - Meta Quest (VR)

Description and Purpose

Supernatural represents a fusion of fitness and VR technology, offering users a path to physical well-being and happiness. Supernatural offers users an immersive fitness experience in a virtual world where dynamic boxing, aerobic, and full-body cardio workouts occur in captivating digital landscapes. These classes breathe excitement into exercise routines, alleviating the monotony of traditional workouts. Additionally, the app prioritises holistic well-being by incorporating guided meditation and recovery exercises following vigorous fitness sessions, aligning with its commitment to both physical and mental health, leaving users feeling physically refreshed and mentally revitalised after each session.

One of Supernatural’s standout features is its commitment to fitness instruction. Users get to interact with live-action coaches who meet them face-to-face in virtual reality. This personal and immersive approach allows for real-time guidance, motivation, and correction, creating a guided and safe fitness experience.

Finally, the Supernatural app improves physical health and unites users by letting them connect, share achievements, and help one another. Social interactions inspire healthy workout routines. The app’s primary objective is to improve physical health and happiness by offering a variety of intense workouts, professional advice, and a deep feeling of a fitness community. Promoting holistic well-being and joy is the overall goal.

Key take-away and innovative features:

• Engaging Fitness and Recovery Classes in VR
• Learning from the Experts
• Building Healthy Routines and Community Engagement

Supernatural is a fitness application within the Metaverse, integrating immersive VR technology with physical exercise and well-being. Its captivating fitness classes, expert coaching, and emphasis on community engagement create a holistic fitness experience that transcends traditional workout routines. As the Metaverse continues to evolve, Supernatural paves the way for a healthier and happier digital future.

Holo Anatomy (demo available here - https://www.microsoft.com/en-us/p/holoanatomy/9nblggh4ntd3?activetab=pivot:overviewtab )

Category - Educational

Platform - Microsoft HoloLens (AR)

Description and Purpose

Holo Anatomy is an educational application that uses the AR technology from Microsoft HoloLens to offer users a way to learn and engage with human anatomy. Holo Anatomy offers users an immersive and spatial learning experience for studying human anatomy. By leveraging holographic technology, the app facilitates 3D visualisation, allowing users to delve into human anatomy with enhanced depth perception and understanding. This immersive approach not only aids in understanding but also enhances memory retention.

Furthermore, Holo Anatomy’s most important feature is its “Lecturer Mode.” Educators and instructors can teach in this mode while interacting with the holographic model. This feature empowers educators to conduct immersive anatomy lessons, making the learning process more engaging and interactive. It transforms traditional lectures into dynamic and participatory experiences.

Lastly, Holo Anatomy facilitates collaborative learning through shared sessions featuring synchronised visual fields, allowing users to explore and discuss anatomical structures in real-time. This collaborative environment enhances engagement and creates opportunities for peer-to-peer learning and in-depth discussions about anatomical structures, functions, and clinical applications. This emphasis on collaboration fosters critical thinking and a deeper understanding of anatomy beyond individual learning.

Key take-away and innovative features:

• Immersive and Spatial Learning
• Lecturer Mode for Interactive Teaching
• Collaborative Exploration

Holo Anatomy is an educational application for the study of human anatomy. Its immersive and spatial learning features, collaborative exploration, and interactive teaching options set it apart. Promoting collaborative discussion and engagement exemplifies the potential of the Metaverse to enhance the way we teach and learn complex subjects.

Discussion, Future prospects and health management and education implications

When reviewing the users’ experience for each of the apps, we came across advantages and disadvantages that were common throughout most apps that use the same technology. We related these to the use of either AR or VR and summarised them in Table 1.

The technologies necessary for the realisation of the Metaverse are presently in developmental stages. They are not all real yet to allow the virtual world many easily write about. Their reality still has distance from the real world for now but glimpses into “the Metaverse” can be accessed exclusively through existing platforms like Spatial, Meta, and, in the near future, Apple, since they recently announced the launch of “Apple Vision,” a mixed reality headset.

The term Metaverse refers to a distinct digital realm that exists in parallel to the physical world. The current state of technology does not currently allow for a realisation of this kind of virtuality. The creation of the Metaverse in the future hinges upon the integration and compatibility of several technologies, including but not limited to AR, VR, MR, haptics, hand/eye tracking, sensors, computer vision, perceptual science, avatars, graphics, networks and clouds, blockchain, robotics, user interactivity and artificial intelligence (AI). And these are only some of the Metaverse enablers.

We urge the scientific community to conduct further research on these technological advancements that enable the creation of the Metaverse. Prominent technology companies such as Meta and Apple have allocated significant resources towards the development of the Metaverse. Notably, Facebook even changed its name to “Meta”. Can this effectively reach a wide audience? Right now, the Metaverse is confined within the platform that allows users to explore it. The headset you have determines what digital ecosystem you are in. Is this the start of many parallel digital worlds like the “Metaverse”, “Appleverse,” and even “Googleverse”? Of not one but many “digital singularities”?

Finally, it is quite likely that these technologies will be significant tools in healthcare. Thus, we emphasise the importance of staying well-informed about them to cultivate competency as a healthcare professional or provider in the upcoming years. As we have done for hundreds of years in the medical field, testing and experimenting with existing tools is the best way to prepare and lead the way into their usage and appropriate usage in health. The reality of virtuality for our patients is needed to deal with mental illness, dementia, anxiety, chronic pain, distance counselling for victims that wish to remain anonymous and many other conditions where the real suffering of humans can be ameliorated if not made to disappear. Health managers, educators and professionals have a role in making the healthcare Metaverse real, which starts by understanding its present reality and experimenting with its thrill.

Conflict of Interest

None