Participatory Development of a 3D Telemedicine system during Covid: the future of remote consultations of a 3D Telemedicine system Covid: the future

The Covid pandemic brought the need for more realistic remote consultations into focus. 2D telemedicine solutions fail to replicate the fluency or authenticity of in-person consultations. This research reports on an international collaboration on the participatory development and first validated clinical use of a novel, real-time 360-degree 3D Telemedicine system worldwide. Development of the system - leveraging Microsoft’s Holoportation TM communication technology – commenced at Canniesburn Plastic Surgery Unit, Glasgow in March 2020. p=0.0002). Safety and clinical concordance (95%) of 3D Telemedicine with a face-to-face consultation were equivalent or exceeded estimates for 2D Telemedicine. One of the ultimate goals of telemedicine is for the quality of remote consultations to get closer to the experience of face-to-face consultations. These data provide the first evidence that Holoportation TM communication technology brings 3D telemedicine closer to this goal than a 2D equivalent.


Methods
Research followed VR CORE guidelines on development of Digital Health trials, placing patients at the heart of the development process. This consisted of three separate studiesa clinician feedback study (23 clinicians, Nov-Dec 2020), a patient feedback study (26 patients, Jul-Oct 2021), and a cohort study focusing on safety and reliability (40 patients, Oct 2021 -Mar 2022). "Lose, Keep and Change" feedback prompts were used to engage patients in the development process and guide incremental improvements.

Results
Participatory testing demonstrated improved patient metrics with 3D in comparison to 2D Telemedicine, including validated measures of satisfaction (p<0.0001), realism or 'presence' (Single Item Presence scale, p<0.0001), and quality (Telehealth Usability Questionnaire, p=0.0002). Safety and clinical concordance (95%) of 3D Telemedicine with a face-to-face consultation were equivalent or exceeded estimates for 2D Telemedicine.

Conclusions
One of the ultimate goals of telemedicine is for the quality of remote consultations to get closer to the experience of face-to-face consultations. These data provide the first evidence that Holoportation TM communication technology brings 3D telemedicine closer to this goal than a 2D equivalent.
Keywords: Telemedicine; 3 dimensional; remote consultation; Plastic Surgery; Covid 19; presence; realism BACKGROUND Real-time 3D Telemedicine has previously been proposed within a research setting only, with constraints on cost, complexity, bandwidth and technology [1,2]. With the Covid Pandemic, use of remote consultation has increased exponentially and has brought the concept of a 3D consultation into focus [3]. Although there appears to be significant theoretical value in assessing surgical patients, as yet no validated clinical data exists on the benefits of such a system in comparison to standard 2D Telemedicine. Here we detail the participatory development of a real-time 3D Telemedicine system, in conjunction with an international group of healthcare and industrial stakeholders. We also discuss the first realworld use of a 3D Telemedicine system with Plastic Surgery patients.

Increasing access to care in Lower to Middle Income Countries
Initial discussions regarding the development of a 3D Telemedicine system, leveraging to initiate an international collaboration on this project. Geospatial mapping set out the early vision, using census data and overland travel times to estimate increased access to reconstructive care [4]. With the global Covid pandemic enforcing the first UK lockdown in March 2020, the focus of the project rapidly pivoted to potential delivery of remote care during Covid.

VR CORE Guidelines on developing Digital Health technologies
The 3D Telemedicine system was co-developed with patients using international guidelines for digital or Virtual Reality (VR) clinical trials, as proposed by the Virtual Reality Clinical Outcomes Research Experts (VR CORE). These guidelines aim to improve methodological quality in digital health technology trials, dividing development into three phases (VR 1 to 3), akin to Clinical Trial Phases 1 to 3 [5]. Participatory development is one of the key elements of these guidelines, focusing on the principles of human-centered design. The importance of this in digital trials has been previously underestimated, as "lack of patient involvement, poor requirement definitions, and non-adaptation to user feedback are some of the common factors that explain failures of digital interventions" [5]. The present study placed patients at the heart of the development process and reports on the Participatory development (VR 1) and early Clinical Trial (VR 2) phases. The objectives of this research were to co-develop a patient-centered 3D Telemedicine system, compare validated outcome measures with a 2D system, assess alignment with an in-person consultation, and to ensure safety, reliability and clinical concordance.

Ethics
NHS Greater Glasgow and Clyde (GGC) R and I approvals GN20HS181/ GN20HS300. NHS GGC governance meetings monitored the project bi-annually. Participants consented in writing. Patient data controlled by NHS GGC. STROBE guidelines followed for reporting of cohort trials.

Approach and Preliminary Work
Research followed the VR CORE guidelines [5] and consisted of preliminary work including focus groups, stakeholder collaborations, equality assessments, and initial prototyping.
Weekly collaboration meetings were held over a 2-year period in UK, Ghana and USA, commencing March 2020. To inform prototype development -a focus group was held with 23 clinicians from Canniesburn Plastic Surgery Unit, Glasgow, UK in June 2020 -exploring desired functions, identification of needs, potential benefits, risks, use during Covid, and implementation (Supplementary Table 1). An Equality Impact Assessment (EQIA) [6] focused on "not leaving anybody behind in a digital world" [7], and collected data on factors that may influence access or use of novel technology, such as deprivation and educational level. This was followed by three separate studies: a clinician feedback study -providing feedback to incrementally improve and shape the 3D Telemedicine sytem prior to patient testing; a patient feedback study -to compare 3D and 2D Telemedicine systems; and a cohort study focusing on safety, reliability and clinical concordance of 3D with face-to-face examination (Flowchart 1).

Flowchart 1 Prototyping and Initial Set up
Initial system set-up took place in September 2020, when a research team from Microsoft Corporation, Redmond, USA travelled to Glasgow, UK.  Table 1).

Patient and Clinician Participatory Development
Patient and clinician feedback was used to shape the development of the 3D Telemedicine system, assess usability of the outcome instruments, and provide data for follow-on trials.
Sample sizes were not required for the participatory development component of this study. Data analysis carried out by a blinded, independent statistical service using R version 3.4.1 [9].

Study 1: Clinician Feedback Study : Iterative improvements in 3D system
Clinician feedback testing focused on optimisation of the prototype system prior to patient testing. Full details are provided in Supplementary Appendix 1. In brief, 23 clinicians were assessed in two batches in November and December 2020, to provide incremental improvements in usability, and assess responsiveness to change between batches. Key findings included a strong correlation between quality of consultation (satisfaction) and the realism or "presence" of the 3D system (Pearson r = 0.8451, p<0.0001, Graph 1). The 3D system was found to have a significantly higher realism or "presence" than the 2D system in batch 2. Improvements were instituted in response to clinician feedback, prior to patient testing (Supplementary Table 1).

Method
Patient feedback testing provided the first real-world data on patients' perceptions of the 3D Telemedicine system in comparison to 2D Telemedicine. A preliminary cohort of six patients was used to test acceptability of the outcome instruments. Longer form scales such as the Presence Questionnaire with 29 items [10] were found to be too long and confusing for patients, and therefore short forms of "presence" testing were considered, with final selection of the validated Single Item Presence scale [11].

Participants
26 patients from Canniesburn Plastic Surgery Unit clinic participated (July -Oct 2021) Table 2). Patients were seen in 3D and 2D Telemedicine by a single clinician, without randomisation (11 patients 2D first, 15 3D first). Each consultation lasted 10 minutes, followed by completion of questionnaires.

Outcome Instruments
Single Item Presence scale asked the key question "To which extent did you feel present in the virtual clinic, as if you were really there?" to provide insights into how closely the 3D clinic aligns with an in-person consultation [12]. Telehealth Usability Questionnaire (TUQ) consists of 21 items covering subdomains of usefulness, ease of use, effectiveness, reliability and satisfaction [12]. Satisfaction measured with 0-100 visual analogue scale [13]. System Usability Scale (SUS) measured usability with an industry standard scale that allows comparison across different technologies [14]. "Lose, Keep and Change" feedback prompts used to engage patients in the development process and guide incremental improvements [15].

3D versus 2D Telemedicine validated outcome measures
3D was rated higher on satisfaction (p<0.0001), presence (p<0.0001), and quality (TUQ, p=0.0002) (Boxplot 1). Mental effort rating scale was equivalent (P=0.2965), scoring highly in both systems suggesting that the 3D system is not more complex to use than normal telemedicine systems. Patients' subjective views on accuracy of diagnosis and ease of positioning their body part for examination were also significantly higher (Table 1). 100% patients preferred 3D (for patient comments see Supplementary Table 3) and usability was rated highly (SUS 87.02). Table 1 Boxplot 1

Feedback Process and Subjective interview
"Lose, Change and Keep" prompts noted limitations related predominantly to quality of 3D spatial resolution in real-time (Supplementary Table 1). Subjective interview was overwhelmingly positive, with patients in particular finding that the 3D system allowed positioning their body part for examination much easier than with 2D (data not shown)

Carry-over Effects
Between-group comparisons of satisfaction, presence and TUQ were done using a general linear model which included subject, group and order factors (3D or 2D system first, to determine any effect of carry-over). No effect of carry-over was noted for these outcome measures (data not shown).

Method
The cohort study aimed to assess safety and reliability. A paired design was used with patients seen in 3D Telemedicine first, followed by face-to-face, by the same clinician to allow assessment of clinical concordance. Ten minutes were given per examination.
Clinicians and patients both completed questionnaires for each consultation.

Sample size
Pilot study with recruitment of 40 patients. Primary and secondary outcomes not designated for pilot studies. Post-study power calculations indicated that with a paired design, 16

Flowchart 2
Outcome Instruments UNN scale, validated for orthopaedic patients, is one of the only scales available designed to assess quality of both telemedicine and face-to-face consultations [16]. NASA TLX is a measure of the workload or "task load" of an activity -assessing subdomains of mental demand, physical demand, temporal demand, performance, effort and frustration [17].

Operational and Clinical Safety
Operational safety issues occurred in 0%, technical issues 20%, reliability issues 7.5%, and image artefacts 12.5%. All issues were temporary and did not curtail the clinical consultation. Clinical concordance of 3D consultations with face-to-face management plan was 95%. 2 cases of clinical discrepancy did not result in clinical safety concerns (Supplementary Table 6).

3D versus Face-to-Face Validated Outcome measures
Clinicians' UNN Sum score was significantly higher for a face-to-face than for 3D consultation (Table 2). Nonetheless, the difference in sum score between 3D and face-toface was only 0.4, with both consultation types approximating to scores between "good" and "very good". The minimal clinically important difference (MCID) has not yet been established for UNN score. The UNN authors suggested a difference of 0.3 as a noninferiority limit in their study, but this appears to be arbitrary and not based on clinical data.
Patients rated the face-to-face higher on satisfaction than 3D, mean 90.08 versus 83.58 respectively (mean difference 6.50 [CI 2.40-10.60]; p=0.0027; paired t test). Patients rated the 3D system higher on satisfaction than clinicians (p<0.0001). This may be related to differences in mental effort (p=0.0042) and task load (NASA TLX, p=0.020).

Improvements over time
Clinician satisfaction, presence (PQ) and usability (SUS) significantly improved over the 2 year development process of these three studies (Graph 2).

Graph 2 Education and Deprivation Level
Deprivation scores -calculated using the Scotland Index of Multiple Deprivation [18] -and education level, did not significantly correlate with satisfaction, usability (SUS), quality (TUQ) or task load (NASA TLX) of the 3D system (Supplementary Table 7).

Harms
No harms occurred during 3D Telemedicine consultations.

3D Telemedicine: The future of remote consultations
This is the first study to examine and demonstrate the potential benefits of 3D Telemedicine in comparison to a 2D equivalent. Previous research has either existed in a research setting only, or has not proven any validated benefits of 3D Telemedicine over existing telemedicine technology [19]. Therefore, until now, the advantages of 3D Telemedicine have been purely speculative. This is also the first clinical system to employ Microsoft's Holoportation TM communication technology -a true 360 degree 3D system that fuses multiple depth cameras -with previous research only employing single depth cameras that do not permit a true 360 degree coverage [19,20].

Placing Patients at the Heart of the Development Process
Development of the 3D system has focused on medical communication from the ground-up, and has not been adapted from business or other user groups. The VR CORE Guidelines, akin to the Model for Assessment of Telemedicine guidelines, place participatory design at the heart of the development process [21]. Clinician and patient feedback testing allowed codevelopment of a system that was fit to context and user group. This incremental, methodical approach directly translated to improved outcome measures, as can be seen by improvements in Clinician satisfaction from 56.6% to 70.1%, and usability (SUS) from 61 (grade D) to 80.5 (grade A) over the development cycle (Graph 2). It is hoped that in using a patient-centered development approach, that this system will be both relevant and embraced by patients in future clinical practice.

3D Telemedicine increases the realism and satisfaction of the remote consultation
Patient feedback testing suggested a strong user preference (100% preferred 3D), satisfaction (p<0.0001), and closer alignment with an "in-person" consultation for the 3D Telemedicine system in comparison to 2D. The Single Item Presence scale asking the question "To which extent did you feel present in the virtual clinic, as if you were really there?" strongly favoured the 3D system (p=0.001). The Telehealth Usability Questionnaire, an overall measure of the quality of telemedicine, favoured the 3D system (p=0.0002).
Subjective interview was very positive, with patients in particular finding positioning their bodypart for examination much easier with the 3D system.

The importance of "presence" or realism in remote Clinical Consultations
Critically, this study points towards real-time 3D telemedicine bringing the remote consultation experience closer than ever towards a face-to-face consultation. This is one of the fundamental goals of telemedicine. Although patients still prefer a face-to-face consultation (higher UNN score/ satisfaction), 3D Telemedicine was rated significantly higher in terms of satisfaction and realism than 2D Telemedicine (Table 1). These study data also highlight the importance of 'presence' or realism of a Telemedicine system. Firstly, 'presence' correlates strongly with satisfaction of the clinical consultation (Graph 1).
Secondly, data from other groups suggests that 'presence' correlates with improved human performance. The more immersive experience provided by 3D Telemedicine may therefore increase the fluency and execution of the consultation [22], and patient satisfaction is seen as a key driver in healthcare systems [23]. Together these provide impetus and rationale to continue research into more immersive forms of Telemedicine.

Clinical Relevance of real-time 3D consultations in Plastic Surgery
Although testing indicated improved key metrics in patient satisfaction and presence or realism, these are relatively abstract terms that are difficult to translate into direct clinical benefits for patients. These potential benefits are numerous -for example, examining patients with flap reconstructions on the side or back of the body, conducting remote physiotherapy without requiring the patient to re-position multiple times, the ability to examine multiple joints from different angles, ease of positioning the patient with limited mobility who is unable to move for the camera, and avoiding having an intermediary to move a camera around the patient. Furthermore, the ability to draw on the 3D patient model allows the clinician to more accurately explain an operation on the actual patient's body, and in doing so provides a more 'personalised medicine' approach. Together, these advantages allow the clinician to conduct a higher fidelity, more fluent consultation than with a 2D system.

Safety and Clinical Concordance
Clinical concordance of the 3D system with in-person consultations (95% in this study) is equivalent or exceeds previous estimates for other surgical Telemedicine consultations.
These include 92% surgeons estimating that their telemedicine decisions are as good as face-to-face [24], and 8.9% of orthopaedic telemedicine patients requiring additional faceto-face consultations (compared to 2.5% in this study) [16]. These data come from nonpaired studies which are considerably less accurate in measuring concordance than the paired design used in the present study, due to the heterogenous nature of medical conditions.

Leave nobody behind in a digital world
The 3D system aimed to not "leave anybody behind in a digital world" and therefore the interface was made as simple as possible for patients. Technological equality and inclusion of the system was demonstrated by high usability of the system irrespective of patient deprivation or education levels (Supplementary Table 7).

Outcome measurement instruments
Given that the key goal of any Telemedicine system is to bring the remote consultation as close as possible to the experience of a face-to-face consultation, "presence" was deemed to be one of the primary research outcomes. Many scales measuring "presence" in digital environments were found to be too lengthy and confusing by patients [10,25].
Questionnaires that work with clinicians or university students will not necessarily translate adequately to patients, with 27% of the UK population having the lowest level of literacy [26]. The Single Item Presence question "To which extent did you feel present in the virtual clinic, as if you were really there?" was clear, simple and resonated strongly with patients [11]. This will therefore form the primary outcome for the follow-on RCT. The Telehealth Usability Questionnaire (TUQ) is a newer, validated questionnaire measuring subdomains of usefulness, ease of use, effectiveness, reliability and satisfaction, giving a metric of the overall quality of the telemedicine system [12]. Although it contains questions on generic topics that do not differentiate between technical systems, such as "Telehealth saves me travelling", it is clear to understand and provides a number of important metrics, and will therefore be used as a secondary outcome in the follow-on trial.

Limitations, Bias and Generalisation
Inherent limitations of these data include lack of randomisation and blinding. Use of validated, generic outcome scales minimizes information bias and allows generalisation of these data to other trials employing the same instruments.

Future Directions
A follow-on VR Phase 3 Randomised trial will provide definitive evidence of clinical benefits.
Concurrently, a test bed has been set up in Ghana to explore increasing access to reconstructive surgical care in LMIC. Health economics, health frameworks, and organisational change will be incorporated in future analyses. Further technical changes will focus on improving Clinician satisfaction by reducing Mental Effort and Task Load (supplementary Appendix 2). This may be aided by improving spatial processing of a 3D object through increased resolution, higher frame rate, reduced lag time, and use of more intuitive, natural interfaces such as the HoloLens [27].

Summary
This research details the participatory development of a 3D Telemedicine system, with the first validated comparative clinical use in patients worldwide. One of the ultimate goals of telemedicine is for the quality of remote consultations to get closer to the experience of face-to-face consultations. Together the data presented here points towards significant potential in bringing the remote consultation closer to this goal, which is of particular relevance to specialities with a strong 3D focus such as Plastic Surgery. In summary: 1) 3D Telemedicine increases the realism of the remote Clinical consultation, and is closer in experience to a face-to-face consultation than a 2D consultation.
2) "Presence" or realism correlates with satisfaction with the consultation. This is a strong driver to invest in technologies that increase the fidelity of telemedicine consultations.

Terminology
Presence, immersion and realism. "Presence" is an experiential, subjective and psychological quality in virtual environments, associated with feelings of "being there".
"Immersion" is associated with objective, technical aspects of virtual systems that help the user feel a sense of presence. "Realism" can be considered a dimension of "presence", with stronger realism leading to increased presence. For the purposes of this paper, which is aimed at a clinical audience, these terms are used interchangeably.        On the right is a standard Telemedicine video call. Note the difference in field of view and ability to position patient to see the right AKA.