Advertisement

Participatory Development of a 3D Telemedicine system during Covid: the future of remote consultations

Open AccessPublished:October 12, 2022DOI:https://doi.org/10.1016/j.bjps.2022.10.012

      Abstract

      Background

      The Covid pandemic brought the need for more realistic remote consultations into focus. 2D telemedicine solutions fail to replicate the fluency or authenticity of in-person consultations. This research reports on an international collaboration on the participatory development and first validated clinical use of a novel, real-time 360-degree 3D Telemedicine system worldwide. Development of the system - leveraging Microsoft's HoloportationTM communication technology – commenced at Canniesburn Plastic Surgery Unit, Glasgow in March 2020.

      Methods

      Research followed VR CORE guidelines on development of Digital Health trials, placing patients at the heart of the development process. This consisted of three separate studies - a clinician feedback study (23 clinicians, Nov-Dec 2020), a patient feedback study (26 patients, Jul-Oct 2021), and a cohort study focusing on safety and reliability (40 patients, Oct 2021 - Mar 2022). “Lose, Keep and Change” feedback prompts were used to engage patients in the development process and guide incremental improvements.

      Results

      Participatory testing demonstrated improved patient metrics with 3D in comparison to 2D Telemedicine, including validated measures of satisfaction (p<0.0001), realism or ‘presence’ (Single Item Presence scale, p<0.0001), and quality (Telehealth Usability Questionnaire, p=0.0002). Safety and clinical concordance (95%) of 3D Telemedicine with a face-to-face consultation were equivalent or exceeded estimates for 2D Telemedicine.

      Conclusions

      One of the ultimate goals of telemedicine is for the quality of remote consultations to get closer to the experience of face-to-face consultations. These data provide the first evidence that HoloportationTM communication technology brings 3D telemedicine closer to this goal than a 2D equivalent.

      Keywords

      BACKGROUND

      Real-time 3D Telemedicine has previously been proposed within a research setting only, with constraints on cost, complexity, bandwidth and technology [
      • Wang G.
      • Xiang W.
      • Pickering M.
      A cross-platform solution for light field based 3D telemedicine.
      ,
      • Stranieri A.
      • Collmann R.
      • Borda A.
      High definition 3D telemedicine: the next frontier?.
      ]. With the Covid Pandemic, use of remote consultation has increased exponentially and has brought the concept of a 3D consultation into focus [
      • McCabe H.M.
      • et al.
      What Matters to Us: Impact of Telemedicine During the Pandemic in the Care of Patients With Sarcoma Across Scotland.
      ]. Although there appears to be significant theoretical value in assessing surgical patients, as yet no validated clinical data exists on the benefits of such a system in comparison to standard 2D Telemedicine. Here we detail the participatory development of a real-time 3D Telemedicine system, in conjunction with an international group of healthcare and industrial stakeholders. We also discuss the first real-world use of a 3D Telemedicine system with Plastic Surgery patients.

      Increasing access to care in Lower to Middle Income Countries

      Initial discussions regarding the development of a 3D Telemedicine system, leveraging Microsoft's Holoportation communication technology, commenced in December 2019 - between Canniesburn Plastic Surgery Unit, Glasgow, UK; Korle Bu Teaching Hospital, Accra, Ghana; and Microsoft Corporation, Redmond, USA. This centered around the potential for increasing access to specialised reconstructive surgical care in Lower to Middle Income countries (LMIC). The research team visited the Ministry of Health, Ghana, in February 2020 to initiate an international collaboration on this project. Geospatial mapping set out the early vision, using census data and overland travel times to estimate increased access to reconstructive care [
      • Lo S.
      • et al.
      Quantifying Inequitable Access to Rapid Burn and Reconstructive Care through Geospatial Mapping.
      ]. With the global Covid pandemic enforcing the first UK lockdown in March 2020, the focus of the project rapidly pivoted to potential delivery of remote care during Covid.

      VR CORE Guidelines on developing Digital Health technologies

      The 3D Telemedicine system was co-developed with patients using international guidelines for digital or Virtual Reality (VR) clinical trials, as proposed by the Virtual Reality Clinical Outcomes Research Experts (VR CORE). These guidelines aim to improve methodological quality in digital health technology trials, dividing development into three phases (VR 1 to 3), akin to Clinical Trial Phases 1 to 3 [
      • Birckhead B.
      • et al.
      Recommendations for Methodology of Virtual Reality Clinical Trials in Health Care by an International Working Group: Iterative Study.
      ]. Participatory development is one of the key elements of these guidelines, focusing on the principles of human-centered design. The importance of this in digital trials has been previously underestimated, as “lack of patient involvement, poor requirement definitions, and non-adaptation to user feedback are some of the common factors that explain failures of digital interventions” [
      • Birckhead B.
      • et al.
      Recommendations for Methodology of Virtual Reality Clinical Trials in Health Care by an International Working Group: Iterative Study.
      ]. The present study placed patients at the heart of the development process and reports on the Participatory development (VR 1) and early Clinical Trial (VR 2) phases. The objectives of this research were to co-develop a patient-centered 3D Telemedicine system, compare validated outcome measures with a 2D system, assess alignment with an in-person consultation, and to ensure safety, reliability and clinical concordance.

      METHODS AND RESULTS

      Ethics

      NHS Greater Glasgow and Clyde (GGC) R and I approvals GN20HS181/ GN20HS300. NHS GGC governance meetings monitored the project bi-annually. Participants consented in writing. Patient data controlled by NHS GGC. STROBE guidelines followed for reporting of cohort trials.

      Approach and Preliminary Work

      Research followed the VR CORE guidelines [
      • Birckhead B.
      • et al.
      Recommendations for Methodology of Virtual Reality Clinical Trials in Health Care by an International Working Group: Iterative Study.
      ] and consisted of preliminary work including focus groups, stakeholder collaborations, equality assessments, and initial prototyping. Weekly collaboration meetings were held over a 2-year period in UK, Ghana and USA, commencing March 2020. To inform prototype development - a focus group was held with 23 clinicians from Canniesburn Plastic Surgery Unit, Glasgow, UK in June 2020 - exploring desired functions, identification of needs, potential benefits, risks, use during Covid, and implementation (Supplementary Table 1). An Equality Impact Assessment (EQIA) [] focused on “not leaving anybody behind in a digital world” [

      Hernandez, K. and T. Roberts. Leaving No One Behind in a Digital World. 2018.

      ], and collected data on factors that may influence access or use of novel technology, such as deprivation and educational level. This was followed by three separate studies: a clinician feedback study – providing feedback to incrementally improve and shape the 3D Telemedicine sytem prior to patient testing; a patient feedback study – to compare 3D and 2D Telemedicine systems; and a cohort study focusing on safety, reliability and clinical concordance of 3D with face-to-face examination (Flowchart 1).
      Flowchart 1

      Prototyping and Initial Set up

      Initial system set-up took place in September 2020, when a research team from Microsoft Corporation, Redmond, USA travelled to Glasgow, UK. The system – inspired by Microsoft's HoloportationTM research [

      Corporation, M. Holoportation. 2022 [Accessed 9th May 2022]; Available from: https://www.microsoft.com/en-us/research/project/holoportation-3/

      ] - consisted of an array of 10 Azure Kinect cameras connected to a Fusion server that fuses each camera's depth output to create a 3D 360-degree model, and a Render server that covers the model in RGB video output. This was linked to a “viewer” room where the patient could be viewed in 360-degrees on a computer screen over the existing hospital network (Figures 1-5, Supplementary Videos 1-3). The “viewer” room was set up in both the test site hospital (West Glasgow Ambulatory Care Hospital, WGACH) and remotely at Canniesburn Plastic Surgery Unit. The WGACH viewer site was used in the present studies (full system features - Supplementary Table 1).
      Figure 1:
      Figure 13D Rig Set up. Multiple Kinect cameras surround the patient within the clinic room. In the centre is a chequerboard used to calibrate the system.
      Figure 2:
      Figure 2Sensate Anterolateral Thigh (ALT) Flap to Above knee amputation (AKA). This patient required resurfacing of an area of skin grafted post-traumatic residual limb, that provided a poor interface with the prosthetic limb. A sensate ALT flap was used to resurface the residual limb.
      Figure 3:
      Figure 3Patient in 3D rig. The same patient with resurfaced AKA sitting in the 3D Telemedicine system. The screen allows him to view the same images as the clinician.
      Figure 4:
      Figure 4Clinician Viewer room. The clinician can see the patient in 3D on the left screen. On the right is a standard Telemedicine video call. Note the difference in field of view and ability to position patient to see the right AKA.
      Figure 5:
      Figure 5Multiple 3D views. These images demonstrate the 3D camera output in real time.
      Supplementary Videos 1-3

      VR PHASE 1: PARTICIPATORY CO-DEVELOPMENT

      Patient and Clinician Participatory Development

      Patient and clinician feedback was used to shape the development of the 3D Telemedicine system, assess usability of the outcome instruments, and provide data for follow-on trials. Sample sizes were not required for the participatory development component of this study. Data analysis carried out by a blinded, independent statistical service using R version 3.4.1 [
      Team, R.C.
      R: A language and environment for statistical computing.
      ].

      Study 1: Clinician Feedback Study : Iterative improvements in 3D system

      Clinician feedback testing focused on optimisation of the prototype system prior to patient testing. Full details are provided in Supplementary Appendix 1. In brief, 23 clinicians were assessed in two batches in November and December 2020, to provide incremental improvements in usability, and assess responsiveness to change between batches. Key findings included a strong correlation between quality of consultation (satisfaction) and the realism or “presence” of the 3D system (Pearson r = 0.8451, p<0.0001, Graph 1). The 3D system was found to have a significantly higher realism or “presence” than the 2D system in batch 2. Improvements were instituted in response to clinician feedback, prior to patient testing (Supplementary Table 1).
      Graph 1:
      Graph 1Correlation of Presence with Satisfaction. Clinician Satisfaction correlation with Presence Questionnaire Score

      Study 2: Patient Feedback Study: 3D versus 2D Telemedicine

      Method

      Patient feedback testing provided the first real-world data on patients’ perceptions of the 3D Telemedicine system in comparison to 2D Telemedicine. A preliminary cohort of six patients was used to test acceptability of the outcome instruments. Longer form scales such as the Presence Questionnaire with 29 items [
      • Witmer B.
      • Jerome C.
      • Singer M.
      The Factor Structure of the Presence Questionnaire.
      ] were found to be too long and confusing for patients, and therefore short forms of “presence” testing were considered, with final selection of the validated Single Item Presence scale [

      Bouchard, S., et al., Reliability and validity of a single-item measure of presence in VR. 2004. 59-61.

      ].

      Participants

      26 patients from Canniesburn Plastic Surgery Unit clinic participated (July - Oct 2021) (Supplementary Table 2). Patients were seen in 3D and 2D Telemedicine by a single clinician, without randomisation (11 patients 2D first, 15 3D first). Each consultation lasted 10 minutes, followed by completion of questionnaires.

      Outcome Instruments

      Single Item Presence scale asked the key question “To which extent did you feel present in the virtual clinic, as if you were really there?” to provide insights into how closely the 3D clinic aligns with an in-person consultation [
      • Parmanto B.
      • et al.
      Development of the Telehealth Usability Questionnaire (TUQ).
      ]. Telehealth Usability Questionnaire (TUQ) consists of 21 items covering subdomains of usefulness, ease of use, effectiveness, reliability and satisfaction [
      • Parmanto B.
      • et al.
      Development of the Telehealth Usability Questionnaire (TUQ).
      ]. Satisfaction measured with 0-100 visual analogue scale [
      • Voutilainen A.
      • et al.
      How to ask about patient satisfaction? The visual analogue scale is less vulnerable to confounding factors and ceiling effect than a symmetric Likert scale.
      ]. System Usability Scale (SUS) measured usability with an industry standard scale that allows comparison across different technologies [
      • Brooke J.
      SUS: A “quick and dirty” usability scale., in Usability Evaluation in Industry.
      ]. “Lose, Keep and Change” feedback prompts used to engage patients in the development process and guide incremental improvements [
      • Frohlich D.M.
      • Lim C.S.C.
      • Ahmed A.
      Keep, lose, change: Prompts for the re-design of product concepts in a focus group setting.
      ].

      Results

      3D versus 2D Telemedicine validated outcome measures

      3D was rated higher on satisfaction (p<0.0001), presence (p<0.0001), and quality (TUQ, p=0.0002) (Boxplot 1). Mental effort rating scale was equivalent (P=0.2965), scoring highly in both systems suggesting that the 3D system is not more complex to use than normal telemedicine systems. Patients’ subjective views on accuracy of diagnosis and ease of positioning their body part for examination were also significantly higher (Table 1). 100% patients preferred 3D (for patient comments see Supplementary Table 3) and usability was rated highly (SUS 87.02).
      Table 1Patient Feedback Study – 3D versus 2D Outcome Measures. Validated outcome measures included satisfaction, Mental Effort Rating Scale, Single Item Presence Scale, System Usability Scale and Telehealth Usability Questionnaire. Non-validated outcome measures included patients’ subjective views on ability of clinician to make an accurate diagnosis, and ease of positioining their body part for examination.
      Category range3D2DMean differenceSignificance
      Satisfaction0-10088.23 (CI 85.21, 91.26)51.35 (CI 43.09, 59.60)36.88 (CI 28.73-45.04)p<0.0001
      dependent t test
      Mental Effort Rating Scale1-9 (lower is better)2.038 (CI 1.377, 2.699)2.462 (CI 1.685, 3.238)0.42 (CI -0.39- 1.24)p=0.2965
      dependent t test
      Single Item Presence Scale0-10080 (CI 74.9, 85.1)52.58 (CI 44.15, 61.0)27.42 (CI 17.24-37.61)p<0.0001
      dependent t test
      Telehealth Usability Quesionnaire0-100
      Raw maximum score of 147 converted to a score out of 100 CI – 95% confidence intervals
      85.31 (CI 80.61, 89.93)76.94 (CI 71.36, 82.52)-8.35 (CI -12,24, -4.45)p=0.0002
      dependent t test
      System Usability Scale (SUS)0-10087.02 (CI 81.69, 92.35)N/AN/AN/A
      Accuracy of diagnosis1-5 (higher is better)4.13 (CI 3.77, 4.50)3.40 (CI 2.94, 3.86)-0.73 (CI -1.20, -0.26)p=0.0254
      dependent t test
      Ease of positioning body part1-54.53 (CI 4.17, 4.90)3.60 (CI 3.01, 4.19)-0.93 (CI -1.49, -0.38)p=0.0157
      dependent t test
      low asterisk dependent t test
      + Raw maximum score of 147 converted to a score out of 100CI – 95% confidence intervals

      Feedback Process and Subjective interview

      “Lose, Change and Keep” prompts noted limitations related predominantly to quality of 3D spatial resolution in real-time (Supplementary Table 1). Subjective interview was overwhelmingly positive, with patients in particular finding that the 3D system allowed positioning their body part for examination much easier than with 2D (data not shown) (Supplementary Video 4).

      Carry-over Effects

      Between-group comparisons of satisfaction, presence and TUQ were done using a general linear model which included subject, group and order factors (3D or 2D system first, to determine any effect of carry-over). No effect of carry-over was noted for these outcome measures (data not shown).

      VR PHASE 2: EARLY CLINICAL EFFICACY AND SAFETY

      Study 3: Cohort Study: 3D Telemedicine versus face-to-face

      Method

      The cohort study aimed to assess safety and reliability. A paired design was used with patients seen in 3D Telemedicine first, followed by face-to-face, by the same clinician to allow assessment of clinical concordance. Ten minutes were given per examination. Clinicians and patients both completed questionnaires for each consultation.

      Sample size

      Pilot study with recruitment of 40 patients. Primary and secondary outcomes not designated for pilot studies. Post-study power calculations indicated that with a paired design, 16 patients would have been sufficient (using primary outcome with University of North Norway UNN score as primary outcome; delta 0.4; SD 0.52; power 0.9)

      Participants

      40 patients and 10 clinicians (5 residents, 2 nurse specialists, 1 physiotherapist, 2 consultants; mean age 37.5; range 28-43; 7F, 3M) from Canniesburn Plastic Surgery Unit participated (Oct 2021 – Mar 2022, Flowchart 2). For inclusion criteria and patient demographics – see Supplementary Tables 4 & 5.
      Flowchart 2

      Outcome Instruments

      UNN scale, validated for orthopaedic patients, is one of the only scales available designed to assess quality of both telemedicine and face-to-face consultations [
      • Buvik A.
      • et al.
      Quality of care for remote orthopaedic consultations using telemedicine: a randomised controlled trial.
      ]. NASA TLX is a measure of the workload or “task load” of an activity - assessing subdomains of mental demand, physical demand, temporal demand, performance, effort and frustration [

      Hart, S.G. and L.E. Staveland, Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research, in Advances in Psychology, P.A. Hancock and N. Meshkati, Editors. 1988, North-Holland. p. 139-183.

      ].

      Results

      Operational and Clinical Safety

      Operational safety issues occurred in 0%, technical issues 20%, reliability issues 7.5%, and image artefacts 12.5%. All issues were temporary and did not curtail the clinical consultation. Clinical concordance of 3D consultations with face-to-face management plan was 95%. 2 cases of clinical discrepancy did not result in clinical safety concerns (Supplementary Table 6).

      3D versus Face-to-Face Validated Outcome measures

      Clinicians’ UNN Sum score was significantly higher for a face-to-face than for 3D consultation (Table 2). Nonetheless, the difference in sum score between 3D and face-to-face was only 0.4, with both consultation types approximating to scores between “good” and “very good”. The minimal clinically important difference (MCID) has not yet been established for UNN score. The UNN authors suggested a difference of 0.3 as a non-inferiority limit in their study, but this appears to be arbitrary and not based on clinical data. Patients rated the face-to-face higher on satisfaction than 3D, mean 90.08 versus 83.58 respectively (mean difference 6.50 [CI 2.40-10.60]; p=0.0027; paired t test).
      Table 2Cohort Study – 3D Telemedicine versus Face-to-Face UNN Score. The UNN score is subdivided into subdomains and co-operation, examination, treatment, informing patient and overall.
      3D TelemedicineFace-to-FaceMean differenceSignificance
      paired t tests used for analysis
      Patient co-operation
      rated from 1-5 where 1=very good, 2= good, 3= neither good nor bad, 4=bad, and 5 = very bad
      1.05 (CI 0.98-1.12)1.03 (CI 0.98-1.08)-0.03 (CI -0.08, 0.03)P=0.33
      Evaluate/examine patient
      rated from 1-5 where 1=very good, 2= good, 3= neither good nor bad, 4=bad, and 5 = very bad
      1.80 (CI 1.55, 2.05)1.03 (CI 0.97, 1.08)-0.78 (CI -1.03, -0.52)P<0.0001
      Treat patient
      rated from 1-5 where 1=very good, 2= good, 3= neither good nor bad, 4=bad, and 5 = very bad
      1.79 (CI 1.46,2.11)1.11 (CI 0.99,1.23)-0.68 (CI -1.00,-0.36)P=0.0002
      Inform patient
      rated from 1-5 where 1=very good, 2= good, 3= neither good nor bad, 4=bad, and 5 = very bad
      1.30 (CI 1.07,1.53)1.18 (CI 1.03, 1.32)-0.13 (CI -0.39, 0.14)P=0.34
      Overall
      rated from 1-5 where 1=very good, 2= good, 3= neither good nor bad, 4=bad, and 5 = very bad
      1.70 (CI 1.45, 1.95)1.18 (CI 1.03, 1.32)-0.58 (CI -0.85, -0.31)P=0.0001
      Sum score1.50 (CI 1.33-1.68)1.10 (CI 1.03-1.16)-0.40 (CI -0.57, -0.24)<0.0001
      low asterisk rated from 1-5 where 1=very good, 2= good, 3= neither good nor bad, 4=bad, and 5 = very bad
      + paired t tests used for analysis

      Differences in Clinician versus Patient outcome measures for 3D system

      Patients rated the 3D system higher on satisfaction than clinicians (p<0.0001). This may be related to differences in mental effort (p=0.0042) and task load (NASA TLX, p=0.020). Usability (SUS) and quality scores (TUQ) were equivalent (Table 3).
      Table 3Cohort study – Clinicians versus Patient outcome measures of the 3D Telemedicine system only
      Category rangeClinicians (n=10)Patients (n=40)Mean differenceSignificance
      Satisfaction0-10070.08 (CI 64.84-75.31)83.58 (CI 79.07-88.08)13.50 (CI 7.67, 19.33)P<0.0001 (paired t test)
      Mental Effort Rating Scale1-9 (lower is better)2.85 (CI 2.34-3.37)1.86 (CI 1.37-2.28)-1.025 (CI -1.71, -0.34)P=0.0042 (paired t test)
      NASA TLX Raw score0-100
      converted to a 0-100 scale
      (lower is better)
      19.08 (CI 14.22, 23.93)12.20 (CI 8.40, 16.00)-6.88 (CI -12.60, -1.16)p=0.020 (paired t test)
      System Usability Scale (SUS)0-10080.5 (CI 68.85-92.15)
      completed only once at end of study by each clinician, not per patient consultation (n=10). Comparative statistical analysis is therefore unpaired
      81.88 (CI 76.83-86.92)1.38 (CI -9.91, -12.66)P=0.8075 (unpaired t test)
      Telehealth Usability Questionnaire (TUQ)0-100
      converted to a 0-100 scale
      84.95 (CI 78.58, 91.32)
      completed only once at end of study by each clinician, not per patient consultation (n=10). Comparative statistical analysis is therefore unpaired
      82.00 (CI 77.48, 86.52)-2.95 (CI -12.43, 6.52)P=0.5341 (unpaired t test)
      Presence Questionnaire (PQ)0-100
      converted to a 0-100 scale
      75.32 (CI 70.38, 80.26)
      completed only once at end of study by each clinician, not per patient consultation (n=10). Comparative statistical analysis is therefore unpaired
      not included in patient questionnaires as overly long for inclusion
      N/AN/A
      low asterisk completed only once at end of study by each clinician, not per patient consultation (n=10). Comparative statistical analysis is therefore unpaired
      low asterisklow asterisk not included in patient questionnaires as overly long for inclusion
      + converted to a 0-100 scale

      Ancillary Analyses

      Improvements over time

      Clinician satisfaction, presence (PQ) and usability (SUS) significantly improved over the 2 year development process of these three studies (Graph 2).
      Graph 2:
      Graph 2Improvements in outcomes during development process. Outcome measurements improved significantly during incremental feedback and development, for ratings of satisfaction, presence (PQ) and usability (SUS) over the development process. 2020 refers to scores from the Clinician Feedback Study Batch 1, 2021 refers to Clinician Feedback Study Batch 2, and 2022 to the Cohort study clinician scores. PQ is converted to a 100 point scale for this graph. 95% CI bars shown. Comparison of 2020 with 2022 scores with unpaired t-test: Satisfaction (p=0.026), PQ (p=0.021), SUS (p=0.017).
      Boxplot 1:
      Boxplot 13D versus 2D patient outcome measures. Satisfaction, Presence (Single Item Presence Scale) and Quality (TUQ) all significantly higher for the 3D system. All scores converted to 0-100 scale.

      Graph 2

      Education and Deprivation Level

      Deprivation scores - calculated using the Scotland Index of Multiple Deprivation [

      Government, S. Scottish Index of Multiple Deprivation 2020 v2 postcode lookup file. 2020 [Accessed 5th June 2022]; Available from: https://www.gov.scot/publications/scottish-index-of-multiple-deprivation-2020v2-postcode-look-up/.

      ] - and education level, did not significantly correlate with satisfaction, usability (SUS), quality (TUQ) or task load (NASA TLX) of the 3D system (Supplementary Table 7).

      Harms

      No harms occurred during 3D Telemedicine consultations.

      Discussion

      3D Telemedicine: The future of remote consultations

      This is the first study to examine and demonstrate the potential benefits of 3D Telemedicine in comparison to a 2D equivalent. Previous research has either existed in a research setting only, or has not proven any validated benefits of 3D Telemedicine over existing telemedicine technology [
      • Sekimoto S.
      • et al.
      Holomedicine: Proof of the Concept of Interactive Three-Dimensional Telemedicine.
      ]. Therefore, until now, the advantages of 3D Telemedicine have been purely speculative. This is also the first clinical system to employ Microsoft's HoloportationTM communication technology - a true 360 degree 3D system that fuses multiple depth cameras - with previous research only employing single depth cameras that do not permit a true 360 degree coverage [
      • Sekimoto S.
      • et al.
      Holomedicine: Proof of the Concept of Interactive Three-Dimensional Telemedicine.
      ,

      Garcia, M. Innovative 3D Telemedicine to Help Keep Astronauts Healthy. 2022 [Accessed 29th May 2022]; Available from: https://www.nasa.gov/feature/innovative-3d-telemedicine-to-help-keep-astronauts-healthy.

      ].

      Placing Patients at the Heart of the Development Process

      Development of the 3D system has focused on medical communication from the ground-up, and has not been adapted from business or other user groups. The VR CORE Guidelines, akin to the Model for Assessment of Telemedicine guidelines, place participatory design at the heart of the development process [
      • Kidholm K.
      • et al.
      A model for assessment of telemedicine applications: mast.
      ]. Clinician and patient feedback testing allowed co-development of a system that was fit to context and user group. This incremental, methodical approach directly translated to improved outcome measures, as can be seen by improvements in Clinician satisfaction from 56.6% to 70.1%, and usability (SUS) from 61 (grade D) to 80.5 (grade A) over the development cycle (Graph 2). It is hoped that in using a patient-centered development approach, that this system will be both relevant and embraced by patients in future clinical practice.

      3D Telemedicine increases the realism and satisfaction of the remote consultation

      Patient feedback testing suggested a strong user preference (100% preferred 3D), satisfaction (p<0.0001), and closer alignment with an “in-person” consultation for the 3D Telemedicine system in comparison to 2D. The Single Item Presence scale asking the question “To which extent did you feel present in the virtual clinic, as if you were really there?” strongly favoured the 3D system (p=0.001). The Telehealth Usability Questionnaire, an overall measure of the quality of telemedicine, favoured the 3D system (p=0.0002). Subjective interview was very positive, with patients in particular finding positioning their bodypart for examination much easier with the 3D system.

      The importance of “presence” or realism in remote Clinical Consultations

      Critically, this study points towards real-time 3D telemedicine bringing the remote consultation experience closer than ever towards a face-to-face consultation. This is one of the fundamental goals of telemedicine. Although patients still prefer a face-to-face consultation (higher UNN score/ satisfaction), 3D Telemedicine was rated significantly higher in terms of satisfaction and realism than 2D Telemedicine (Table 1). These study data also highlight the importance of ‘presence’ or realism of a Telemedicine system. Firstly, ‘presence’ correlates strongly with satisfaction of the clinical consultation (Graph 1). Secondly, data from other groups suggests that ‘presence’ correlates with improved human performance. The more immersive experience provided by 3D Telemedicine may therefore increase the fluency and execution of the consultation [
      • Kincaid J.S.a.J.
      The relationship between presence and performance in virtual simulation training.
      ], and patient satisfaction is seen as a key driver in healthcare systems [
      • Manary M.P.
      • et al.
      The patient experience and health outcomes.
      ]. Together these provide impetus and rationale to continue research into more immersive forms of Telemedicine.

      Clinical Relevance of real-time 3D consultations in Plastic Surgery

      Although testing indicated improved key metrics in patient satisfaction and presence or realism, these are relatively abstract terms that are difficult to translate into direct clinical benefits for patients. These potential benefits are numerous - for example, examining patients with flap reconstructions on the side or back of the body, conducting remote physiotherapy without requiring the patient to re-position multiple times, the ability to examine multiple joints from different angles, ease of positioning the patient with limited mobility who is unable to move for the camera, and avoiding having an intermediary to move a camera around the patient. Furthermore, the ability to draw on the 3D patient model allows the clinician to more accurately explain an operation on the actual patient's body, and in doing so provides a more ‘personalised medicine’ approach. Together, these advantages allow the clinician to conduct a higher fidelity, more fluent consultation than with a 2D system.

      Safety and Clinical Concordance

      Clinical concordance of the 3D system with in-person consultations (95% in this study) is equivalent or exceeds previous estimates for other surgical Telemedicine consultations. These include 92% surgeons estimating that their telemedicine decisions are as good as face-to-face [
      • Aarnio P.
      • et al.
      A prospective study of teleconferencing for orthopaedic consultations.
      ], and 8.9% of orthopaedic telemedicine patients requiring additional face-to-face consultations (compared to 2.5% in this study) [
      • Buvik A.
      • et al.
      Quality of care for remote orthopaedic consultations using telemedicine: a randomised controlled trial.
      ]. These data come from non-paired studies which are considerably less accurate in measuring concordance than the paired design used in the present study, due to the heterogenous nature of medical conditions.

      Leave nobody behind in a digital world

      The 3D system aimed to not “leave anybody behind in a digital world” and therefore the interface was made as simple as possible for patients. Technological equality and inclusion of the system was demonstrated by high usability of the system irrespective of patient deprivation or education levels (Supplementary Table 7).

      Outcome measurement instruments

      Given that the key goal of any Telemedicine system is to bring the remote consultation as close as possible to the experience of a face-to-face consultation, “presence” was deemed to be one of the primary research outcomes. Many scales measuring “presence” in digital environments were found to be too lengthy and confusing by patients [
      • Witmer B.
      • Jerome C.
      • Singer M.
      The Factor Structure of the Presence Questionnaire.
      ,
      • Slater M.
      • Usoh M.
      • Steed A.
      Depth of Presence in Virtual Environments.
      ]. Questionnaires that work with clinicians or university students will not necessarily translate adequately to patients, with 27% of the UK population having the lowest level of literacy [

      Trust, N.L. Adult Literacy. 2021 [Accessed 5th June 2021]; Available from: https://literacytrust.org.uk/parents-and-families/adult-literacy/.

      ]. The Single Item Presence question “To which extent did you feel present in the virtual clinic, as if you were really there?” was clear, simple and resonated strongly with patients [

      Bouchard, S., et al., Reliability and validity of a single-item measure of presence in VR. 2004. 59-61.

      ]. This will therefore form the primary outcome for the follow-on RCT. The Telehealth Usability Questionnaire (TUQ) is a newer, validated questionnaire measuring subdomains of usefulness, ease of use, effectiveness, reliability and satisfaction, giving a metric of the overall quality of the telemedicine system [
      • Parmanto B.
      • et al.
      Development of the Telehealth Usability Questionnaire (TUQ).
      ]. Although it contains questions on generic topics that do not differentiate between technical systems, such as “Telehealth saves me travelling”, it is clear to understand and provides a number of important metrics, and will therefore be used as a secondary outcome in the follow-on trial.

      Limitations, Bias and Generalisation

      Inherent limitations of these data include lack of randomisation and blinding. Use of validated, generic outcome scales minimizes information bias and allows generalisation of these data to other trials employing the same instruments.

      Future Directions

      A follow-on VR Phase 3 Randomised trial will provide definitive evidence of clinical benefits. Concurrently, a test bed has been set up in Ghana to explore increasing access to reconstructive surgical care in LMIC. Health economics, health frameworks, and organisational change will be incorporated in future analyses. Further technical changes will focus on improving Clinician satisfaction by reducing Mental Effort and Task Load (supplementary Appendix 2). This may be aided by improving spatial processing of a 3D object through increased resolution, higher frame rate, reduced lag time, and use of more intuitive, natural interfaces such as the HoloLens [
      • Jeffri N.F.S.
      • Awang Rambli D.R.
      A review of augmented reality systems and their effects on mental workload and task performance.
      ].

      Summary

      This research details the participatory development of a 3D Telemedicine system, with the first validated comparative clinical use in patients worldwide. One of the ultimate goals of telemedicine is for the quality of remote consultations to get closer to the experience of face-to-face consultations. Together the data presented here points towards significant potential in bringing the remote consultation closer to this goal, which is of particular relevance to specialities with a strong 3D focus such as Plastic Surgery. In summary:
      • 1)
        3D Telemedicine increases the realism of the remote Clinical consultation, and is closer in experience to a face-to-face consultation than a 2D consultation.
      • 2)
        “Presence” or realism correlates with satisfaction with the consultation. This is a strong driver to invest in technologies that increase the fidelity of telemedicine consultations.
      • 3)
        3D Telemedicine appears to significantly increase patient satisfaction in comparison to a standard 2D consultation.
      • 4)
        Safety and clinical concordance are acceptable for 3D Telemedicine - equivalent or exceeding estimates for 2D Telemedicine

      Funding

      Medical Research Scotland CVG-1742-2020, Global Challenges Research Fund  SFC1236-105, Jean Brown Bequest, NHS GGC endowments Covid Fund.

      Ethical Approval

      NHS Greater Glasgow and Clyde Research and Innovation board approvals GN20HS181/GN20HS300. ClinicalTrials.gov identifiers NCT05267197/NCT04444323. Designated a Covid priority study by the Health Research Authority, UK. All patient data controlled by NHS GGC.

      Terminology

      Presence, immersion and realism. “Presence” is an experiential, subjective and psychological quality in virtual environments, associated with feelings of “being there”. “Immersion” is associated with objective, technical aspects of virtual systems that help the user feel a sense of presence. “Realism” can be considered a dimension of “presence”, with stronger realism leading to increased presence. For the purposes of this paper, which is aimed at a clinical audience, these terms are used interchangeably.

      Patient Consents

      Patient consents were obtained for publication of identifiable photographs and videos

      Conflict of Interest

      The authors have no disclosures.

      Acknowledgements

      An extended research team included many others at the West of Scotland NHS Innovations Hub, UK; Korle Bu Hospital, Ghana; and Microsoft Corporation, Redmond, USA.
      Presented at: BBC News 13th July 2022 and BBC Good Morning Scotland Radio 13th July 2022; American Society of Plastic Surgeons meeting, Atlanta, USA October 2021; British Association Plastic and Reconstructive Surgeons, Celtic Meeting, Dunblane September 2021; Future Surgery Show, London, November 2021.

      Appendix. Supplementary materials

      References

        • Wang G.
        • Xiang W.
        • Pickering M.
        A cross-platform solution for light field based 3D telemedicine.
        Comput Methods Programs Biomed. 2016; 125: 103-116
        • Stranieri A.
        • Collmann R.
        • Borda A.
        High definition 3D telemedicine: the next frontier?.
        Stud Health Technol Inform. 2012; 182: 133-141
        • McCabe H.M.
        • et al.
        What Matters to Us: Impact of Telemedicine During the Pandemic in the Care of Patients With Sarcoma Across Scotland.
        JCO Glob Oncol. 2021; 7: 1067-1073
        • Lo S.
        • et al.
        Quantifying Inequitable Access to Rapid Burn and Reconstructive Care through Geospatial Mapping.
        Plast Reconstr Surg Glob Open. 2020; 8: e3069
        • Birckhead B.
        • et al.
        Recommendations for Methodology of Virtual Reality Clinical Trials in Health Care by an International Working Group: Iterative Study.
        JMIR Ment Health. 2019; 6: e11973
      1. GGC, N. Equality Impact Assessments. 2022 [Accessed 7th June 2022]; Available from: https://www.nhsggc.scot/your-health/equalities-in-health/meeting-the-requirements-of-equality-legislation/equality-impact-assessments/.

      2. Hernandez, K. and T. Roberts. Leaving No One Behind in a Digital World. 2018.

      3. Corporation, M. Holoportation. 2022 [Accessed 9th May 2022]; Available from: https://www.microsoft.com/en-us/research/project/holoportation-3/

        • Team, R.C.
        R: A language and environment for statistical computing.
        R Foundation for Statistical Computing, Vienna, Austria2020
        • Witmer B.
        • Jerome C.
        • Singer M.
        The Factor Structure of the Presence Questionnaire.
        Presence. 2005; 14: 298-312
      4. Bouchard, S., et al., Reliability and validity of a single-item measure of presence in VR. 2004. 59-61.

        • Parmanto B.
        • et al.
        Development of the Telehealth Usability Questionnaire (TUQ).
        International journal of telerehabilitation. 2016; 8: 3-10
        • Voutilainen A.
        • et al.
        How to ask about patient satisfaction? The visual analogue scale is less vulnerable to confounding factors and ceiling effect than a symmetric Likert scale.
        J Adv Nurs. 2016; 72: 946-957
        • Brooke J.
        SUS: A “quick and dirty” usability scale., in Usability Evaluation in Industry.
        Taylor and Francis., London1996: 189-194 (B.T. P. W. Jordan, B. A. Weerdmeester, & A. L. McClelland Editor)
        • Frohlich D.M.
        • Lim C.S.C.
        • Ahmed A.
        Keep, lose, change: Prompts for the re-design of product concepts in a focus group setting.
        CoDesign. 2014; 10: 80-95
        • Buvik A.
        • et al.
        Quality of care for remote orthopaedic consultations using telemedicine: a randomised controlled trial.
        BMC Health Serv Res. 2016; 16: 483
      5. Hart, S.G. and L.E. Staveland, Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research, in Advances in Psychology, P.A. Hancock and N. Meshkati, Editors. 1988, North-Holland. p. 139-183.

      6. Government, S. Scottish Index of Multiple Deprivation 2020 v2 postcode lookup file. 2020 [Accessed 5th June 2022]; Available from: https://www.gov.scot/publications/scottish-index-of-multiple-deprivation-2020v2-postcode-look-up/.

        • Sekimoto S.
        • et al.
        Holomedicine: Proof of the Concept of Interactive Three-Dimensional Telemedicine.
        Mov Disord. 2020; 35: 1719-1720
      7. Garcia, M. Innovative 3D Telemedicine to Help Keep Astronauts Healthy. 2022 [Accessed 29th May 2022]; Available from: https://www.nasa.gov/feature/innovative-3d-telemedicine-to-help-keep-astronauts-healthy.

        • Kidholm K.
        • et al.
        A model for assessment of telemedicine applications: mast.
        Int J Technol Assess Health Care. 2012; 28: 44-51
        • Kincaid J.S.a.J.
        The relationship between presence and performance in virtual simulation training.
        Open Journal of Modelling and Simulation. 2015; 3: 41-48
        • Manary M.P.
        • et al.
        The patient experience and health outcomes.
        N Engl J Med. 2013; 368: 201-203
        • Aarnio P.
        • et al.
        A prospective study of teleconferencing for orthopaedic consultations.
        J Telemed Telecare. 1999; 5: 62-66
        • Slater M.
        • Usoh M.
        • Steed A.
        Depth of Presence in Virtual Environments.
        Presence. 1994; 3: 130-144
      8. Trust, N.L. Adult Literacy. 2021 [Accessed 5th June 2021]; Available from: https://literacytrust.org.uk/parents-and-families/adult-literacy/.

        • Jeffri N.F.S.
        • Awang Rambli D.R.
        A review of augmented reality systems and their effects on mental workload and task performance.
        Heliyon. 2021; 7: e06277