Comparing the Everi Labs AI Copilot and Paper Documentation: Task Time and Usability of a P&O Patient Assessment UI Prototype

Sammanfattning

Background: Prosthetics and orthotics (P&O) clinicians in low-resource settings rely heavily on paper-based documentation, which contributes to workflow inefficiencies, data loss, and a heavy documentation burden. Electronic medical records (EMRs) and AI-assisted tools offer potential solutions, but their usability and efficiency in specialized P&O workflows remain underexamined. Objective: This study evaluated whether an iteratively designed, AI-assisted EMR prototype (Everi Labs AI Copilot) could improve task times and perceived usability compared to paper-based documentation for P&O clinicians in the Philippines. Methods: A crossover mixed-methods design was used. Ten (10) ISPO-certified prosthetist-orthotists completed simulated patient assessments under three conditions: paper, prototype v1, and prototype v2. Task completion time and System Usability Scale (SUS) scores were collected. Qualitative feedback was gathered through focus group discussions and analyzed thematically to inform design decisions of the iterated prototype. Results: The v1 prototype was 38.31% slower than paper (36.42 vs. 26.34 minutes; SUS = 60.00). After one iteration informed by user feedback, v2 achieved slightly slower total task time than the paper (+1.97%) and was faster in SOAP note creation (-21.30%). The SUS score improved 16 points to 76.00, indicating good usability. Welch's ANOVA showed no statistically significant differences (p > 0.05), but effect sizes were large (ηp² > 0.14 for both time and SUS), suggesting meaningful differences that a larger sample would likely confirm. Three qualitative themes emerged: the burdens of paper-based documentation (data loss, fragmentation, storage demands), issues with v1 (AI transcription errors, connectivity problems, terminology mismatches), and desired features for v2 (visual prompts, structured inputs, data security, accessibility options), which were used to inform changes to the UI prototype. Conclusion: Within the UI development cycle, a user-centered, iteratively designed AI-assisted EMR achieved good usability and total task time parity with paper. The study demonstrates that structured formative evaluation with limited resources can produce actionable design improvements, supporting the potential of purpose-built digital tools to reduce documentation burden in P&O practice.