A Robotic 3D Perception System for Operating Room Environment Awareness
release_heqjtzmpcvek3c7jignw7yvmu4
by
Zhaoshuo Li, Amirreza Shaban, Jean-Gabriel Simard, Dinesh Rabindran,
Simon DiMaio, Omid Mohareri
2020
Abstract
Purpose: We describe a 3D multi-view perception system for the da Vinci
surgical system to enable Operating room (OR) scene understanding and context
awareness.
Methods: Our proposed system is comprised of four Time-of-Flight (ToF)
cameras rigidly attached to strategic locations on the daVinci Xi patient side
cart (PSC). The cameras are registered to the robot's kinematic chain by
performing a one-time calibration routine and therefore, information from all
cameras can be fused and represented in one common coordinate frame. Based on
this architecture, a multi-view 3D scene semantic segmentation algorithm is
created to enable recognition of common and salient objects/equipment and
surgical activities in a da Vinci OR. Our proposed 3D semantic segmentation
method has been trained and validated on a novel densely annotated dataset that
has been captured from clinical scenarios.
Results: The results show that our proposed architecture has acceptable
registration error (3.3%±1.4% of object-camera distance) and can robustly
improve scene segmentation performance (mean Intersection Over Union - mIOU)
for less frequently appearing classes (> 0.013) compared to a single-view
method.
Conclusion: We present the first dynamic multi-view perception system with a
novel segmentation architecture, which can be used as a building block
technology for applications such as surgical workflow analysis, automation of
surgical sub-tasks and advanced guidance systems.
In text/plain
format
Archived Files and Locations
application/pdf 2.8 MB
file_orb3m4hzcbay7kfmkyynwbzkry
|
arxiv.org (repository) web.archive.org (webarchive) |
2003.09487v2
access all versions, variants, and formats of this works (eg, pre-prints)