Objective:
This study demonstrates intravascular micro-agent visualization by utilizing robotic ultrasound-based tracking and visual servoing in clinically-relevant scenarios.
Methods:
Visual servoing path is planned intraoperatively using a body surface point cloud acquired with a 3D camera and the vessel reconstructed from ultrasound (US) images, where both the camera and the US probe are attached to the robot end-effector. Developed machine vision algorithms are used for detection of micro-agents from minimal size of 250
$\boldsymbol{\mu }$
m inside the vessel contour and tracking with error recovery. Finally, real-time positions of the micro-agents are used for servoing of the robot with the attached US probe. Constant contact between the US probe and the surface of the body is accomplished by means of impedance control.
Results:
Breathing motion is compensated to keep constant contact between the US probe and the body surface, with minimal measured force of 2.02 N. Anthropomorphic phantom vessels are segmented with an Intersection-Over-Union (IOU) score of 0.93
$\pm$
0.05, while micro-agent tracking is performed with up to 99.8% success rate at 28–36 frames per second. Path planning, tracking and visual servoing are realized over 80 mm and 120 mm long surface paths.
Conclusion:
Experiments performed using anthropomorphic surfaces, biological tissue, simulation of physiological movement and simulation of fluid flow through the vessels indicate that robust visualization and tracking of micro-agents involving human patients is an achievable goal.
