Medical training, help, and telecommunication using interactive
3D graphics as visual explanations
Gernoth Grunst, GMD
The project cluster SCENE at GMD applies 3D graphics and animation
systems in order to develop functional models of the human body. These "dynamic
anatomy" models offer potentialities of interactive explorations which
build up visual expertise, graphic background understanding, and practical
medical abilities. The interested student or physician can this way experience
otherwise unobtrusive phenomena like the morphodynamics of the beating heart.
A first softwaretype which makes use of this potential are medical training
systems with different degrees of interactivity.
The 4D Heart Explorer is a PC based multimedia training unit which presents
its user a beating heart model in different visualisation modes. The interactive
model can be rotated, slowed down, transparency and elements like the ventricles
can be switched. This way the trainee can build up a graphic conception
of the outer and inner heart structures from different viewpoints.
The next step in training and technology is the EchoSim hard- and software
system. This environment couples a dummy patient torso and ultrasound transducer
with a virtual reality scenario of the breast, beating heart, and a simulated
ultrasound scan. The system is actually running on Silicon Graphic workstations.
The link between the physical transducer head and the virtual ultrasound
scan is provided by an electromagnetic position sensor (Polhemus Fasttrack).
The user of the system can exercise real echocardiographic examination by
positioning the transducer on the model torso. The visual orientation can
rely on different views of the scenario. Besides a semitransparent continuous
view of the chest, heart, and the "slicing" ultrasound plane a
synchronised view of the actually sliced inner heart structures can be selected.
In order to actively acquire the standard views of echocardiography the
system features a sophisticated self control facility. Recordings of trials
can be compared with the appropriate positionings. Subtle details with relevance
for diagnosis can thus be evaluated by the trainee or the accompanying expert.
The spatial and diagnostic orientations of EchoSim shall even be exploited
in actual examinations of a patient. To achieve this goal the project CARDI-ASSIST
(funded by the EU Telematics health care program) shall align the 3D model
scenario with real 3D ultrasound data sets of actual patients. This calibration
or registration procedure shall be based on at least three salient "landmarks"
in the ultrasound images which can be selected even by inexperienced cardiologists.
The points will be matched onto the model heart. Geometric orientation and
size will be adapted. After the calibration procedure the physician can
rely on the model scene in order to explore the patient's image data.
In order to adapt the standard heart model to an actual patient's heart
different structural and haemodynamic pathologies can be inserted. This
task is rendered possible through the tool TiDas. TiDas furthermore supports
the optimal visual presentation of the diagnostic findings in the heart
model.
The integrated pathological 3D data set of a patient with the related pathological
3D model is a very effective means of telemedical communication between
cardiologists and cardiac surgeons. On the one hand a graphic visual context
supports the evaluation of the ultrasound images, on the other hand both
partners can intuitively choose their pertinent view of certain pathologies.
Diagnosis and therapy require different visual cues. In CARDI-ASSIST the
telemedical exchange scenario will furthermore be enhanced by interactive
means of multimedia communication like voice annotation and tools for marking
and changing views of the aligned data set an model scene.
The whole training, orientation and communication environment can be seen
as a very effective enabling system. In order to impart a medical in depth
understanding of certain pathologies the exchange of image data and related
verbal comments is insufficient. The SCENE approach however seems to be
a very promising attempt to raise the level of medical know how and efficiency.
Especially societies which have not yet an advanced level of medical education
can immediately profit from such enabling systems.