Application Of Three-Dimensional Voxel Fusion Technique To Multi-Modality Fusion Of Anatomical And Functional Images

Image fusion with various anatomical and functional data has become a practical procedure in radiation oncology clinic. It has been a challenge to fuse anatomical images(CT/MRI) with functional images(PET) in low resolution due to the low signal to noise ratio. Different fusion strategies have been reported, including fuzzy logic method, special fiducial markers, and even integrated imaging hardware to generate fused images. Here, we apply the 3D voxel fusion technique to achieve multi-modality fusion of anatomical and functional images. The 3D voxel fusion technique, previously developed at our Branch, NCI/NIH, is based on a special algorithm with a volume rendering board(VP1000) in a PC computer. This technique can handle up to 4 image modalities simultaneously at real-time speed. Its superior visual representation and straightforward operation make this technique unique for image fusion in the form of virtual reality. With RGB color band(mono-color representation), the fusion result can be clearly identified by the homogeneity of the color distribution. With pre-defined RGBA lookup tables for given image modalities(pseudo-color representation), tissue of interest can be selectively shown and thus is useful to find anatomic landmark for fusion and to verify fusion result as quality assurance. The 3D voxel fusion technique has been used to fuse anatomical image(CT) with high quality and functional image(PET) with low resolution. More often than not, the functional image has signal only in certain voxels, including those with tumor activity. The 3D voxel fusion has shown its superiority in selectively viewing and fusing the tissue of interest, where valid voxel contents exist in both image modalities. The 3D voxel fusion can also identify distortional error present in the fusing image modalities. In this case, the fusion criteria will focus on the tissue of interest, while ignoring less relevant voxels, and the fusion can be done manually by rotating and/or translating the images around. By changing W/L in the lookup table and/or the transparency of the voxel content, this technique provides a fast way to find and fuse the anatomic landmark present in multiple image modalities. As an example, the fusion result of CT/PET images with distortional error is shown in Fig.1, where PET voxel(green or dark grey) is fused with corresponding CT voxel(orange, or grey) selectively based on CT lookup table. The fused voxels become yellow(or light grey). The most outside CT muscle voxels are adjusted to transparent, avoiding their interference with interior viewing yet leaving trace line to show their existence. The bended leg voxels in PET are ignored because of the identified distortion. When fusing different anatomical images(CT/MRI), owing to the high resolution of the voxel objects, the fusion accuracy can be improved to be the same accuracy as that of individual image modality(namely, single voxel or sub mm). The 3D voxel fusion technique with superior 3D visualization offers practical means to do multi-modality image fusion efficiently and accurately in clinic. It can help physician to selectively view and fuse anatomic landmark and identify if a systematic error resides in the fusing image modalities. It has proven to be useful to fuse anatomical and functional images.