Register for our upcoming hands-on workshop on LangChain | June 17th | Virtual

X-Ray Colorization and 3D X-Ray image Reconstruction

Author(s): Vibhav Patil


Medical errors are the leading cause of mortality in the medical field and are a substantial contributor to the increased medical cost. Radiologists play an integral role in the interpretation and diagnosis of X- ray images. Diagnostic errors are bound to happen as they examine and interpret large numbers of X-rays. Generally over 40% of diagnostic errors lead to increase in medical costs, incorrect treatment or even death. Such diagnostic errors or “miss” in primary or critical findings, lead to incorrect diagnosis. One of the reasons could be due to lack of X- Ray clarity and visibility. Solutions to address such problems are to colorize the X- Ray and convert X-Ray image from 2D to 3D which would help the radiologist to interpret the X-Rays easily in less time due to improved clarity, visibility and due to almost real life like image.In this paper we have used the Computer Vision techniques for Colorization of X-Ray image and also to convert the 2D X-Ray image to 3D image. X- Ray colorization is done using Generator Adversarial Networks (GANS) specifically Pix2Pix GANS. GANS consists of the Generator part which generates the image and a Discriminator part which will differentiate between generated image and an original image to ensure conversion to realistic images. This combination of Generator and Discriminator networks will produce real and life like images. We have tried to implement this network combination to colorize the grayscale images so that Radiologists can easily detect any abnormality in the X-Ray with good precision. 2D to 3D Image reconstruction involves transforming a 2D X- Ray to a 3D X-Ray using Computer Vision. A depth map is generated for the 2D X-ray image which would give a depth axis or a third dimension to the otherwise 2D image using OpenCV library. Once the depth map is generated it would become easy to plot the 3D dimensional X-Ray images and view it from different angles so that the Radiologist’s don’t miss out on abnormalities.