Integrating Patient Digital Photographs with Medical Imaging Examinations Senthil Ramamurthy & Pamela Bhatti & Chesnal D. Arepalli & Mohamed Salama & James M. Provenzale & Srini Tridandapani Published online: 14 February 2013 # The Author(s) 2013. This article is published with open access at Springerlink.com Abstract We introduce the concept, benefits, and general architecture for acquiring, storing, and displaying digital pho- tographs along with medical imaging examinations. We also discuss a specific implementation built around an Android- based system for simultaneously acquiring digital photo- graphs along with portable radiographs. By an innovative application of radiofrequency identification technology to radiographic cassettes, the system is able to maintain a tight relationship between these photographs and the radiographs within the picture archiving and communications system (PACS) environment. We provide a cost analysis demonstrat- ing the economic feasibility of this technology. Since our architecture naturally integrates with patient identification methods, we also address patient privacy issues. Keywords Patient identification . Electronicmedicalrecords . Medical imaging . Medical errors . Digital camera . DICOM . PACS Introduction Patient safety issues have gained prominence in the national dialog in the USA particularly since the publication of the 2001 Institute of Medicine’s report on quality [9]. The Joint Commission on Accreditation of Healthcare Organizations (JCAHO) in its 2010 National Patient Safety Goals (NPSG) provides a specific requirement (NPSG.01.01.01) that at least two patient identifiers be used when providing care, treatment, and services [13]. The rationale is that “wrong-patient errors occur in virtually all stages of diagnosis and treatment… Acceptable identifiers may be the individual’s name, an assigned identification number, telephone number, or other person-specific number” [13]. Meanwhile, the National Quality Forum [18], with support from the Agency for Healthcare Research and Quality, has spe- cifically endorsed, in its “30 Safe Practices for Better Health Care Fact Sheet,” the use of standardized proto- cols to prevent mislabeling of radiographs. One can easily see that many of the acceptable identifiers noted in the JCAHO NPSG requirements can be problematic, particularly if patients are unconscious, uncooperative, or noncommunicative for various reasons. On the other hand, human beings have been hardwired to use the human face as an identification device for millennia, and this identification A very preliminary version of this work was presented at the IEEE Engineering and Medicine and Biology Society’s Conference on Biomedical Engineering and Sciences (IECBES) 2010 and at the 2012 SIIM meeting. S. Ramamurthy : C. D. Arepalli : M. Salama : J. M. Provenzale : S. Tridandapani (*) Department of Radiology and Imaging Sciences, Winship Cancer Institute, Emory University School of Medicine, 1701 Uppergate Drive NE, Suite 5018, Atlanta, GA 30322, USA e-mail: stridan@emory.edu S. Ramamurthy e-mail: sramamu@emory.edu C. D. Arepalli e-mail: carepal@emory.edu M. Salama e-mail: mo.salama@emoryhealthcare.org J. M. Provenzale e-mail: jmprove@emory.edu S. Ramamurthy : P. Bhatti : S. Tridandapani School of Electrical and Computer Engineering, Georgia Institute of Technology, 777 Atlantic Drive NW, Atlanta, GA 30332, USA e-mail: pamela.bhatti@ece.gatech.edu J. M. Provenzale Department of Radiology, Duke University Medical Center, Durham, NC 27710, USA J Digit Imaging (2013) 26:875–885 DOI 10.1007/s10278-013-9579-6 device remains strong even if the patient is unconscious or uncooperative. To minimize or prevent mislabeling of medical imaging studies, we introduce the concept of obtaining digital photo- graphs of patients simultaneously with all medical imaging studies. These digital photographs will be small additions to the imaging study similar to the scout or localizer images that are performed with CT studies. We do not intend these digital photographs to entirely replace numerical identifiers, but rather we envision that they would supplement and strengthen these identifiers. However, in some cases, such as unconscious trauma patients, these photographs may indeed be the only available identifiers. Two parallel developments currently make our proposed technique a contender for serious consideration in electronic healthcare delivery systems: 1. Recent advances in charge coupled device (CCD) and complementary metal oxide semiconductor (CMOS) camera technologies have made it possible to miniatur- ize these relatively inexpensive devices, such that digital cameras capable of 12–16 megapixel resolution occupy less than a square centimeter. At the same time, memory costs have continued to drop and the addition of digital photographic data to an imaging study has negligible cost associated with it when compared with the overall cost of the study. 2. For more than two decades, the development of the digital imaging and communications in medicine (DICOM) standard has allowed for integrating imaging data from various modalities into hospital information systems [11, 14] and has provided the ability to present integrated image data to radiologists and other physi- cians [10]. Thus, the technical foundation exists, mak- ing our novel concept a feasible one. The distinguishing feature of our technique is that we consider point-of- care photographic imaging, that is, photographs will be obtained simultaneously with every instance of acquisi- tion of diagnostic imaging. The ideal implementation of this photography technique would require the cooperation of equipment vendors, that is, manufacturers of radiography, ultrasound, computed tomog- raphy, and magnetic resonance imaging equipment. These vendors will all have to integrate cameras in their equipment. However, there is an installed base of tens to hundreds of thousands of imaging devices, which would require some form of retrofitting for such a technique to work. In this paper, we describe the general architecture for achieving this inte- gration in an existing picture archiving and communications system (PACS). We also describe our prototype for retrofitting a camera system on an existing portable conventional radiog- raphy (CR) machine. Thus, we provide an end-to-end imple- mentation from image acquisition, to transmission, to storage, and to display, for retrofitting a camera system on an existing portable CR machine. The rest of this paper is organized as follows. In the “Motivations for and Advantages of the Proposed Concept” section, we discuss the motivations for and advantages of the proposed concept. In the “Implementation Strategies: Clinical Perspective” section, we consider implementation strategies for a variety of imaging modalities from a clinical perspective, and potential privacy concerns are explored in depth in the “Potential Privacy Concerns of Gathering Photographic Data” section. In the “Architecture for Integrating Photography with a Portable Radiography Machine” section, we discuss the specifics of the hardware architecture that we have designed to integrate a camera with a portable radiography machine, and the back-end processing that is required to integrate the digital photographs with the DICOM radiographic images; cost considerations are also discussed in this section. Finally, in the “Conclusions and Future Work” section, we provide conclusions. Motivations for and Advantages of the Proposed Concept There are two significant advantages of incorporating photo- graphs with imaging studies: 1. Decreasing medical errors: Medical errors are not an insignificant source of adverse clinical outcomes and medical complications, which add significantly to health care costs [9]. In particular, imaging studies are prone to mislabeling and misidentification errors. Such errors can cause medical problems for both the patient whose demographic information was tagged to the study and the patient to whom the images belong. While advances in PACS may lead to improved workflow and increased efficiency and throughput, medical mistakes may also become more prevalent [15]. A number of such mislabeled cases are identified at the time of image interpretation by the radiologist, when a current study is compared with an older study purport- ing to be from the same individual. The radiographs or the scout/localizer images (in the case of CT examina- tions) from the new and old study may show some obvious differences particularly if the body habitus of the patients in the two comparative examinations are quite different or if there are different medical support hardware between the two studies [5]. However, when the two imaged individuals have similar physiques, then determining that the old and new studies do not belong to the same patient can be challenging. We believe that obtaining a patient’s facial digital photograph simultaneously with the diagnostic images 876 J Digit Imaging (2013) 26:875–885 can significantly increase the detection rate of misla- beled studies, thereby decreasing medical error. This will also increase interpreting physicians’ efficiency and throughput since they will have to spend less time looking for anatomical landmarks. Aakre et al. noted that plain radiographic errors were reduced from 2.4 to 0.7 %, but not eliminated, after the introduction of bar code scanners to automatically gen- erate patients’ demographic information and examina- tion dates on the computed radiography modality via a DICOM modality work list [1]. Their intervention re- quired either the patient or the technologist to verify the demographic information, which is a potential source of errors. We believe that automatically adding digital photographs will help reduce this error rate further. 2. Improved diagnostic capabilities: In addition to facial digital photography, obtaining a digital photograph of the area that is imaged can, in many cases, add to the diagnostic value of the imaging studies. For example, with portable chest radiographs, it is often unclear if the many medical lines and tubes pro- jected over the patient are outside the patient or inside the patient, or what parts of such devices are outside or inside the patient. Quite often, such ambiguity requires a call by the radiologist to the clinical service taking care of the patient for clarification; the radiologist’s time is one of the more expensive costs associated with an otherwise simple study. A digital photograph of the chest may show por- tions of some of these lines and tubes outside the patient and thus provide additional clues and improve the diag- nostic value of the imaging study and interpretation. The digital facial photograph may also add to the diag- nostic value of the study. Quite often, standing orders for obtaining daily portable radiographs in the intensive care units are placed with generic indications, such as “check lines and tubes” or “evaluate endotracheal tube.” Many times, these generic indications propagate to the study requi- sitions that are submitted even after the questioned lines and tube have been removed from the patient. A digital facial photograph may show if tubes such as nasogastric tubes, orogastric tubes, or endotracheal tubes are present or absent in the patient. Such additional information can dramatically speed up the interpretation of portable chest radiographs. Another area of radiology where photographs of the affected region can aid tremendously in diagnosis is trauma imaging. Showing the entry and exit wounds of gunshot victims or the presence of foreign objects that protrude outside the patient can aid in the diagnostic accuracy of CT examinations by calling attention to these entities. A potential further advantage of such an identification system is that it could eventually be entirely automated given the significant progress that is being reported in the area of computerized face recognition techniques. Bowyer et al. [6] and Zhao et al. [25] have surveyed numerous techniques, with developers of these techniques reporting a greater than 90 % recognition rate. Indeed, O’Toole et al. have demonstrated the superiority of several state-of-the-art face recognition algorithms over human capabilities in de- termining whether pairs of face images, taken under differ- ent illumination conditions, were pictures of the same person or of different people [20]. Thus, face recognition technologies may serve as an additive safeguard to human face matching capabilities, just as other computer-aided diagnostic technologies are assisting radiologists in a variety of clinical conditions [12] including the detection of pulmo- nary nodules [2, 4], osseous metastases [19], and colorectal polyps [3]. Implementation Strategies: Clinical Perspective To avoid the possibility of tagging the wrong patient’s photograph with the imaging study, an important require- ment of this type of integration is to ensure “point-of-care” imaging, that is, the photographic information is obtained either simultaneously or as close in time as possible with the medical image acquisition. Implementation strategies for several modalities are discussed below: Digital Radiography and Portable Conventional Radiography Currently, a light source is used to illuminate and set the field-of-view of digital radiographs and portable radiographs by technologists. It is relatively straightforward to integrate CCD and CMOS cameras into these imaging devices so that photographs of the field of view are obtained simultaneously with the radiographs. In addition to obtain- ing a photograph that may provide useful diagnostic infor- mation, this system may also be employed for better positioning of the patient and improving the field of view of the radiograph. Additionally, a second CCD or CMOS camera may also be employed in the X-ray machine tower to simultaneously obtain a facial photograph of the patient. Of course photographs of the chest, for example, with portable radiography will be obtained with the patient wear- ing a hospital gown. The objective is to retain the patient’s modesty to the extent possible and not add a new step in the workflow. As a result, some of the lines and tubes may be obscured by the clothing; however, much of the overlying hardware may still be visible and provide some useful information. Facial photographs may be limited as identification tools, for example in trauma or postsurgical patients, whose faces are covered with dressing. However, the presence and pattern of such dressing may itself serve as an identification tool and allow us to recognize when a mislabeling error occurs. J Digit Imaging (2013) 26:875–885 877 Ultrasound CCD or CMOS cameras measuring a square centimeter or less are available and these can be integrated with the ultrasound transducers or a separate port could be used. These cameras can then be used to obtain both facial photographs and photographs of the affected body part that is being imaged. Already magnetic tracking markers are being embedded into ultrasound (US) transducers to allow for real-time registration of US images in three-dimensional space with imaging from other modalities. Thus embedding a camera in a US transducer should be feasible. Of course, some judgment is called for in what body part photography is useful and clinically acceptable, and clinical standards for this can be eventually established. Obviously, correlative photographic imaging is neither clinically rele- vant in many cases nor ethically acceptable when consider- ing studies such as ultrasound imaging of female pelvis or of the male prostate. However, in some cases, photographs of the affected part may turn out to be an important medico legal documentation tool. For example, radiologists are of- ten unable to perform ultrasound imaging of some organs because of overlying dressing and bandages, and it may be useful to document these with a photograph. Software locks can be provided to prevent acquisition of ultrasonic images by the technologist until a facial photograph of the patient is first obtained, thus ensuring compliance with this workflow modification. CT, MRI, PET Digital cameras can be integrated with the CT gantry or embedded within the MRI scanner, and these cameras can obtain digital photographs of both the face and the body part being imaged. At our institution, we have already installed video cam- eras in the MRI, PET-CT, and PET-MRI suites for the purposes of monitoring patients, particularly those patients receiving moderate sedation. These monitoring cameras are necessary since nursing personnel cannot be in the room, especially when the X-ray tube is active. Such monitoring cameras could easily be converted into recording devices integrated with the imaging equipment. PACS and Viewing Workstations Our technique is useful only if the photographs are readily available on a PACs viewing station. We envision this photograph will be treated just like the scout or localizer films in CT or MRI studies. These scouts should be “clickable” for enlargement. Most importantly, we should have the ability to view simulta- neously and compare photographs obtained from different studies of the same patient. A possible hanging protocol for displaying portable chest radiographs, obtained along with photographs, is shown in Fig. 1. The DICOM standard already has a standard for the storage and display for visible light (VL) images. Freely available software—the SimpleDICOM Suite, has been developed to allow importation of nonradiologic images into the PACS [7]. The key element of their approach is that the VL images require an additional workflow step and the patient must be assigned a mock event within the Radiology Information System to represent the additional images in the PACS; the VL images are thus not integrated with the radiologic images and are not part of one study. At the same time, it has been shown by PET-CT and more recently by PET-MRI implementations that multiple modal- ities can be integrated seamlessly both at the data acquisition phase and at the display phase. Thus, implementation of our technique for simultaneous acquisition and display of pho- tographic and medical imaging data is feasible with existing technologies. Thus, the technology is available for integration of digital cameras for acquisition of patient facial photographs at the point-of-care of medical imaging for a wide variety of modalities. Potential Privacy Concerns of Gathering Photographic Data Patient privacy concerns will potentially be raised as issues with obtaining, storing, and displaying digital photographs with the medical images. There are several reasons why most patients will likely not object to this minor intrusion, if it could even be considered an in- trusion, on their privacy. First, there is a significant safety issue that benefits the patient, and most patients would be happy to provide more information if it could potentially lead to a more accurate diagnosis. Second, most healthcare institutions have multiple video cameras in the hallways as a security measure, and patients’ presence and movements are already being recorded at various locations. Most patients do not enter a healthcare facility wearing a veil, and most of them are seen by a number of healthcare workers including physicians, nurses, technologists, and transporters. With photographic record- ing, their external physical appearance will be seen by one more physician—the radiologist. Third, photographic data is no different from all of the demographic data, such as contact information, social secu- rity number, and date of birth that are already being collect- ed from all patients at most medical facilities. The photographic data to be gathered will be secured just like the individually identifiable health information including demographic data that is attached to medical imaging data, and is protected under the Health Insurance Portability and Accountability Act of 1996. This data will be available only to medical personnel who are charged with the care of the patient. 878 J Digit Imaging (2013) 26:875–885 Fourth, imaging modalities such as CT already collect enormous amounts of data, and with currently available sophisticated 3D volume and surface rendering techni- ques, it is possible to recreate the external appearance of a patient. In fact, Cesarani et al. have produced a model of the face of a man who lived nearly 3,000 years ago from CT data of a mummy [8]. Interestingly, “defacing” algorithms for neuroimages are required to preserve subject privacy for research projects involving large-scale collaboration with neuroimaging data [21]. Thus, with photographs, patients are not really giving us access to any new information that cannot already be derived from the data they currently provide us. Finally, the data we intend to gather is an externally visible feature of humans and does not involve other sophis- ticated data such as retinal scans or fingerprints or any other data that could be misused. Architecture for Integrating Photography with a Portable Radiography Machine Emory Prototype Of course, the optimal implementation for the technique we have discussed thus far would require vendors of medical imaging equipment to integrate digital cameras into their devices to ensure simultaneous capture of this multimodal data. On the other hand, there is an installed base of several hundred thousand imaging devices throughout the country, and it would be difficult to justify replacing all these devices with newer devices simply for the ability to obtain digital photographs simultaneously. Thus, inexpensive, snap-on solutions need to be developed. The critical features of these snap-on solutions are (1) the photographs must be obtained nearly simultaneously with the medical images and (2) there should be a tight integration between the photograph and the medical images within the PACS environment. At Emory University Hospital and affiliated hospitals and clinics, we are currently constructing a snap-on solution built around an open-source ARM-based development board running the Android-operating system (ARM Ltd, Cambridge, UK). In Fig. 2, a system-level architectural diagram of our implementation is shown, wherein the new elements in our architecture are shown with dashed borders, and the subsystems in the existing architecture are shown with solid line borders. In the existing environment, each modality, such as CT, MRI, and US, has a wired Ethernet link to the PACS server allowing for transmission of the patient demographic information and the medical images. All of our portable X-ray machines in routine use are cassette-based CR machines. A cassette processor serves to convert each X-ray image into DICOM format. To asso- ciate each study with the patient, each cassette is marked with a unique barcode, the Plate_ID, which is added to the DICOM header by the cassette processor. These processors also allow the technologist to add the patient demographic information from the work list. The DICOM file is then transmitted to the PACS server from the processor via a wired Ethernet link. Our new architecture adds two main hardware compo- nents (Fig. 2): an android-based camera device (ABCD) and an integration server (IS). Digital photographs are captured by the ABCD and transmitted along with a time-stamp and a device code (or Plate_ID in the case of CR) via wireless links to an IS. The IS was developed to efficiently integrate the photographs with medical images in DICOM format. Android-Based Camera Device Our first ABCD implementation has been developed as a custom device using off-the-shelf components (Fig. 3). The Fig. 1 An example display showing a current radiograph–photograph combination (left) and a prior radiograph–photograph combination (right) from two different patients. For privacy reasons, the patients’ eyes have been masked. Reprinted with permission from the American Journal of Roentgenology [22] J Digit Imaging (2013) 26:875–885 879 Android platform (Texas Instruments; Dallas, TX, USA) was chosen for ease of implementation and the ability to leverage existing applications in the Android market. The device is built around a BeagleBoard (Texas Instruments) initially for deployment with a portable CR machine (Fig. 4). We are affixing all of the X-ray-sensing plates (cassettes) used with the portable CR machines with radio- frequency identification (RFID) tags that correspond to the barcodes on the cassettes, i.e., the Plate_ID. As described later, these Plate_IDs will allow us to link the photographs with the radiographs. These passive 125 KHz RFID tags feature RFID integrated circuits based on the EM4001 ISO standard, with the corresponding DICOM tag 0018,1004. The RFID tags offer a read range of 10 cm; that is, the tags are read when they are brought in close proximity to the reader. The read range has been deliberately chosen to be very small for two reasons: (1) longer range RFID readers consume more power; (2) to prevent cross-talk among the cassette RFID tags. It is not uncommon for technologists to take up to 12 cassettes when they go to the ICUs and in-patient floors to obtain portable X-rays, and this can create interference (cross talk) among the cassette RFID tags. We are currently exploring RFID readers with a larger range employing highly directional antennas that can work up to distances of 6 ft for the next generation prototype. The camera used in our solution is an Aptina ¼ CMOS Sensor (Aptina Imaging Corp., San Jose, CA, USA), which is capable of 3 megapixel resolution and is mounted on a Leopardboard 365 3 M camera board (Leopard Imaging Inc., Fremont, CA, USA). To ensure that the digital photo- graph is obtained simultaneously with the diagnostic image, a standard instrumentation bus is used. More specifically, when the trigger for acquisition of a radiograph is activated, it is received via a general purpose input/output pin that triggers the ABCD to capture a photograph with the camera. PACS ABCD: Android- based Camera Device Wi-Fi Link Ethernet Link ABCD ABCD ABCD ABCD Processor Integration Server GPIO Fig. 2 System level architecture. The building blocks in the existing environment are shown with solid borders and the building blocks in our new environment are shown with dashed borders IS ARM microprocessor Network 802.11b/g/n Bluetooth Bluetooth enabled RFID reader Trigger from imaging modality GPIO RFID Radiographic Cassettes Fig. 3 Android-based camera device block diagram. IS integration server, GPIO general purpose input/output Fig. 4 Prototype ABCD 880 J Digit Imaging (2013) 26:875–885 At the same time, the ABCD employs a bluetooth- enabled RFID reader to capture the unique identifier for the cassette. The photograph and the RFID informa- tion are both transmitted to the platform that we have built to integrate the photographs with the medical images in DICOM format, the IS (Fig. 5). An IEEE 802.11 b/g wireless module will leverage existing enterprise-wide wireless infrastructure to connect to the IS. The ABCD runs rowboat (a port of Android to 2.3 (gingerbread) to BeagleBoard XM). Table 1 summarizes the cost of our development ABCD system totaling 568 USD. We expect the cost of the final snap-on solution to be much less and on the order of 100 USD. Our initial hardware development goal was to rapidly realize a functional development platform with full debugging capabilities. As a result, we integrated off-the-shelf components that provide well above the functionality required in the final optimized system. Options to scale the cost down include replac- ing the BeagleBoard XM with a low-cost microproces- sor, eliminating the display monitor that is not essential in the final product, and selecting a lower cost RFID reader. We also expect the footprint of the ABCD to dramatically scale down once the system is further optimized and integrated at the component level. Integration Server The integration software has been developed in C++ readily leveraging the DCMTK libraries from DICOM to imple- ment the DICOM standard. The IS process flow is shown in the right half of Fig. 5. The IS has bidirectional communi- cation with the PACS server. The IS queries and retrieves recent studies for each modality from the PACS. In this illustration, reference is only made to the CR list, but the process is similar for other modalities. Once the IS receives a photo with a Plate_ID, it compares this Plate_ID with the Plate_IDs (DICOM tag 0018,1004) from the headers of the DICOM images of the retrieved studies from PACS until a match is found. Since the cassettes are reused, the Plate_ID is not unique, and this creates an ambiguous relationship between the digital photograph and the imaging study as shown in Fig. 6. However, the combination of the Plate_ID and time of acquisition is still unique, thus a Time_Stamp is also generated by the ABCD for each photograph. This requires all the ABCDs to be synchronized in time. The ABCDs use Network Time Protocol (NTP) to synchronize their times with a NTP Server running on the IS. Once a match is found, the photograph is converted from JPEG format to DICOM format and a new series is created with a study-matched subject photo. This series is then sent to PACS where it becomes a part of the imaging study or folder. Extending the implementation of this technique for non- portable, stand-alone equipment, such as CT and MR scan- ners, is much easier since departmental Wi-Fi equipment is always within range. Further, the time stamps for the photo- graphs and the medical images can be perfectly synchro- nized by the time of acquisition. Acquire subject photo Retrieve recent CR studies from PACS Identify Plate_ID from DICOM header Match photo to CR study based on Plate_ID and Time_Stamp Identify X-ray cassette using RFID Tag photo with X-ray Plate_ID and Time_Stamp Send photo to Integration Server Create new series with study-matched subject photo Convert photograph to DICOM Send new series to PACS ABCD: Android-Based Camera Device Integration Server PACS Trigger from imaging modality Fig. 5 Process-flow diagram for the ABCD and IS Table 1 Cost of the various ABCD components Component function Specific component used Cost (USD) Microprocessor BeagleBoard XM 149 Camera 3 M Camera, Leopard Imaging 40 RFID Reader RFID USB Reader, Serial IO 179 Communication (WiFi) BeagleBoard Expansion V2 200 J Digit Imaging (2013) 26:875–885 881 Integrated Display of Digital Photographs and Radiographs At the interpreting workstation, hanging protocols treat the photographs like a separate series within the study. A potential hanging protocol is shown in Fig. 1, where a portable radio- graph obtained simultaneously with a photograph is displayed along with a prior radiograph–photograph combination. In this example from [22], the radiograph on the left shows an 81 year-old man with aortic valve replacement status post coro- nary artery bypass grafting and aortic valve replacement; the characteristic median sternotomy wires can be seen with prop- er window and level settings on the workstation. The radio- graph on the right, which is the comparison (“previous”) radiograph from three days prior, shows an 89 year-old white man with aortic stenosis admitted for aortic valve replacement surgery; the radiograph also shows a calcified aortic knob and calcified mediastinal lymph nodes not seen in the patient on the left. In addition, given a difference of only three days between the two radiographs, it is unlikely that the post-operative changes would show median sternotomy wires only and no support lines and tubes. The photographs, despite being edited to protect patient identity for this report, clearly show differ- ences in facial hair and baldness between the two patients. System Requirements and Cost Considerations The average sizes of various medical imaging studies range from 8 MB to 1 GB. A few representative studies have sizes as follows [17]: chest radiograph, 8 MB; CT abdomen, 150 MB; CT heart, 1 GB; MRI abdomen, 15–50 MB; whole body PET, 10 MB; heart PET, 24 MB; standard US, 12.5 MB/s; Doppler US, 37.5 MB/s. Considering that a 3 megapixel camera can provide a JPEG compressed picture for under 0.5 MB, the overhead of adding a single photograph to a medical imaging study ranges from 12.5 % for a one-view chest radiograph down to 0.05 % for a heart CT. Currently, a MB of memory costs less than 0.01 USD and the cost of storing a photograph is thus negligible relative to the cost of the examination. A basic 3 megapixel camera costs around 10 USD, espe- cially since no sophisticated focusing capabilities are re- quired for our purpose; the object distance is almost always fixed for each machine. The most inexpensive por- table X-ray machine costs around 40,000 USD and a PET- CT scanner costs anywhere from 2 to 3 million USD to be installed. Thus, the added cost of digital photography in new medical imaging equipment is miniscule. Furthermore, it should be possible to develop snap-on kits which can be used to retrofit existing imaging equipment at a cost of about 200 USD or less per kit. As an example, at Emory’s affiliated hospitals and clinics, currently approximately 137 imaging devices are deployed (Table 2). The machines include PET-CT, MRI, US, gamma cameras, and portable X-ray machines. We estimate that retro- fitting all of these machines would cost less than 30,000 USD. In 2010, approximately 481,000 imaging examinations were performed on approximately 142,000 patients at these centers. Assuming that these cameras only last 1 year before requiring replacement (that is, grossly underestimating the longevity of these devices), the cost per examination is projected to be less about 0.02 USD including memory costs, and likely even less since these devices are expected to last more than 1 year. Plate_ID: 123456 Plate_ID: 123456 Plate_ID: 123456 Acq Time: 14:00 Imaging Study 1 Plate_ID: 123456 Imaging Study 2 Plate_ID: 123456 Imaging Study 1 Plate_ID: 123456 Acq Time: 14:00 Imaging Study 2 Plate_ID: 123456 Acq Time: 16:00 Plate_ID: 123456 Acq Time: 14:00 + + + + a b Fig. 6 a Ambiguous relationship among the two imaging studies when the same cassette is reused for one of the studies. b The addition of acquisition time, Time_Stamp, to the Plate_ID removes the ambiguity and restores the one- to-one relationship 882 J Digit Imaging (2013) 26:875–885 It is quite difficult to predict the number of mislabeled cases in any center. Many mislabeling errors may simply never be discovered. In other cases, mislabeled examina- tions may be discovered, either by the technologists or the radiologists shortly after the examinations are performed, and may be fixed promptly before the images or the inter- pretation enters the patients’ permanent medical records. These cases may not be reported since no clinician has seen the images or the radiologists’ interpretation, and there are thus no clinical consequences. Quite often, however, mis- labeling is discovered days, weeks, or months later, when a patient undergoes a subsequent examination and it is noted that the body habitus does not match between the com- parative examinations or some discrepancy regarding supportive hardware, such as presence or absence of a pacer/defibrillator device, is noted. While it is difficult to estimate the extent of the impact that our proposal will have on patient safety, it should not be difficult to argue that 30,000 USD would be a fairly inexpensive investment if even one major complication or death were to be prevented out of 480,000 medical imaging studies performed at our institution annually. We now project national costs. According to Mettler et al., nationally in 2006 about 400,000,000 imaging exami- nations involving ionizing radiation (including diagnostic radiographic and fluoroscopic studies, interventional proce- dures, CT scanning, and nuclear medicine studies) were performed [16], roughly 1,000 times the number of exami- nations being performed at our institution. If we include MRI and ultrasound examinations, nationally the number of imaging examinations is probably close to 500,000,000. Extrapolating the ratio of imaging examinations from our institution to the nation, we can estimate the number of imaging devices to be at least 145,000. Note that this is an underestimate and does not include devices such as echo- cardiography machines, which do not form part of the Radiology Department at our institution, and are thus not being counted. Likewise, various other imaging modalities such as endoscopes, which can benefit from integrated facial photographic imaging, are not included in our estimates. Additionally, non-imaging medical diagnostic devices such as electrocardiography machines could also benefit from photographic imaging as an identification tool. Thus, the market potential for such a technique is quite large. Conclusions and Future Work We have presented a relatively straightforward approach that intuitively should reduce mislabeling of medical imag- ing examinations and thus result in a reduction in medical errors. The technique employs digital facial photography at the point-of-care of medical image acquisition and integra- tes this data with the imaging data. These photographs would serve as powerful identification tools. The method can be applied to all imaging modalities including X-ray, CT, MRI, ultrasound and PET. Digital photography at the time of medical imaging also provides supplemental clinical information that can enhance the diagnostic capability of the medical imaging study. This technique is not limited to diagnostic medical imaging, and can be easily translated to other applications such as electrocardiography or any other method where electronic patient data is gathered. Thus, our concept can be exercised at a host of point-of-care data collection points resulting in a more robust patient identifi- cation and authentication function in integrated healthcare information systems. We reiterate that this technique is intended to strengthen other existing identification methods, and there is a limita- tion that patient appearances may change with time. Fur- thermore, we note that when considering the first imaging examination for a patient, the matching photograph may have to be obtained from the patient’s electronic medical record since a prior photograph will not exist in PACS. Another limitation is the availability or lack of color mon- itors for PACS workstations, which can affect the visualiza- tion of patient facial features such as skin color and tone. This problem will vanish in the future as color monitors increasingly replace grayscale ones. We have described the hardware and software architec- tural framework required to integrate photography with medical imaging examination. Specifically, we developed the ABCD, which was targeted toward portable radiography machines and enables simultaneous acquisition of digital facial photographs along with portable chest radiographs. Several research questions must be addressed before the idea of integrated photography with medical imaging can Table 2 Number of devices and number of examinations performed in 2010 classified by modality at Emory University’s affiliated hospitals and clinics Modality Number of devices Number of examinations performed annually PET, PET/CT 6 9,553 MRI 12 44,341 CT 12 89,967 SPECT/CT and gamma cameras 8 9,540 Mammography 11 32,866 Ultrasound 17 39,635 Radiography and radiofluoroscopy 35 151,804 Portable X-ray 17 81,268 Portable C-arm 19 7,230 Interventional radiology suites 8 14,647 Totals 145 480,852 J Digit Imaging (2013) 26:875–885 883 become a clinically acceptable and useful tool. First, if we want to minimize technologist interaction and thus reduce the time the technologist would spend with this new modification, then the camera must be positioned automatically to take the facial photographs. In the case of chest radiographs, it is fairly easy to mount a fixed camera on the radiography unit tower so that with a high degree of certainty the patient’s face can be captured. Moveable or steerable mounts are available for cameras and intelligence can be built into the system so that the camera can be dynamically pointed to the expected loca- tion of the patient’s face depending on the type of examination that the technologist has entered into the system. For example, if a chest radiograph is being obtained, then the camera may be positioned to take a picture about 15° superior to the angle of the radiography tower. If an abdominal radiograph is being obtained, then a greater angle between the tower and the camera unit may be required. Intelligent face tracking cameras are already commonly available; systems that can be trained to obtain images of other body parts need to be designed. Second, face recognition systems have matured signifi- cantly [6, 25], and such systems may further help inter- preters quickly identify wrong patient errors. Such face recognition systems can be embedded in multiple portions of the imaging chain. Finally, perhaps the biggest challenge is to evaluate the clinical impact of adding patient photographs. While these photographs may help with identifying wrong patients, they may lead to unintended consequences: (1) photographs may distract the reader and impair reader efficiency; (2) photo- graphs may provide conflicting information relative to the medical images and confuse the interpreter; and (3) the interpretations may become more subjective. Indeed, pre- liminary work by Turner and Hadas-Halpern [23] suggested that subjectively radiologists felt more empathy towards patients when their photographs were shown along with CT examinations, but it was also noted that the reports become objectively longer and a greater number of inciden- tal findings were reported. A survey reported in an abstract by Weiss and Safdar [24] revealed that 67 % of surveyed radiologists were not in favor of including photographs with medical images. It is unclear what radiologists’ responses would be if they are presented with an actual working tool. These issues deserve further investigation and are the sub- ject of a forthcoming paper [22]. Acknowledgments Dale Walker and Jessie Knighton provided data for Table 2. Diana Fouts and Eric Jablonowski provided assistance with the figures. Srini Tridandapani, PhD MD, was supported in part by the PHS grant (UL1 RR025008, KL2 RF025009) from the Clinical and Translational Science Award program, National Institutes of Health, National Center for Research Resources and in part by Award Number K23EB013221 from the National Institute of Biomedical Imaging And Bioengineering. Pamela Bhatti, PhD, was supported in part by the PHS grant (UL1 RR025008, KL2 RR025009) from the Clinical and Translational Science Award program, National Institutes of Health, National Center for Research Resources. The content is solely the responsibility of the authors and does not necessarily repre- sent the official views of the National Institute of Biomedical Imaging and Bioengineering, the National Center for Research Resources, or the National Institutes of Health. Open Access This article is distributed under the terms of the Creative Commons Attribution License which permits any use, distribution, and reproduction in any medium, provided the original author(s) and the source are credited. References 1. Aakre K, Johnson C: Plain-radiographic image labeling: a process to improve clinical outcomes. Journal of the American College of Radiology 3(12):934–953, 2006 2. Awai K, Murao K, Ozawa A, et al: Pulmonary nodules at chest CT: effect of computer-aided diagnosis on radiologists’ detection per- formance. Radiology 230(2):347–352, 2004 3. Baker M, Bogoni L, Obuchowski N, et al: Computer-aided detection of colorectal polyps: can it improve sensitivity of less-experienced readers? Preliminary findings. Radiology 245(1):140–149, 2007 4. Beigelman-Aubry C, Raffy P, Yang W, Castellino R, Grenier P: Computer-aided detection of solid lung nodules on follow-up MDCT screening: evaluation of detection, tracking, and reading time. American Journal of Roentgenology 189(4):948–955, 2007 5. Bhalla M, Noble E, McLean F, Norris D, et al: Radiographic features useful for establishing patient identity from improperly labeled portable chest radiographs. Journal of Thoracic Imaging 9(1):35–40, 1994. Win 6. Bowyer K, Chang K, Flynn P: A survey of approaches and challenges in 3D and multi-modal 3D+2D face recognition. Com- puter Vision and Image Understanding 101(1):1–15, 2006 7. Branstetter B, Uttecht S, Lionetti D, Chang P: SimpleDICOM suite: personal productivity tools for managing DICOM objects. RadioGraphics 27(5):1523–1530, 2007 8. Cesarani F, Martina F, Grilletto R, et al: Facial reconstruction of a wrapped Egyptian mummy using MDCT. American Journal of Radiology 183(3):755–758, 2004 9. Committee on Quality of Health Care in America: Institute of Medicine, crossing the quality chasm: a new health system for the 21st century. National Academy Press, Washington, DC, 2001 10. Dayhoff R, Kuzmak P: Providing complete multimedia patient data to consulting radiologists, other specialists, and the refer- ring clinician. Journal of Digital Imaging 11(3):134–136, 1998. Suppl. 1 11. Dayhoff R, Maloney D, Kuzmak P, Shepard B: Integrating medical images into hospital information systems. Journal of Digital Imag- ing 4(2):87–93, 1991. Suppl. 1 12. Doi K: Current status and future potential of computer-aided diagnosis in medical imaging. The British Journal of Radiology 78, Spec. No. 1:S3–S19, 2005 13. Joint Commission on Accreditation of Healthcare Organizations 2010 National patient safety goals. Available at http://jointcommission.org/ 2012_npsgs_slides. Accessed on Jun 13, 2012. 14. Kuzmak P, Dayhoff R: The use of digital imaging and com- munications in medicine (DICOM) in the integration of imag- ing into the electronic patient record at the Department of Veterans Affairs. Journal of Digital Imaging 13(2):133–137, 2000 15. Lobo-Stratton G, Mercer T, Polman R: Patient exam data reconciliation tool. Journal of Digital Imaging 20(3):314– 319, 2007 884 J Digit Imaging (2013) 26:875–885 http://jointcommission.org/2012_npsgs_slides http://jointcommission.org/2012_npsgs_slides 16. Mettler Jr, F, Bhargavan M, Faulkner K, et al: Radiologic and nuclear medicine studies in the United States and worldwide: frequency, radiation dose, and comparison with other radiation sources—1950–2007. Radiology 253(2):520–531, 2009 17. Nait-Ali A, Cavaro-Menard C Eds: Compression of biomedical images and signals. Wiley, London, 2003 18. National Quality Forum. 30 Safe practices for better health care fact sheet. Available at http://www.ahrq.gov/qual/30safe.htm. Accessed on Aug. 13, 2011. 19. O’Connor S, Yao J, Summers R: Lytic metastases in thoracolum- bar spine: computer-aided detection at CT—preliminary study. Radiology 242(3):811–816, 2007 20. O’Toole J, Phillips P, Jiang F, et al: Face recognition algorithms surpass humans matching faces over changes in illumination. IEEE Transactions on Pattern Analysis and Machine Intelligence 29(9):1642–1646, 2007 21. Schimke N, Kuehler M, Hale J: Preserving privacy in structural neuro- images. Lecture Notes in Computer Science 6818:301–308, 2011 22. S. Tridandapani, S. Ramamurthy, S. Galgano, J. Provenzale. In- creasing rate of detection of wrong patient radiographs: use of photographs obtained at time of radiography. American Journal of Roentgenology 2013. In press. 23. Y. Turner, I. Hadas-Halpern. “The effects of including a patient’s photograph to the radiographic examination.” In: Radiological Society of North America scientific assembly and annual meeting. Oak Brook, Ill: Radiological Society of North America, 2008; 576. 24. Weiss F, Safdar N: Do digital pictures add to the interpretation of diagnostic imaging: a survey along the spectrum of experience. American Journal of Roentgenology 194:A4–A7, 2010 25. Zhao W, Chellappa R, Rosenfeld A: Face recognition: a literature survey. ACM Computing Surveys 35(4):399–458, 2003 J Digit Imaging (2013) 26:875–885 885 http://www.ahrq.gov/qual/30safe.htm Integrating Patient Digital Photographs with Medical Imaging Examinations Abstract Introduction Motivations for and Advantages of the Proposed Concept Implementation Strategies: Clinical Perspective Potential Privacy Concerns of Gathering Photographic Data Architecture for Integrating Photography with a Portable Radiography Machine Emory Prototype Android-Based Camera Device Integration Server Integrated Display of Digital Photographs and Radiographs System Requirements and Cost Considerations Conclusions and Future Work References