key: cord-0035956-5nc8f3a4 authors: Pathegama, Mahinda P.; Göl, Özdemir title: Interactive Real-time Image Analysis System for Distant Operation date: 2005 journal: Information Processing and Security Systems DOI: 10.1007/0-387-26325-x_3 sha: a84ab3275d255662f3709efaba15d7394959073c doc_id: 35956 cord_uid: 5nc8f3a4 This paper reports on the development and implementation of an integrated and interactive system for cell analysis featuring remote operation and real-time analysis for generating analytical data from microscopic images. The system consists of a number of image processing modules implemented in a virtual instrumentation environment, combined with novel techniques developed for thinning and local edge-gap filling in the cell image segmentation process. These approaches, integrated with advances in networking, have been initially applied to viral feature analysis in SARS-CoV microscopy. Real-time operation through the user-interface of the proposed system generates quantitative results for remote clients. The rapidity and viability of operation permit the investigation of mutant viral agents on the basis of their morphological cell features. Advancement of image processing techniques is a welcome adjunct in electronmicroscopic cell analysis and is the object of increasing interest in research for the enhancement of accuracy in disease diagnosis. Electron microscopy is notable for its rapidity of operation and open view that permit detection and identification of emergent viral agents [1] . Electron microscopy does not require specific reagents for, or prior knowledge of, a particular agent, but can nevertheless categorise a pathogen on the basis of its morphological cell features [2] [3] . However, full exploitation of this potential requires the coordinated application of electron microscopy together with other frontline technologies [1] . The emergence and rapid spread of SARS (Severe Acute Respiratory Syndrome) has dramatically emphasised the need for both collaborative international networks [4] and linking of personnel from distant locations with laboratories monitoring and analysing test samples of the causative agent by electron microscopy. Remote monitoring and analysis is thus beneficial for the diagnosis of disease by experts in distant locations as it offers the opportunity for collaborative involvement in this type of disease surveillance. Remote access to stored medical images in telemedicine enables the transmission of images with patient information over the Internet. Although many system developments have the ability to transmit hospital data on patients through the communication protocol TCP/IP [5] , the combined facilities of real-time operation and analysis are yet to be developed for many applications. Remote users would derive considerable benefit from a system endowed with facilities enabling image acquisition, camera control, real-time monitoring, automated pattern recognition for cell feature extraction and quantitative result generation. This paper reports on the experience gained from developing and implementing an integrated and interactive system for remote cell analysis in order to produce quantitative data. System viability is demonstrated by applying it -in the first instance -to SARS-CoV identification. 2 System Features Upon activation, the proposed system first seeks to confirm the availability of a camera coupled to the microscope and then checks the system's ability to acquire images continuously. If none is available, the program searches in image databases for any image which has been stored by the operator. The image acquisition card used in this system is PCI-1490, which enables highresolution, measurement-quality images to be obtained from a RS-170 standard camera. Images are acquired at double-speed with 60 frames per second for progressive scan and interlaced perception. This "frame-grabber" has three independent DMA (Data Memory Access) controllers with each controller performing scatter-gather DMA, which means that the DMA controller reconfigures on the fly, and thus performs continuous image transfers to either contiguous or fragmented buffers. Four external triggers are used for pulse generation to send commands via digital I/O lines embedded in the board for remote control of the camera connected to a frame-grabber. Additional digital outputs are used to transmit pulses for camera zooming, focusing, lighting and directional turning, so as to offer more functionality over the Internet to remote users. The acquisition method used in this implementation utilises relatively inexpensive digitised electron microscopy instead of robotic electron microscopes having embedded acquisition devices. It is suggested [1] that diagnostic electron microscopy will be neither expensive nor difficult to perform if implemented in a diagnostic network. Machine vision utilised in the proposed system enables realtime monitoring of cell images, which are obtained from cameras coupled to electron microscope. Medical diagnosis, based on machine vision, is becoming increasingly used in medical practice. Advantages of machine vision over human vision arise from the fact that machine vision precludes factors which can lessen diagnostic accuracy such as personal bias, physiological factors and ambient conditions. For instance, receptive cells in the human eye may respond in a particular way to the edge information of a particular image when processing image information. Direct visual inspection of medical images may result in the false interpretation of object information, if edge effects are present in the image.. An example of this situation is given in Fig. 1 which shows an edge-effect included in the image of a partly occluded cell. This type of phenomenon is referred to as the Craik-O'Brien-Cornsweet Effect [6] . The proposed system implemented for remote operation performs analysis tasks, which can be controlled through graphical user interfaces (GUIs). The main GUI includes live image acquisition, image enhancing, processing and analysis based on image processing techniques including morphological transformations. The main interface programmatically links various analytical programs, which have separate control panels appearing as sub-windows within the main GUI. Various sub-windows, example of which is given in Fig. 2 , are employed in the remote system to make the analysis easier, providing user-friendly interactive operation. The implemented system utilises client-server technology by a LabVIEW® Server [8] , which enables the detection and acceptance of the network connection from the browser, the retrieval of programs and the transmission of data between the server and clients via the communication protocol TCP/IP. Each processing step of the system -implemented with virtual instrumentation programs (Vis) -performs objective tasks, calling Vis remotely through userinterfaces and ActiveX automation. The embedded data socket technology [8] shares the live data over the Internet. The data socket delivers the data in a variety of formats, such as strings of text, arrays of data as spreadsheets and sound files. Separate data socket connections added to each required terminal in the Vis make the data available to remote locations through their own data socket connections. These connections in each VI enable remote users to manipulate the controls through the GUI and to receive new values through the data socket as they flash up on the client machine. The server publishes GUIs as PNG formatted images as these compress graphics better than JPEG compression technique that may cause loss of detail in images. Enhanced security controls deny access to unauthorised users. In addition, the processed images and data can be stored, with later access to the database facilitating observation and further analysis by experts. Health management services are enhanced by the availability of remote systems as they provide 24-hour access although staff may only be available during daytime. This means that at night-time in one location, the system can be operated by staff in another country where it is daytime. Operation of the system at a distance also facilitates the reduction of bio-safety risks in the case of viral disease. For instance, when a SARS case re-emerged in Singapore, investigation showed that the SARS patient had most likely acquired the infection from the laboratory where the patient had worked [11] . The image processing techniques used for the remote analysis constitute the key elements of this system. A series of image processing techniques implemented in the Vis and related sub-Vis can readily be controlled from a distant location using the system. The techniques used aid in image enhancement and provide feature highlighting needed for quantitative report generation on each cell object detected in the microscopic image. The first processing step extracts the value pane from the image displayed on the interface by effective grey-level morphological transformation. As the output images provided by each step continue into the next step, a copy of each output image is numbered and stored in temporary memory, enabling other desired actions such as subtraction, addition and comparison with other output images to be carried out. The Gaussian filter used with a 3x3 kernel as a smoothing filter recalculates each pixel value based on the coefficients of the convolution kernel. The next step of the procedure performs edge detection using non-linear spatial filters namely, Gradient, Sobel, Prewitt, Roberts, Differentiation and Sigma operators, which can be selected from the user-interface as desired. The Sobel operator is set as default to facilitate automatic processing. Thresholding applied to the output image creates a binary image. Thresholding is always a subjective process and sometimes produces more information than is needed for the consequent binary image, leaving noisy particles or less information on edge detail than is needed. These deficiencies and the remaining superfluous incomplete cells on image corners are rectified by the following novel approaches. The preliminary image processing steps implemented in the VI environment employ mathematical morphology [9] . The morphological operation applied to the procedure uses structuring elements, which systematically move across the entire image, matching representative pixels with corresponding pixels, either retaining or deleting pixels to suit the application. This template acts as a criterion for removing any features that do not match the template pixels, within some tolerance. The template used to remove each small particle uses eight-connectivity, which specifies how the algorithm determines whether an adjacent pixel belongs to the same particle or to a different one. When the system uses the input image in Fig 3(a) , the processing step provides the image as in Fig. 3 (b) that include incomplete cell objects touching the image border. The superfluous elements are deleted by using the structuring element in the automated process, since they will not count as cells for the final quantitative analysis. After this step and binary image inversion, the application of the thinning process converts the remaining thick boundary to a one pixel thin boundary, at the same time deleting the remaining pixels. Fig. 3(c) shows the output edged image obtained by thinning, which overlays the first copy previously created in the temporary memory. Cell images having a cluttered background usually produce open edges in the cell boundaries. When these edge-gaps are monitored, the GUI enables the user to switch on the local edge-filling process, which displays a sub-control panel. Cell boundaries with even a one-pixel open gap will not be labelled and will not be given due recognition in the analysis procedure to continue. The process of edge-gap filling [10] creates new pixel information on corresponding empty neighbours using directional sensitivity information of edgeend pixels. This procedure searches the entire image, finding edge-end pixels row by row. The relevant direction of each of the detected pixels takes into consideration the known alignment of neighbouring pixels, setting the direction of the incomplete pixel as opposite to that of the neighbouring cell direction. Using the proposed system, the SARS-CoV microscopic image shown in Fig. 4 (a) has been analysed from a distant location over the Internet. The image processing steps, including the edge filling, provide the required contiguous boundaries. Fig. 4(b) shows the output boundary image which overlays the base image by weighted averaging. The labelling operation, which is the next step performed by Vis, groups the same types of particles and applies colours to the objects having contiguous boundaries, as shown in Fig. 4(c) . The Vis of the remote system detect the spatial characteristics of the labelled objects in Fig. 4 (c) and measure each object. The Vis return a set of measurements as an array of coefficients recording relevant values for each object. The cell boundary (Bilayer) is detected as Object 2 with the background being detected as Object 1. The highlighted boundaries of N-protein (viral nucleocapsids) are detected as Objects 3 to 8, and their measured data are placed in a spreadsheet. Detected Object 10 is the lOOnm scale given by electron microscopy and stands for the calibration of the system's units into required units; in this case in nanometres. The generated data show that the SARS-CoV (Object 2) has a longest distance of 96nm across the image and a perimeter of 314nm. A previous SARS study stated that the examination of negative-stain electron microscopy shown in Fig. 4 (a) revealed that SARS-CoV has a diameter of 80nm to 140 nm [2] . The Vis embedded in the system generate multiple parameters to identify and classify the objects allowing speedier investigation of detected cell features. The system produces a variety of measurements, including shape, orientation, longest segment, movement of inertia, etc. The movement of inertia gives a representation of the distribution of the pixels in the object with respect to its centre of gravity. For example, the Vis convert the detected object to a circle having the same perimeter and then produce a value which is half the radius of the new circle, being the hydraulic radius for the detected object. The data sheet generated on the client window shows that hydraulic radius of the bilayer in the example shown in Fig. 4(c) is 19.8nm. The Waddel disk diameter of 89.8nm given for the bilayer is derived from the diameter of the circle with the same area as the object. 4 Conclusion The integrated system described affords the ability to perform remote analysis and report generation for cell analysis, as demonstrated in the case of a negativestained SARS-CoV microscopic image. The system functions rapidly, producing reliable measurements followed by the cell image analysis procedures implemented on a virtual instrumentation platform. Electronic information and communication technologies embedded in the system will support collaborating healthcare professionals separated by distance. Electron microscopy for rapid diagnosis of infectious agents in emergent situations A novel coronavirus associated with severe acute respiratory syndrome A morbillivirus that caused fatal disease in horses and humans Severe Acute Respiratory Syndrome (SARS) Construction of a system for the access, storage and exploitation of data and medical images generated in radiology information systems (RIS) Visual Perception Coronavirus LabVIEW measurement manual Image analysis and mathematical morphology An artificial neural process for edge-linking in biological cell image analysis Update: Review panel concludes that laboratory was source of SARS-CoV infection in Singapore patient