key: cord-0151008-dwa6ajlk authors: Zhang, Jiawei; Li, Chen; Rahaman, Md Mamunur; Yao, Yudong; Ma, Pingli; Zhang, Jinghua; Zhao, Xin; Jiang, Tao; Grzegorzek, Marcin title: A Comprehensive Survey with Quantitative Comparison of Image Analysis Methods for Microorganism Biovolume Measurements date: 2022-02-18 journal: nan DOI: nan sha: f9824d4f5f5d576a1a9288faf57aa97630606890 doc_id: 151008 cord_uid: dwa6ajlk With the acceleration of urbanization and living standards, microorganisms play increasingly important roles in industrial production, bio-technique, and food safety testing. Microorganism biovolume measurements are one of the essential parts of microbial analysis. However, traditional manual measurement methods are time-consuming and challenging to measure the characteristics precisely. With the development of digital image processing techniques, the characteristics of the microbial population can be detected and quantified. The changing trend can be adjusted in time and provided a basis for the improvement. The applications of the microorganism biovolume measurement method have developed since the 1980s. More than 62 articles are reviewed in this study, and the articles are grouped by digital image segmentation methods with periods. This study has high research significance and application value, which can be referred to microbial researchers to have a comprehensive understanding of microorganism biovolume measurements using digital image analysis methods and potential applications. Microorganisms are kinds of tiny organisms that are distributed all around the world. They exist in people's daily life but cannot be discovered by naked eyes. Various microscopes are designed to observe the microorganism precisely [97] . Microorganisms are classified into a large number of types based on different classification standards. Generally, microorganisms consist of bacteria, viruses, fungi, and algae. Bacteria are the most common unicellular organisms globally, which have a tiny size and simple biological structure. They lack nuclei, cytoskeletons, and membranous, and most of them are decomposers that can decompose dead organisms into simple inorganic substances and release energy, such as the works of Bacillus subtilis. Viruses are kinds of microorganisms with a simple structure that can infect other organisms. It contains nucleic acid, such as ribonucleic acid (RNA) and deoxyribonucleic acid (DNA). The healthy cells are parasitized and produce the nutrition for virus propagation [37] . Viruses are harmful to human and agricultural production. For instance, the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has caused significant loss of life and property all over the world [10, 91] . Fungi are kinds of eukaryotic, sporogenic, chloroplast-free eukaryotes that contain molds, yeasts and other mushrooms known. The spores are produced by sexual and asexual propagation. Fungi play an essential role in agriculture and industrial production [142] . Algae are a group of eukaryotes of the Protista. Most of them are aquatic, without vascular bundles, and can carry out photosynthesis. Some algae are single-celled dinoflagellates, while others aggregate into colonies. Algae play essential roles in food production, such as Undaria pinnatifida and Laminaria japonica, which can be used for medical purposes, like preventing and treating goitre [35] . Microorganisms are in various forms. Some are corrupt, which can cause food decomposition and adverse changes to organizational structure, but some are beneficial. For example, healthy people have a large number of bacteria in the stomach. The normal flora contains hundreds of species of bacteria. These bacteria are mutually dependent and mutualistic in the gut environment, and the disorder of bacteria can lead to diarrhea. However, some microorganisms are harmful to human health and society production. Acquired immune deficiency syndrome (AIDS) is a dangerous infectious disease caused by the human immunodeficiency virus (HIV), which can attack the body's immune system. The CD4T lymphocytes system, which is an essential part of the human immune system, is attacked violently, the cells are destroyed in large numbers, and causes the loss of immune function in the human body; SARS-CoV-2 broke out in 2019 [69, 119] , and it continues to be widely spread. More than 120,000,000 people have been infected globally till March 17th, 2021, which has caused huge economic and productivity losses [141] . Because of microorganisms' vital role globally, related microbial research is continuously developing. Microorganism quantification is one of the most important parts of microbial research. It is widely used in food and water safety test, biomedical test and environmental surveillance [95] . There are two main methods for microorganism biovolume measurement one is manual measurement, and the other is computer image analysis-based measurement [122] . Manual measurement mainly includes the biovolume and dry cell weight (DCW) measurement methods. The biovolume measurement method reflects the growth state of microorganisms by measuring the volume of mycelia contained in a certain volume of culture medium. It has the advantage of being fast and straightforward, but the deviation is relatively large. The culture medium usually contains nonbacterial solid substances that can cause measurement errors. Moreover, it does not apply to the medium for mash fermentation. DCW method is applied by collecting a unit volume of microbial culture solution through centrifugation and repeatedly washing the microorgan-isms with water. After frequent pressure or vacuum drying, the total biomass of the culture can be calculated by accurate weighing. In an environment with high target microorganism content and less non-bacterial particle impurities, it is more accurate, but it is cumbersome and time-consuming, which is not convenient for online analysis. The advantages of manual measurements are that when the sample is clear with fewer impurities, the measurement speed is rapid, resulting in higher accuracy. Nevertheless, when the sample is turbid, the impurities cannot be excluded completely, and the measurement system cannot distinguish impurity debris from target microorganisms, so the target microorganisms cannot be measured accurately. Moreover, the measurement results are subjective, and the staff's workload is as heavy and frustrating, probably leading to deviation due to fatigue [29] . An automatic computer-aided image analysis system can significantly reduce the workload and enhance accuracy [138, 150] . The different microorganism measurement methods are applied because of the different microorganism species and microscopic imaging methods. Some images with larger magnification and more scattered colonies can be measured by image analysis. In Fig. 1 , an image of a Candida yeast cell is shown. It has shown a clear boundary that can be separated. After that, the measurement of parameters such as area and perimeter can be obtained readily, and the number of objects can be counted precisely. The corresponding survey of microorganism counting approaches can be found in our latest work [148] for detail. The approaches to microorganism biovolume measurement are different from those of microorganism counting. Biovolumes can be measured from each species or genus's geometric shape and size, including categorizing some genera by morphology. However, in some cases, the biovolume cannot be measured by using relationships between microorganism counts and the biovolume of each identified microorganism species or genus [126] . Some images are blurred with smaller magnification and not clear enough for separation. For example, algae and fungi are entangled with each other that cannot be separated. Biovolume measurement can be used for quantification for this purpose. In Fig. 2 , an image of fungal hyphae with complex distribution is illustrated. In Fig. 3 , the mycelia image is shown. The classical measurement method of mycelial morphology relies on inaccurate and time-consuming manual measurements from images because of the random distribution of entangled mycelial. Moreover, the method cannot measure the hyphal branches individually. Digital Image Processing (DIP) is applied for image denoising, image enhancement, image segmentation and feature extraction by using a computer. DIP is firstly used in the 1950s, the computer is designed and can be applied to process graphics information to improve the image resolution [59] . DIP has been widely used in many fields. Agricultural and forestry departments can detect pesticides in vegetables, identify the rice varieties, design various piking robots and monitor the management of diseases and insect pests [8] . The water conservancy department can prevent the disaster of water in advance through remote sensing image analysis [136] . Image segmentation technology, image description method and pattern recognition technology are used to identify and detect fire disaster [111] . The Fig. 3 The mycelia images obtained through the program (in [140] fig.1 ) traffic controlling department can use DIP to monitor the road condition, recognize license plate number, and violate detection [45] . Medical departments can use amounts of DIP technology to automatically classify and segment various diseases [84, 86, 93, 154, 94] . The professional equipment is necessary for microbial researches to achieve the better precision. However, the use of DIP can reduce the equipment costs [46] . Meanwhile, DIP has been applied in the field of microorganism biovolume measurement widely. The trend of development is shown in Fig. 4 . It can be seen that the use of DIP in the field of microorganism biovolume measurement maintains an upward trend. Since 1980s, DIP has been applied to microorganism biovolume measurement. From 1980 to 1995, the development are relatively slow. The developments of microorganism biovolume measurement are relatively rapid from 1995 to 2010. However, after 2010, the biovolume measurement related research entered the platform period. According to the content of the papers, summed up two possible reasons for the development situation. At first, due to the development of the algorithm, especially the development of deep learning, more accurate image segmentation can be achieved. For example, the segmentation of adherent colonies can lead to more accurate microorganism counting. Secondly, it is very difficult to obtain the three-dimensional data only from the image. Image analysis can obtain the surface areas of microorganisms, but cannot accurately calculate their volume. Accurate quantification will be a challenging task without a more accurate instrument. Since microorganism counting and biovolume measurement suit different cases: In our previous work [148] , there were more than 132 papers for microorganism counting; in this study, there are 58 papers for biovolume measurement. Among all these papers, only four papers are overlapping [117, 60, 38, 41] , both for microorganism counting and biovolume measurement, indicating that they are two different research topics. Due to the biovolume measurement is an essential topic in the research of microorganisms, plenty of literature reviews are published based on the abundant research, which are summed up as follows: Bölter et al. [20] describes the research progress of microbial direct microorganism counting method, and discusses the technology and application of each direction from two aspects of microorganism counting and biovolume measurement. This review discusses data analysis and error propagation, which are widely neglected in the quantitative study of microbial ecology, and compares different microbial staining techniques and their applications. This review discusses the common staining methods and compares the advantages of each method. In this review, there are 3 papers are about calculating the error propagation and more than 23 papers are about biovolume measurement. Qiu et al. [117] reviews the application of both size measurement and counting for bacteria. The automatic flow analysis technology and classical counting methods are introduced. At first, the plate counting method is the earliest developed method, then the immunofluorescence microscopic method and most probable number (MPN) are wildly used. With the development of computer technologies, DIP has shown the extremely convenience for microbial counting analysis. In the field of cell measurement, the different microorganisms can be detected and classified by using the water flow image method. However, the microorganisms with small size are difficult to be detected. The application of resistance change method can precisely measure the number and size of the individual cell, but the classification is difficult. Another measurement method with fast speed and high accuracy is flow cytometry. However, the concentrations of samples are limited to be high. There are more than 25 papers are summarized to illustrate the application of the size measurement of cell. Gracias et al. [60] summarized the application of chromogenic and impedance method for microorganism classification and counting. The classical microbial measurement methods for food safety testing are illustrated. There are more than 28 papers are about microorganism measurement methods. Daims et al. [38] describes the microbial quantification methods based on DIP. The material difference of microbial counting and microbial biovolume measurement is the identification of a single cell. Thresholding and edge detection are most widely applied methods by using DIP. And the development of local thresholding can focus on the region of interest and obtain the better segmentation result comparing with global thresholding. There are 6 papers are about automatic cell counting, 14 papers are about quantification of cell size and 11 papers are about biomass quantification. Dazzo et al. [40] describes the application of CMEIAS (Center for Microbial Ecology Image Analysis System) computer-assisted microscopy for data extraction from images after accurately segmented, that can provide 63 different insights into the ecophysiology of microbial populations and communities within biofilms and other habitats. There are eight quantitative assessments topics are proposed, that are morphological diversity as an indicator of impacts that substratum physicochemistries have on biofilm community structure and dominance-rarity relationships among populations, morphotype specific distributions of biovolume body size that relate microbial allometric scaling, metabolic activity and growth physiology, fractal geometry of optimal cellular positioning for efficient utilization of allocated nutrient resources, morphotype-specific stress responses to starvation, environmental disturbance and bacteriovory predation, patterns of spatial distribution indicating positive and negative cell-cell interactions affecting their colonization behaviour, and significant improvements to increase the accuracy of color-discriminated ecophysiology. More than 50 papers are used to describe quantification of biofilms. Costa et al. [33] describes the development of biological wastewater treatment (WWT) using image analysis. This review introduces the fields of application of image analysis based WWT, such as aerobic wastewater treatment biological processes, calculation of sludge settling ability, measurement of sludge contents, detection of toxic compounds, biomass physiology analysis, full-scale WWT plants, aerobic granulation detection, quantification of the physical characteristics of the anaerobic granular sludge, analysis the behavior of anaerobic granules and biomass activity research. More than 110 papers are used to list the application of image analysis in WWT field but there is no definite descrip-tion for microorganism counting or biovolume measurement, and there is no mention for technique methods. Dazzo et al. [41] introduced the application of CMEIAS for microbial counting and biovolume measurement based on DIP. The shape-adaptable methods can be applied to measure the length and width of cell for the calculation of cell size. Then, a supervised tree is applied for individual cell classification. After that, a k-Nearest Neighbour (kNN) classifier is used to cluster the complex particles, such as spherical coccus and branched filament. Moreover, several parameters are calculated to location the spatial distribution of microorganisms in the biofilm. The extracted data can be analyzed by geospatial statistics, that is used to define the biogeography of microbial cells during the development of biofilm. And the colonization behavior models are generated by using the cell-to-cell interactions intensity. In the fourth part, the image segmentation algorithms are applied to show the color and spatial relationships of the foreground pixels. Finally, the applications of CMEIAS is summarized that contains the image analysis for morphological diversity, filamentous microbial morphotypes, ecophysiological attributes linked to accurate measures of biovolume body mass, spatial pattern analysis and its relationship to microbial biofilm ecology and color segmentation tool for cell-cell communication. More than 35 papers are about measurement of biovolume. Li et al. [90] introduces the development of microbial analysis based on DIP and amounts of methods used for microorganism classification are summarized. In this survey, the microorganisms are grouped based on their application domains, including Agricultural Microorganisms (AMs), Food Microorganisms (FMs), Environmental Microorganisms (EMs), Industrial Microorganisms (IMs), Medical Microorganisms (MMs), Waterborne Microorganisms (WMs) and Scientific Microorganisms (SMs). They analyse the properties of the methods below and the practical conditions of the microorganism classification tasks jointly, the methods include image preprocessing methods such as image segmentation, feature extraction methods, post-processing methods, feature fusion methods and classification methods. More than 60 related papers are summarized to illustrate different microbial classification methods. This review is a comprehensive microorganism classification paper and uses plenty of literatures for quoting, but there is no significant description for microorganism counting or biovolume measurement. The comparation of several related works and their contribution to biovolume measurement are shown in Fig. 5 . It is observed from the previous studies that the explanations about the current technologies of microor- ganism research are objective and detailed. However, the targeted research about biovolume measurement is limited. Therefore, additional research on this aspect needs to be completed. Since microorganism quantification plays a vital role in microbial research, this review focuses on applying microorganism biovolume measurement, and the prospects of the methods. This review has high research significance and application value for microbiological researchers and computer vision researchers. There are more than 62 papers are summarized in this survey about biovolume measurement. The flow chart of microbial biovolume measurement is shown in Fig. 6 to elaborate on the methods. The approach contains five steps: microbiological data acquisition, microscopic image, image preprocessing, microorganism biovolume measurement methods, and evaluation methods. The microorganisms are composed of the following seven types according to the different application domains, including agricultural microorganisms, environmental microorganisms, food microorganisms, industrial microorganisms, medical microorganisms, waterborne microorganisms and other microorganisms. Then, the digital images are obtained after staining and slicing by using professional equipment [57] . The following part is pre-processing, including image denoising and contrast enhancement, which is helpful for image segmentation. The next step is biovolume measurement methods [85] . Biovolume measurement aims to measure the morphological parameters and calculate the biovolume. The most significant part of microorganism biovolume measurement is image segmentation. Finally, the evaluation methods are proposed. However, the results obtained in these studies are area and volume, that are difficult to be compared with the results obtained by manual methods. So the proper evaluation methods can help to analyze the performance of the image analysis methods [11] . This study proposes a comprehensive review of microorganism biovolume measurement using image analysis. The research of microbial application was first developed in 1980, and the articles, including research articles and review articles, are summarized. Moreover, the application of microorganism biovolume measurement in various circumstances are discussed. More than 62 papers are selected from the initial paper dataset, and the structure of the systematic review is shown in Fig. 7 . The review is organized as follows: In Sec. 2, the DIP and evaluation approaches, which are widely applied for microorganism biovolume measurement, are summarized. In Sec. 3, the relevant research on DIP-based microorganism measurement are grouped and summarized. In Sec.4, the quantitative analysis of different DIP approaches for microorganism biovolume measure-ment is summarized. Then in Sec. 5, the commonly used approaches are summarized. Finally, in Sec. 6, this review is concluded by summarizing the whole paper. This review structure is clear enough to help researchers comprehend the DIP-based microorganism biovolume measurement's current situation. The application of DIP technology to microbial biovolume measurement can promote stability and higher accuracy by comparing it with traditional measurement. This chapter introduces the commonly used DIP technologies applied in microorganism biovolume measurement. In the process of image generation and transmission, the image quality is often degraded due to the interference The systematic flow chart of paper selection for our work and influence of various noises, which will have a negative impact on the subsequent image processing and image visual effect. Therefore, in order to suppress noise and improve image quality, denoising methods need to be applied. The main methods to remove noise are spatial filter, transform-domain filter and so on. Spatial filtering directly performs data operation on the original image and processes the grey value of the pixel, such as the Median filter and low-pass filter [85] . The transform-domain filter contains Fourier transform and wavelet transform [107] . Image segmentation is an important method in digital image processing and computer vision, which contains threshold segmentation, edge detection and region extraction. Threshold segmentation Thresholding is one of the classical methods for image segmentation, which has the advantages of simple calculation, high efficiency and fast speed [155] . The selection of the threshold is the core algorithm of threshold segmentation [128] . There are two main methods, the first one is iterative thresholding method, which can obtain the good results for the images with high global contrast [109] . Another one is the maximum inter class variance based on Otsu method, which can obtain the satisfactory result for most images [106] . This method is considered to be the best method for automatic threshold selection because it is easy to calculate and is not affected by the change of image contrast and brightness [144] . Edge detection based segmentation In the DIP, edge detection can reduce the amount of data and maintain the essential architecture of the image [63] . Edge detection includes gradient operator based method, second-order differential operator based method, LOG edge detection method and Canny edge detection method. The gradient operator based methods contain Roberts operator, Sobel operator, Prewitt operator, Kirsch operator and Robinson operator, which are the most widely applied for edge detection [58] . Sobel operator detection method has a good performance when image have a wide variety of noise, but the edge location is not accurate and the edge of the image is more than one pixel. Region based segmentation Among the region extraction methods, watershed segmentation provides excellent results and widely used for segmentation. It belongs to a segmentation method based on region growth. The calculation process of the watershed is an iterative labeling operation, which is sensitive to feeble contours [134] . The commonly used morphological operations include dilation, erosion, close operation, open operation and so on, which are usually applied in pre-and post-processing in DIP [105] . The satisfactory results can be obtained after morphological operations. The commonly used features in biovolume measurement consist area, perimeter, length and width. The area can be obtained by calculating the inner pixels, the perimeter can be calculated by counting the contour pixels. The length and width can be obtained from the features of the central axis [156] . Evaluation is an essential part of DIP, which can help researchers find the reasons that affect the result. True positive (TP), false negative (FN), false positive (FP) and true negative (TN) are four basic metrics in image classification. Commonly used evaluation metrics are shown in Table 1 . Accuracy is the proportion of correctly classified samples from all samples [98] . TP, FN, FP and TN can be composed as confusion matrix [64] . The columns of confusion matrix are predicted results and rows are real classification. Precision is the probability of samples which are correct predicted from the predicted positive samples. Recall is the proportion of positive prediction from all positive samples [24] . F1-score is the weighted harmonic average of precision and recall [28] . Pixel accuracy (PA) is a basic evaluation method in image segmentation, that is the proportion of pixels that are classified correctly. The mean pixel accuracy (MPA) is the improved methods of PA, which performs more objective [147] . Dice is applied to measure the inner similarity between segmented image and GT [12] . Intersection over union (IoU), also named Jaccard, is the proportion of intersection and union of the prediction and ground truth [121] . Evaluation is one of the most essential parts of DIP, which is the performance measurement of the model. The results of evaluation can affect the performance of microorganism biovolume measurement directly. Microorganisms come in various types, and different microorganism biovolume measurement methods show different performances. The microorganism biovolume measurement methods are suitable when colonies are adherent. In this chapter, the current applications are summarized and structured as follows: firstly, the semiautomatic methods, gray level histogram methods, thresholding methods, morphological methods, edge detection methods, and third-party tools for bacteria biovolume measurement are summarized. After that, the image enhancement-based methods, thresholding methods, machine learning-based methods, edge detectionbased methods, region connection-based methods, morphological methods, and third-party tools methods of other microorganisms measurement methods are summarized, including animal, alga, plant, fungi, spore, virus, and protozoa. Bacteria is one of the essential parts of microorganisms, and the datasets of bacteria are relatively abundant, which are usually used in microbial research. In [76] , an scanning electron micrograph based dialogue program is developed. The system needs users to mark the width and length of all bacteria and then calculates the biovolume increment of bacteria. The total time required could be reduced from 1 day to 2.5 h by using a M6520 microcomputer program. In [25] , the mean value of the RGB histogram and the ratio of red to blue intensity components are used to determine the concentration of bacteria biomass. The sum of the mean values of the RGB distributions are linearly related to the total biomass protein concentration. In [96, 112] , biofilm is applied for reconstruction and quantification of bacteria biovolume. First, a median filter [96] and contrast stretching [112] are applied for noise reduction, and then the image contrast is enhanced by using histogram stretching. After that, the method of histogram equalization and a high pass filter [112] is applied to enhance images. In [96] , the dead biofilm patches are eliminated by comparing the patch morphology with substrate morphology, and three metrics are then computed for the individual biofilm patches: patch area, patch perimeter, and circularity, which can be applied for bacteria colony biovolume measurement. The result shows that the patch morphology is useful for investigating the relationships between surface morphology with biofilm growth and determining the surface morphology and biofilm shape. In [112] , the geographical information systems (GIS) are used to reconstruct the biofilm images and measure the biovolume of bacteria. The 3-D reconstruction of the bacteria volume is shown in Fig. 8 . In [18, 65, 54] , thresholding is applied for bacteria biovolume measurement. First, contour enhancement [18] is applied for pre-processing. Then the thresholding methods with fixed value are applied for determination of bacteria biomass. The system can detect and measure the objects after segmentation and erosion operation, which provides estimates of cell number, mean volume and biovolume with standard errors of about 5%. In [65] , the connected-volume filtration can erase debris outside the region of interest after thresholding. Finally, the quantity of elements can be measured. In [54] , the useless particles are removed after thresholding segmentation. Then the wavelet transformation and densitometric techniques are applied to quantitatively estimate the growth of cells. The measurements of densitometric techniques are based on the measurements of the area occupied by the cells, the maximum and average cell grey-scale tones. It shows the predominance of larger wavelet coefficients for the images with a higher biomass content or number of cells. The processed images are shown in Fig. 9 . In [104, 125] , Otsu Thresholding is applied for bacteria biovolume measurement. Both methods use confocal laser scanning microscopy (CLSM) images of bacteria for quantification. First, Otsu Thresholding is applied for CLSM images segmentation. Then in [104] , the parameters of biofilm are obtained by calculating the binary image stacks with connected volume filtration (CVF). The number of fore-ground pixels are multiplied by the volume of voxel in a stack, that represents the biovolume. The volume of voxel is the square of the pixel size multiplied by the scanned step size. In [125] , the color channels are connected with the connected volume filtration matrix (1 = connected-biofilm bacteria and 0 = either unconnected bacteria or background noise) after Otsu Thresholding. The same pixel location of dead and live bacteria is compared. If both live and dead channels are 1, the dead channel remains 1 and the live channel converts to 0. Moreover, if the CVF matrix has the same value as the color channel, this pixel is labeled as connected bacteria. On the contrary, if the value of color channel is one, but the CVF ma-trix has a value of 0, the particular pixel is labeled as unconnected bacteria. Furthermore, if the CVF matrix has the same value of 0 as the color channel, this pixel is labeled as noise. Finally, the green and red channel points are summed up to calculate the percentage of dead and live bacteria populations. The result is shown in Fig. 10 . Fig. 10 The result images after processing (In [125] fig.4) In [22] , the adaptive thresholding is used to determine the structure of bacterial biofilm. The images of bacteria biofilm are obtained using CLSM, then an adaptive thresholding is used for image segmentation. Finally, the image structure analyzer is used to extract the features of biofilm and the thickness of biofilm is calculated for biovolume quantification. In [100] , an image analysis system is used to determine the biovolume of filamentous bacteria. The image processing program consists of three stages: image pretreatment, segmentation and debris elimination. Finally, the morphological parameters are measured to calculate the total recognized aggregates area per volume, the number of recognized aggregates per image, percentage number of recognized aggregates and recognized aggregates area percentage. In [21] , the images are separated into three types that contains cocci, rods and others, then different formulas are used to calculate the biovolume of each type. The integration of volume performs better than simple addition. In [124] , more than 30 biofilm samples are captured for measurements of several parameters, such as the area of biosurface, biovolume, and biothickness. After that, the biofilm area on each slice is obtained for measurement of biovolume. The stack of 39 biofilm images in a 15-µm biofilm is shown in Fig. 11 . Finally, the measurement of the area can be obtained by calculating the heterogeneous growth. In [127] , the contrast of images are enhanced and the channel is separated into blue and green. Then, the Marr-Hildreth filter is used for the detection of objects in green channel. After that, the useless debris are removed. Finally, the area and fiber length of fungi and bacteria in the gut of earthworm are measured. In [49] , the marine microorganisms are measured by using the 'Artek 810' image analyzer (Artek Systems Corp.,Farmingdale,N.Y.), and a variable gray-level is set to obtain the binary images. The area and perimeter are measured to estimate the abundance of marine organisms. In [80, 74, 9, 137] , the 'IBAS' (Kontron Inc., Eching, West Germany) series image processors are applied for biovolume measurement of bacteria. In [80, 74] , 'IBAS 2000' image processor is applied to measure the area of darkfield bacteria images [80] ,microcolony count, area and size distribution [74] by enhancing the cell contrast. The moving objects can be highlighted using the motility of bacteria difference images of exercised images with initial images. The result shows that computer enhanced microscopy (CEM) is better than direct microbial observation in terms of speed, precision, and quantitative data collection. In [9] , the intensity changing is corrected by using a low-pass filter. Then, 'IBAS 2000' is used to count the Staphylococci and calculate the surface area. In [137] , discriminate the area using different threshold after contrast and contour enhance-ment, and then the 'IBAS II' is applied to calculate the total particulate matter area of copepods. In [77] , the morphological processor is used to enhance the bacteria image and the 'Quanitmet 570' computer system (Leica, Cambridge, United Kingdom) is used for image processing. Then images are binarized using threshold and the calculation of biovolumes is guided by a numerical integration method by following the trapezoidal rule. In [55] , 'Image Pro Plus version 4.5' is used for quantification of bacteria. The cluster of bright pixels can be recognized as cells, whose area can be measured automatically by using 'Image Pro Plus'. The data is stored as normalized accumulated distribution of cell. Finally the cumulative cell distribution is measured. In [108] , 'bioImage_L' is used to determine the baseline physiology of dental plaque grown in a mini-flow cell system. First, the CTC (red) and Syto24 (green) are used to stain the biofilm image. Then, a Gaussian low-pass filter and Otsu Thresholding are applied for image segmentation. After that, the green channel and red channel are reconstructed in three dimensions that is shown in Fig. 12 . Finally, the biovolume of dental plaque is measured and the results on the viability of dental plaque bacteria indicates that the biovolume of the subpopulation of microbes with undamaged cell membranes accounts for 96% ± 2% of the total biofilm biovolume. In [115] , 'ImageJ' v1.41 (US National Institutes of Health) is used to determine the total biomass and the individual biomass. The calculation of total biomass is the sum of all living cells biovolume, and the individual biomass is the biovolume of single cell. Once the CLSM images are obtained, the ImageJ v1.41 software is applied for green and blue pixel enumeration of each image. Then the images are transformed to binary im-ages using an automatic threshold. The final fluorescence image of blue channel is determined by subtracting the green channel from initial image, on the contrast, the fluorescence image of green channel is determined by subtracting the blue channel from the initial image. Then the image is smoothed by median filtering. The ratio of the blue or green threshold voxels to the total threshold voxels can represent the ratio of live and dead cells. Finally, the biovolume of bacteria is measured and multiplied by a conversion factor to convert it to biomass. The process of image analysis is shown in Fig. 13 . In [56] , the 'ImageJ' and 'CellC' are applied for bacteria counting in ballast water. The size and shape of individual cells or the intensity of fluorescently labelled microorganisms are measured with quantum dots or fluorochromes. The user can adjust the parameters of the modules in software to adapt various experimental environments, then the measurement results can be obtained from hundreds of images. Fig. 13 The process of image analysis for CLSM images (In [115] fig.1) In [51] , the CMEIAS, which is proposed in [39] , is used for biovolume measurement. The shape and morphological features are extracted by using CMEIAS to calculate cellular biovolume. The accuracy and error rates are calculated, which contains seventeen distinct methods for calculation biovolume by comparing with the ground truth using DIP. It is proved that this improved technology of biovolume measurement has high accuracy in the determination of the selection of algorithm, which is applied for biovolume computation of morphologically diverse communities captured by classical microscope at single cell resolution. By reviewing the works of image analysis based on bacteria biovolume measurement and referring to Table 2 , we find that: -Development trend: The development of image analysis methods based on bacteria biovolume measurement started in the 1980s and developed rapidly in the 2010s, showing the increasing importance of bacteria research in human society. With the continuous improvement of DIP and pattern recognition, the bacteria biovolume measurement approaches will have appreciable development in the future. -Measurement techniques: The medial filter and contrast enhancement are the most frequently used preprocessing method. Image segmentation methods are thresholding and Otsu thresholding. In [31] , the 'Equalize/Best fit' method is used to enhance the contrast of phytoplankton images for obtaining good detection of filaments. The measurement of biovolume is determined by the volume calculation of a cylinder, which contains measurement for trichome and section of trichome. The diameter represents the minimum diameter and the length represents the maximum diameter. In [103, 17, 130, 48, 4, 32] , thresholding approaches are applied for microorganism biovolume measurement. First, a median filter [48] , contour enhancement and contrast enhancement [130] are applied for denoising and image enhancement. Then, the local adaptive background correction [48] is used for spatial threshold changing, which caused by the inhomogeneity of the illumination and fluorescence while imaging. After that, a high-pass filter 'Black tophat' [4] is used to extract black detail from a background with variable intensity. In [103] , a threshold is used to remove the background and the lines of fungi images are enhanced, then thinning operation is used to keep the hyphae into one-pixel width and calculate the total length. The manual method and automated method have obtained very close results. In [17] , a threshold is used to convert grey images into binary images. The average particle thickness is used to convert the area (two-dimensional) into biovolume (three-dimensional). The resulting linear relationships are found to be significant in homogeneous samples like the spheres and diatoms. In [130] , thresholding is used to separate the colony from the background and then the features are extracted. Then the area and biovolume of colony are calculated. The result shows that image processing method can be applied in interface cultivation to measure the growth conditions of colony. In [48] , thresholding is used to quantify the plankton and improve the accuracy of measurement. The images are automatically segmented into filaments and background by binarization with a single intensity threshold. After that, the segmented images are thinned and the total filament length is measured. The result shows that the filamentous bacteria can be quantified automatically, that has high accuracy and robustness. In [4] , a thresholding based method is used for automated measurement of filamentous cyanobacteria. The holes within the object are filled after thresholding. Finally, the parameters such as area and perimeter of all particles are measured, which are applied for the calculation of curve length and curve width. By comparing with the manual measurement procedure, the error of filament length calculations is less than 1%. In [32] , digital image processing is used for quantification of alga. First, the RGB images of alga are obtained form a CCD camera. The color in RGB channel of region of interest (ROI) are segmented and filtered by using two algorithms. The colored images are converted to gray-level images in first algorithm and the weights of RGB channels are obtained separately. Then the second algorithm is applied for color feature extraction. Finally, the densities of cell are calculated and the correlation coefficient between the method and manual measurement method is 0.997. In [73] , an artificial neural network is applied for algal biovolume measurement. First, a neighborhood-averaging filtering method is used to remove the noises of algal images. The filtered light distribution image is converted to a contour image by grouping the original 8bit gray scale into a desired value. Then, an artificial neural network model is used to relate the cell density in the photobioreactor with the digitized images. The present study shows that image analysis techniques can be used to measure light intensity in a photobioreactor, and then the cell concentration in a photobioreactor can be predicted with high accuracy by analyzing the image data using a neural network model. In [6] , SVM is used for plankton biovolume estimation. First, SVM is applied for plankton classification. Then, three methods are designed for biovolume measurement. In the first method, for the spherical shape particles, the projected area of each object is applied for biovolume calculation. In the second method, the measurement of biovolume is considered as the calculation of a evolution volume, which is designed manually, and then the width and length are measured for calculation. In the third method, the particles are classified based on shape features and DIP is applied for biovolume measurement. The accuracy of proposed method for plankton classification can be improved up to 86%. In [52] , an automatic cyanobacterial quantification system is developed using pattern recognition and machine learning. First, the initial images are denoised and the regions of interest are extracted, which have seven main steps. First, adaptive brightness and contrast adjustment are applied for exposure minimization. Then a Sobel based bidimensional convolution is used for contrast enhancement. After that, thresholding is applied to convert images to binary images for enhancement of connected elements in the image. Moreover, contours closing and holes filling are applied to extract explicit parts from all closed shapes. The noises and spurious objects are erased to reduce computational complexity, and the objects with low roundness value are eliminated based on roundness filter. Finally the extracted objects are thinned and pruned for debris elimination. After image processing, the filamentous objects in the image are processed as several points. And the crossing and interrupted filaments are recombined for accurate measurement. Afterwards, the parameters of filament are measured and 17 parameters are selected for ma-chine learning based on a Random Forest. The proposed method is shown in Fig. 14 . In [1] , the higher order local autocorrelational masks are used to extract plankton with their contours. The canonical correlation analysis is used to determine the area of plankton after plankton classification. The result shows that this method can measure the size of all plankton in the image with not contour labelling. In [82] , image analysis is used to estimate the cell density and biomass concentration of fungi. First, the total difference between background and foreground are minimized to obtain the threshold, that allows satisfactory result while adjusting for variations in background intensity. After that, an octagonal filter is applied for region connection, which can help to remain the tiny objects after subsequent denoising operations. Then the height and width filter are applied to remove fungi clumps. Finally, the 50 measured values of mycelium width in the stochastic selected images are averaged to estimate the cell diameter. The process of image analysis method is shown in Fig. 15 . Fig. 15 The processing steps of image analysis system for area determination of fungi (In [82] fig.2) In [2] , DIP is used for determine the biovolume of cyanobacteria. The biovolume of aggregates is calculated by dividing the whole aggregate area with average area in the same image. As for curved and coiled filaments, the images are segmented to obtain the straight parts. Then the biovolume of images are calculated and added with other measurement results, that is used for determination of total biovolume of cyanobacteria. In [81] , quantitative image analysis is applied for quantification of microorganisms in waste water. Firstly, the binary images of the biomass of aggregate and the filamentous bacteria are obtained by using DIP, which contains image preprocessing, image segmentation and denoising. Then, the size of biomass of aggregate and filamentous bacteria are calculated by using this DIP system from the binary image obtained. The biomass of aggregate can be calculated by using the projected area, and the filamentous of biomass can be measured by using the total filament length. In [3] , 'NIH-Image 1.62' (National Institute of Health, Bethesda, Md., USA) image analysis system is applied for biomass and biovolume determination of zooplankton. The major and minor axis of individual cell are measured and the corresponding volumes of the revolution ellipsoids are calculated. The proposed method can accurately estimate the biovolume and number of zooplankton based on DIP. In [101] , a digital image analysis software called 'Mi-croImage 4.0' (Media Cybernetics for OLYMPUS) is used for fungi measurement. The median and high-pass Gauss filter are used to remove noises. Then the images are segmented in HSI channel and the projected areas are estimated. Moreover, the weighed mean value of segmented region to total area are calculated as the zone fraction. The result shows that the digital image analysis of R. oligosporus hyphae can obtain the fast and accurate segmentation results. In [34] , 'KS400 software' (Kontron Electronic GMB, Carl Zeiss MicroImaging, Inc., Thornwood, NY) is used to measure the biomass growth of fungi. A high pass filter is used to obtain the binary images, then the binary filters are used to obtain the features of images for biovolume measurement. In [44] , 'KS400' is used for monitoring the biomass growth of fungi. First, the binary filters are applied to obtain morphological data. After that, a high-pass filter is applied to enhance the contrast of boundaries, and the images is binarized by using a fixed threshold. Finally, the debris are pruned and the biovolume are measured. In [132] , 'ImageJ' v1.33f (US National Institutes of Health) is used to determine the biomass of cyanobacteria based on image analysis. The threshold is calculated by 'ImageJ' that can separate the cyanobacteria clearly and then measure the biomass of cyanobacteria. It can be observed the significant differences between the manual measurement method and the proposed method (p<0.001), with a 95% confidence interval (2.31, 4.14) . The results show that the proposed method has higher accuracy when comparing with traditional manual method, particularly when the filamentous of samples have high density. In [15] , 'ImageJ' is used for quantification of filamentous fungal. Firstly, the uneven illumination is adjusted by filtering. Then, the image is converted to a 2-dimentional signal and the high-pass filtering is applied to correct the background greyscale changes. After that, the gray-level threshold is calculated by 'ImageJ' and applied to obtain the binary image. Then the watershed algorithm is applied to separate the clustered objects and a single binary close operation is applied to erase any tiny crevices of images. Finally, the image is skeletonized and measured for determination of biovolume. In [131] , 'ImageJ' 1.37v is applied to determine the biomass of cyanobacteria. First, a total of 4103 confocal images corresponding to 156 stacks were obtained. Then the threshold is used to transform to binary images and the ratio of the biovolume of object to sediment are calculated. To determine the individual biomass, a type of cyanobacteria in the stack is eliminated separately. Finally, each stack of biomass is measured. In [114] , the image analysis software 'LUCIA G/F' (Laboratory Imaging) is used for the measurement of the size and morphological features of microorganisms. There are two channels are used to correct possible pixel shift that caused by different excitation wavelengths. The first one is designed to detect DAPI stained particles by using UV-excitation, and another one is designed to detect hybrid bacteria by using blue excitation. Then, the 'AND' operation is applied for combination of binary images obtained by two channels. After that, a 'Mexican Hat' type edge detection filter is applied and the morphological operations like 'opening' and 'eroding' are applied to decrease the size of particles of processed image. Finally, the morphological parameters are measured for determination of biovolume. In [5] , 'Scion Image' 4.0.2 is used for quantification of alga. Thresholding is used to obtain binary image. After that, the area that covered by algae is dividing by the area of total image, which is calculated to represent the percentage area of alga. The number of zero pixels can be counted to quantify the area of the specific color channel in the binary image. Finally, the total number of chlorophyll is compared with the obtained result to verify the accuracy of this system. In [133] , 'ZooScan Integrated System' is used for quantification of zooplankton and the regression between image parameters with biovolume is obtained. The segmented image obtained by 'ZooScan' system is used for measurement of 46 different parameters including size, shape, gray level and the fractal dimension. The area is calculated with the size of the projection on a horizontal plane while the volume is calculated by assuming the shape of zooplankton is spheroid. In [102] , AutoCAD is applied to create the digital models of phytoplankton, then the parameters including volume and mass of object can be measured. Firstly, the cross-sectional outline of the phytoplankton images is detected. Then, the outline is revolved around the central axis to create the diatom disc. After that, holes are removed from the disc to meet the real distribution of areolae. Finally, other structural components of the phytoplankton are added. The creating process of 3Dmodel is shown in Fig. 16 . Moreover, nine representative 3D-models are created in different size. To measure the biovolume of the phytoplankton, the proposed models are fit with a regression model to get the mathematical parameters of biovolume measurement. In [99] , Opera Phenix software is applied for imaging and measuring the biovolume of phytoplankton. After data acquiring, the images are processed in two methods. First, a 3D projection is rendered from the discrete confocal images, and the software can measure the bio- Fig. 16 The creating process of 3D-model for phytoplankton biovolume measurement (In [102] fig.2) volume of detected region of cells. Second, a 2D projection can be converted from the confocal images, which has the same view of a light microscope. Finally, the biovolume of phytoplankton can be measured. In [23] , Blender 2.79 is applied to create the 3-D model of algae and measure the biovolume. After modelling, a software tool, NeuroMorph, is applied to measure the area of the surface by calculating all area of each quadrangulated polygon of the mesh. Finally, the biovolume can be calculated. In [13] , Minitab version 16 (Minitab Inc., USA) is applied to determine the biovolume of Mycoplasma fermentans. First, a median filter is used for noise removing in each slice of images. Then, the filtered images are segmented based on thresholding to separate the microcolonies and realize the visualisations of 3D isosurface. After that, Amira, version 5.4 software (Visualisation Sciences Group, USA), is applied to measure the biovolume of attached biofilm cells. Finally, MATLAB (Version R2013b, The MathWorks Inc., USA) is applied to measure the holes between cells. The measurement of channel diameter sizes is shown in Fig. 17 . In [47] , the biovolume estimation of cyanobacterial colonies by using NeuroMorf software is compared with the other two methods. The first method is the traditional microorganism counting determined by experts. The second one is based on morphological approach, measuring the sphere packing, cell size and cell distance. Then, an approach based on regression is applied to calculate the relationship between cell number and the result of biovolume measurement. The ration between the measured biovolume using estimating and 3D modelling can be applied as the bias of estimation. Finally, ordinary least square analysis is applied to study the relationship between the colony volumes and overall cell biovolume in the colonies. By summarizing the relevant research on DIP based on other microorganism's biovolume measurements and referring to Table 3 , it can be seen that: -Development trend: The DIP-based measurement for other microorganisms methods began in the 1980s and developed in the 2000s. It can be seen that the researches on other microorganism biovolume measurement have a similar development trend to related research on bacteria. However, bacteria research is relatively abundant and comprehensive, which may be caused by the limited dataset of other microorganisms such as fungi, alga, and viruses. On the other hand, compared with the hyphae of alga, bacteria have a relatively simple structure, which makes it challenging to segment completely. -Measurement techniques: The most frequently used pre-processing method are the medial filter and contrast enhancement. Image segmentation methods are thresholding and Otsu Thresholding. In order to compare the performance of the DIP approaches mentioned above for microorganism biovolume measurement, a benchmark database is applied for the experiment of quantitative analysis. There are few open-source databases of the microorganism with adherent mycelia [153] . Therefore, we downloaded and adjusted the yeast database proposed in [42] to meet the requirements of biovolume measurement. An image with and without a mask and an adjusted mask of the yeast database is shown in Fig. 18 . By reviewing the biovolume measurement approaches proposed in Sec. 3, DIP techniques are commonly applied in image pre-processing and segmentation tasks. The outcomes of different approaches can be evaluated systematically, which can be further used to analyze the measurement performance quantitatively. Moreover, with the development of deep learning, we found that there is no existing work on the deep learning-based biovolume measurement method. Hence several deep learning image segmentation methods are also applied for biovolume measurement and compared with classical segmentation methods. This section is organized as follows: In Section 4.1, several image pre-processing methods, such as filtering and morphological operations, are compared. In Section 4.2, several segmentation approaches, which contain classical segmentation and deep learning based segmentation, are applied and evaluated to show their performance in the task of biovolume measurement. Digital Image pre-processing methods are applied to deal with the noise and enhance the contrast. Gaussian filter is one of the linear filters, which can effectively suppress noise and smooth the image [108, 101] . The process of the Gaussian filter is similar to the mean filter. It takes the mean value of pixels in the filter window as the output. The mean filter window value is 1, but the value of the Gaussian filter decreases with the increase of distance from the center of the window. Therefore, the Gaussian filter performs less fuzzy to the image than a mean filter. Another widely applied filter is the median filter, which replaces the value of the center pixel as the median value of the sorted pixel in the filter window [96, 115, 48] . To show the denoising abilities of three filters above, the salt and pepper noise with a ratio of 0.3 and Gaussian noise with a mean of 0 and a variance of 0.01 are added to the images, respectively. Then, the images are denoised with three filters above. The images after filtering with 3×3 mean filter, 3×3 median filter and Gaussian filter with the cut-off frequency of 50 are shown in Fig. 19 . Moreover, mean square error (MSE) and signal noise ratio (SNR) [50] are applied to evaluate the performance of denoising, which is shown in Table 4 . By comparing the denoising results, the images after Median filtering have lowest MSE and highest SNR in both cases, indicating the process of Median filtering can maintain more information and suppress more noises. However, the results after Mean filtering and Gauss filtering seem unsatisfactory because of noises and blur contours, and showed low contrast between the objects and background. Morphological close operation can also be applied for denoising and debris erasing [132, 52] . The particles smaller than the size of kernel can be eliminated after close operation. The yeast images after close operation are shown in Fig. 20 . The noise of images are apparently reduced, but the contours of yeast cells are relatively fuzzy. Histogram equalization is one of the most used approaches for image enhancement [112, 96] . It stretches the histogram of the images as an approximately uniform distribution to enhance the contrast of the images. Histogram equalization can make the brightness distributes better on the histogram. It can enhance the local contrast without affecting the overall contrast, and histogram equalization achieve this function by effectively extending the commonly used brightness. The yeast images after global histogram equalization are shown in Fig. 21 . The result is unsatisfactory because the noises spread all around the image, which makes the computer challenging to focus on the foreground and the background. In the task of image analysis based microorganism biovolume measurement, the segmentation performance determines whether the measurement result is good or not. The approaches based on thresholding can perform well when the foreground and background are in different range of gray scale, and they can easily be applied due to the small computation cost. The comparative results of thresholding (value=115) and Otsu Thresholding are shown in Fig. 22 . It can be seen that both results have satisfactory segmentation performance, but the results after thresholding have more noise than the Otsu Thresholding results obviously, which need to be post-processed. By reviewing the works of biovolume measurement, edge detection approaches are also widely applied [52, 127, 114] . The core algorithm of edge detection is to determine the edge pixels in the image first rather than connect these pixels to form the region boundary. Sobel operator is a discrete differentiation operator, which can be applied to calculate the approximate gradient of the grayscale image [52] . The pixel with a large gradient is more likely to be the edge. The Sobel operator can calculate the derivative in the horizontal and vertical directions, but it is relatively sensitive to the noise. Hence, the pre-processing of denoising is necessary. The canny operator is a multi-level detection algorithm that contains denoising, gradient calculation, non-maximal suppression, and the method of double thresholding [14] . Non-maximum suppression is an edge thinning method to retain the local maximum gradients and suppress the other gradients. The canny operator performs well for most of the images. In the Marr-Hildreth operator, a Gaussian filter is applied first for smoothing, and then the Laplacian is applied for image enhancement [127] . Finally, the point of intersection between zero axis and the second derivative line can be determined as the edge. The comparative results of Sobel, Canny and Marr-Hildreth are shown in Fig. 23 . The segmentation results of Sobel operator have more noise obviously, and the Marr-Hildreth results have the problem of over-detection, which means many inner pixels are classified as edge. Canny operator performs best by comparing with the other approaches. Watershed is a segmentation method based on region growing [15] . The gray level of all pixels are sorted and then the growing region is labelled based on local minimum. The results of watershed is shown in Fig. 24 . It can be seen that the watershed approach has the problem of over-segmentation,hence it may meet the requirements of adherent objects separation, but should not be applied for global segmentation. Since the development of deep learning, several excellent backbones (such as SegNet and U-Net) have been widely applied for image segmentation and can obtain satisfactory results [151, 123] . The first 13-layer convolutional networks of VGG16 are applied in SegNet as an encoder, and SoftMax is applied for classification after the decoder. U-Net is an end-to-end net of image segmentation with an encoder and a decoder. It was designed for medical image segmentation, which meets the similar requirements of biovolume measurement as both of them are microscopic images, and the datasets are limited. SegNet and U-Net train the yeast dataset for 60 epochs with a batch size of 8. The CNN-based models can be trained independently without relying on feature engineering, and the models can obtain the expected results, showing strong generalization abilities. The most-used pre-processing approach in deep learning is data augmentation to avoid overfitting and to increase generalization capabilities, but a limited preprocessing technique is used for denoising [53] . Here, SegNet and U-Net are end-to-end models trained without any pre-processing or pre-training. The segmentation results of SegNet and U-Net are shown in Fig. 25 . The results show that U-Net has better performance for biovolume measurement, which means that concatenated operation of encoder and decoder can improve the precision of segmentation. To quantitative analyze the performance of segmentation approaches above, several classical indices are applied for evaluation, including Accuracy, Dice, Jaccard, Precision and Hausdorff distance. The definition of Accuracy, Dice, Jaccard (IoU) and Precision are summarized in Table 1 . Hausdorff distance is applied to measure the Euclidean distance between the predicted and GT images [70] . Dice is more sensitive to the inner filling of the mask, while Hausdorff Distance is more sensitive to the separated boundary. The comparative results of different approaches is shown in Table. 5. It is observed that the segmentation performance of deep learning models are much more better than the classical segmentation approaches, obviously. Canny operator achieves the best result in classical segmentation methods, which is much more better than Sobel and Marr-Hildreth operators. U-Net is the best model in the task of microorganism biovolume measurement, indicating that the architecture of encoder-to-decoder and concatenation operation can indeed improve the capability of image segmentation. The microorganism biovolume measurement methods based on DIP are summarized in Sect. 3. By reviewing all the approaches, image pre-processing and segmentation are the most import tasks in biovolume measurement. The performances of these methods are summarized and analyzed, including their advantages and potential applications. Most microorganisms are mirrored as colorless and have the same color as the environment. So, the color feature is not worthy of image segmentation. Moreover, due to the inhomogeneity of the illumination and imaging noise, pre-processing approaches can be used for denoising and contrast enhancement. At first, the images are converted to gray-level images to eliminate the worthless color feature. After that, the Intensity of images with Hue-Saturation-Intensity (HSI) can be changed individually, that has no influence to color of images, such as the works in [32] , [101] . Secondly, the Gaussian low-pass filter and linear transformation can be applied to reduce the influence of uneven illumination, which is caused by uneven shading, such as the work in [132] . Thirdly, denoising is necessary for preparation of subsequent image segmentation. The median filter and Gaussian filter perform well for noise removal, which are widely applied. Gaussian filter is a linear smoothing filter to eliminate Gaussian noise, which is widely applied in image denoising. Gaussian filtering denoising is the weighted average of the gray value of the whole image. The value of each pixel is calculated by the weighted average of its own value and the value of the neighbor pixels. Median filter is a non-linear smoothing filter, which replaces the gray value of the pixel as the median gray value of neighbor pixels. The comparison of Gaussian filter and Median filter is illustrated in Fig. 19 , which shows the Gaussian filter with proper parameters performs best in the task of image denoising. The application of morphological close operation can remove the useless debris. The performance of close operation is shown in Fig. 20 , the noise of image is erased and the contour lines are smoothed, but the edges are relatively fuzzy, which makes it difficult for segmentation. High-pass filter can also be used for denoising, such as the work in [44] , the performance is shown in Fig. 26 . Finally, the colorless microorganisms have low contrast resolution, which are inconspicuous in environment. Contours with high contrast can perform better in image segmentation. One of the most frequentlyused global methods is gray level histogram equalization, which has satisfactory result in most images, such as the work in [112] . Histogram equalization can indeed improve the contrast of the foreground and the back-ground, which is shown in Fig. 20 . It shows that the contour lines of yeast cells are enhanced and easy to identify, which is helpful for image segmentation. However, the image noises in the background are enhanced meanwhile, hence the denoising operation is necessary to reduce the influence of noises. In [52] , contours closing, holes filling and Sobel operation are applied to make solid and unambiguous areas for contour enhancement, the performance is shown in Fig. 27 . Image segmentation is the most vital task in microorganism biovolume measurement. The colony region needs to be segmented from the whole image for subsequent processing. At first, thresholding based image segmentation methods are widely used for microorganism biovolume measurement. Global thresholding can easily segment the regions of colony after proper pre-processing, such as the works in [54] . The results of global thresholding on a benchmark dataset are shown in Fig. 22 . By comparing with the ground truth, the contour lines of yeast cells are segmented precisely, but some inner holes are wrongly classified as the foreground. Hence, the global thresholding can obtain the satisfactory segmentation results in general, but there are still some problems in detail. Secondly, the performance of Otsu Thresholding, such as the works in [104] , [125] and [108] , seems more satisfactory with fast computing speed. However, the process of noise removal is necessary before Otsu Thresholding calculation because of the high sensitivity. The performance of Otsu Thresholding based image segmentation is shown in Fig. 28 . The results of Otsu Thresholding on a benchmark dataset are shown in Fig. 22 . By comparing with the segmentation results of global thresholding, Otsu Thresholding based segmentation have less noise and the result is more similar to the ground truth, which is proved in Thirdly, the approaches of edge detection are widely spread in microorganism biovolume measurement. Sobel edge detection method performs well in microorganism segmentation, such as the work in [52] . The combination method of Gaussian filter and Laplacian filter, that is, Marr-Hildreth operator, performs well for image denoising and contour enhancement, such as the work in [127] . In [114] , a Mexican hat filter is applied for edge detection, which is the second derivative of the Gauss function. The advantage of the Mexican Hat wavelet transform is that it has good localization in the time domain and frequency domain. It is symmetric and can be used for the continuous wavelet transform. And it is close to the spatial response characteristics of human eyes. The comparison results based on the proposed edge detection approaches on a benchmark dataset are shown in Fig. 23 . It shows that Sobel operator based edge detection is sensitive to noises, which means lots of noises of the background are recognized as the foreground. Hence the denoising operation is necessary before Sobel filtering. The segmentation results of Marr-Hildreth operator exist the problem of oversegmentation. Many inner pixels are identified as the edge and the results remain a lot of noises. Canny operator based segmentation performs best, most of the noises are erased and the contour lines are relatively clear and accurate. The quantitative analysis in Table. 5 also proves that Canny operator based segmentation has the highest accuracy and the lowest Hausdorff distance, which shows it has the highest similarity with ground truth. Finally, the adherent colonies need to be separated, which is usually achieved by using watershed algorithm. Separation of attached colonies is necessary when the biovolume need to be calculated individually, such as the work in [15] . The segmentation results based on watershed is shown in Fig. 24 . Watershed can segment the images into several individual parts and the contour lines can be detected accurately, indicating it can perform well in the task of microorganism counting (meet the requirement of separate the clustered colony into individual cells). However, the problems of oversegmentation still need to be solved. Classification is necessary when measuring the biovolume of microorganisms. According to supervised learning, the support vector machine (SVM) is one of the classical linear classifiers, such as the work in [6] . The morphological features are obtained for training, and the classification groups are shown in Fig. 29 . SVM performs well with small samples and can be applied for high-dimensional classification using different kernels. Fig. 29 The classification groups with their morphological features (In [6] fig. 2) Secondly, an artificial neural network (ANN) is a self-learning non-linear network with strong fault tolerance. In [73] , an ANN is used to relate the cell density in the photobioreactor with the digitized images. ANN can learn the classification mechanism automatically, which can be applied to obtain the satisfactory results, but the learning process is unobservable and the output is hard to interpret. Thirdly, roundness filter is applied to remove useless debris after segmentation, such as the work in [82] . In [52] , the roundness value of remaining elements is estimated to erase all objects with a roundness value > 0.4. When the target particles are circular, the particles with low roundness value should be erased. However, when the target particles are non-circular, such as fungi and alga, the particles with high roundness value should be erased. The performance is shown in Fig. 30 . The microorganism biovolume measurement approaches based on DIP can not only be referred by microbial measurement researches, but also referred by other DIP fields. For example, such as the environmental microorganism classification [75, 152, 78, 89] , blood cell classification [135] , classification for different types of microorganisms [90, 75, 88] , microorganism image generation [143] , microorganism object detection [87] , medical image classification [120, 118] , medical object tracking [157, 26] and human health monitoring [92, 129, 68] . After that, the segmentation methods for microorganisms can be referred to by DIP researchers, such as stem cell segmentation [67] , near-duplicate detection [139] , image enhancement [116] , cancer cell segmentation [27] , and environmental microorganism segmentation [151, 79, 149] . Moreover, microscopic image processing performs an essential role in industrial analysis, such as the monitoring for wastewater [7] , beef carcass evaluation [36] , monitoring of bacteria in milk [113] , monitoring flames in an industrial boiler [146] , softwood lumber grading [16] and so on. Microorganism quantification consists of microorganism biovolume measurement and microorganism colony counting, and the colony counting can be regarded as a measurement approach with higher accuracy. Object counting is one of the most essential parts of computer vision, which can be modified and applied in microorganism quantification. In [145] , the reverse perspective network is applied for object counting to solve the problem of input im-age scale variation. The perspective estimator can calculate the perspective parameter and the coordinate transformer can convert the images to similar size. The weight of ground truth is promoted for training by using an adversarial network. The proposed method can obtain the satisfactory result for the dataset with large variation of scale. The frame work of proposed method is shown in Fig. 31 . Fig. 31 The framework of reverse perspective network(In [145] fig. 4) In [72] , the traditional detection based methods are replaced by the density estimation based crowd counting methods, which can detect the density of crowd with high performance and high precision. However, the encoder and decoder of traditional CNN based density estimation methods have low correlation, which is difficult to remain the spatial information of the image. The Trellis Encoder-Decoder networks including multi-path decoder and encoder, is proposed to improve the accuracy of counting. The structure of network is shown in Fig. 32 . Only two pooling layers are applied to reduce the loss of spatial pixel accuracy. In addition, a multiscale encoder is designed to improve the adaptability of network to the large variation of the object size. In [30] , an image-leve supervision based image segmentation method is proposed for object counting. The traditional object counting approaches based on regression can obtain the counting results with high accuracy. However, the results can only reflect the global counting but cannot locate the objects that are counted. The Image-level lower-count (ILC) supervised density map estimation is proposed in this article for crowd counting and segmentation. The architecture of the network is shown in Fig. 33 . In addition, the development of an attention mechanism can be applied in image processing to focus on the region of interest (ROI). Spatial transformer networks (STN) can locate the region of interest and transform it into an ideal image by affine transformation, and then put it into the neural network for training [71] . STN consists of Localisation net, Grid generator and Sam- Fig. 32 The architecture of Trellis Encoder-Decoder network(In [72] fig. 1 ) Fig. 33 The architecture of object counting method(In [30] fig. 3) pler. The Localisation net can be applied for calculation of affine transformation parameters. The Grid generator can find the mapping between output and input features, and the corresponding relation of pixel coordinates is obtained. The Sampler can select input features based on the position mapping and transformation parameters, and output the image with bilinear interpolation. The architecture of STN is shown in Fig. 34 . Another network based on attention mechanism is Squeeze-and-Excitation Networks (SENET). The application of attention mechanism in STN is based on the spatial domain, while SENET is based on the channel domain. SENET can learn the correlation of each feature channel, and the weight value of each channel is calculated, so that the following network can improve the selected significant feature channels and the unworthy feature channels are restrained. The structure of SENET is shown in Fig. 35 . The satisfactory ROI extraction method provided by the attention mechanism can support a possibility to extract the target microorganism region in an image with a complex environment. Fig. 35 The structure of one Squeeze-and-Excitation block in SENET (In [66] fig. 1) Furthermore, Transformer is a type of self-attention mechanism-based deep neural network that is wildly used in natural language processing (NLP). A transformer has been developed to satisfy computer vision because of the strong representation ability recently. The use of Transformer performs better by comparing with other networks, such as CNN, which has shown strong competitiveness [62] . A transformer has a more robust ability of global information representation than CNNs, making it possible to describe the microorganism structure in a complete image. Especially, Vision Transform (ViT) is one of the most remarkable visual transformer method till now, which directly applies sequences of image patches (with position information) as input first. The ViT projects the patches to the original transformer encoder and classifies the images with a multi-head attention mechanism as it does in NLP tasks. The framework of ViT is shown in Fig. 36 . Fig. 36 The structure of the ViT(In [43] fig. 1) Finally, the development of microorganism biovolume measurement based on DIP has high correlation with the development of relevant researches, such as the technique of immunofluorescence staining for cells [83] , confocal laser scanning microscope (CLSM) imaging [19] , epifluorescence microscope equipment [110] , and so on. In this paper, a comprehensive review of DIP based microorganism biovolume measurement is proposed. The current methods are generalized and summarized, and grouped as bacteria and other microorganisms, such as fungi, alga and so on. In each type of microorganism summarization, the methods are grouped based on image segmentation algorithm, including classical thresholding methods, self-designed methods, and third-party tools. In Sect. 3.1.1 and Sect. 3.2.1, it can be seen that the classical approaches are developed first, from 1980s to 2000s. By reviewing the classical methods, the performances are limited and unsatisfied. In Sect. 3.1.1 and Sect. 3.2.1, the development of self-designed methods for biovolume measurement is rapid. Self-designed methods can process the microorganism data by using specific approaches to match their different features, which show satisfactory result by comparing with the classical methods. In Sect. 3.1.3 and Sect. 3.2.3, a mass of professional DIP systems are applied for microorganism biovolume measurement, such as 'ImageJ' and 'CellC'. The integration software are more professional and performs well in specific field of biovolume measurement, which shows the microorganism biovolume measurement is increasing important. In conclusion, the prosperous development of DIP based microorganism biovolume measurement methods show the extensive research potential in microbial research. After that, in Sect.4, the most used image pre-processing and segmentation approaches are quantitatively analyzed to show their performance in the task of microorganism biovolume measurement. Finally, the widely applied microorganism biovolume measurement approaches such as image pre-processing, image segmentation, and connected domain detection are analyzed in Sect. 5. Design and testing of an underwater microscope and image processing system for the study of zooplankton distribution Image analysis for qualitative and quantitative evaluation of planktic cyanobacteria Estimating zooplankton biomass through image analysis Automated measurements of filamentous cyanobacteria by digital image analysis Image analyses-based nondisruptive method to quantify algal growth on concrete surfaces Improvement of plankton biovolume estimates derived from imagebased automatic sampling devices: application to Flow-CAM Activated sludge monitoring of a wastewater treatment plant using image analysis and partial least squares regression Rapid quantification of staphylococci adhered to titanium surfaces using image analyzed epifluorescence microscopy The proximal origin of SARS-CoV-2 Automatic image classification for the urinoculture screening Validate conference paper using dice coefficient Visualisation and biovolume quantification in the characterisation of biofilm formation in Mycoplasma fermentans An algorithm for counting microorganisms in digital images Morphological quantification of filamentous fungal development using membrane immobilization and automatic image analysis Softwood lumber grading through on-line multivariate image analysis techniques Image analysis as a tool for measuring particulate matter concentrations and gut content, body size, and clearance rates of estuarine copepods: validation and application Automatic determination of bacterioplankton biomass by image analysis Fully automatic determination of soil bacterium numbers, cell volumes, and frequencies of dividing cells by confocal laser scanning microscopy and image analysis Enumeration and biovolume determination of microbial cells-a methodological review and recommendations for applications in ecological research Determination of bacterial biovolume with epifluorescence microscopy: comparison of size distributions from image analysis and size classifications Sequential development of biofilm spatial structure of Pseudomonas Aeruginosa tagged with SYTO9/PI Biovolume and surface area calculations for microalgae, using realistic 3D models. Science of The Total Environment The relationship between recall and precision Control of a Thiobacillus denitrificans bioreactor using machine vision SVIA dataset: A new dataset of microscopic videos and images for computer-aided sperm analysis Automated segmentation, classification, and tracking of cancer cell nuclei in time-lapse microscopy The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation Urine sediment examination: a comparison of automated urinalysis systems and manual microscopy Object counting and instance segmentation with image-level supervision Evaluating biomass of Baltic filamentous cyanobacteria by image analysis Evaluation of Isochrysis galbana (clone T-ISO) cell numbers by digital image analysis of color intensity Quantitative image analysis for the characterization of microbial aggregates in biological wastewater treatment: a review Digital image processing as a tool to monitor biomass growth in Aspergillus niger 3T5B8 solid-state fermentation: preliminary results Micro-algae. Manual on the production and use of live food for aquaculture Beef carcass evaluation by use of a video image analysis system Origin and evolution of pathogenic coronaviruses Quantification of uncultured microorganisms by fluorescence microscopy and digital image analysis CMEIAS Quadrat Maker: a digital software tool to optimize grid dimensions and produce quadrat images for landscape ecology spatial analysis In situ ecophysiology of microbial biofilm communities analyzed by CMEIAS computer-assisted microscopy at singlecell resolution Use of CMEIAS image analysis software to accurately compute attributes of cell size, morphology, spatial aggregation and color segmentation that signify in situ ecophysiological adaptations in microbial biofilm communities A convolutional neural network segments yeast microscopy images with high accuracy An image is worth 16x16 words: Transformers for image recognition at scale Lipase production in solid-state fermentation monitoring biomass growth of Aspergillus niger using digital image processing Car license plate recognition using color features of Persian license plates Digital image processing techniques Uncertainties of cell number estimation in cyanobacterial colonies and the potential use of sphere packing Determination of the filamentous cyanobacteria Planktothrix rubescens in environmental water samples using an image processing system MacImage: a user-friendly image-analysis system for the accurate mensuration of marine organisms Brief review of image denoising techniques. Visual Computing for Industry Accuracy of biovolume formulas for cmeias computer-assisted microscopy and body size analysis of morphologically diverse microbial populations and communities ACQUA: Automated Cyanobacterial Quantification Algorithm for toxic filamentous genera using spline curves, pattern recognition and machine learning A review on deep learning techniques applied to semantic segmentation Application of the wavelet image analysis technique to monitor cell concentration in bioprocesses Quantification of cell size distribution as applied to the growth of Corynebacterium glutamicum Optimizing the heat exchange in ballast water treatment using software monitoring methods Automated immunofluorescence for enumeration of selected taxa in supragingival dental plaque Digital image processing Digital image processing using MATLAB A review of conventional detection and enumeration methods for pathogenic bacteria in food Cell identification and sizing using digital image analysis for estimation of cell biomass in High Rate Algal Ponds A Survey on Visual Transformer Image segmentation techniques. Computer vision, graphics, and image processing The derivation of global estimates from a confusion matrix Quantification of biofilm structures by the novel computer program Squeeze-and-excitation networks Stem cell microscopic image segmentation using supervised normalized cuts Sleep stage classification for child patients using DeConvolutional Neural Network The continuing 2019-nCoV epidemic threat of novel coronaviruses to global health-The latest 2019 novel coronavirus outbreak in Wuhan Comparing images using the Hausdorff distance Spatial transformer networks Crowd counting and density estimation by trellis encoder-decoder networks Image analysis of light distribution in a photobioreactor Effect of laminar flow velocity on the kinetics of surface recolonization by Mot+ and Mot-Pseudomonas fluorescens Environmental microorganism classification using conditional random fields and deep convolutional neural networks Microcomputer-assisted biomass determination of plankton bacteria on scanning electron micrographs Automated confocal laser scanning microscopy and semiautomated image processing for analysis of biofilms A New Pairwise Deep Learning Feature For Environmental Microorganism Image Analysis A state-of-the-art survey for microorganism image segmentation methods and future potential Computer-enhanced darkfield microscopy for the quantitative analysis of bacterial growth and behavior on surfaces Microbialbased evaluation of foaming events in full-scale wastewater treatment plants by microscopy survey and quantitative image analysis An image analysis technique to estimate the cell density and biomass concentration of Trichoderma reesei Multiplex immunofluorescence staining and image analysis assay for diffuse large B cell lymphoma A review for cervical histopathology image analysis using machine vision approaches A review of clustering methods in microorganism image analysis A comprehensive review of computer-aided whole-slide image analysis: from datasets to feature extraction, segmentation, classification, and detection approaches A State-of-the-art Survey of Object Detection Techniques in Microorganism Image Analysis: from Traditional Image Processing and Classical Machine Learning to Current Application of content-based image analysis to environmental microorganism classification Environmental microbiology aided by content-based image analysis A survey for the applications of content-based microscopic image analysis in microorganism classification domains A SARS-CoV-2 Microscopic Image Dataset with Ground Truth Images and Visual Features Comparison of feature learning methods for human activity recognition using wearable sensors A Comprehensive Review of Markov Random Field and Conditional Random Field Approaches in Pathology Image Analysis. Archives of Computational Methods in Engineering Is the aspect ratio of cells important in deep learning? A robust comparison of deep learning methods for multi-scale cytopathology cell image classification: From convolutional neural networks to visual transformers Highthroughput imaging of bacterial colonies grown on filter plates with application to serum bactericidal assays A method for rapid analysis of biofilm morphology and coverage on glass and polished and brushed stainless steel Brock biology of microorganisms Discrimination, classification, identification of microorganisms using FTIR spectroscopy and chemometrics. Vibrational spectroscopy Phytoplankton carbon and nitrogen biomass estimates are robust to volume measurement method and growth environment Dilution and magnification effects on image analysis applications in activated sludge characterization Physiological properties and enzymatic activities of Rhizopus oligosporus in solid state fermentations On the matter of phytoplankton: A novel method using 3D computer models to calculate biovolume of microorganisms Automated image analyss method to determine fungal biomass in soils and on solid matrices Analysis of a marine phototrophic biofilm by confocal laser scanning microscopy using the new image quantification software PHLIP Multiscale morphological segmentation of gray-scale images A threshold selection method from gray-level histograms Two denoising methods by wavelet transform Image analysis software based on color segmentation for characterization of viability and physiological activity of biofilms An iterative thresholding algorithm for image segmentation Automated enumeration of groups of marine picoplankton after fluorescence in situ hybridization Infrared image sensor for fire location Reconstruction and computation of microscale biovolumes using geographical information systems: potential difficulties Semi-automated counting of bacteria and somatic cells in milk using epifluorescence microscopy and television image analysis New image analysis tool to study biomass and morphotypes of three major bacterioplankton groups in an alpine lake Viability and biomass of Micrococcus luteus DE2008 at different salinity concentrations determined by specific fluorochromes and CLSM-image analysis A Comprehensive Overview of Image Enhancement Techniques Advance in measured techniquesof aquatic bacterial counting and cell sizes A survey for cervical cytopathology image analysis using deep learning Identification of COVID-19 samples from chest X-Ray images using deep learning: A comparison of transfer learning approaches DeepCervix: A deep learning-based framework for the classification of cervical cells using hybrid deep feature fusion techniques Optimizing intersection-overunion in deep neural networks for image segmentation A review of methods for the detection of pathogenic microorganisms Machine learning and deep learning based computational approaches in automatic microorganisms image recognition: methodologies, challenges, and developments Three-dimensional quantification of soil biofilms using image analysis Enhanced analysis of bacteria susceptibility in connected biofilms Cyanobacteria species dominance and diversity in three Australian drinking water reservoirs Origins and fate of fungi and bacteria in the gut of Lumbricus terrestris L. studied by image analysis A survey on global thresholding methods for mapping open water body using Sentinel-2 satellite imagery and normalized difference water index Towards large-scale multimedia retrieval enriched by knowledge about human interpretation Morphological analysis of monascuson surface fermentation Confocal laser scanning microscopy image analysis for cyanobacterial biomass determined at microscale level in different microbial mats A new method based on image analysis for determining cyanobacterial biomass by CLSM in stratified benthic sediments Relationship between shape parameters and dry weight of the dominant zooplankton in Jiaozhou bay based on image method Quantitative analysis of watershed geomorphology A neural-networkbased approach to white blood cell classification. The scientific world journal ALOS/PALSAR Image Processing Using Dinsar and Log Ratio for Flood Early Detection in Jakarta Based on Land Subsidences Measuring selectivity of feeding by estuarine copepods using image analysis combined with microscopic and Coulter counting Automatic recognition of cancerous cells using mathematical morphology A review on nearduplicate detection of images using computer vision techniques Fullyautomatic measurement of mycelial morphology by image analysis Coronavirus covid-19 global cases by the center for systems science and engineering (csse) at johns hopkins university (jhu) Introduction to fungi An enhanced framework of generative adversarial networks (EF-GANs) for environmental microorganism image augmentation with limited rotation-invariant training data Characteristic analysis of Otsu threshold and its applications Reverse perspective network for perspective-aware object counting Monitoring flames in an industrial boiler using multivariate image analysis Image segmentation evaluation: A survey of unsupervised methods. computer vision and image understanding A comprehensive review of image analysis methods for microorganism counting: from classical image processing to deep learning approaches An application of Pixel Interval Down-sampling (PID) for dense tiny microorganism counting on environmental microorganism images Applications of Artificial Neural Networks in Microorganism Image Analysis: A Comprehensive Review from Conventional Multilayer Perceptron to Popular Convolutional Neural Network and Potential Visual Transformer LCU-Net: A Novel Low-cost U-Net for Environmental Microorganism Image Segmentation. Pattern Recognition p A Comparative Study of Deep Learning Classification Methods on a Small Environmental Microorganism Image Dataset (EMDS-6): From Convolutional Neural Networks to Visual Transformers EMDS-6: Environmental Microorganism Image Dataset Sixth Version for Image Denoising, Segmentation, Feature Extraction, Classification, and Detection Method Evaluation A comprehensive review for breast histopathology image analysis using classical and deep neural networks An image segmentation algorithm in image processing based on threshold segmentation Morphological features of cell death TOD-CNN: An effective convolutional neural network for tiny object detection in sperm videos This work is supported by the "Natural Science Foundation of China" (No. 61806047), "Sichuan Science and Technology Program" (No. 2021YFH0069, 2021YFQ0057, 2022YFS0565). We thank Miss Zixian Li and Mr. Guoxian Li for their important discussion.