key: cord-0179396-sxxl3rfz
authors: Son, Raku; Yamazawa, Kenji; Oguchi, Akiko; Suga, Mitsuo; Tamura, Masaru; Murakawa, Yasuhiro; Kume, Satoshi
title: Morphomics via Next-generation Electron Microscopy
date: 2021-11-29
journal: nan
DOI: nan
sha: aed0ad5b1eadfbad9a3c6ef84dd7a400bcb9261b
doc_id: 179396
cord_uid: sxxl3rfz

The living body is composed of innumerable fine and complex structures and although these structures have been studied in the past, a vast amount of information pertaining to them still remains unknown. When attempting to observe these ultra-structures, the use of electron microscopy (EM) has become indispensable. However, conventional EM settings are limited to a narrow tissue area that can bias observations. Recently, new trends in EM research have emerged that provide coverage of far broader, nano-scale fields of view for two-dimensional wide areas and three-dimensional large volumes. Together with cutting-edge bioimage informatics conducted via deep learning, such techniques have accelerated the quantification of complex morphological images. Moreover, these advances have led to the comprehensive acquisition and quantification of cellular morphology, which is now treated as a new omics science termed 'morphomics'. Moreover, by incorporating these new methodologies, the field of traditional pathology is expected to advance, potentially with the identification of previously unknown structures, quantification of rare events, reclassification of diseases and automatic diagnosis of diseases. In this review, we discuss these technological and analytical advances, which have arisen from the need to analyse nano-scale bioimages, in detail, as well as focusing on state-of-art image analysis involving deep learning.

It is said that 'a picture is worth a thousand words'; in line with this sentiment, scientists have been developing tools and techniques to visualise biological specimens for around 400 years. Since Robert Hooke first published 'Micrographia' with beautiful illustrations of cells and living organisms in 1665 [1], the use of light microscopy has led to many important discoveries, not only of various microorganisms, including Mycobacterium tuberculosis [2] and Treponema pallidum [3] , but also of cellular components, such as red blood cells [4] , capillary vessels [5, 6], brain neurons [7-9] and intracellular structures including nuclei [10] .

In 1932, electron microscopy (EM) invented by Von M. Knoll and Ernst Ruska [11, 12] advanced imaging in the field of biology, with one example being the microscopic observation of Escherichia coli at a magnification of 10,000 times [13] . In the 1940s, EM enabled the discovery of virus and phage particles [14, 15] , which stimulated the later development of virology. Initially, the application of EM in biological research was difficult because of the lack of histological techniques [16] ; however, with the development of fixing and staining methods using aldehydes and heavy metals [17] [18] [19] [20] [21] [22] [23] [24] [25] [26] [27] [28] [29] [30] , EM was applied more broadly to histology [31] . The use of EM has also revealed a variety of cellular functions such as autophagy [32] , slit structures in kidney glomeruli [33] and undifferentiated cell states in induced pluripotent stem cells [34, 35] . Thus, the technological developments in EM revealed a new world of intracellular nano-metre-scale histology that brought with it major biological insights.

In the last decade, two important trends in EM research have primarily emerged: (1) coverage of a two-dimensional (2D) wide-range field for simultaneous capture of many cells and/or whole tissues at high resolution [36, 37] ; (2) three-dimensional (3D) resolution, which provides a volumetric viewpoint and reveals the stereoscopic morphology of whole cells and the intercellular connections in tissues [37] [38] [39] . These developments potentially facilitate the handling of large bioimaging datasets and/or the collection of comprehensive morphological data from biological specimens [40] [41] [42] . Thus, EM has gained attention as a potential new omics modality. In this review, we discuss the application of 'big data' analysis to nano-scale bioimages and we highlight the use of deep learning for state-of-art image analysis.

EM produces outstanding images of membranous cellular structures that maintain cellular morphology and contribute to intracellular/extracellular functions including intracellular transport, phagocytosis and migration [43, 44] . The cell forms a thin and flexible lipid bilayer boundary that holds both hydrophilic and lipophilic solutes, such as DNA/RNA, proteins, glycogen, lipid droplets and minerals, inside the cell [45] . Lipid membranes are also located in cellular organelles such as the nucleus, Golgi apparatus, mitochondria and endoplasmic reticulum [45, 46] . EM can capture the membrane structures, and the cytologic images show the localisation and distribution of cellular components and/or their cellular dynamics.

In practice, the structure and localisation of cellular components and organelles can be captured at the nano-scale using chemically fixed biosamples and a resin-embedded ultra-thinsectioning EM method [47, 48] as follows. To preserve the characteristic microstructures of cell membranes and cellular solutes, specimens must undergo double fixation using a glutaraldehyde agent and osmium tetroxide [49, 50] . Osmium staining is used to selectively visualise intracellular structures as the chemical covalently binds to osmiophilic materials, such as unsaturated fatty acids and biomolecules with unsaturated bonds [29, 51, 52], thereby conferring some electron density to the osmiophilic substrates. Additional staining using heavy metals, such as lead [19, 25] , acetic uranyl [23, 53] , gadolinium [54] and neodymium [55] , is also performed to improve the contrast of intracellular components. For the preparation of bulk tape-collecting ultramicrotome (ATUM), a device that automatically collects serial tissue sections using an ultramicrotome and magnetic polyimide tape [79, 80] . The ATUM devices are mainly used in connectome research, which aims to elucidate the network of neurons in the entire brain [81, 82] , but they may be used in other research fields. Recently, automated ultramicrotome techniques and diamond knives particularly for use in continuous section preparation have been developed [83] ; thus, several hundred to one thousand serial sections can be prepared and collected. Further details are outlined in Section 5.

Current ultramicrotomes are limited in terms the width of sections that can be cut (1-2 mm) and the range of movement of the z-axis of the device. Challenges also exist with highthroughput section preparation and multi-specimen processing. To resolve such problems, the Yagishita Giken Co., Ltd. (Wako, Japan) and RIKEN research groups are developing an ultrathin section preparation system that can prepare and collect large-area ultra-thin sections by combining machine elements and machining tools for precision cutting. For example, a microtome equipped with a new cutting technique is being developed; in this ultramicrotome system, the aim is to control the cutting feed axis and infeed axis with high repeatability and positioning performance. The recovery system for ultra-thin sections should be able to numerically control the running speed to match the stable tape running speed and the intermittent generation speed of the sections. A synthetic diamond knife has also been developed with a blade width of 10 mm, a blade angle of 50° and stability in terms of material and quality. To obtain stable ultra-thin sections, it is necessary to optimise the processing conditions by observing the cutting surface properties of the specimen and measuring the cutting resistance force. In time, additional researchers from various fields are expected to contribute to morphology research and accelerate the development of technology in this field.

In biological and medical fields, EM observations typically involve bioimaging of stained thin sections of plastic-embedded samples using transmission electron microscopy (TEM) [84, 85] . In TEM, the device accelerates an electron beam with an extremely short wavelength and irradiates the thin section. Through detection of the transmitted and forwardscattered electrons through the thin section [86], a 2D projected magnified image of the specimen can be obtained at a sub-nano-scale resolution (Fig. 1a) . Specimens such as bulk tissues must be sufficiently thin to allow electrons to pass through. The ultra-thin sections are typically ≤100-nm-thin because thicker sections cause inelastic electron scattering [87] . Such sections are placed on a metal mesh grid for observation [88] .

Some limitations exist when attempting to prepare ultra-thin slices suitable for TEM observation. First, the metal mesh itself interferes with observations of overlapping tissues [88, 89] . Second, the brittle slices are prone to breakage, which restricts imaging time. Third, since large specimens do not fit in a single field of vision (FOV) of the microscope [85, 90], a controlled system is required to automate imaging process [91] and a handling system is needed for large-sized digital images [42] . These limitations have restricted the use of conventional TEM when observing narrow areas or a relatively small set of cells [84] . Consequently, the current application of EM in clinical diagnosis is limited to assisting diagnosis of renal diseases, undifferentiated tumours, metabolic diseases that mainly affect the muscles or nerves and diseases with unknown aetiologies [85, 92] .

Some progress has been made in overcoming the limitations associated with TEM observations. First, a large-sized window and tough supporting film with uniform thickness have been developed to assist with observing wide-range areas [37, 93] . One such supporting film, the LUXfilm® support film, is a highly transmissive and robust film that is better suited for automatic TEM workflows; however, it produces substantial noise without any noise reduction [94] . Konyuba et al. proposed a large-scale silicon nitride window chip deposited using low-pressure chemical vapour deposition as a new support grid for wide-area TEM imaging [95] . This chip is mesh-free, which allows wide-area support for the specimen without creating imaging interference.

A large number of digital TEM images can be captured using an auto-acquisition system with the device [96, 97] . When the physical movement on the microscope stage is not sufficiently precise to obtain the required imaging resolution [98] , computational registration and stitching techniques of digital images can be used; these reconstruct single-captured widearea images from individual tile images [90, [99] [100] [101] [102] [103] [104] [105] [106] [107] . These tiled images are also known as montage or mosaic images. Toyooka et al. reported the use of wide-area TEM imaging with a tiled scan of a whole plant cell; this technique successfully produced 3,000-5,000 digital images with the desired range of observation and comprehensive detection of plant organelles [91, 108, 109] . Bock et al. reported electron micrographs of an entire 120,000 × 80,000 pixel thin section of the mouse visual cortex using controlled automated x-y stage motion and image acquisition [37] . Faas et al. performed large-scale EM analysis known as virtual nanoscopy, a methodology for ultra-structurally mapping regions of cells and tissues as large as 1 mm2 at a nano-metre resolution [36] . Lamers et al. imaged human severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2)-infected intestinal organoids autonomously using virtual nanoscopy slides and TEM tomography [110] . As shown in Fig. 2a , through wide-range TEM imaging, including TEM [JEM-1400/Matataki Flash Camera (2,048 × 2,048 pixels), Jeol Ltd., Japan] with a silicon nitride window chip and an automated montage system on Limitless Panorama software (Jeol Ltd., Japan), it was possible to obtain a view of mouse glomeruli that consists of 8,500 tiled images. Using this technique not only preserves the conventional resolution required to capture the basement membrane of the glomeruli and podocyte foot effacement but also enables imaging of multiple glomeruli within the same captured view.

The nanotomy project (http://www.nanotomy.org/) and its related works provide systematic virtual nanoscopy studies mainly using scanning-type TEM in which the electron probe is scanned across the sample and the transmitted signals are detected point- Large-scale EM with energy-dispersive X-ray (EDX) analysis enables the acquisition of elemental composition patterns from the surface of samples as well as the visualisation of traditional grey-scale EM images for composition-based interpretation [128, 129] . EDX analysis of the rat pancreas has been used to distinguish, for example, cytoplasmic mitochondria and granules via elemental fingerprinting [120]; thus, analysis of disease tissues may also be possible using composition-based EM. In the objective identification of human pancreas cell types with type 1 diabetes, element maps of granule content produced using EDX analysis have provided data on the elemental variation of granule content within each of the aforementioned cell types [112] . Thus, EDX analysis enables unbiased fingerprinting of cell types and the functionalities of each cell can be inferred from elemental fingerprinting.

Scanning EM (SEM) involves the use of a different type of electron microscope to that used for TEM (Fig. 1b) . According to reports, SEM was developed in 1965 [130, 131] , about 30 years after TEM. In SEM, the incident electron probe scans across the surface of a specimen in a raster fashion [132] , and the interaction between the relatively heavy elements containing the sample and the impacted electrons produces various types of emissions including secondary electrons, backscattered electrons and characteristic X-rays [132] . By detecting such emission types, SEM creates images that reflect the topological contrast or compositional information of specimens as signal intensities in the digital images [132, 133] as an electron source are known as field-emission scanning electron microscopes; they produce high-resolution images because of the smaller spot size from these emitters [132], the negative stage bias potential [161] and the improved sensitivity of the multiple electron detector system [132, 162, 163], even when the ultra-thin tissue sections are <100 nm. The backscattered electron detection by SEM when using resin-embedded ultra-thin sections provides a reverse contrast of the view that is conventionally possible when using TEM [164, 165] . Although the contrast of the backscattered images is reversed, the quality of the images is sufficient to enable general morphological analysis from TEM observations [85, [166] [167] [168] [169] [170] [171] [172] [173] . Furthermore, SEM observations provide high-resolution histological images that are independent of section thickness through regulation of acceleration voltage. Observation of tissue sections through a combination of SEM and resin-embedded sectioning is also known as the section SEM method. In addition, other SEM methods, such as helium ion scanning microscopy [174] [175] [176] [177] Recently, the popularity of SEM has lowered the accessibility of the usage and potentially lowered the operating costs [183] . For surface observation with the SEM, various shapes of sample stands can be used provided that they are not made of electrically charged materials. A silicon wafer is a typical base used for biological SEM specimen observation [184]; indeed, huge specimen bases, e.g. 10-cm-diameter wafers, are available. Silicon wafers also adhere well to ultra-thin sections. Sections scooped up on the wafer can be stably stored even when the section is large [50] . In addition, glass slides that are inherently prone to static electricity can be used under conductive treatment by applying a metal coating to the slides before SEM observations are performed [185, 186] . Because SEM has typically been used for surface observation of bulk samples, the sample storage space is designed to have a large XYZ dynamic range. Such advantages of the conventional technique can be fully exploited with the improvement of SEM devices. In other words, the stable fixation of the sample on a sample board has made it possible to observe samples over a long period. The large XY dynamic range facilitates the introduction of relatively large sections (i.e. several millimetres), multiple sample sets and hundreds of sections into the instrument at the same time. The use of the range in the z-direction has made it possible to include microfabrication methods, such as knife cutting and laser cutting, in the sample chamber of the scanning electron microscope [164, 187, 188] .

With the aim of practically producing a fish-eye perspective view, early panoramic imaging with SEM was performed in the 1990s [189] . To convert the mosaic images of SEM into a combined image, montage capturing software and image stitching algorithms were developed, similar to existing software for microscopy images [102, 103, [190] [191] [192] [193] [194] [195] . For widerange SEM imaging, Brantner et al. demonstrated large-area and high-resolution mosaic imaging of a 2.5 × 1.8-mm mouse spinal cord resin section (a biologically relevant scale) using the workflow of Chipscanner's laser interferometer stage, FOV mapping and an image stitching technique [196] . Kataoka et al. indicated that stitching SEM enabled the observation of an entire pulmonary alveolus with influenza virus particles in a resin section [197] . More et al. applied a montage SEM imaging technique to quantify the number, myelination and size of axons in the rat fascicle using a computer-assisted axon identification and analysis method [198] . Maeda et al. reported the results of cell counting with autophagy-like vacuoles in wide tissue fields (~600 images in a total area of 0.25 mm2) of the mouse cortex using an automatic acquisition system for tiling SEM images [50]. Kume et al. reported an imaging database of wide-range montage SEM images and their metadata for various tissues, including those from the kidneys, liver and brain cortex region of rodents and human cultured cells [42] . Figure 2b shows imaging data obtained using wide-area montaged SEM images of a rat liver. We integrated more than 1110 images to reconstruct the rat liver leaflet in this large-area image (1 x 0.6 mm). Strikingly, we were able to observe the whole liver lobule while preserving the spatial resolution in EM. The image information obtained using wide-area EM is substantial, which makes interactive visualisation difficult. To solve this problem, image data is converted to the Deep Zoom Image format, which is a layered format with different resolutions on a pyramid structure; this allows interactive zooming in and out for improved visualisation using web software such as Google street view and OpenSeadragon [36, 42, 100]. The use of wide-area EM imaging avoids arbitrary selection of target regions in experimental or diagnostic specimens, and it enables the efficient and comprehensive observation of biological tissues in a time-efficient manner without susceptible bias.

SEM imaging is sometimes more time consuming than TEM imaging due to raster fashion scanning. Thus, methods for speeding up SEM imaging have been developed as follows: (1) image capture with a higher speed single beam, (2) imaging different sections in parallel on multiple EM devices and (3) parallelised imaging of the same section using multiple scanning beams. As a method of parallelised imaging, Eberle et al. demonstrated a throughput imaging technique with multibeam SEM [199] [200] [201] . In this system, 61 electron beams are scanned over the sample with one global scanner and secondary electron signals are acquired for each scan position of each beam [199] [200] [201] . The multibeam SEM then produces 61 montaged SEM images simultaneously as a hexagonal FOV. In the resultant images, all membranes of neural tissue were clearly visible and intracellular organelles were distinguishable [199] . One hexagonal FOV is used to image a region of about 100 µm2; however, by performing montage imaging, a region ≥1 mm2 can be imaged [199] . Pereira et al. reported that a surface area of 5.7 mm2 could be imaged in a human femoral neck tissue sample, resulting in 897 hexagonally shaped multibeam FOVs comprising ~55,000 high-resolution image tiles and 75,000 megapixels [100]. Multibeam SEM with 196 electron beams has also been developed; this SEM device was designed to detect transmission electrons and backscattered electrons [202] . Thus, multibeam EM systems contribute to high-speed collection of digital images. The applications of multibeam SEM include a much wider-range 2D imaging in addition to 3D EM analysis and brain connectomics research [81, 82, 203, 204] .

To obtain histological and cellular images of targeted 2D regions, the use of ultra-thin section EM techniques with resin-embedded samples is widely accepted and has led to new biomedical discoveries [31] . Indeed, a cellular image obtained from only one tissue section contains substantial biological and medical information. The steric and complex communication of many cells allows living tissues to exhibit and maintain their function [81, 205, 206] . Occasionally, the appearance of characteristic cells and compositions in diseases serves as a biomarker for disease identification [112, 207] . However, the thickness of ultra-thin sections is 50-200 nm; assuming that the actual size of a cell is ~10 µm, one ultra-thin section can be used to interpret cellular events in around one-fiftieth to one-two hundredth of the total cell volume. In most cases, even within the same cell, the shape of the cell nucleus differs greatly depending on the cutting angle and position of the cross section (Fig. 3a) . In other words, when a cell image is observed in a cross section, it is difficult to precisely describe whole-cell morphology. In addition, there is an ongoing debate among researchers as to whether the cellular view obtained from extremely thin sections contains artificial cutting bias such as compression. In such cases, visualising the entire morphology of target cells or tissue regions in 3D resolution is required. To realise 3D-directed resolution in EM techniques, observing multiple tomographic images for each cross section one-by-one is a reasonable method [208, 209] . Thus, it is expected that the generalisation of stereoscopic EM techniques will lead to a deeper understanding that would otherwise not be obtained using conventional 2D EM techniques. Here, we reviewed the morphomics techniques used to obtain volumetric EM images (Fig. 4) .

Focused ion beam SEM (FIB-SEM) is used to observe the surface of a specimen milled by an ion beam on the sample stage of the scanning electron microscope [210] . By repeatedly and alternatively exposing and imaging the new top surface, serial images are captured, although the cutting surfaces cannot be preserved (Fig. 4a ). FIB-SEM offers the best z-axis resolution at 4-5 nm; thus, it is suitable for mesoscale observations such as for the observation of cellular organelles [211, 212] .

FIB-SEM was originally applied in the 1990s [213] , at which time the area covered by ion beams was far smaller than it is today. As the area of observation is enlarged in FIB-SEM, it is commonly applied to various biological samples. Moreover, the outstanding z-axis resolution of FIB-SEM has seen it applied for observations of intracellular events and organelles [214] 

Serial block-face SEM (SBF-SEM) is used to observe an exposed sample surface cut using a built-in diamond knife [232] . Compared with FIB-SEM, SBF-SEM facilitates the handling of a broader area as well as faster sample sectioning. SBF-SEM produces 1,000 3D EM images, but it cannot preserve processed sections (as with FIB-SEM) (Fig. 4b) .

The prototype of SBF-SEM was produced in 2004 by Denk et al. [233] . They not only showed the power of the technique to reconstruct 3D tissue nano-structures but also directed visualisation of neural circuit reconstructions in neuroscience research [164, 234] . In brain research, the largest mammalian cerebral cortex dataset yielded a reconstruction ~300-fold larger than that in previous reports, which allowed the analysis of axonal patterns [235] . In kidney analysis, Ichimura et al. revealed novel 3D structures in rat podocytes [206, 236] [247] and cultured cells [248] [249] [250] [251] .

Array tomography (AT) is also used to achieve stereoscopic EM (Fig. 4c ). In the AT method, serial ultra-thin sections are prepared from a resin-embedded block using an ultramicrotome and then the same site for each section is observed sequentially using TEM or (primarily) SEM [252] . A continuous tomographic image is then reconstructed to obtain the 3D tomography. Unlike other methods, the AT method is notable for its capacity to preserve thin sections, which could then be re-observed later. The resolution of the z-axis in the AT method is the thickness of the section, which is approximately 50-100 nm. Combining the AT method with SEM potentially allows for wide-area volumetric observations [40] . This technique is also known as the serial-section SEM method [184] .

The idea for creating serial sections dates back to the 1950s, soon after the first ultramicrotome became available [77] . Ribbon-like serial sections were transferred to supporting grids for observation. Later, in a 1970s report on 3D construction of the juxtaglomerular apparatus of the rat kidney, 500 serial sections were obtained and TEM observations were successful [208] ; however, 3D illustrations were limited. Subsequently, TEM was used for 3D EM, especially until the 2000s. Serial-section TEM (ssTEM) images have been acquired from serial sections of differentiating monocytes [78], neuron connections [253] , yeast cells [254] and the endoplasmic reticulum [255] . Notable results of AT and TEM combined include whole-imaging of an adult Drosophila brain using a custom high-throughput ssTEM platform developed by Zheng et al. [256] . This volumetric morphology obtained by ssTEM has contributed to mapping brain-spanning circuits and accelerated research in the field of neuroscience.

In parallel with the development of ssTEM, AT combined with SEM was proposed in 2007 [257] . This combination method has been used to study varicella-zoster virus-infected cells [258] , the Golgi apparatus in different cell types [259] and sorted immune cells [260] . In addition, we could successfully generate a 3D volume EM image of a human leukaemia cell and the macula densa in the distal tubules of a mouse kidney glomerulus using AT combined with SEM (Fig 3bc) . The SEM-based serial-sectioning method is suitable for relatively large samples because it collects larger-area serial thin sections onto the silicon substrate or glass slide [184, 261] . However, the AT method is generally challenging because it is difficult to manually prepare continuous ultra-thin sections of hundreds to thousands of samples. To improve the AT technique, customised AT methods have been developed such as magnetic collection of ultra-thin sections [203] . Among these, the tape collection system using ATUM has facilitated automatic collection of tissue serial sections and volumetric SEM [79, 267] . This ATUM-based AT-SEM method was used to clarify the sub-volume of the mouse neocortex from ~2,000 serial sections [81] as well as all myelinated axons of the zebrafish brain from 16,000 serial sections [268] . Morgan et al. imaged 10,000 sections of the mouse visual thalamus, which were produced using ATUM to a thickness of 30 nm, with an imaging volume for the dataset of 0.8 mm × 0.8 mm × . Moreover, the use of such large-scale stereoscopic EM techniques to analyse the microstructures of pathological conditions is expected to improve our understanding of disease-specific structures that could not be obtained using conventional EM techniques.

To develop high-throughput TEM imaging, Graham et al. used a tape-based, reel-toreel pipeline that combines automated serial sectioning and a TEM-compatible tape substrate, GridTape [275] . This acquisition platform provides nano-metre-resolution imaging at fast rates via TEM. Based on this pipeline, multiple-scope parallel imaging using a 50-MP camera has enabled image acquisition of a >1-mm3 volume of the mouse neocortex, spanning four different visual areas at synaptic resolution, in less than 6 months; in turn, this has yielded a >2-petabyte dataset from over 26,500 ultra-thin tissue sections [265] . In addition, Phelps et al. applied GridTape-based serial-section TEM imaging to acquire a synapse-resolution dataset containing an adult female Drosophila ventral nerve cord [264] . The complete connectivity maps provided a deeper understanding of how the nervous system controls the locomotor rhythms underlying swimming and crawling [264] . Since TEM offers much faster imaging compared with that of SEM, this research could be applied in areas that require broad observation with precise imaging.

The body can be understood more deeply if tissue functions and macromolecular fingerprinting can be estimated at the nano-level, which is sometimes difficult to achieve using only EM-based morphomics. For example, distinguishing between excitatory and inhibitory neurons cannot be achieved based only on morphology [276, 277] . To resolve this issue, correlative EM, which combines EM and other imaging tools, can be used to better understand molecular functions and other factors. Correlative EM is also useful for screening or targeting specific structures, especially when targeting cellular markers. A well-established example is the combination of light microscopy and EM, known as correlative light and electron microscopy (CLEM) [278] . The idea behind CLEM was first proposed in the 1980s [213] . Samples are initially imaged using a light microscope to detect histological morphologies or fluorescence signals, after which they are subjected to EM imaging with a nano-resolution. Correlation imaging is achieved either by sharing the same FOV for both modalities or by sample transfer in tandem [278] .

To screen for structures of interest, Ronchi et al. developed a workflow that combined fluorescent labelling and FIB-SEM, which enabled correlative targeted imaging of animal mammary gland organoids, tracheal terminal cells and ovarian follicular cells [225] . CLEM has also been applied to the mouse brain [252, [279] [280] [281] [282] [283] and the ferret brain [284] as well as whole model organisms [83], cultured cells [227] and various tissues [232] . In addition, CLEM could be used for in vivo multicolour imaging, known as Brainbow [285] [286] [287] .

When targeting specific structures, Trzaskoma et al. applied CLEM to reveal 3D chromatin folding [288] ; they combined DNA fluorescent in situ hybridisation with SBF-SEM. Oorschot et al. published a workflow integrating the Tokuyasu technique to preserve the antigenicity of proteins and investigated neural stem and progenitor cell populations [276] . In addition, 3D CLEM combined with the CryoChem technique allows for quality ultra-structural preservation that is broadly applicable to cultured cells and tissue samples [289] . The CLEM method can even be used to target specific proteins under transgenic conditions via the engineered peroxidase gene APEX2 [290] [291] [292] [293] , which serves as a labelling probe in both EM and light microscopy [294] . This APEX2 system has been successfully implemented to track lysosomes in dendrites [295] and to visualise the localisation of endoplasmic reticulum chaperonin [296] , the outer endoplasmic reticulum and mitochondrial membrane [297] , membrane proteins [298] and multicolour labelling of peroxidases [299] . Recently, the APEX-Gold method, which has high sensitivity, was used as a genetic tagging in 3D EM [300] .

For correlation of live cell imaging, Fermie et al. analysed the dynamics of individual GFP-positive structures in HeLa cells and then correlated these with images from FIB-SEM [301] . Thus, they overcame the limitations of EM, i.e. that EM cannot visualise live cells. Betzig et al. first introduced a combination of super-resolution light microscopy and EM (superresolution CLEM) to image specific target proteins in the thin sections of lysosomes and mitochondria [302] . Currently, super-resolution CLEM can achieve a resolution of 20-50 nm, although the distortion of the sample becomes a problem at <10-nm resolutions [303] . This technique was also utilised to visualise the Golgi apparatus [304, 305] , mitochondria [306, 307] and other organelles [306] .

X-ray computed tomography (CT) has been applied to biological tissues or cells to obtain almost single cell-level morphology data [308] . In practical terms, observing cells that are approximately 10 µm in size requires sub-micro-resolution potential in the CT device. When observing intracellular structures, the use of synchrotron radiation X-ray is necessary. Although, at present, single-cell imaging with a CT device remains a special case, in this section we discuss cellular tissue analysis, including single-cell imaging, using CT devices and further correlative CT-EM.

The following are a few examples of X-ray CT applied in biology research to date. In the early 1980s, the micro-CT technique was developed to achieve 3D observations at micrometre resolutions [309] . This technique can be used to obtain a projection image of a sample by irradiating it with X-rays with wavelengths of ~1 pm to 10 nm. Compared with the 1-2 mm resolution in conventional medical CT scans, micro-CT tomography results in a higher spatial resolution of 1-50 µm (generally approximately 5 µm resolution per voxel) [310, 311] . Because the spatial resolution depends on the focal spot size of the X-ray source [312, 313] , relatively small sample pieces (<10 mm in size) can be used for micrometre-level resolution with micro-CT. Moreover, micro-CT imaging provides high contrast results, especially in tissues with high or low X-ray permeability (e.g. the lungs or bones, respectively), without the need for special sample preparation. However, particularly in soft tissues, including the brain and renal cortex, suitable staining techniques are required to increase absorption-based contrast of tissue structures [314] [315] [316] . In several studies, micro-CT has been applied to visualise juvenile coral [317] , small organisms [318] [319] [320] [321] , nano-material in lung tissues [322] , mammalian brain tissue [310, 323] , rodents kidney nephrons [316, 324, 325] , mouse liver structures [326] , mouse embryos [327] [328] [329] [330] [331] [332] and human placenta [333] . In addition, the phasecontrast approach of X-ray CT can be applied generally to unstained specimens such as mouse kidneys [334] , the human heart [335] , human brain tissue (cerebellum) [336] and plant germination [337] . Overall, this modality has become a promising method used in morphomics.

For 3D non-destructive targeting of a region of interest in a specimen, EM analysis correlated with the X-ray CT modality has been proposed. Several reports of correlative CT and other microscopic techniques have included EM [338] [339] [340] [341] . In particular, correlative micro-CT and EM has been applied to clarify neural 3D structures in mouse brain tissues [342] . Silver impregnation staining applied to neurons can also feasibly be used in correlative workflows [343, 344] . Karreman et al. demonstrated the in vivo tracking of single tumour cells using multimodal imaging including X-ray CT and EM [345] , which is expected to have broad applications in various biological fields.

Some CT devices even offer sub-micro-resolution [346] , i.e. nano-CT, which is sometimes used as a synchrotron radiation-based CT setting and a use of soft X-ray with relatively low penetrating power. Nano-CT has been applied to reconstruct the neural network in Drosophila or the rodent brain [347] [348] [349] . In addition, this resolution can achieve cell-level observations [316, 326] that could further facilitate correlative analysis in combination with EM. Interestingly, Kuan et al. demonstrated that X-ray holographic nano-tomography can be used to image large-scale volumes with sub-100-nm resolution in Drosophila melanogaster and mouse nervous tissue, thereby enabling a close reproduction of the EM images [350] . Moreover, multiple scanning technique can comprehensively catalogue mechanosensory neurons and trace individual motor axons from muscles to the central nervous system [350] . Nano-scale X-ray CT can then bridge a key gap that helps move toward EM resolutions. Furthermore, the integration of nano-scale CT and EM has been used to study mitochondrial morphology in relation to drug resistance in human colon carcinoma cells [351] . A parallel-beams CT method can achieve much faster image acquisition and comparable or even better resolution [352] . In addition, synchrotron-based CT has been used for cellular-level analysis of bacteria [353] , yeast [354, 355] , mammalian cells [355, 356] , neuroanatomy [357] , renal microvasculature [358] and human bones [359, 360] . Overall, the X-ray CT modality has the potential to be used not only for regional targeting prior to correlative EM analysis but also for morphomics analysis of parenchymal morphology with nano-scale resolution.

The morphome or biological morphome refers to the totality of the morphological features in a species [42, [361] [362] [363] [364] . The morphome is expressed as the sum of a species' molecular dynamics [362, 363] including its DNA (genome), gene expression (transcriptome) and metabolic (metabolome) information such as lipids and sugars (Fig. 5) . It also refers to morphological phenotypes. Most morphological data are imaging data, which at first glance differs from the genomic sequences and numerical data that are mainly used as omics data in molecular biology [364] . Thus, different approaches are required to handle morphome data.

As EM device technologies have advanced [132], the acquisition speed of EM imaging has dramatically accelerated and EM-based imaging techniques have been used to study the complexity of organisms at high-level 2D and 3D resolutions. Indeed, imaging data can be produced at a level comparable with genome data obtained via next-generation sequencing. For example, wide-area imaging produced using single-beam SEM can acquire several tens of gigabytes of data in a single day of imaging [42], whereas 2D/3D imaging produced using multibeam SEM can acquire hundreds of gigabytes or a terabyte scale of image data [201] . Moreover, the latest EM methods, such as high-speed TEM methods [265] and ATUM-based AT and multibeam SEM methods [274] , can generate petabyte-scale image data.

These levels of imaging data can be used to systemically measure and quantify large morphological fingerprints and diverse biological phenomena. Further quantitative morphology analysis could be applied to study biological functions. Such comprehensive approaches have led to the treatment of the resultant large imaging datasets as new omics information, which is termed morphomics (Fig. 5) . This involves the integration of comprehensive (big) morphology data and bioimaging informatics, which will result in the discovery of unknown features, but there is still a bottleneck of imaging data mining. In the following subsections, we discuss such morphome analysis, outline imaging data operations and highlight image data analysis using deep learning.

Just over 20 years ago, film photographs were still in mainstream use as EM images [365], whereas the EM images of today are high-resolution digital images. In this period of development, advances in infrastructural technologies, such as digital image-archiving, faster network communication, improved computing performance and increased storage disk capacity, have facilitated the acquisition of large-scale digital bioimage data and enabled the practical handling and processing of images [366] . It is now possible to operate with hundreds of gigabytes or terabytes of images even in a laboratory setting. Efforts are being made in the field of bioimaging data operations to handle such huge image datasets for data repositories, data sharing and reuse in a standardised manner [367] .

Open-source data and data accessibility are critical to the sharing of bioimaging data [368]; fortunately, worldwide access to data has become possible via the Internet [369, 370]. However, it is necessary to construct a descriptive format and data repository, so-called metadata and an image database, respectively, prior to distributing bioimaging data [371] . The Open Microscopy Environment (OME) consortium is working to produce imaging metadata and public image archives in the medical and life sciences [372, 373] . The OME is an opensource software framework developed to address standards for sharing image data and analysis results [368, 374] . Within its framework, OME Remote Objects, an open-source interoperability toolset for biological imaging data [375] , and OME metadata [376-379] have been developed to manage multidimensional and heterogeneous imaging data mainly from light optical microscopy. However, standardised metadata that describes EM experiments, including bioresources, measurement conditions and image formats, has yet to be developed; thus, integrated analysis of imaging data with other metadata has remained difficult [380]. Therefore, we previously proposed the development of microscopy metadata to describe EM experiments and their image datasets based on the data model of OME metadata [369, 380], and we offered a combination of an ontology-based imaging metadatabase and an image viewer, which were distributed in a machine-readable web form [42] . At present, metadata arrangements for bioimaging, including EM, have been discussed internationally toward the reuse of microscopy data [381] . In future work, the application to medical imaging research, such as MRI and PET/CT, for human subjects will also be important [382, 383] .

In May 2021, the Global BioImaging (GBI) consortium proposed criteria for globally applicable guidelines related to the tools and resources of open image data in the fields of biological and biomedical imaging [367] . The GBI also founded international non-boundaries to develop common imaging and data standards that promote data sharing and open data, as well as world-class training programmes and repositories of image data analysis tools for use by imaging scientists. Furthermore, the 'Quality Assessment and Reproducibility for Instruments and Images in Light Microscopy' initiative recently proposed the establishment of guidelines for quality assessment and reproducibility related to microscopy instruments and images [384] . These activities are expected not only to improve the overall quality and reproducibility of data across the microscopic bioimaging field but also to enable handling of huge bioimage datasets in a standardised manner. In addition, the barriers to data sharing of EM images should also be reduced.

As As a public archive of EM images, the Electron Microscopy Data Bank [387, 388] was launched in 2002 to provide a public repository of mainly electron cryo-microscopy volume maps and tomograms including macromolecular structures, such as proteins and their ligand complexes, and subcellular structures. In 2016, the Electron Microscopy Public Image Archive (EMPIAR; https://www.ebi.ac.uk/empiar/) [389] was published. This archive became a public resource for raw images underpinning 3D cryo-EM maps and tomograms. Most EMPIAR datasets are particle images and 3D tomograms of macromolecules obtained using cryo-EM, but they currently contain several 3D EM datasets of epoxy-embed tissue and cell samples obtained using SBF-SEM or FIB-SEM. For example, a 3D imaging dataset of the HeLa cell line obtained using SBF-SEM (EMPIAR-10094) [251, 390] consists of 518 cross-sectional images with a size of 8,192 × 8,192 pixels; the dataset is nearly 130 gigabytes in size, indicating that the EMPIAR covers a wide range of biological samples. Notably, all data archived in the EMPIAR is under CC0 licence and can be re-used freely without any conditions or restrictions.

To date, the development of these public bioimage resources is at an early stage and further accumulation of imaging data and development of integrated platforms is highly desirable. Representative list of current public EM datasets is summarised in Supplementary Table 1. In the past, providing large-scale open-source genomic sequence data helped advance our understanding of genomics. Accordingly, increasing the availability of imaging datasets will further stimulate imaging research and the development of novel imaging technology.

Little progress has been made in the methods used to analyse EM bioimages over time; conventionally, every image was examined manually [391] . EM bioimages possess a low signal-to-noise ratio, black-and-white contrast and a variety of morphological features. When using classical image analysis methods, it has been difficult to recognise and decode EM bioimages. For example, in semantic segmentation, which is used to extract a particular region in an image, the use of classical deductive methods [392] [393] [394] has failed to identify a mathematical solution for the morphological features of a particular region among various other morphologies. In most EM research cases, comprehensive quantification, including automatic segmentation, could not be achieved even after acquiring large-scale image datasets. In an attempt to resolve these issues, current best practice is to apply cutting-edge informatics or artificial intelligence (AI) approaches [395, 396] to quantify the microstructures in EM bioimages.

Image analysis via AI techniques, such as machine learning (ML) and deep learning (DL), has received substantial attention in the fields of biomedicine [397] [398] [399] and imaging research [400, 401] . Inductive analysis using supervised data has been used to identify characteristic changes in morphology [402] . Initially, such AI techniques were applied in neuroanatomy research. Kaynig et al. also demonstrated fully automatic 3D segmentation of thin, elongated, cell membrane structures of dendrites for 30 sections in TEM images by finding features from the images and constructing a features classifier using a random forest, a ML method that uses ensemble learning [403] ; this tool was supplied as a plug-in of ImageJ/Fiji [404, 405] . Turaga et al. presented a ML algorithm for training a classifier to produce affinity graphs, representing the x-, y-and z-direction information, which can be used to segment the EM images of neural tissue [406] . Subsequently, they reported an affinity graph computation that used a four-layered convolutional neural network (CNN) trained with the supervised dataset; this resulted in 3D reconstructions of neurites with ~90% segmentation accuracy in a 3D EM dataset of rabbit retina tissue [407] . These were the original applications of ML and DL in connectomics studies. Subsequently, AI technologies have become indispensable for bioimage analysis. For instance, DL models, which use an exquisite combination of multilayered CNN for learning [408, 409] , are the current state-of-the-art technology in image recognition [401, 410] ; they can extract morphological features, such as the cell body and nucleus, from cellular images via complex networks [411, 412] .

For quantification of bioimaging data by DL, the U-Net model was proposed by Ronneberger et al. [413, 414] ; the model has encoder and decoder parts with multi-layered CNNs and contracting paths between the encoder and decoder. U-Net targets segmentation tasks in a small number of image datasets with large feature types that are unique to the bioimage dataset. The U-Net model substantially improves performance in segmentation tasks such as cell division tracking and neuronal cell membrane segmentation [414] . Many derivative models, such as FusionNet [415] , enhanced U-Net [416] , U-Net-Id [417] , SCAU-Net [418] and Dual ResUNet [419] , have subsequently been proposed that reportedly improve performance compared with that of the original U-Net model. In addition, 3D applicable models have been developed as an extension to 3D volume data [420] [421] [422] . Therefore, the U-Net method was a leading technique that has served as a foundation for biological image analysis. For further details, refer to the review by Siddique et al. [423] , which comprehensively discusses the U-Net models and derived models.

To use ML and DL techniques universally, user interface tools, such as QuPath [424] , Microscopy Image Browser [425] , NuSeT [426] , UNI-EM [427] and DeepMIB [428] , have also been developed. Additionally, the CDeep3M tool can perform image segmentation in cloud computing [429] . A generalist DL model for cellular segmentation, so-called Cellpose, was proposed, which can be precisely applied to the segmentation of cells from a wide range of image types [430] . The DL model has also been ported to R, and typical models for segmentation are available in the ANTsX ecosystem [431] . In addition, we have recently begun distributing supervised bioimage datasets in the R array format that can be used in the analytical workflow in the R environment as the BioImageDbs project via the Bioconductor ExperimentHub platform [432] . Wei et al. introduced the MitoEM dataset, a 3D mitochondria instance segmentation dataset with approximately 40,000 instances and 30 µm3 volumes brain cortices [433] .

In contrast to these successes in bioimage data analysis, automated segmentation tasks related to EM images remain challenging because the texture and intensity variation are generally similar [434] . A benchmark report comparing published seven models for EM images showed that their performances were still highly variable [434] . In recent years, however, successful research cases of EM image analysis using DL have been reported, including in neuroanatomy research and from other fields. In neuroanatomy research, the 3D segmentation of neurites using the DeepEM3D model achieved close to human-level performance [435] . Lee et al. reported a residual symmetric U-Net architecture that achieved an approximately 2%-3% error rate for an EM image dataset of mouse neurites [81] in the SNEMI3D challenge; thus, the system surpassed the human accuracy value provided at that time [436] . Januszewski et al. used a flood-filling network (FFN) to trace neurons in a dataset from a zebra finch brain obtained using SBF-SEM; they achieved high-precision automated reconstruction of neurons with a mean error-free neurite path length of 1.1 mm [437] . Sherida et al. reported that the addition of local shape descriptors promotes affinity-based segmentation methods to a level that is on par with that of the current state-of-the-art system for neuron segmentation based on FFN [438] . Other researchers have attempted segmentation analysis of cell bodies and cell nuclei [251, 390, 439, 440] as well as organelles [441] [442] [443] [444] in EM images. For multiple segmentation tasks with EM images, the transfer of learning from pre-trained models using the CEM500K dataset was effective for the transferability of learned features, indicating that a large amount of training data is important for encoding bioimages [445] . Comprehensive quantification of multiple organelles in whole cells using DL segmentation has been reported for serial cross-sections of the mouse liver [446] and for cultured cells [231] , which suggests the future possibilities for cell biology that may arise from intracellular morphomics.

Importantly, the DL technique has been applied to more than just quantitative analysis. A recent report described a new data reduction and compression scheme (ReCoDe) that converts the raw data from EM images into 100 times more minor data [447] . Furthermore, image generation models, such as CycleGAN [448, 449] , have been applied to EM images as denoising [94, [450] [451] [452] and image super-resolution techniques [453] [454] [455] [456] . The superresolution DL technique has been applied to an active acquisition pipeline of SEM imaging [457] . Interestingly, image transformation techniques can also reproduce tissue-stained images from unstained or other stained images [284, [458] [459] [460] [461] [462] . To expand limited datasets, various proposals have been made for image data augmentation in DL [463] . In future research, these conversion techniques may also be widely applicable to EM images.

The use of DL in pathology has also advanced remarkably. Using AI to analyse tissue sections is often referred to as computational pathology [397] . Cases of its application have been reported in relation to the pathological diagnosis of various cancer types, e.g. breast cancer [464] [465] [466] , bladder cancer [419] , renal cell carcinoma [467] , non-small cell lung cancer [468] , skin cancer [469] and gastrointestinal cancer [470] , with such techniques potentially increasing the precision of oncology results [471] . For kidney disease assessments, the relationship between renal histology and the prognosis/severity of renal disease has been examined using image recognition and comprehensive segmentation of the constituent tissues of renal samples including human renal biopsies [472] [473] [474] [475] .

Intriguingly, recent DL results have been associated with biological functions such as the prediction of gene expression patterns [476, 477] and genetic mutations from histological images [478, 479] . Prediction of genetic mutation patterns in lung cancer [468] , breast cancer [480] and haematologic cancer [481] has been achieved using morphology image recognition, and this enabled severity classification. Digital imaging studies have been conducted to explore the relationship between histology and gene expression patterns for the prediction of genetic variation associated with tissue morphology [482] , the RNA-Seq profiles of tumours [483] and multifactorial site-specific signatures of tissue submitting sites [484] from whole-slide images, as well as diverse molecular phenotypes identified by the expression of immune checkpoint proteins in tumours [485] . Future developments may lead to the prediction of gene expression at the nano-scale level.

To ensure that DL becomes more widely available, two weaknesses must be addressed. First, the manual annotations required for supervised learning are time consuming and costly. For example, for an EM dataset of 1 million µm3, the cost to manually annotate all images will be 10 million dollars [486] . However, using an unsupervised learning method, which learns from the data without any pre-annotated labels [487] , is one means of solving this problem. Although such an approach without annotations is still rare, some studies have included autoencoders for unsupervised learning that have achieved an acceptable performance in histological image recognition [488] [489] [490] . Second, the computational process along with the heavy optimisation of many parameters remains a 'black box', which is not interpretable by humans. Currently, the appropriateness of a network model can only be evaluated using the numerical value of its prediction performance. One solution is to use human-interpretable image features to predict outcomes [485] . In addition, post-DL analysis should be directed at determining which biologically meaningful information can be extracted from the quantification of morphological features.

The development EM has placed nano-scale imaging data at the centre of morphological analysis. Large 2D and 3D datasets can be acquired with a millimetre-wide range at a nanometre resolution. However, previous morphological studies involving big data analysis have been limited to analysis of the individual dataset, sometimes of a single dataset of normal tissue without any comparison. Although comparisons of several morphological features have been conducted in simple datasets, it is currently challenging to compare across datasets or among unbiased structural features.

Morphomics currently stands at means to create a 'reference morphology' or reference EM datasets at an early stage in the development of genomics studies. Only if a solid 'reference morphology', or at least a standardised workflow including sample preparation, microscopic settings, data storage, data normalisation and image analysis, is established, will it be possible to conduct larger scale comparative studies that could have major biological implications. The use of bioinformatics methods and imaging databases will accelerate this process. In addition, further multi-omics analysis techniques that can bridge the gap between morphomics and other omics data will be powerful tools. In particular, we expect the development of revolutionary methodologies that combine large-scale EM data analysis techniques with analysis of genomics data such as gene mutation and expression data. By incorporating these new methodologies, the field of pathology is expected to progress rapidly, which might include the identification of previously unknown structures, the quantification of rare events, the reclassification of diseases and the automatic diagnosis of diseases. Furthermore, the amount of data that can be analysed is expected to increase dramatically with the development of automatic AI analysis.

Wide-area 2D EM imaging along with large-scale 3D EM image acquisition and its reconstruction for biological tissues are currently defined as next-generation electron microscopy techniques; however, these tools are now more commonly being used to produce massive morphomics datasets. To maximise the utilisation of morphomics data, the general use of DL methods and post-DL analysis will be essential for comprehensively quantifying cellular morphology. Overall, these advanced techniques can be expected to deepen our understanding of living tissues and cells. Biological systems consist of various molecular components such as genes and transcripts (the genome), proteins, lipids, sugars, amino acids and other metabolic components. Quantitative datasets of comprehensive biometric information are usually treated as omics data in biology or bioinformatics. The resulting pattern of molecular information forms the complex structures of tissue and cells in an organism, which is referred to as the morphological features or biological morphome. The nano-scale organisation of the morphome is accessible using the morphomics approaches, such as a large-scale EM, CLEM combined with light microscopy, and CT method, described in this review. 

Multiple-beam scanning electron microscopy

High-resolution, high-throughput imaging with a multibeam scanning electron microscope

Multi-beam scanning electron microscopy for highthroughput imaging in connectomics research

Transmission electron imaging in the delft multibeam scanning electron microscope 1

MagC, magnetic collection of ultrathin sections for volumetric correlative light and electron microscopy

Mission (im)possible -mapping the brain becomes a reality

3-d EM exploration of the hepatic microarchitecture -lessons learned from large-volume in situ serial sectioning

Morphological process of podocyte development revealed by block-face scanning electron microscopy

Developmental stages of tertiary lymphoid tissue reflect local injury and inflammation in mouse and human kidneys

The ultrastructure of the juxtaglomerular apparatus as disclosed by threedimensional reconstructions from serial sections. The anatomical relationship between the tubular and vascular components

The innervation of the juxtaglomerular apparatus and surrounding tubules: A quantitative analysis by serial section electron microscopy

Electron and ion imaging of gland cells using the FIB/SEM system

Three-dimensional ultrastructural imaging and quantitative analysis of the periodontal ligament

Comparing 3D ultrastructure of presynaptic and postsynaptic mitochondria

An application of scanned focused ion beam milling to studies on the internal morphology of small arthropods

ER-to-golgi protein delivery through an interwoven, tubular network extending from ER

Patterns of organelle ontogeny through a cell cycle revealed by whole-cell reconstructions using 3D electron microscopy

3D correlative light and electron microscopy of cultured cells using serial blockface scanning electron microscopy

Progressive sheet-to-tubule transformation is a general mechanism for endoplasmic reticulum partitioning in dividing mammalian cells

Origins of enterovirus replication organelles established by whole-cell electron microscopy

Deep learning for automatic segmentation of the nuclear envelope in electron microscopy data, trained with volunteer segmentations

Q&a: Array tomography

Synaptogenesis of the calyx of held: Rapid onset of function and one-to-one morphological innervation

Smart specimen preparation for freeze substitution and serial ultrathin sectioning of yeast cells

3D tomography reveals connections between the phagophore and endoplasmic reticulum

A complete electron microscopy volume of the brain of adult <em>drosophila melanogaster</em>

Array tomography: A new tool for imaging the molecular architecture and ultrastructure of neural circuits

3D reconstruction of VZV infected cell nuclei and PML nuclear cages by serial section array scanning electron microscopy and electron tomography

Three-dimensional shape of the Golgi apparatus in different cell types: serial section scanning electron microscopy of the osmiumimpregnated Golgi apparatus †

Array tomography: Characterizing FAC-sorted populations of zebrafish immune cells by their 3D ultrastructure

Double staining method for array tomography using scanning electron microscopy

TEM, SEM, and STEM-based immuno-CLEM workflows offer complementary advantages

Label-free 3D-CLEM using endogenous tissue landmarks

Microglia remodel synapses by presynaptic trogocytosis and spine head filopodia induction

Nanobody immunostaining for correlated light and electron microscopy with preservation of ultrastructure

SuperCLEM: an accessible correlative light and electron microscopy approach for investigation of neurons and glia in vitro

Targeting functionally characterized synaptic architecture using inherent fiducials and 3D correlative microscopy

Transgenic strategies for combinatorial expression of fluorescent proteins in the nervous system

Brainbow: New Resources and Emerging Biological Applications for Multicolor Genetic Labeling and Analysis

Zebrabow: multispectral cell labeling for cell tracing and lineage analysis in zebrafish

Ultrastructural visualization of 3D chromatin folding using volume electron microscopy and DNA in situ hybridization

High-quality ultrastructural preservation using cryofixation for 3D electron microscopy of genetically labeled tissues

Directed evolution of APEX2 for electron microscopy and proximity labeling

Electron microscopy using the genetically encoded APEX2 tag in cultured mammalian cells

Ultrastructural localisation of protein interactions using conditionally stable nanobodies

Directed evolution of split APEX2 peroxidase

APEX2-tagging of sigma 1-receptor indicates subcellular protein topology with cytosolic n-terminus and ER luminal c-terminus

Activity-dependent trafficking of lysosomes in dendrites and dendritic spines

APEX2-enhanced electron microscopy distinguishes sigma-1 receptor localization in the nucleoplasmic reticulum

Proteomic mapping of cytosol-facing outer mitochondrial and ER membranes in living human cells by proximity biotinylation. eLife 6:e24463

APEX fingerprinting reveals the subcellular localization of proteins of interest

Multicolor electron microscopy for simultaneous visualization of multiple molecular species

A robust method for particulate detection of a genetic tag for 3D electron microscopy

Single organelle dynamics linked to 3D structure by correlative live-cell imaging and 3D electron microscopy

Imaging intracellular fluorescent proteins at nanometer resolution

The 2018 correlative microscopy techniques roadmap

Visualization of secretory cargo transport within the Golgi apparatus

Cargo sorting zones in the trans-golgi network visualized by super-resolution confocal live imaging microscopy in plants

Correlative three-dimensional superresolution and block-face electron microscopy of whole vitreously frozen cells

2020) mEosEM withstands osmium staining and epon embedding for super-resolution CLEM

Nanoscale x-ray imaging

Abstracts fifth annual scientific meeting of the american society for bone and mineral research june 5-7

The ascent of 3D x-ray microscopy in the laboratory

Micro-CT -a digital 3D microstructural voyage into scaffolds: A systematic review of the reported methods and results

The effect of focal spot size on the spatial resolution of variable resolution x-ray CT scanner

Spatial resolution characterization of a x-ray microCT system

MicroCT for developmental biology: A versatile tool for highcontrast 3D imaging at histological resolutions

A quantitative comparison of micro-CT preparations in dipteran flies

Three-dimensional virtual histology enabled through cytoplasm-specific x-ray stain for microscopic and nanoscopic computed tomography

Ocean acidification causes structural deformities in juvenile coral skeletons

MicroCT for comparative morphology: Simple staining methods allow high-contrast 3D imaging of diverse non-mineralized animal tissues

Micro-computed tomography and histology to explore internal morphology in decapod larvae

Biological applications of x-ray microtomography: Imaging microanatomy, molecular expression and organismal diversity

3D x-ray ultra-microscopy of bone tissue

Multi-scale x-ray computed tomography to detect and localize metal-based nanomaterials in lung tissues of in vivo exposed mice

A micro-CT-based method for quantitative brain lesion characterization and electrode localization

Global heterogeneity of glomerular volume distribution in early diabetic nephropathy

Visualization of three-dimensional nephron structure with microcomputed tomography

Nucleus-specific x-ray stain for 3D virtual histology

Rapid 3D phenotyping of cardiovascular development in mouse embryos by micro-CT with iodine staining

High-resolution µCT of a mouse embryo using a compact laser-driven x-ray betatron source

Three-dimensional microCT imaging of mouse development from early post-implantation to early postnatal stages

Overdosage of Hand2 causes limb and heart defects in the human chromosomal disorder partial trisomy distal 4q

High-throughput discovery of novel developmental phenotypes

Structural stabilization of tissue for embryo phenotyping using micro-CT with iodine staining

A massively multi-scale approach to characterizing tissue architecture by synchrotron micro-CT applied to the human placenta

X-ray phase tomography with near-field speckles for three-dimensional virtual histology

Intact imaging of human heart structure using x-ray phase-contrast tomography

Three-dimensional virtual histology of human cerebellum by x-ray phase-contrast tomography

Visualization of internal 3D structure of small live seed on germination by laboratory-based x-ray microscopy with phase contrast computed tomography

A correlative approach for combining microCT, light and transmission electron microscopy in a single 3D scenario

A correlative method for imaging identical regions of samples by micro-CT, light microscopy, and electron microscopy: Imaging adipose tissue in a model system

Size and specimen-dependent strategy for xray micro-ct and tem correlative analysis of nervous system samples

X-ray microscopy as an approach to increasing accuracy and efficiency of serial block-face imaging for correlated light and electron microscopy of biological specimens

X-ray microtomography in biology

Golgi-cox staining step by step

Fast and precise targeting of single tumor cells in vivo by multimodal correlative microscopy

Myoanatomy of the velvet worm leg revealed by laboratory-based nanofocus x-ray source tomography

Three-dimensional network of drosophila brain hemisphere

Three-dimensional x-ray visualization of axonal tracts in mouse brain hemisphere

Optimising complementary soft tissue synchrotron x-ray microtomography for reversibly-stained central nervous system samples

Dense neuronal reconstruction through xray holographic nano-tomography

3D quantitative and ultrastructural analysis of mitochondria in a model of doxorubicin sensitive and resistant human colon carcinoma cells

X-ray computed tomography in life sciences

3D ultrastructural organization of whole chlamydomonas reinhardtii cells studied by nanoscale soft x-ray tomography

X-ray tomography generates 3-d reconstructions of the yeast, saccharomyces cerevisiae, at 60-nm resolution

Switchable resolution in soft x-ray tomography of single cells

Biological soft X-ray tomography on beamline 2.1 at the Advanced Light Source

Quantifying mesoscale neuroanatomy using x-ray microtomography

Differential synchrotron x-ray imaging markers based on the renal microvasculature for tubulointerstitial lesions and glomerulopathy

Osteocyte-directed bone demineralization along canaliculi

Nanoscale x-ray microscopic imaging of mammalian mineralized tissue

Automative morphome analysis of medical-biological images

Systems biology in 3D space -enter the morphome

REMBI: Recommended metadata for biological images-enabling reuse of microscopy data in biology

OME Ontology: A Novel Data and Tool Integration Methodology for Multi-Modal Imaging in the Life Sciences. Proceeding of SWAT4HCLS

Correlated multimodal imaging in life sciences: Expanding the biomedical horizon

A community-driven initiative to establish guidelines for quality assessment and reproducibility for instruments and images in light microscopy

Chromatin arranges in chains of mesoscale domains with nanoscale functional topography independent of cohesin

Meiotic cellular rejuvenation is coupled to nuclear remodeling in budding yeast

New electron microscopy database and deposition system

EMDataBank unified data resource for 3DEM

EMPIAR: A public archive for raw electron microscopy image data

Volumetric semantic instance segmentation of the plasma membrane of HeLa cells

<B>integrative method for three-dimensional imaging of the entire golgi apparatus by combining thiamine pyrophosphatase cytochemistry and array tomography using backscattered electron-mode scanning electron micr</b><b>oscopy </b>

Chapter 5 -overview and fundamentals of medical image segmentation

An application of optimized otsu multi-threshold segmentation based on fireworks algorithm in cement SEM image

Comparative study on automated cell nuclei segmentation methods for cytology pleural effusion images

Deep learning in bioinformatics

Democratising deep learning for microscopy with ZeroCostDL4Mic

Deep learning in histopathology: The path to the clinic

Digital pathology and computational image analysis in nephropathology

Deep neural network models for computational histopathology: A survey

A bird's-eye view of deep learning in bioimage analysis

A survey on deep learning in medical image analysis

Pathology image analysis using segmentation deep learning algorithms

Neuron geometry extraction by perceptual grouping in ssTEM images

Trainable Weka Segmentation: a machine learning tool for microscopy pixel classification

The ImageJ ecosystem: An open platform for biomedical image analysis

Maximin affinity learning of image segmentation

Convolutional Networks Can Learn to Generate Affinity Graphs for Image Segmentation

Mitosis detection in breast cancer histology images with deep neural networks

Automated training of deep convolutional neural networks for cell segmentation

Translational AI and deep learning in diagnostic pathology

Leukocyte recognition with convolutional neural network

An automatic nucleus segmentation and CNN model based classification method of white blood cell

U-net: Convolutional networks for biomedical image segmentation

U-net: Deep learning for cell counting, detection, and morphometry

FusionNet: A deep fully residual convolutional neural network for image segmentation in connectomics

Microscopy cell nuclei segmentation with enhanced u-net

U-net-id, an instance segmentation model for building extraction from satellite images-case study in the joanópolis city

SCAU-net: Spatial-channel attention u-net for gland segmentation

Zebrafish embryo vessel segmentation using a novel dual ResUNet model

3D u-net: Learning dense volumetric segmentation from sparse annotation

Hippocampal segmentation for brains with extensive atrophy using three-dimensional convolutional neural networks

Two-level training of a 3D U-Net for accurate segmentation of the intra-cochlear anatomy in head CTs with limited ground truth training data

U-net and its variants for medical image segmentation: A review of theory and applications

QuPath: Open source software for digital pathology image analysis

Microscopy image browser: A platform for segmentation and analysis of multidimensional datasets

NuSeT: A deep learning tool for reliably separating and analyzing crowded cells

UNI-EM: An environment for deep neural network-based automated segmentation of neuronal electron microscopic images

DeepMIB: User-friendly and open-source software for training of deep learning network for biological image segmentation

CDeep3M-plug-and-play cloud-based deep learning for image segmentation

Cellpose: A generalist algorithm for cellular segmentation

The ANTsX ecosystem for quantitative biological and medical imaging

BioImageDbs: Bio-and biomedical imaging dataset for machine learning and deep learning

MitoEM dataset: Large-scale 3D mitochondria instance segmentation from EM images

EM-net: Deep learning for electron microscopy image segmentation

DeepEM3D: approaching human-level performance on 3D anisotropic EM image segmentation

High-precision automated reconstruction of neurons with flood-filling networks

Local shape descriptors for neuron segmentation

Semantic segmentation of HeLa cells: An objective comparison between one traditional algorithm and four deep-learning architectures

Practical method of cell segmentation in electron microscope image stack using deep convolutional neural network☆

Convolutional neural network pruning to accelerate membrane segmentation in electron microscopy

Automatic mitochondria segmentation for EM data using a 3D supervised convolutional network

Mitochondria segmentation in electron microscopy volumes using deep convolutional neural network

Reducing manual operation time to obtain a segmentation learning model for volume electron microscopy using stepwise deep learning with manual correction

CEM500K, a large-scale heterogeneous unlabeled cellular electron microscopy image dataset for deep learning

Three-dimensional ATUM-SEM reconstruction and analysis of hepatic endoplasmic reticulum-organelle interactions

A data reduction and compression description for high throughput time-resolved electron microscopy

Unpaired image-to-image translation using cycle-consistent adversarial networks

Generative adversarial networks: An overview

Noise2Atom: Unsupervised denoising for scanning transmission electron microscopy images

Deep denoising for scientific discovery: A case study in electron microscopy

Denoising-based image compression for connectomics

Deep learning-based point-scanning superresolution imaging

Resolution enhancement in scanning electron microscopy using deep learning

Segmentation-enhanced CycleGAN

Super-resolution for asymmetric resolution of FIB-SEM 3D imaging using AI with deep learning

(eds) Medical image computing and computer assisted intervention -MICCAI 2020

Effective immunohistochemistry pathology microscopy image generation using CycleGAN

Digital synthesis of histological stains using micro-structured and multiplexed virtual staining of label-free tissue

Cell image segmentation by integrating Pix2pixs for each class

Use of Deep Learning to Develop and Analyze Computational Hematoxylin and Eosin Staining of Prostate Core Biopsy Images for Tumor Diagnosis

Deep learning-based transformation of h&e stained tissues into special stains

A survey on image data augmentation for deep learning

Deep learning for identifying metastatic breast cancer

Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer

Clinical-grade computational pathology using weakly supervised deep learning on whole slide images

Pan-renal cell carcinoma classification and survival prediction from histopathology images using deep learning

Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning

Dermatologist-level classification of skin cancer with deep neural networks

Deep learning can predict microsatellite instability directly from histology in gastrointestinal cancer

Artificial intelligence in digital pathology -new tools for diagnosis and precision oncology

Association of pathological fibrosis with renal survival using deep neural networks

Deep learningbased histopathologic assessment of kidney tissue

Deep learningbased segmentation and quantification in experimental kidney histopathology

Development and evaluation of deep learning&#x2013;based segmentation of histologic structures in the kidney cortex with multiple histologic stains

Deep learning predicts gene expression as an intermediate data modality to identify susceptibility patterns in <em>mycobacterium tuberculosis</em> infected diversity outbred mice

Spatial transcriptomics inferred from pathology whole-slide images links tumor heterogeneity to survival in breast and lung cancer

Deep learning in cancer pathology: A new generation of clinical biomarkers

Deep learning of histopathological features for the prediction of tumour molecular genetics

Prediction of BRCA gene mutation in breast cancer based on deep learning and histopathology images

Machine learning of bone marrow histopathology identifies genetic and clinical determinants in patients with MDS

Joint analysis of expression levels and histological images identifies genes associated with tissue morphology

A deep learning model to predict RNA-seq expression of tumours from whole slide images

The impact of site-specific digital histology signatures on deep learning model accuracy and bias

Human-interpretable image features derived from densely mapped cancer pathology slides predict diverse molecular phenotypes

Connectomic reconstruction of the inner plexiform layer in the mouse retina

A guide to machine learning for biologists

Automated acquisition of explainable knowledge from unannotated histopathology images

Disentangled autoencoder for cross-stain feature extraction in pathology image analysis

Unsupervised detection of cancerous regions in histology imagery using image-to-image translation

This study was partially supported by the RIKEN engineering network, the RIKEN aging project, the JSPS KAKENHI (Grant Numbers 18K19766 and 15K16536), Prof. Osafune memorial scholarship (the Japanese Society of Microscopy) and the Strategic Core Technology Advancement Program (Supporting Industry Program; SAPOIN) funded by the Ministry of Economy, Trade and Industry in Japan. We thank a member of the SAPOIN project in RIKEN for the regular and helpful discussion as well as Ms. Yuka Watahiki (Frontiers of Innovative Research in Science and Technology (FIRST), Konan University, Kobe, Japan) for her technical assistance with cell segmentation. The English language in the manuscript was reviewed by Enago (https://www.enago.jp/).

The authors declare that they have no conflicts of interest.Author contributions RS and SK designed the contents of the review article. AO provided the EM dataset. RS, KY, AO, MT, MS, YM and SK wrote the manuscript. RS and SK edited the entire manuscript.