key: cord-0054181-i3vt7px2
authors: Chandrasekaran, Srinivas Niranj; Ceulemans, Hugo; Boyd, Justin D.; Carpenter, Anne E.
title: Image-based profiling for drug discovery: due for a machine-learning upgrade?
date: 2020-12-22
journal: Nat Rev Drug Discov
DOI: 10.1038/s41573-020-00117-w
sha: 9e863f91f3db9e8c2b070c7095a37c7ce2a567fb
doc_id: 54181
cord_uid: i3vt7px2

Image-based profiling is a maturing strategy by which the rich information present in biological images is reduced to a multidimensional profile, a collection of extracted image-based features. These profiles can be mined for relevant patterns, revealing unexpected biological activity that is useful for many steps in the drug discovery process. Such applications include identifying disease-associated screenable phenotypes, understanding disease mechanisms and predicting a drug’s activity, toxicity or mechanism of action. Several of these applications have been recently validated and have moved into production mode within academia and the pharmaceutical industry. Some of these have yielded disappointing results in practice but are now of renewed interest due to improved machine-learning strategies that better leverage image-based information. Although challenges remain, novel computational technologies such as deep learning and single-cell methods that better capture the biological information in images hold promise for accelerating drug discovery.

In a drug development programme, evaluation of the efficacy and safety of all candidate compounds in humans, or even rodents, is ethically and practically unfeasible. Therefore, simpler model systems (cells, tissues and small model organisms) are used to map clinical efficacy and safety to a screening-amenable molecular target, pathway or phenotype in the process of screening 1 (Box 1). The design of a screening assay balances on the one hand p r a c t i c ality a n d a f f o r d a bi lity, a l l o w ing b r o ad e x p lo r a tion o f c he mi cal space, and on the other hand biological relevance to the disorder being studied or to possible safety concerns. Thus, the output, or readout, of a screening assay is typically chosen to be one or a few readily interpretable features that reflect biology already understood to be relevant for efficacy or safety 2 . Multiple such assays are used to test thousands to millions of small molecules to identify and triage hits (that is, attractive molecular starting points). The assays are also then used to advance hits to more drug-like leads and finally to optimize leads before preclinical development.

Profiling is an alternative strategy to screening. The word 'profiling' has two meanings: representing a sample by a profile (that is, a collection of features), and making predictions about a sample based on such a representation. Profiling aims to capture a wide variety of features, few or none of which may have previously validated relevance to a disease or potential treatment. It may, therefore, reveal unanticipated biology at play. Profiling often relies on the same or similar model systems as screening assays (for example, fluorescently stained cells), but profiling represents these model systems with a more comprehensive set of features [3] [4] [5] . More features can be powerful -for example, to uncover a previously unexpected mechanism of the disease or to provide sensitive quality control for the stability of an assay system -but they can also add unhelpful noise or be more difficult to interpret than a carefully selected screening feature.

Feature profiles can be constructed in many ways using many assay types 6 . Readouts can be generated from panels of separate assays; examples widely used in industry 7 include cell viability across cell line panels 8 , enzymatic activity across a kinase panel 9 and binding to a panel of safety-relevant targets 10 . More cost-efficient profiles can be acquired in a single multiplexed assay 11 by combining one of several kinds of high-dimensional readout technologies with cell-based model systems. Proteomic profiling and metabolomic profiling would be very powerful but are too low throughput due to their cost [12] [13] [14] [15] . Increasingly affordable sequencing approaches enable transcriptional profiling 16 to rival bead-based alternatives 17, 18 ; they also underlie highly multiplexed cell viability assays 19 .

Measuring the levels of a large number of proteins in a sample, sometimes including their post-translationally modified forms.

Measuring the levels of a large number of metabolites in a sample.

To date, image-based profiling using automated microscopy is the least expensive of these high-dimensional profiling techniques, and it inherently offers single-cell resolution, which can capture important heterogeneous cell behaviours. Computer vision techniques have advanced dramatically in the past few years, enabling the extraction of a huge quantity of unbiased morphological information from images. Cell-based microscopy assays have also advanced (Box 2), with assays such as Cell Painting inexpensively combining multiple stains in a robust assay yielding single-cell profiles composed of thousands of features 20 . The few reported comparative analyses indicate that image-based profiling may capture more biological information than high-throughput transcriptional profiling 21, 22 .

Profiling is not a widely deployed strategy at the primary screening phase in the pharmaceutical industry, where well-validated, bespoke screening assays remain the preferred approach. However, profiling does play a role at other stages of the drug discovery process. In the discovery stages before screening -including target identification, target validation, phenotype discovery and assay development -up to thousands of compounds are profiled in exploratory assays that combine imaging, primary or induced pluripotent stem cell-derived cells and/or genetic editing 1, 23, 24 . As a consequence, profiling in the early stages of drug discovery can reveal key biological readouts that can be used for the subsequent phase of screening at high throughput.

Image-based profiling is also regularly applied downstream of screening to the hits that emerge from target-based, pathway-based or phenotypic screening and validation. It then functions as an unbiased secondary assay, applicable to hits from any screen and in any disorder. At minimum, profiling can organize hits into groups with biologically similar effects. At best, it can hint at a compound's mechanism of action (MOA) and previously unsuspected off-target activities. However, it requires a considerable interpretation effort to derive these actionable insights from phenotypic features. With conventional analytical approaches, this interpretation step has been too onerous to scale up to screening full libraries.

Recently, machine-learning models were trained to predict the outcomes of hundreds of assays from a set of existing high-content images that were originally collected at high throughput for an image-based screen with a focused readout 25 . The study illustrated how machine learning can leverage side information -in this case, a large volume of assay activity labels. In another study, machine learning showed potential for image-based lead optimization 26 . Both of these studies were led by researchers at large pharmaceutical companies and reveal a renewed interest in profiling-inspired drug discovery. Not coincidentally, both studies were enabled by machine learning.

A new wave of biotechnology companies, including insitro and Recursion, are developing image-based profiling strategies with the explicit intent to feed a machine-learning effort 27, 28 . They rely on moderately sized libraries and unbiased image-based profiling for phenotype discovery and primary screening, rather than on extensive decks of compounds in more time-consuming, customized assays. This enables more rapid exploration of hundreds of model systems that emulate disease states with genetic perturbations, even though some disease states may be less well represented in a generic cell system without customized assay readouts (Box 2). Machine learning is deployed to learn to map chemical, genetic or pathological perturbations to their in vitro, in vivo or clinical effects and to transform raw profiles into a screening assay, reducing the need for expert-crafted screening assays.

In this Review, we focus on applications of imagebased profiling that are immediately applicable to the drug discovery process. We begin with a brief introduction to image-based profiling assays and analysis approaches and then discuss the status, successes and limitations of various pharmaceutical applications of this technology.

Image-based profiling does not require specialized equipment or reagents. All that is needed are images of biological samples that represent different cases (for example, categories of human patients) or treatment conditions (for example, chemical, genetic, time-point or other perturbations of the biological system) (Fig. 1) .

First, the biological samples are prepared. This is typically done in arrayed multiwell plate format, although living cell microarrays and pooled imaging strategies are higher-throughput options 29 . Next, the samples are subjected to treatment conditions of interest and incubated. The samples are imaged, typically after fixation and staining, although one can instead conduct live, time-lapse imaging 30, 31 and/or use label-free techniques such as those that predict a staining pattern from brightfield images using machine learning 32, 33 .

The images are processed to extract features, which are aggregated into profiles. Methods for this step Box 1 | Approaches to screening screening is the workhorse of modern drug discovery; 'screening' describes testing many potential drugs in an assay that in some way detects an impact on a disease. a screening assay can detect engagement with a pre-identified disease-related protein, known as a target (in target-based assays), or a predefined molecular output event such as phosphorylation, translocation or gene activation for a disease-related pathway (in pathway-based assays), or a change in a molecularly agnostic disease-related phenotype (in phenotypic assays). Pathway-based assays, while target agnostic, are as reductive as target-based assays; they tend to be relatively simple, relying on a cultured cell line and one or a few prespecified molecular readouts. Molecularly agnostic phenotypic screens use a broader definition of phenotype; they aim to replicate the human disease state as closely as possible in an assay format that is nevertheless efficiently screenable. some screens can be exceptionally complex, such as testing each compound's ability to alter the function of 3D organoids derived from patient cells. interest in phenotypic screens has fluctuated, driven initially by the expectation of improved validation in the clinic, but dampened by disappointment with the lack of evidence thereof 1, 23, 24, [164] [165] [166] [167] .

Designing a screenable assay involves selecting a combination of a model system, stimulus and readouts that aim to maximize clinical trial success. a popular guiding strategy in the industry called the 'rule of three', for example, specifies (1) designing the model system to best mimic the disease condition, (2) selecting a stimulus to produce a disease-associated response and (3) choosing readouts as the most proximal quantifiable features that reflect the functional consequence of disease 168 .

(MoA). The description of how a compound interacts with a target and affects a biological system.

Further available measurements or metadata about samples that indirectly improve predictive performance.

Values for particular parameters in a given set of samples. For example, each compound in a dataset might have a mechanism of action label or a toxicity label.

The process of narrowing down compounds after hit expansion to those with desired activity.

images captured from a sample without using any fluorescent illumination light.

www.nature.com/nrd are developing rapidly, in particular moving from expert-defined feature extraction to data-driven deep learning 34, 35 . Finally, the extracted profiles are analysed for biologically meaningful similarities and differences in a portion of the computational workflow that differs depending on which application has been chosen.

In essence, any set of images can be used for image-based profiling (Box 2). The workflow described above requires adaptation depending on the samples, perturbations, stains and imaging modality used, but the overall strategy is the same. That said, profiling will be most powerful if each image contains a large number of instances (for example, cells or organisms) and a large amount of visible information about each instance (for example, high-resolution images of multiple corresponding stains).

Challenges in profile analysis. Thousands of features can be extracted from images, but not all are equally useful for a given task. Different disease states or different compound mechanisms of action (MOAs) are best captured by reading out changes in distinct combinations of features, such as the shape and size of cell structures or the intensity and texture of various stains. An extensive image-based profile encompasses many feature combinations, overlapping or not, that together contain the information to document a broad spectrum of diseases and MOAs at once.

Notably, for a given goal, the presence of the relevant features is often not enough; we also need differential weighting: the amplification of features that are relevant to the goal and the suppression of features that are not. Importantly, the more extensive the profile, the more biological information it can potentially encode, but also the harder it is to extract the information for any given individual task from among irrelevant information. This phenomenon is referred to as the curse of dimensionality: the masking of the task-specific signal derived from comparatively few relevant features by the cumulative noise (related to technical variation) and confounding signal (related to unrelated biology) across the many other features. The problem can be addressed by combining features by weighted aggregation and/or more powerful representations produced by machine learning, which can have various levels of supervision (Box 3). In a way, an assay developer applies feature weighting when exploring and optimizing biological models, reagents and readouts for a screening assay. Computationally extracting relevant information from feature weighting of a profile can be thought of as the virtual development of a profile-based screening assay.

Since the first proposals to leverage image-based profiling for drug discovery [36] [37] [38] , the analysis of image-based profiles has largely relied on unsophisticated unsupervised clustering of the profiles, including some profiles for well-known controls. Profiles are used to place each sample into a single, shared high-dimensional representation space (panels a-c of the figure in Box 3) -also referred to as a (weighted) feature or phenotypic space -where the closeness between pairs of compounds corresponds to their profile similarity. 'Shared' here refers to adjusting all features in the same way, regardless of which samples or classes are being studied.

The nearest-neighbour strategy is an approach to add an element of supervision. It predicts the biological activity or MOA of a query compound on the basis of that of a reference compound with a suitably similar profile. The biggest advantage of the nearest-neighbour approach is its information efficiency: in contrast to more advanced methods, it can be applied even if only a handful of reference compounds with existing annotation are available. Nevertheless, like those advanced methods, the approach improves with more annotated references. Importantly, reference compounds must be sufficiently active, fairly selective and, of course, accurately annotated -which is not assured even for the current marketed pharmacopoeia. As a result, the nearest-neighbour strategy has limited power.

Feature adjustment, transformation and normalization techniques are recommended to increase signalto-noise ratios and thereby mitigate the curse of dimensionality 39 (panel b of the figure in Box 3). These techniques, however, affect only the shared representation space in which samples are placed. Hence, they do not remedy the vulnerability of unsupervised approaches to confounding signals from sample biases Supervision in machine learning, supervised learning aims for the system to predict the correct answers for each input, on the basis of examples. By contrast, in unsupervised learning the goal is to learn useful representations of each sample such that the similarities and differences among them can be observed.

there are two approaches to choosing the staining conditions and biological model system for image-based profiling: customized versus unbiased. in the customized approach, one chooses a model system and fluorescent markers that are thought to be associated with specific disease properties. the unbiased approach uses a more generic model system (for example, a particular cultured cell line) and a more general set of stains, regardless of the disease under study. although customization tends to more reliably provide information relevant to a disorder, there is growing evidence that unbiased marker sets can detect changes in a substantial proportion of biological pathways, even in a single assay using a single cultured cell line 22, 25, 59, 87 .

the most commonly used unbiased assay for image-based profiling is Cell Painting 20,104 , whereby six inexpensive dyes are used to stain eight cell organelles and components, which are imaged in five channels that each capture fluorescent light of a particular wavelength. Developed by the laboratories of stuart schreiber and anne Carpenter, the assay captures several thousand metrics for each imaged cell. although alternatives exist 169 , most of the current publicly available image-based profiling data were obtained by Cell Painting 82, 170 .

Customized image-based assays for profiling can range from a straightforward combination of stains for markers thought to be relevant to a disease to much higher complexity. For example, there are several techniques for performing multiple rounds of staining and destaining, which can yield dozens of channels/biomarkers detected per field of view [171] [172] [173] [174] . imaging mass cytometry also offers substantial multiplexing capacity, but requires expensive, slow instrumentation 175, 176 . Multiplexed image profiles can also be created by combining readouts from multiple separate assays, each treated with a different fluorescent probe 177 , although preparing multiple wells per sample limits this approach to relatively small numbers of samples.

the vast majority of image-based profiling is cell based for practical reasons, but animal-based screening is possible and powerful, primarily in the worm Caenorhabditis elegans 178 and zebrafish 179 . Most image-based profiles use static images as input, but notable exceptions exist; successful studies include identifying antipsychotic-like compounds in zebrafish behavioural assays 69 , identifying dynamic mitotic phenotypes 31 and detecting drug and growth surface response in time-lapse imaging of neutrophil granulocytes from patients with asthma 180 . while standardizing on a single image-based profiling assay such as Cell Painting allows profiles across experiments and research groups to be usefully shared, these examples demonstrate compelling reasons to choose alternative staining and imaging protocols and biological systems.

or polypharmacology (Box 4) in the shared representation space. Classical feature selection methods reduce thousands of raw features to hundreds according to feature correlation, but typically without a dramatic increase of discriminative power. The first step in the analysis of an image-based profiling study is often a further reduction of the high-dimensional phenotypic space to a 2D or 3D representation for visualization purposes using computational methods such as principal component analysis (PCA), t-distributed stochastic neighbour embedding (t-SNE), and uniform manifold approximation and projection (UMAP) [40] [41] [42] . Although such visualizations are a non-optimal representation of the rich information of high-dimensional profiles, they can provide a first sense of the similarity patterns in the experiment.

A substantial proportion of known bioactive compounds yield a significant change in image-based profiles: 68% in one study using the Cell Painting assay 22 . Yet, due to the curse of dimensionality, a limited number of MOAs can be readily distinguished by the nearestneighbour strategy: typically only one or two dozen form well-delineated clusters, and rarely are new clusters identified. The commonly observed clusters tend to represent compounds with disruptive mechanisms, such as histone deacetylase inhibition and microtubule interference. The ability of image-based profiling to yield distinctive clusters of compounds for other, often subtler mechanisms is limited and has not increased much over the years 43 . In a recent study where 15 reporter cell lines were evaluated, even after feature selection only 20 of 83 MOA classes were readily distinguishable by image-based profiles in the best single cell line, and 41 were readily distinguishable, cumulatively across all 15 cell lines 44 .

Driven by the recent availability of everlarger volumes of images, the field has begun to turn to machine learning, and specifically deep learning, to improve extraction of relevant signal from profiles 45 . In essence, a deep neural network is a concatenation of multiple layers, each trained to reweight and transform data to achieve a goal, such as detecting objects in images or predicting the compound to which a cell has been exposed. An input layer introduces data such as image-based profiles or raw images into the network. Each layer then reshapes the data as it flows through the network. Finally, the output layer exports the completely transformed data.

These techniques thus learn a more sophisticated representation than simple reweighting. Self-supervised methods focus on information that can be learnt from different documentations of the same object (perturbation, sample or cell type) (panel c of the figure in Box 3). For example, if the same microscopy field is documented with two views with different stains, a method can be trained to predict one view from the other; in the process, it learns to filter out the noise in either view to leave only the robust signal 46 It includes six stains labelling eight cellular components, which are imaged in five channels 20 . ER, endoplasmic reticulum.

The property of a compound whereby it interacts with more than a single target.

A machine-learning architecture whereby features of a sample (for example, image pixels or image-derived metrics) are fed into a network of nodes, which collectively learn to produce the correct answer for that sample by each node adjusting its contribution (weight) to the final answer.

www.nature.com/nrd views (that is, replicates) that document a specific perturbation (for example, chemical or genetic) from those documenting any other perturbations 47, 48 . These machine-learning methods require more substantial volumes of images for training but position all samples in a cleaner shared representation space than unsupervised methods. The more samples that are included, the more mechanisms and biological processes this shared representation space will try to simultaneously capture, challenging the representation's discriminatory power. Like unsupervised or self-supervised approaches, supervised methods require a large volume of images for training, but in addition they need training data, including training data points with activity labels for predefined tasks, and they benefit from side information. If large label sets are available that annotate compounds with their activities in validated assays, supervised methods can be trained, for example, to deconvolute the relevant signal from raw images or image-based profiles, including the output of self-supervised methods 25 . In contrast to unsupervised or self-supervised approaches, which learn a single representation space to position samples irrespectively of tasks of interest, the output layers of advanced supervised approaches produce Box 3 | increasing discriminative ability in phenotypic space various computational strategies can improve results in image-based profiling, shown here as a theoretical example for identifying the mechanisms of action (MOas) for a set of compounds. each tested sample (in this case, each compound) is represented as a dot in the phenotypic space, where distances between dots reflect the similarity of images of cells treated with the compounds. even with the best strategies, many MOa classes will not be readily detectable or distinguishable. the strategies shown in panels a-c in the figure are useful to get a quick view of clusters of samples in a given dataset even if no sample annotations (for example, MOa labels) are available to indicate what each cluster represents. if these strategies are to be used to assign an MOa class to each sample, the approach would be called 'semisupervised', because after creation of this shared space, close proximity to compounds with known MOas (if any) would be used to assign MOas.

if raw features are extracted from images and placed into phenotypic space with no adjustment, samples do not typically form noticeable clusters (panel a). all features are equally weighted, such that those most relevant to the task at hand are typically drowned out by irrelevant, noisy or redundant features.

an unsupervised machine-learning method can select the appropriate weights for each feature in order to emphasize important ones and suppress noisy or redundant ones (panel b). this process can be performed by techniques ranging from principal component analysis to autoencoders and is depicted as individual features becoming darker or lighter in the profiles shown below the scatterplot. this weighting typically allows classes of compounds with strong morphological phenotypes (pink and green, here) to be distinguished.

self-supervision strategies leverage redundant image information (for example, multiple replicates per perturbation, different channels for each microscopy view or multiple instances per cell type) to distil a robust signal (panel c). the phenotypic space of reweighted features learned by the network often yields a better discrimination of MOa classes.

when MOa annotations or assay activity values are available, the problem becomes supervised (panel d). although one can learn a common (shared) representation using supervision (not shown), in practice this does not typically work well given the scale of MOa annotations available. instead, here, for each MOa class, features in the profile are specifically weighted by training on samples with known activity in a corresponding assay. this can be especially helpful to handle polypharmacology; in our example, the blue and yellow-green compounds are distinguishable from other compounds via individual models in panel d but not by any shared model (panel a, b or c). this could result from all compounds in the blue and yellow-green classes having the MOA 'X', but half of these compounds may have an additional MOA, 'Y', whereas the other half have a different additional MOA, 'Z'. Training separate models for MOAs 'X', 'Y' and 'Z' allows the features to be weighted so that each model ignores features associated with the other MOAs and more readily focuses on the targeted MOa. Drug development companies are increasingly treating their existing data as a resource to be leveraged for machine learning. The promise of improved profile resolution through rich data and machine learning is currently rekindling industrial interest in phenotypic profiling. Although the pay-off may be great, serious machine-learning effort comes at a high cost, largely in terms of recruiting and retaining highly sought-after experts and giving them the time and computational resources to build and maintain suitable software. Machine-learning strategies also risk problems such as overfitting, technical artefacts or confounders in the data, and bias 49,50 (Box 4). The ideal experimental set-up is often cost-prohibitive, and avoiding all possible confounding factors is impossible. Thus, practically speaking, practitioners must carefully design a combined experimental/data strategy that diminishes the risks; this requires deep understanding of the problem. One mitigant of machine-learning confounders in drug discovery is that for many of the applications we will discuss, the goal is simply to triage compounds that will subsequently be tested thoroughly, such that false positives do not proceed. Nevertheless, in biomedicine, machine-learning methods that are interpretable -that is, where one can learn what features of samples are being used in decision-making -can sometimes ensure that technical artefacts are not the source of the signal and can provide insight into disease and drug mechanisms.

Profile-based phenotype discovery and screening It takes many months to years to develop a conventional image-based assay for screening, even after years of basic research into the mechanisms of a disease. One time-consuming step is hypothesizing and engineering one or a few relevant assay readouts, most often the staining of a molecule or other cell component. The time spent on this step can be dramatically reduced by using a generic staining approach such as Cell Painting (Box 2); applying image-based profiling to such images typically yields more extensive feature sets than customized staining. In conventional image-based assay development, researchers also spend considerable effort selecting suitable conditions for testing a drug's impact on putative disease-associated cell phenotypes, which might include the cell type and culture conditions, appropriate stimuli and the duration of drug exposure. Here too, timelines can be cut by taking a more generic approach; for instance, one that relies on a simple cultured cell system and standardized assay conditions that are not customized to the diseases at hand. However, developing customized assay readouts with a careful selection of sample material, stimuli and time points may yield a higher probability of finding a disease-relevant phenotype, perhaps justifying much longer assay development timelines. Furthermore, many disease programmes will involve highly customized assays in the long run as secondary assays, so one could argue these may as well be developed up front for primary screening unless their throughput is limiting.

Regardless of where an image-based assay is on the spectrum from unbiased to customized, the typical steps to profile-based phenotype discovery and screening (also known as signature discovery and signature-based screening) are as follows.

• Prepare sets of biological samples that represent the disease state and the healthy state via strategies described in detail later in this section (TABle 1) . • Capture image-based profiles and attempt to identify any reproducible phenotypic difference between the diseased and healthy samples. This phenotypic difference will become the screening objective -that is, the phenotypic assay readout. This readout might be a single feature extracted from a single image channel (in essence, a conventional high-content assay), or it might be a multifeature profile that discriminates between the diseased and healthy states. Machine learning and side information may be required to filter out confounding signals and noise. The discovery of novel phenotypes associated with a disease may itself yield new mechanistic insights into the disorder. • Optionally, simplify the assay (for example, remove unnecessary fluorescent markers) to reduce its cost, or add markers that serve a useful triaging function for hits. • Use the identified processed phenotype or profile to (a) test thousands to millions of chemicals for their ability to reverse the disease morphology to resemble the healthy state or (b) virtually query an existing dataset of image-based profiles from chemical perturbations of healthy cells to identify those whose perturbation yields the 'opposite' (anticorrelated)

apart from the curse of dimensionality (the masking of a signal of interest by the noise and confounding signal of other features in a profile), all high-dimensional profiling methods, including imaging, face additional challenges. technical artefacts such as batch effects and plate layout effects typically make images from particular locations on a plate or from particular batches look more similar to each other than to images of the same sample in different plate locations or different batches 181 . even different lineages of a given cultured cell line can be distinguished on the basis of image-based profiles 61 . although arguably the ability to detect technical artefacts speaks to the sensitivity of image-based assays, how to best mitigate the effects remains unresolved.

when one is identifying disease phenotypes from patient samples, there is a risk of confounding by various genetic or sample biases that may not be relevant for the disease. For example, samples from diseased patients might have a different demographic balance or might be more affected by particular technical artefacts than controls, such that features associated with the disease samples lack true predictive power. it is difficult and expensive to carefully identify and control for all possible confounding factors, increase sample sizes and/or test separately obtained cohorts, so these present real risks.

For compounds, a unique challenge is polypharmacology (that is, the simultaneous engagement of a small molecule with multiple targets or processes). Historically, successful drugs have been thought to be highly specific in their activity, but it has become clear that polypharmacology is the rule rather than the exception, even for marketed drugs 182, 183 . Polypharmacology results in profiles that convolve multiple signals, only one or two of which may be relevant for the intended activity, while the other signals may indicate favourable, unfavourable or undefined effects of the compound. attempting to identify the mechanisms of a polypharmacological compound can be a challenge: at best, its profile shows strong similarity to the single most prominent mechanism it features; at worst, the profile becomes suspended in a poorly interpretable 'no-man's land'.

www.nature.com/nrd phenotype, indicating a favourable impact on the same pathways as are impacted by the disease. In addition, compounds that produce the same (correlated) profile as the disease can potentially provide useful mechanistic information. • Optionally, identify or validate novel targets for the disorder by (a) testing a genome-scale set of genetic perturbations for their ability to modify the disease-related phenotype or (b) virtually querying an existing genome-scale dataset of image-based profiles from genetic perturbations of healthy cells to identify or validate genes whose perturbation yields the same (correlated) or opposite (anticorrelated) phenotype. Novel, validated targets could then be fed into conventional target-based drug discovery pipelines.

Identifying a disease-associated phenotype. The first step, identifying a disease-associated phenotype in images, is crucial 51 . Several strategies exist for identifying a cellular disease state with a profile that differs from that of the healthy state (TABle 1) . First, patient-derived cells are a physiologically relevant choice, assuming a sufficient number of independent patients are available to yield confidence that phenotypic differences are associated with the disease rather than due to the inherent morphological variability of cell lines across patients. Caution must be exercised, as high-dimensional profiles are prone to confounding factors (Box 4), whereby features that seemingly distinguish between healthy and diseased states may in fact reflect age, genetic, exposure or sample biases that are not relevant to the disease. Nevertheless, many reproducible image-based phenotypes have been discovered, often inadvertently, as scientists stained and visually examined cells, typically using common markers such as organelle dyes. For example, unusual mitochondrial structure was identified in fibroblasts and lymphocytes from patients with bipolar disorder 52 and in fibroblasts from patients with Leigh syndrome 53 , and normal human fibroblasts can be differentiated from Huntington disease fibroblasts using only tubulin staining 54 . Image-based profiling offers a way to scale-up and systematize this kind of serendipitous discovery. A second approach to identifying a disease-associated phenotype is especially suited to disorders caused by loss-of-function mutations in single genes. A gene's expression is decreased using RNAi or CRISPR and then the morphological impact on cells is examined, taking care to identify off-target effects 55 . This approach was used to detect an impact on cell structure (as detected by stains for DNA, actin and VE-cadherin), which was obvious by eye, of RNAi knockdown of CCM2, the gene associated with the loss-of-function disorder cerebral cavernous malformation 56 . Through screening, the researchers identified small molecules that reverse the phenotype. Remarkably, compounds chosen by a computational analysis outperformed those chosen by eye in secondary physiological experiments. This study, conducted at the University of Utah, led to the launch of the biotechnology company Recursion. The company has since identified hundreds of disease-associated image-based phenotypes available for parallel screening and has placed four drugs into clinical trials. In a similar strategy, an academic research team took a comprehensive approach to investigate alleles from genome sequencing studies: in adipocytes differentiated in vitro, they ablated 125 genes associated with type 2 diabetes and clustered the resulting image-based profiles, identifying novel lipodystrophy genes 57 . Another team mutated zebrafish orthologues near 132 schizophrenia-associated alleles and created behavioural and brain structural image-based profiles, prioritizing candidates for further study 58 .

A third approach to phenotype identification for screening, gene overexpression, is especially suited for testing alleles in protein-coding regions of genes that are known or hypothesized to cause disorders. The mutant form of a given protein is exogenously expressed in cells, and its image-based profile is compared with that of cells overexpressing the wild-type form 59 . Although overexpression may itself impact the cell's structure and function, observing a differential signature between wild-type and mutant overexpression yields a strong hypothesis for disease-related impact. The Taipale laboratory at the University of Toronto is using this strategy for a set of monogenic disorders, with the twist that each disease-associated protein is flag-tagged so that disease-associated changes in protein localization can be detected in addition to changes in cell morphology (M. Taipale and J. Lacoste, personal communication). One could envision this strategy becoming the routine next step after every sequencing study that yields long lists of hypothesized disease-associated variants: generate appropriate genetic reagents for all alleles and test their image-based impact in a suitable cell line. This approach might allow clustering of alleles into different functional groups, such that a subset of each group could be studied in more disease-specific assays. Already, image-based profiling of gene overexpression is showing promise in determining the impact 60 . A fourth strategy is to genetically modify cells, which, unlike overexpression, allows interrogation of both coding and non-coding variants. For now, the single-cell cloning procedures needed for gene editing techniques are too slow and inconsistent for large-scale use, and lineage artefacts may confound accurate phenotype detection 61 . Nevertheless, the approach can work well for testing individual alleles. Recently a change in image-based profile was detected for cancer cells genetically modified to express mutant focal adhesion kinase (FAK) versus its wild-type form 62 . This genetic perturbation was intended to mimic pharmacological treatment targeted to FAK and simplify the pathway engagement while avoiding off-target concerns with FAK small-molecule inhibitors. The study authors then screened small molecules to identify putative synergistic combinations with FAK inhibitors, identifying histone deacetylase inhibitors as potential novel kinase inhibitor drug combinations for cancer. It may soon be feasible to rapidly and systematically genetically engineer cell lines containing disease-associated alleles in their endogenous locations, which opens the door for this approach to be conducted more systematically. One can even engineer model organisms to carry human disease-associated alleles to create 'avatars' for identifying image-based phenotypes and subsequent testing of compounds [63] [64] [65] [66] [67] .

A fifth approach to identify a screenable phenotype is to identify changes in image-based profiles associated with existing drugs for a particular disease of interest, then use those phenotypes to identify new candidate drugs and often their MOA as well. This approach has identified new behavioural phenotypes and potential therapeutics in zebrafish time-lapse imaging using existing psychotropics 68 , antipsychotics 69 , appetite modulators 70 and anaesthetics 71 as the query compounds. Image-based profiling also revealed morphological changes associated with 61 structurally diverse free fatty acids, identifying those that are associated with lipotoxicity in insulin-secreting pancreatic β-cells (N. Wieder and A. Greka, personal communication).

These five strategies for identifying screenable phenotypes differ in terms of the ease of assay development, the degree of customization required and how closely they reflect the human disease state. At one end of the spectrum, one can choose a single cell type and set of staining conditions to search for many disease phenotypes in parallel. In this case, the assay development time for each disorder is zero, but many phenotypes will be missed due to a lack of a suitable stain or appropriate biological conditions. Still, the simplicity, cost-efficiency and scalability of the strategy make it an attractive approach: although customized follow-up validation assays must still be created, they can be more expensive and of lower throughput, and they need to be made only for the subset of disorders showing promising hits. By contrast, there will be more chance of identifying a phenotype for any particular disease if the assay is more specifically tailored to the disorder, although this increases the assay development time. For example, one could explore several healthy and diseased systems to be imaged, such as several cell lines, primary cell types, mixtures of co-cultured cells or even differentiated cell cultures. In addition to more physiologically relevant cell/organism systems, stains can be customized to reflect several biological processes associated with the disorder. At the limit, these customizations end up as equivalent to conventional phenotypic assay development.

It should be noted that reversing disease-associated phenotypes identified via the strategies described above will not always yield effective drugs. First, cultured cells do not reflect the full intricacies seen in a human organism; many disease mechanisms are not cell autonomous, and many drugs will have different effects in patients' whole bodies, especially considering their different genetic backgrounds and environmental exposure. These limitations are shared with all biochemical or cell-based drug screening methods and are fairly well appreciated. However, profile-based phenotype discovery introduces additional concerns. For example, phenotypes detected as described above might be completely incidental to the phenotype that causes symptoms for patients, such that drugs reversing the phenotype are ineffective. Even worse, the detected phenotype might reflect the cells' attempt to mitigate the impact of the disease perturbation, such that drugs reversing the phenotype would aggravate the condition in patients. Nevertheless, the speed that profiling offers allows extra time to rule out these kinds of problems.

The drug industry as a whole has begun to adopt image-based profiling to inform target identification and validation, phenotype discovery and assay development before screening. However, for screening itself, wherever possible, the industry tends to prefer a customized assay focused on a molecularly defined target or pathway that adequately reproduces the profile-based findings. Focused assays facilitate determining structure-activity relationships (SARs), selecting mechanistically inspired biomarkers of efficacy and identifying on-target liabilities that correlate with desired on-target activity. Moreover, contrary to expectations, compounds identified in early mRNA and image profile-based phenotypic screening efforts do not seem to have been validated in animal models more often than compounds identified in more straightforward target or pathway-based screening approaches -although, admittedly, no systematic analysis has been reported to date. Large pharmaceutical companies also tend to focus on selected disease areas, and in those areas they prefer to maximize the chances of success by screening large compound libraries, which can be more efficiently accommodated in streamlined customized assays, especially if profiling will nevertheless eventually be used in follow-up assays. However, thousands of diseases fall out of industry scope. As a result, biotechnology start-ups (Recursion and insitro, most notably 27 ) are exploring this space. As mentioned in the introduction, they rely on profile-based phenotypic screening of smaller libraries in generic assays to more efficiently evaluate the potential of compound intervention across that less frequently studied disease spectrum.

In the drug discovery process, screening is followed by lead generation, in which hundreds of screening hits are narrowed down to just a few lead candidates. The various applications of image-based profiling for lead generation can use either unbiased assays (which could be systematically applied across disease areas) or customized assays with relevant biomarkers (which can be tailored to particular disease areas) as described in the previous section and Box 2.

During lead generation, two complementary activitieshit expansion and lead optimization -are pursued to triage and modify compounds for the most favourable attributes for further drug development. Conventionally, lead generation is guided by several individual assays, including the primary screening assay. These typically have simple and rapid readouts, reflecting single effects of molecules, such as enzyme activity, reporter expression, aggregation of a target protein or cell viability. Image-based profiling can be a tractable path for lead optimization, because it is quick, is sensitive and covers a broad, though not comprehensive, range of biology. Rather than relying on multiple assay readouts to select compounds with desired qualities, image-based profiling presents the possibility of selecting compounds with several desired qualities using a single assay 72 . Moreover, image-based profiling can quickly and comprehensively establish relationships among hits into biological clusters, independently of compound structure, providing direction for prioritization 37, 73, 74 . In addition, hits can be triaged quickly across disease-specific cell types with different genetic backgrounds using the same image-based profiling assay to assess the efficacy of compounds 75, 76 .

Recently, several SAR studies applied image-based profiling using Cell Painting to assess the biological activity of newly synthesized compounds and build diversity sets for focused libraries [77] [78] [79] [80] . The quick deployment and richness of the readouts enabled researchers to efficiently examine the effects of compound modifications across a wide range of biological activity. The unbiased assay proved to be sensitive enough to distinguish the effects of stereochemistry on activity. Furthermore, because an unbiased image-based profiling assay covers such a broad spectrum of activity, modifications that act on different mechanisms can be identified 78,79 -an advantage over conventional SAR studies.

Because these readouts reflect the consequences of molecular modifications for biological activity, this approach can be used to examine biological activity of hits from targeted screening, provided the cellular model system contains the molecular target in a biologically relevant context. Importantly, one must confirm that observed effects are indeed the result of target modification by the test compound and not the result of cellular adaptation via an alternative mechanism or polypharmacology. Furthermore, and more challenging, is the possibility of no observed response. The absence of a response may not be due to the absence of a target or target binding, but could be due to complex cellular pharmacologya combination of compound concentration, duration of treatment and cellular metabolism -which yields an observable morphological profile similar to no-treatment controls.

In this context, it is helpful to understand the relative association of image-based features with the desired phenotype or with off-target activities. However, this interpretation step is still a challenge beyond simplistic cases in which a small number of image-based features are strongly affected and easily interpretable, such as a change in protein localization. Although it has yet to be demonstrated for lead optimization, machine learning is emerging as a powerful tool to deconvolute multiplexed profiles of studied hits into virtual assays for distinct biological activities (as described in the next section), which can be favourable, neutral or unfavourable. Whether translated into interpretable activities by nearest-reference or by more advanced machine-learning approaches, the rich information in image-based profiles for newly synthesized compounds can be used to track the incremental effects of subsequent compound modifications during lead optimization.

Predicting assay activity. Compared with traditional hit expansion approaches, machine learning offers some attractive alternatives that make use of unbiased image data and provide an advantage over conventional SAR studies. By nature, image-based prediction can generate hits that are structurally diverse because predictions are based on activity in a biological system rather than chemical structure. Also, being inherently multiplexed, image-based methods are potentially capable of predicting the activity of hit compounds in assays unrelated to the assay that generated the image-based profiles.

This hypothesis was verified by a multi-institution team using image-based profiles from Janssen to successfully predict the activity of structurally diverse hit compounds in screening assays. In two validation studies, the strategy yielded a 60-fold to 250-fold increase in hit rates compared with the original screening assays 25 . Using the program CellProfiler to perform classical image segmentation and feature extraction, they leveraged supervised machine learning to predict the activity of a large set of compounds in given assays, based on archived imaging data on those compounds in an unrelated assay. Building on this research, a later team constructed a novel network architecture, GapNet, to predict ChEMBL-derived 81 compound annotations for a 30,000-compound Cell Painting dataset 82 : 32% of the assays could be well predicted 83 . In addition to the practical implication -that many expensive screens might be replaced effectively with a computational prediction step -this result also indicates that a substantial proportion of biological pathways of interest are captured in a single imaging assay. Furthermore, by combining image profiles with compound structural information, machine learning was able to predict active chemical structures de novo 26 .

If these machine-learning strategies work well in practice, one could envision the sizes of primary screens being so substantially reduced in the future as to fundamentally change the typical drug discovery process, with

The selection of compounds that were not tested in the primary screen, to broaden the diversity of the chemical space for hit selection. Compounds are selected on the basis of similarities in structure or biological activity to candidate hits.

An iterative process for lead optimization in which assays are applied to determine the effect of successive structural modifications to a compound on activity.

Nature reviews | Drug Discovery heavier reliance on computational predictions rather than experimentation in early stages.

Predicting toxicity. Identifying a compound's liabilities during lead generation is as important as assessing its efficacy; roughly 17% of phase III trial failures are due to safety concerns 84 . A simple cell-based assay such as Cell Painting, which could readily be scaled to an entire million-compound collection, is appealing for toxicity prediction given widespread enthusiasm for reducing the use of intact animals for testing both pharmaceuticals and environmental chemicals 85, 86 .

Recently, a team at the US Environmental Protection Agency used image-based profiles from the Cell Painting assay to characterize selected chemicals' bioactivity and toxicity 87 . One nuance of toxicity testing is that the dose at which effects are seen is paramount to discerning whether the compound will be toxic in humans. By comparing the toxic concentration thresholds for each chemical generated with other methods with thresholds generated with image-based profiling, the team concluded that image-based profiling is a viable, cost-effective alternative for chemical safety assessments. Image-based profiling of full dose-response curves for large-scale chemical libraries (for example, six dose points for millions of compounds) would require significant investment, but subsets of doses or subsets of compounds are readily feasible. Cell Painting's ability to predict various specific cell health assay readouts that provide mechanistic information, such as stalling in various cell cycle states 88 , provides further motivation for the collection of large-scale Cell Painting data on compounds for pharmaceutical, agricultural and environmental use.

Other methods combine image-based profiling with machine learning to identify the MOAs of toxicants in different types of cells. Image-based features can predict nephrotoxicity of drugs, chemicals and toxicants targeting human renal proximal tubular cells 89 . A high-throughput in vitro phenotypic profiling for toxicity prediction (HIPPTox) system predicted pulmonotoxicity more accurately than cell viability assays in human lung cell lines 90 . Image-based profiling identified rosmarinic acid as a candidate with cardioprotective effects against the toxicity of doxorubicin 91 . This approach also revealed the toxic effects of bisphenol A and its analogues on a testicular cell co-culture model by simultaneously measuring multiple adverse end points such as changes to nuclear morphology and cytoskeletal structure 92 .

Identifying the MOA Elucidating the MOA of a drug provides a deeper understanding of its biological activity 68, [93] [94] [95] [96] , increases its chances of clinical approval and enables the design of novel drugs 97 , and provides insights into the potential to separate unfavourable effects from favourable effects if they are driven by different targets 98 .

Identifying the MOA and/or targets of a hit or lead presents a major challenge, particularly for candidates arising from phenotypic rather than target-based screens. No single experimental technology can definitively identify a compound's MOA. Instead, the most likely MOA is often inferred from a combination of several complementary methods 99 . Proteomics approaches can be used to identify unknown protein targets after pull-down with a tagged compound or if an untagged compound shifts the target's stability 100 . Assay panels can readily identify the engagement of a compound with a set of predefined targets such as kinases, but only a fraction of the druggable proteome can be queried in this way, and every assay has its limitations. Finally, phenotypic profiling, including image-based profiling, can also contribute to MOA identification, as we describe in this section.

There are three broad categories of image-based profiling approaches to determining the MOA. The first approach, known as guilt-by-association with annotated compounds, involves comparing image-based profiles of compounds with unknown MOAs with those of well-annotated compounds to identify neighbours 101 . In this approach, image-based profiles are clustered according to the assumption that drugs with similar MOAs generate similar phenotypic signatures 102 . This strategy has been successfully implemented across studies spanning more than a decade 73,75,102-106 . However, as described in the section Analysis techniques evolve, image-based profiles and classical methods have effectively grouped compounds for many classes of MOAs but are ineffective for other classes, and polypharmacology is confounding. Nevertheless, there are many examples of novel MOA discoveries being made by this route, including identifying novel inhibitory activity of silmitasertib 107 , the MOA of autoquin (an autophagy inhibitor with a non-protein target) 108 , the translation inhibition of phenomycin 109 , the activity of synthesized pseudo-natural products, pyranofuropryridones, which remained inactive in other common assays 110 , and useful components of natural products [111] [112] [113] [114] .

We expect that in coming years this guilt-byassociation strategy will become increasingly sophisticated by leveraging advancements in machine learning, availability of more extensive reference annotation, integration with other data sources and optimization of cell lines. As the choice of cell line informs MOA prediction accuracy 21 , effort has been made towards engineering reporter cell lines 43 and identifying cell line-invariant features 115 . Others have interrogated genetic heterogeneity, finding distinct morphological responses to serotonin modulators across breast cancer cell lines 116 . The integration of image and transcriptome data into ensemble approaches has shown promise to improve MOA determination for synthetic small molecules, natural products and identified bioactive metabolites 21, 117 . Image-based profiles of cells treated with natural products have also been combined with mass spectral features of the same natural products to discover a novel family that causes endoplasmic reticulum stress 118 . If image-based and other profile types carry complementary information, then creating and integrating both kinds of profiles will be useful; if they are instead largely redundant, then researchers could acquire only one modality and build translators to convert one type of profile to the other 119 .

In the second approach, known as guilt-by-association with perturbed genes, the image-based profile of the drug is matched to that of cells perturbed with a specific genetic reagent, thereby yielding a hypothesis for the drug's target and MOA 120 . As pioneered with yeast mutants [121] [122] [123] , this approach has shown success in cell-based, image-based profiling experiments, although it is not yet widely used. Genetic perturbation can be achieved by a variety of techniques. Historically, perturbation by small interfering RNA (siRNA) has been most common [124] [125] [126] , although seed effects can be confounding 127 . Another strategy is to use CRISPR-Cas9 (ReF. 128 ) to knock out genes of interest as it is less susceptible to undesirable and confounding off-target effects of siRNA 55, 129 . Overexpressing genes is another option that can yield distinctive image-based profiles 59 , which in theory could be matched to a chemical's image-based profile. Most chemical compounds inhibit a protein's function, but it is currently unclear whether matching a suppressed gene more effectively determines the MOA as opposed to looking for a profile that is anticorrelated to that of an overexpressed gene.

The third approach, which is based on rescue experiments, might be more accurate than the similaritymatching strategies described above, but would require new, large-scale experiments for each query molecule rather than a simple computational matching exercise. In a rescue experiment to determine the MOA, cells treated with a given drug that induces a particular image-based profile would be treated with genetic perturbations to identify any that can reverse the profile, such that the cells resemble untreated cells. There are currently no published examples of this approach, although pooled optical profiling puts this type of experiment closer to reality by offering the capability to test hundreds to thousands of genetic reagents in parallel 29 .

As mentioned in the section Analysis techniques evolve, most image-based MOA prediction studies use a nearest-neighbour approach. This procedure assumes that each drug has a single MOA, but it is now well appreciated that few chemicals impact only a single protein within a cell. Instead, polypharmacology, including off-target effects, is commonly seen (Box 4). One or two of these targets may be relevant in the context of the disease of interest, but others may also be reflected in the unweighted profile and obscure the profiles resulting from the targets of interest. For certain MOAs, particularly those with a strong and broad impact on cellular morphology, clusters of compounds are easily identified with straightforward methods 130 but such methods fail for many MOA classes. Hence there is a need for MOA determination methods that account for polypharmacology. Deep learning-based approaches have been developed 25, 83 that are inherently well suited for tackling this problem, as they can produce MOA-specific, reweighted image-based profiles, thereby deconvoluting the complex phenotypes arising from polypharmacology 44 . We are only beginning to scratch the surface of image-based profiling of single perturbations, but testing pairs of reagents has begun for limited sets of small molecules 131 and genetic perturbations 132 . Larger such datasets should lead to a better understanding of how to identify and deconvolute polypharmacology in profiles. Regardless of how MOA hypotheses arise, ideally more than one structurally different and accurately annotated compound targeting the same target/pathway will be available to lend evidence to a hypothesis.

Beyond drug discovery Outside the scope of this Review are many other successes of image-based profiling applied to a wide variety of important biological phenomena and cellular structures, including identifying relevant genes via functional genomics and studying cell responses to growth topologies and differentiation factors [133] [134] [135] [136] [137] [138] . Although identifying biomarkers of disease or drug response can be a first step towards drug discovery and a helpful aid in clinical trials, we do not cover in this Review the tremendous strides that have been made in image-based diagnostics, in some cases by using deep learning on unlabelled samples to create label-free diagnostics 139 . Examples range from suggesting a diagnosis 140 to predicting patient outcomes [141] [142] [143] or even molecular phenotypes 144, 145 . Lastly, we do not cover interventional, personalized medicine applications here, where patient samples are treated with various potential therapies and imaging is used to measure responses. Examples include the image-based profiling of the response of a bacterial strain isolated from a patient to various antibiotics 146 , of organoids derived from a patient with cystic fibrosis to drugs 147 and of tumour cells or organoids to various chemotherapies [148] [149] [150] .

We fully expect advancements in the field of image-based profiling -both computational and biological -will progress rapidly in the next 5 years as the approach gains attention.

On the computational side, deep learning is already beginning to accelerate drug discovery by tackling diverse problems in the process 151 , and image-based profiling will be among the major beneficiaries of advancements in computer vision and predictive algorithms [152] [153] [154] . Deep learning can process raw microscopy images to produce representations that are better suited for downstream analysis and interpretation; cells or cellular substructures can be identified more accurately 155, 156 , and improved image-based descriptors can be derived during feature extraction 157 . Deep learning may therefore eventually replace classical image processing and feature extraction algorithms, such as those in the currently most commonly used software program, CellProfiler 158 , or one of its commercial counterparts. Deep networks have been trained for the interpretation of image-based profiles: they can recognize the biological states of imaged cells 159 or predict the biological activities in validated assays for imaged compounds 25 . Convolutional neural networks can also integrate bespoke feature extraction and interpretive tasks in a single process 160 . Emerging anecdotal evidence suggests that provided with enough activity labels, these single-step, end-to-end networks have a predictive performance superior to that of conventional feature extraction Nature reviews | Drug Discovery and profiling. This may stem from deep learning's ability to learn specialized composite image features beyond the ones predefined by feature extraction software. Alternatively, it may reflect an implicit encoding of the heterogeneity of single cells within a microscopy image 134 . Capturing single-cell heterogeneity after feature extraction has indeed been found to improve image-based phenotypic clustering 161 . Finally, the flexible architecture of neural networks enables information to flow in from alternative data sources and formats, as input, or as side information.

On the biology side, drug discovery scientists are adopting increasingly complex model systems for their image-based profiling, such as differentiated cell types, tissues, organoids and whole model organisms. These assay systems may more routinely become higher-resolution and multidimensional systems, including 3D and time-lapse image capture. Sequencing-based barcoding methods are enabling larger-scale genetic perturbation libraries to be profiled by imaging 29 . We also expect that data generation and machine-learning approaches will become increasingly intertwined. For example, networks can learn to predict fluorescence patterns from transmitted light images 33 when provided with enough training pairs. This may in the future enable cost-efficient label-free image capture.

The ability to make broad activity predictions on the basis of microscopy screen images that were originally collected for a single mechanism of interest 25 has created an appetite to combine similar approaches with the richer profiles from multiplexed generic staining protocols such as Cell Painting 83 . Biotechnology and pharmaceutical companies are already investing in generating purpose-built image sets that document genetic and compound-induced perturbations. This investment occurs in proprietary settings, but also in the context of public-private partnerships involving multiple pharmaceutical companies, which will ultimately boost the availability of such datasets in the public domain 27 . It remains to be seen whether image-based profiles have reproducibility problems as recently suggested for mRNA profiles in the largest single-site public dataset 162 , and whether machine-learning approaches can reconcile image-based profiles across experimental batches and data generation sites. Nevertheless, we hope that increased availability of high-quality queryable image datasets 163 paired with side information on imaged compounds, genetic perturbations or disease models will in turn inspire the design of yet more powerful machine-learning methods, driving a virtuous circle of discovery.

Published online xx xx xxxx

How phenotypic screening influenced drug discovery: lessons from five years of practice

Increasing the content of high-content screening: an overview

Filling the drug discovery gap: is high-content screening the missing link?

Microscopybased high-content screening

Applications in image-based profiling of perturbations

Multiparametric assays for accelerating early drug discovery

Applications of deep-learning in exploiting large-scale and heterogeneous compound data in industrial pharmaceutical research

Morphological and immunocytochemical characteristics of human tumor cell lines for use in a disease-oriented anticancer drug screen

Profiling prediction of kinase inhibitors: toward the virtual assay

Reducing safety-related drug attrition: the use of in vitro pharmacological profiling

Connecting phenotype and chemotype: high-content discovery strategies for natural products research

A library of phosphoproteomic and chromatin signatures for characterizing cellular responses to drug perturbations

Multidimensional profiling of drug-treated cells by imaging mass cytometry

Frontiers of high-throughput metabolomics

A framework for large-scale metabolome drug profiling links coenzyme A metabolism to the toxicity of anti-cancer drug dichloroacetate

DRUG-seq for miniaturized high-throughput transcriptome profiling in drug discovery

Connectivity mapping: methods and applications

A next generation connectivity map: L1000 platform and the first 1,000,000 profiles

Discovering the anticancer potential of non-oncology drugs by systematic viability profiling

Cell Painting, a high-content imagebased assay for morphological profiling using multiplexed fluorescent dyes

Evaluation of gene expression and phenotypic profiling data as quantitative descriptors for predicting drug targets and mechanisms of action

Toward performance-diverse small-molecule libraries for cell-based phenotypic screening using multiplexed high-dimensional profiling

Primary cell-based phenotypic assays to pharmacologically and genetically study fibrotic diseases in vitro

A stem cell-based approach to cartilage repair

This study applies machine learning to microscopy images designed in the context of a specific screening campaign to inform on the activity of compounds in a broad spectrum of biological processes

This study proposes an approach to image-based lead optimization: a generative adversarial network is trained on small-molecule structures with and without matching image-based profiles to generate new small-molecule structures that mimic the morphological profiles induced by overexpression of 12 individual genes

Machine learning brings cell imaging promises into focus

Optical pooled screens in human

Apolar and polar transitions drive the conversion between amoeboid and mesenchymal shapes in melanoma cells

Automated 3D light-sheet screening with high spatiotemporal resolution reveals mitotic phenotypes

In silico labeling: predicting fluorescent labels in unlabeled images

Label-free prediction of three-dimensional fluorescence images from transmitted-light microscopy

Machine learning and computer vision approaches for phenotypic profiling

Data-analysis strategies for image-based cell profiling

Members of the CytoData Society across most laboratories in the field of image-based profiling come together to define best practices for informatics; remarkably, these are still relatively up to date

Advances in molecular labeling, high throughput imaging and machine intelligence portend powerful functional cellular biochemistry tools

This article arguably launched the field of imagebased profiling, as it presents the high-dimensional analysis of images of cells in the presence of chemical perturbations

High content screening applied to large-scale cell biology

Improving phenotypic measurements in high-content imaging screens

Visualizing data using t-SNE

UMAP: uniform manifold approximation and projection for dimension reduction

On lines and planes of closest fit to systems of points in space

Improving drug discovery with highcontent phenotypic screens by systematic selection of reporter cell lines

This study quantifies the improvement in the ability of image-based profiling using Cell Painting to distinguish

use of data from all 15 cell lines distinguished only about half of the MOAs

Machine learning and image-based profiling in drug discovery

Learning unsupervised feature representations for single cell microscopy images with paired cell inpainting

A brief introduction to weakly supervised learning

Weakly supervised learning of single-cell feature embeddings

Avoiding common pitfalls in machine learning omic data science

Three pitfalls to avoid in machine learning

High-throughput imaging for the discovery of cellular mechanisms of disease

Abnormalities in mitochondrial structure in cells from patients with bipolar disorder

Quantifying small molecule phenotypic effects using mitochondrial morphofunctional fingerprinting and machine learning

A patient-derived cellular model for Huntington's disease reveals phenotypes at clinically relevant CAG lengths

Evaluation of RNAi and CRISPR technologies by large-scale gene expression profiling in the Connectivity Map

Strategy for identifying repurposed drugs for the treatment of cerebral cavernous malformation

Discovering metabolic disease gene interactions by correlated effects on cellular morphology

Phenotypic landscape of schizophrenia-associated genes defines candidates and their shared functions

Systematic morphological profiling of human gene and allele function via cell painting

High-throughput phenotyping of lung cancer somatic mutations

Genetic and transcriptional evolution alters cancer cell line drug response

A synergistic anti-cancer FAK and HDAC inhibitor combination discovered by a novel chemical-genetic high-content phenotypic screen

An assessment of the translational relevance of Drosophila in drug discovery

Zebrafish avatars towards personalized medicine-a comparative review between avatar models

Fishing for cures: the alLURE of using zebrafish to develop precision oncology therapies

Treating Rett syndrome: from mouse models to human therapies

Chemical genetic discovery of targets and anti-targets for cancer polypharmacology

Zebrafish behavioral profiling links drugs to biological targets and rest/wake regulation

Zebrafish behavioral profiling identifies multitarget antipsychotic-like compounds

High-throughput screening for selective appetite modulators: a multibehavioral and translational drug discovery strategy

Zebrafish behavioural profiling identifies GABA and serotonin receptor ligands related to sedation and paradoxical excitation

Differentiation and visualization of diverse cellular phenotypic responses in primary highcontent screening

Image-based multivariate profiling of drug responses from single cells

A grand challenge. 2. Phenotypic profiling of a natural product library on Parkinson's patient-derived cells

High-content phenotypic profiling of drug response signatures across distinct cancer cells

High-content phenotypic profiling in esophageal adenocarcinoma identifies selectively active pharmacological classes of drugs for repurposing and chemical starting points for novel drug discovery

Real-time biological annotation of synthetic compounds

This study launched many further examples using Cell Painting interactively with synthetic chemistry for medicinal chemistry and SAR determination

Divergent synthesis and real-time biological annotation of optically active tetrahydrocyclopenta[c]pyranone derivatives

Synergistic effects of stereochemistry and appendages on the performance diversity of a collection of synthetic compounds

A scaffold-diversity synthesis of biologically intriguing cyclic sulfonamides

The ChEMBL database in 2017

A dataset of images and morphological profiles of 30,000 small-molecule treatments using the Cell Painting assay

Accurate prediction of biological assays with high-throughput microscopy images and convolutional networks

Failure of investigational drugs in late-stage clinical development and publication of trial results

The next generation blueprint of computational toxicology at the US Environmental Protection Agency

Utility of in vitro bioactivity as a lower bound estimate of in vivo adverse effect levels and in risk-based prioritization

Bioactivity screening of environmental chemicals using imaging-based high-throughput phenotypic profiling

This careful analysis describes the use of Cell Painting to report on various aspects of toxicity

Predicting cell health phenotypes using image-based morphology profiling

High-throughput imaging-based nephrotoxicity prediction for xenobiotics with diverse chemical structures

Building predictive in vitro pulmonary toxicity assays using high-throughput imaging and artificial intelligence

Rosmarinic acid as a candidate in a phenotypic profiling cardio-/cytotoxicity cell model induced by doxorubicin

High-content image-based single-cell phenotypic analysis for the testicular toxicity prediction induced by bisphenol A and its analogs bisphenol S, bisphenol AF, and tetrabromobisphenol A in a threedimensional testicular cell co-culture model

Morphological deconvolution of beta-lactam polyspecificity in E. coli

An unbiased cell morphology-based screen for new, biologically active small molecules

Diversity focused semisyntheses of tetronate polyether ionophores

Small molecule inhibitor of mitotic spindle bipolarity identified in a phenotype-based screen

Mechanism matters

Identifying off-target effects and hidden phenotypes of drugs in human cells

Target identification and mechanism of action in chemical biology and drug discovery

Target deconvolution from phenotype-based drug discovery by using chemical proteomics approaches

Large-scale cytological profiling for functional analysis of bioactive compounds

Characterizing heterogeneous cellular responses to perturbations

Comparison of methods for imagebased profiling of cellular morphological responses to small-molecule treatment

Multiplex cytological profiling assay to measure diverse cellular States

Profiling of the effects of antifungal agents on yeast cells based on morphometric analysis

Morphobase, an encyclopedic cell morphology database, and its use for drug target identification

Linking phenotypes and modes of action through high-content screen fingerprints

Image-based morphological profiling identifies a lysosomotropic, iron-sequestering autophagy inhibitor

Structure and function of the bacterial protein toxin phenomycin

Design, synthesis, and phenotypic profiling of pyrano-furo-pyridone pseudo natural products

Rapid inhibition profiling identifies a keystone target in the nucleotide biosynthesis pathway

Function-first' lead discovery: mode of action profiling of natural product libraries using image-based screening

Phenotype-guided natural products discovery using cytological profiling

Bioactive potential of marine macroalgae from the central red sea (Saudi Arabia) assessed by high-throughput imaging-based phenotypic profiling

Domain-invariant features for mechanism of action prediction in a multi-cell-line drug screen

High content phenotypic screening identifies serotonin receptor modulators with selective activity upon breast cancer cell cycle and cytokine signaling pathways

High-throughput functional annotation of natural products by integrated activity profiling

Integration of high-content screening and untargeted metabolomics for comprehensive functional annotation of natural product libraries

Systematic exploration of cell morphological phenotypes associated with a transcriptomic query

A chemical-genetic interaction map of small molecules using high-throughput imaging in cancer cells

This is a pioneering study in yeast demonstrating successful identification of small-molecule mimics of gene signatures (and vice versa for target identification) using image-based data

Plant-derived antifungal agent poacic acid targets β-1,3-glucan

Vanillin inhibits translation and induces messenger ribonucleoprotein (mRNP) granule formation in saccharomyces cerevisiae: application and validation of high-content, image-based profiling

Integration of chemical and RNAi multiparametric profiles identifies triggers of intracellular mycobacterial killing

Deducing the mechanism of action of compounds identified in phenotypic screens by integrating their multiparametric profiles with a reference genetic screen

Parallel chemical genetic and genome-wide RNAi screens identify cytokinesis inhibitors and targets

Morphological profiles of RNAi-induced gene knockdown are highly reproducible but dominated by seed effects

A programmable dual-RNA-guided DNA endonuclease in adaptive bacterial immunity

Expression profiling reveals off-target gene regulation by RNAi

Integrating high-content screening and ligand-target prediction to identify mechanism of action

Mapping the perturbome network of cellular perturbations

A map of directional genetic interactions in a metazoan cell

A screen for morphological complexity identifies regulators of switch-like transitions between discrete cell shapes

Systematic genetics and single-cell imaging reveal widespread morphological pleiotropy and cell-to-cell variability

Yeast proteome dynamics from single cell imaging and automated analysis

Phenotypic profiling of the human genome by time-lapse microscopy reveals cell division genes

An algorithm-based topographical biomaterials library to instruct cell fate

Image-derived models of cell organization changes during differentiation of PC12 cells

Leveraging machine vision in cell-based diagnostics to do more with less

Clinically applicable deep learning for diagnosis and referral in retinal disease

An image informatics pipeline for imaging mass cytometry to characterize the immune landscape in pre-and on-treatment immune therapy and its application in recurrent platinium-resistant epithelial ovarian cancer

Predicting cancer outcomes from histology and genomics using convolutional networks

The single-cell pathology landscape of breast cancer

Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning

Joint analysis of gene expression levels and histological images identifies genes associated with tissue morphology

Bacterial cytological profiling (BCP) as a rapid and accurate antimicrobial susceptibility testing method for staphylococcus aureus

A functional CFTR assay using primary cystic fibrosis intestinal organoids

Image-based ex-vivo drug screening for patients with aggressive haematological malignancies: interim results from a single-arm, open-label, pilot study

The Tumor Profiler Study: integrated, multi-omic, functional tumor profiling for clinical decision support

Multiparametric phenotyping of compound effects on patient derived organoids

The rise of deep learning in drug discovery

Computer vision for high content screening

Deep learning for cellular image analysis

From observing to predicting single-cell structure and function with highthroughput/high-content microscopy

GRUU-Net: integrated convolutional and gated recurrent neural network for cell segmentation

Nucleus segmentation across imaging experiments: the 2018 Data Science Bowl

Phenotypic profiling of high throughput imaging screens with generic deep convolutional features

CellProfiler 3.0: next-generation image processing for biology

Reconstructing cell cycle and disease progression using deep learning

Transfer learning with deep convolutional neural networks for classifying cellular morphological changes

Capturing single-cell heterogeneity via data fusion improves image-based profiling

Evaluation of connectivity map shows limited reproducibility in drug repositioning

A call for public archives for biological image data

The contribution of mechanistic understanding to phenotypic screening for first-in-class medicines

The discovery of first-in-class drugs: origins and evolution

Next-generation phenotypic screening

How were new medicines discovered?

Developing predictive assays: the phenotypic screening 'rule of 3

macrophage cytological profiling and anti-inflammatory drug discovery

Highly multiplexed immunofluorescence imaging of human tissues and tumors using t-CyCIF and conventional optical microscopes

Highly multiplexed single-cell analysis of formalin-fixed, paraffin-embedded cancer tissue

Multiplex staining by sequential immunostaining and antibody removal on routine tissue sections

SIMPLE: a sequential immunoperoxidase labeling and erasing method

Mass cytometry imaging for the study of human diseases-applications and data analysis strategies

Spatial single-cell profiling of intracellular metabolomes in situ

Fluopack screening platform for unbiased cellular phenotype profiling

A phenotypic caenorhabditis elegans screen identifies a selective suppressor of antipsychotic-induced hyperphagia

Measuring drug absorption improves interpretation of behavioral responses in a larval zebrafish locomotor assay for predicting seizure liability

Combining morphological and migration profiles of in vitro time-lapse data

Batch equalization with a generative adversarial network

Computational polypharmacology comes of age

Polypharmacology by design: a medicinal chemist's perspective on multitargeting compounds

The authors appreciate helpful comments from S. Jaensch, S. Singh, J. Caicedo, N. Rindtorff and all members of the Carpenter laboratory. The authors acknowledge funding support for S.N.C. and A.E.C. from the US National Institutes of Health (R35 GM122547 to A.E.C.).

The authors contributed equally to all aspects of the article.

H.C. and J.D.B. are employed by Janssen and Pfizer, respectively. A.E.C. is on the Scientific and Technical Advisory Board of, has optional ownership interest in and receives income from Recursion. S.N.C. declares no conflict of interest.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.