key: cord-0873830-muz2dy18
authors: Konovalov, Kirill A.; Unarta, Ilona Christy; Cao, Siqin; Goonetilleke, Eshani C.; Huang, Xuhui
title: Markov State Models to Study the Functional Dynamics of Proteins in the Wake of Machine Learning
date: 2021-08-04
journal: JACS Au
DOI: 10.1021/jacsau.1c00254
sha: 356ff9103e41cd6f42b4c34e4f55f50a64c21d31
doc_id: 873830
cord_uid: muz2dy18

[Image: see text] Markov state models (MSMs) based on molecular dynamics (MD) simulations are routinely employed to study protein folding, however, their application to functional conformational changes of biomolecules is still limited. In the past few years, the field of computational chemistry has experienced a surge of advancements stemming from machine learning algorithms, and MSMs have not been left out. Unlike global processes, such as protein folding, the application of MSMs to functional conformational changes is challenging because they mostly consist of localized structural transitions. Therefore, it is critical to properly select a subset of structural features that can describe the slowest dynamics of these functional conformational changes. To address this challenge, we recommend several automatic feature selection methods such as Spectral-OASIS. To identify states in MSMs, the chosen features can be subject to dimensionality reduction methods such as TICA or deep learning based VAMPNets to project MD conformations onto a few collective variables for subsequent clustering. Another challenge for the application of MSMs to the study of functional conformational changes is the ability to comprehend their biophysical mechanisms, as MSMs built for these processes often require a large number of states. We recommend the recently developed quasi-MSMs (qMSMs) to address this issue. Compared to MSMs, qMSMs encode the non-Markovian dynamics via the generalized master equation and can significantly reduce the number of states. As a result, qMSMs can be built with a handful of states to facilitate the interpretation of functional conformational changes. In the wake of machine learning, we believe that the rapid advancement in the MSM methodology will lead to their wider application in studying functional conformational changes of biomolecules.

Biological macromolecules often exert their functions through conformational changes: 1−3 i.e., dynamic transitions between metastable conformational states. For example, the SARS-CoV-2 spike protein complex undergoes dramatic opening during recognition of the human ACE-2 receptor, 4 RNA polymerases continuously translocate on the DNA template during gene transcription, 5 and Src kinases' activation-loop needs to open to make their active site accessible. 6 In this Perspective, we distinguish between these functional conformational changes and global conformational changes. As functional conformational changes mostly involve slow, often hierarchical, collective transitions of protein loops and specific domains, 7 it is often sufficient to describe the functionally relevant motions using only a subset of structural features (e.g., certain residue−residue distances, torsion angles, etc.). This is in sharp contrast to conformational changes, such as complete protein folding, in which the whole structure undergoes drastic changes involving a complete set of structural features. 8−10 Delineating mechanisms of functional change is crucial to our understanding of numerous fundamental biological processes and facilitating rational drug design.

Functional conformational changes can be studied in fine detail by all-atom molecular dynamics (MD) simulations. However, the time scales accessible to MD simulations of complex biomolecules (at microseconds or shorter) remain orders of magnitude shorter than those of functional conformational changes (millisecond or longer). In recent years, Markov state models (MSMs) have become a popular approach to bridge this time scale gap by predicting long-time scale dynamics based on numerous short MD simulations. 11 −32 MSMs have been widely applied to study global conformational changes, such as the folding of small proteins (e.g., NTL9 33 and FiP35 WW domain 34 ) and the dynamics of intrinsically disordered peptides (e.g., hIAPP 35 ). In these studies, the entire structure is used to describe these global conformational changes 11, 22 (e.g., pairwise distances between all C α atoms). This is not the case for complex and localized functional conformational changes of large biomolecular complexes, where it is often difficult to precisely pinpoint parts of the system relevant to function, and even more difficult to choose an appropriate set of structural features to describe them. 36, 37 In early MSM studies, researchers mainly chose structural features based on their a priori physical understanding of the system (e.g., distances between ligand and binding pockets for protein−ligand recognition 24 or DNA/ RNA and their surrounding protein motifs for RNA polymerase translocation 5 ). This renders the construction of MSMs to study functional conformational changes time-consuming and challenging. In the past several years, novel machine learning algorithms, especially deep neural networks, have been introduced to the MSM community, 38−42 promising to aid MSM construction for such complex problems.

In this Perspective, we first briefly review the MSM theory and highlight two major challenges specific to MSMs of functional mechanisms of large biomolecular complexes (section 2). We then introduce a state-of-the-art protocol for the MSM construction to study functional conformational changes together with a few examples of its recent application (section 3). Next, we discuss in detail several recently developed machine learning algorithms in our recommended protocol to address these two challenges (section 4): Algorithms for the identification of proper structural features and collective variables (CVs) to describe localized functional conformational changes of interest (e.g., Spectral-oASIS, 39 feature importance selection, 40 variational approach to Markov process neural network (VAMPNets), 41 and state-free reversible VAMPNets (SRVs) 42 ); and methods to produce models containing a handful of states to facilitate the interpretation of biological mechanisms (e.g., quasi-MSM (qMSM) based on the Generalized Master Equation (GME) framework 43 ) (section 5). We hope that this Perspective will encourage researchers to apply MSMs to study challenging problems related to biomolecular functional conformational changes and other dynamic systems.

THEIR APPLICATION TO FUNCTIONAL CONFORMATIONAL CHANGES MSMs are a powerful tool that can combine disparate short MD simulations at local equilibrium to model long-time scale dynamics of complex conformational changes. Specifically, MSMs partition the conformational space into metastable states, such that intrastate transitions are fast but interstate transitions are slow. This separation of time scales ensures an MSM is Markovian (i.e., that the probability of transitioning from state i to state j depends only on the identity of i and not any previously visited state) and allows MSMs to be built from many short simulations. These probabilities can then be propagated to give long-time scale dynamics:

where Δt corresponds to the lag time, P(nΔt) is a vector of state populations at time nΔt, and T is the transition probability matrix.

One of the key challenges in MSM construction is correctly identifying kinetically metastable states, which requires selecting a protein's structural features that can properly describe the slowest dynamics of conformational changes. With these chosen structural features, dimensionality reduction methods can be applied to obtain CVs, and then clustering algorithms can be used to group MD conformations into metastable states. However, it is not trivial to identify proper structural features that can describe the localized, but often complex, conformational changes underlining the function. For example, RNA polymerase II (Pol II) will translocate backward (backtrack) on the DNA template to allow the cleavage of the misincorporated nucleotide, which is a critical step to maintain accurate gene transcription. Exhaustive featurization of this system is infeasible due to its large size (e.g., the Pol II complex contains ∼3600 residues, and the consideration of distances between all the C α atoms will lead to nearly 13 million features). Furthermore, noise due to thermal fluctuations, especially from parts of the system that do not participate in backtracking, could compromise the quality of the MSM. In early studies, features were often selected manually based on researchers' prior knowledge of the system. For example, in the MSM studies of Pol II backtracking, 44 distances between atom pairs (695 interatomic distances) that are sensitive to the backtracking of Pol II were chosen based on physical intuition, which contain backtracked RNA and DNA nucleotides, critical bridge helix residues, and two Tyr residues which are known to stabilize the nucleotide bases during backtracking. With recently developed machine learning methods, automatic selection of features becomes feasible, and we recommend a few such methods in section 4.

Another challenge for MSMs lies in the comprehension of biophysical mechanisms of functional conformational changes, as MSMs built for these processes often contain hundreds or even more states. 5,23,45−49 In an MSM, the lag time must be long enough to allow transitions among states to become Markovian (or memoryless), and the memory of these transitions is mainly determined by dynamic relaxation within each state. In practice, this is challenging as the lag time is bound by the length of MD simulations available to estimate transition probabilities (T). To render the models Markovian, successful application of MSMs for functional conformational changes often contain at least hundreds of states, so that each state is sufficiently small and has relatively fast relaxation dynamics to allow affordable lag times. To address this challenge, we recommend the recently developed qMSM, 43 which can accurately predict dynamics from models containing a small number of states by explicitly considering the memory of protein dynamics (see section 5).

TO STUDY FUNCTIONAL CONFORMATIONAL CHANGES Figure 1 summarizes our recommended protocol for constructing MSMs to study how biomolecules dynamically transition between metastable states to perform their functions. In this protocol, the initial paths connecting known states (e.g., structures obtained from X-ray crystallography or cryo-EM) are first generated via approaches such as targeted MD, 50 Onsager−Machlup action-based conformational state annealing (Action-CSA), 51 Climber, 52 or coarse-grained MD simulations 53, 54 and are further optimized using the String method 55 or traveling-salesman-based automatic path searching (TAPS) 56 ( Figure 1A ). Extensive MD simulations are then initiated from conformations along these optimized initial pathways ( Figure 1B ). Next, structural features (e.g., interatomic distances, torsion angles, etc.) that can describe functional conformational changes are selected ( Figure 1C ). Here, we recommend Spectral-oASIS, 39 feature importance selection, 40 or automatic mutual information noise omission (AMINO) 57 to automatically select a proper set of features. As shown in Figure 1D , dimensionality reduction algorithms (e.g., time-lagged independent component analysis (TICA), 58 VAMPNets, 41 or SRVs 42 ) can then be applied to find a few CVs. MD conformations projected onto these CVs are then grouped into microstates using various clustering algorithms. 59−61 The microstate-MSM is then built and validated using the Chapman−Kolmogorov test 13, 16 ( Figure 1E ). The Chapman−Kolmogorov test can be performed by directly examining if eq 1 is satisfied: 16, 62 i.e., if time evolutions of state populations (P(nΔt)) obtained from MD simulations agree with the prediction of an MSM via the replication of the transition probability matrix ([T(Δt)] n P(0)). Another implementation of the Chapman−Kolmogorov test is to compare the probabilities for the system to stay in a given state between the predictions of MSMs and those obtained from MD simulations. 13 The cross-validation tools are recommended to avoid overfitting and to select optimal parameters from the previous steps (e.g., feature sets, number of CVs, and number of microstates). With cross-validation, the model is constructed on part of the original data and then tested on the remaining data. Models built with various parameters can be scored with objective metrics such as the generalized matrix Rayleigh quotient (GMRQ) 63 or the VAMP-2 29 score, allowing the selection of optimal parameters. Grounded on the variational principle for conformational dynamics, both GMRQ and VAMP-2 scores favor the models that yield slower dynamics. In particular, GMRQ 63 can be computed from the eigenvalues of the transition probability matrix, while VAMP-2 scores can be obtained from the time-lagged covariance matrix of input features. 29 If the conformational sampling is not sufficient to build a Markovian microstate-MSM, we suggest performing adaptive sampling 64−66 and repeating the previous steps ( Figure 1B −E) until the model is valid. In an adaptive sampling strategy developed by Bowman and co-workers, 67 additional sampling is initiated from conformations selected based on a function (e.g., the solvent accessible surface area of the solute) that balances exploration and exploitation of the previously sampled states.

Finally, the microstates can be lumped into a few metastable macrostates by grouping those microstates that can interconvert quickly. This step can be achieved via kinetic lumping algorithms, 70−76 and the resulting macrostate MSM can greatly aid the interpretation of biological mechanisms ( Figure 1F ). It is challenging to build a Markovian macrostate-MSM since the lag time cannot exceed the length of the MD trajectories. Therefore, we recommend using qMSMs 43 that encode non-Markovian dynamics via the GME formalism to build these macrostate models. In recent years, MSMs have been successfully applied to study various protein functional conformational changes. 5,45−49,77−80 For example, Da and co-workers constructed MSMs that revealed that thymine DNA glycosylases translocate along double-stranded DNA via a rotation-coupled sliding model in order to detect DNA lesions. 68 To build their MSMs, they followed the protocol in Figure 1 but chose the structural features based on physical intuition. Their MSM identified two parallel pathways over nine macrostates, where state 5 (S5) is the specific interrogating complex with a mismatched base pair (Figure 2A ). In another study, Shukla and co-workers applied MSMs to reveal a rocker switch mechanism in a substrate exchange cycle of a membrane transport protein, the bacterial NO 3 − /NO 2 − antiporter NarK. 69 From the MSM-weighted free energy landscape, a series of important conformations during the substrate exchange cycle were identified ( Figure 2B ). Based on the MSM, they discovered that the exchange of NO 3 − and NO 2 − is ensured by the closure of space between two arginine residues in the binding site of the antiporter. More recently, Bowman and co-workers 4 have constructed an MSM from over 1 ms of MD simulations to describe the opening of the SARS-CoV-2 spike protein complex, and reveal cryptic pockets during this process as potential drug targets.

As discussed in section 2, it is challenging to efficiently select a subset of protein's structural features that describe localized functional conformational changes. For this purpose, internal coordinates such as distances, contacts, and dihedral angles are generally superior to Cartesian coordinates (being independent of the overall translation and rotation of the system). 81 Properly selected structural features serve as the input for dimensionality-reduction methods, and MD conformations can then be clustered into metastable states at reduced dimensions ( Figure 1C−E) . In this section, we introduce a few recently developed methods that could help achieve automatic feature selection and dimensionality reduction for the construction of MSMs to study functional conformational changes. 

Spectral-oASIS is particularly useful for automatically selecting features for MSM construction. 39 This method is based on the Nystrom matrix operation theory, which can approximately reconstruct the time-lagged covariance matrix of all input features while using only a subset of features as input. Given an initial input feature set, Spectral-oASIS samples a subset of these features that best reconstructs the leading eigenfunctions of the time-lagged covariance matrix obtained from MD simulations, yielding a sparse solution to the generalized eigenvalue problem ( Figure 3A ). An optimal subset of features can then be selected based on the variational principle, i.e., the ability of the reconstructed matrix to reproduce the slowest time scales of the original matrix ( Figure 3B ). Using a benzamidine-trypsin binding process as an example, Clementi and co-workers 39 demonstrated that an initial feature set of approximately 25 000 features can be reduced 5-fold while still accurately describing the slowest dynamic mode, which corresponds to the flipping of Trp215 to open the active site ( Figure 3C) . Notably, Sparse-TICA 82 is similar to Spectral-oASIS in the sense that they both aim to find a subset of input features that can best approximate leading eigenfunctions of the time-lagged covariance matrix; however, Sparse-TICA uses a regularization approach rather than the Nystrom reconstruction adopted in Spectral-oASIS. 39 Sparse-TICA has also been successfully applied to study a functional conformational change of an opioid receptor, where 10 out of 4,400 features were chosen to build the MSMs. 84 Stock and co-workers 40 developed an alternative method (we refer to it as "feature importance selection") to automatically select essential features by ranking their importance in the ability to explain the labeling of the dynamics (e.g., index of metastable states). This method is based on training decision trees and only requires an input feature and the labeling of MD conformations ( Figure 3D ). The set of essential features can then be constructed by iteratively extracting the most important feature in the tree ( Figure 3E ). They demonstrated that their chosen essential features can well explain the functional dynamics of T4 lysozyme ( Figure 3F ). This approach has also been applied to select features prior to MSM construction in a study of ancestral mutations that activate the extracellular signalregulated kinase (ERK2), 85 in which they successfully identify the most informative features (inter-residue contacts) that can distinguish the mutant from the WT protein. AMINO is another method that holds the potential to select nonredundant features for functional conformational changes, 57 even though it has yet to be applied in the MSM construction. By clustering the features using a mutual information-based metric, Tiwary and co-workers demonstrated that AMINO can achieve a significant reduction in features to describe a protein−ligand binding process: i.e., a set of 428 features containing all possible distance between protein C α atoms and the ligand was reduced to just 8, allowing accurate computation of ligand binding free energy. 57 

TICA is one of the most popular methods to perform dimensionality reduction in the MSM construction, which performs the eigen decomposition of the time-lagged covariance matrix. 58, 86 The leading eigenvectors (so-called time-lagged independent components, TICs) are linear approximations to the slowest dynamic modes of the system. When applying TICA to study functional conformational changes, we recommend using the subset of structural features chosen by Spectral-oASIS and other methods described in the previous section. 87 Furthermore, we suggest using crossvalidation tools, such as GMRQ 63 or VAMP-2 score, 28 to choose the optimal hyperparameters for the TICA analysis (e.g., number of TICs and TICA lag time). 68, 69, 88 Emerging deep Learning Algorithms for Feature Selection and Dimensionality Reduction VAMPNets developed by Noéand co-workers are among the first deep learning architectures for MSM construction. 41 VAMPNets adopt two encoder networks in parallel together with a specific loss function (i.e., the VAMP-2 29 score) based on the variational principle of the conformational dynamics. As shown in Figure 4A , the VAMP-2 score (R 2 ) is computed based on the output of the encoder lobes: R 2 = ∥C 00 −1/2 C 01 C 11

where C 00 and C 11 are the covariance matrices of the functions output by each of the two encoder lobes and C 01 is the cross-covariance between lobes (i.e., timelagged covariance). The general implementation of VAMPNets is not restricted to equilibrium data and thus does not enforce the detailed balance. To facilitate its application to equilibrium sampling, Ferguson and co-workers 42 designed a variation of VAMPNets, so-called state-free reversible VAMPNets or SRV, enforcing the detailed balance by transforming the time-lagged covariance matrices into symmetric matrices. More recently, the Wu and Noégroups developed a version of VAMPNets by imposing the reversibility by introducing additional constraint variables. 89 These VAMPNets-based deep learning algorithms can be used for dimensionality reduction to output a few CVs for subsequent MSM construction. Indeed, SRV has been successfully applied to construct MSMs to study the folding of the Trp-cage protein, where they chose all C α −C α distances ( Figure 4B ) as input, and output seven CVs to perform clustering to group MD conformations into 100 states. 83 Compared to TICA with the same input features, SRV is able to identify an additional slow dynamic mode. Specifically, MSMs built from top CVs obtained from SRV successfully identified a dynamic mode that corresponds to the transition from a molten globule to an α-helix-like state with proline residues facing outward (denoted as a trapped intermediate state that precludes folding 83 ), while MSMs built from top TICs failed to capture this dynamic mode (see the middle panel of Figure 4C ). Furthermore, SRV was shown to be more robust than TICA for dimensionality reduction in the crossvalidation test ( Figure 4D ). In theory, when applied to study functional conformational changes, these VAMPNets-based methods could achieve the goal of simultaneously selecting input features (interatomic distances, dihedral angles, etc.) and identifying their proper combinations to form CVs through the optimization of numerous parameters and their nonlinear combinations in the deep neural networks. However, considering the large number of input features and the localized nature of functional conformational changes, we anticipate that it will not be a trivial task for VAMPNets-based methods to achieve the above-mentioned goal. Therefore, we still suggest preselecting features when applying VAMPNets to study functional conformational changes.

As discussed in section 2, MSMs of protein dynamics with a small number of states often suffer from non-Markovianity due to the limited length of lag time, which is bound by relatively short MD simulations. To address this challenge, we have developed the qMSM method based on the GME formalism, 43 in which memory kernels of protein dynamics are explicitly calculated and the dynamics are propagated with a discretized GME (eq 2) where memory kernels (K(mΔt)) can be obtained iteratively from the transition probability matrix T(t) and their derivatives T(t) at time points t = 0, Δt,... nΔt (Δt is the saving interval of MD trajectories) as well as all K(t) at previous time points. τ K = n K Δt corresponds to the time until the memory kernels are relaxed to zero. qMSMs and MSMs adopt the same state decomposition. However, rather than using the transition probability matrix as in an MSM, qMSM models the dynamics using the transition tensors: K(t) (i.e., each transition element is associated with a memory kernel curve; see Figure 5A and B for memory kernels of a simple three-state model as an example).

For the folding of a small protein (the Fip35 WW domain), we show that qMSMs (consisting of four states) can be built from MD simulations that are an order of magnitude shorter than those required by an MSM. 43 We expect that this advantage will be more prominent for the studies of functional conformational changes of more complex biomolecular systems. Recently, qMSMs have been successfully applied to elucidate the dynamics of a large functional conformational change of the bacterial RNA Polymerase (RNAP) transcription complex: i.e., the opening of the RNAP clamp. 90 Bacterial RNAP has a shape that resembles a crab claw with two pincers: clamp and β-lobe (see yellow and magenta regions, respectively in Figure 5C ). The opening and closing of the clamp are crucial for the initiation of bacterial gene transcription, and inhibition of the RNAP clamp opening provides a promising target for the development of antibiotics (e.g., Myxopyronin). Using qMSMs, we identified two intermediate states during the clamp opening, and our fourstate qMSM predicts that the clamp opening process occurs at millisecond time scales ( Figure 5C ). For this system, qMSM greatly outperforms MSMs. For example, qMSMs with τ k = 30 ns can already reproduce the dynamics of the original MD simulations, while MSMs predict significantly faster dynamics than MD simulations ( Figure 5D ). Consistently, MSM (τ = 30 ns) predicts around 6-fold shorter mean first passage times (MFPTs) than qMSM (τ k = 30 ns, Figure 5E ). Therefore, qMSMs have substantial advantages over MSMs in interpreting biological mechanisms by yielding models with a handful of states.

Notably, Tiwary and co-workers recently developed another algorithm based on the long short-term memory (LSTM) model to consider the memory functions of protein conformational dynamics. 91 This approach is based on a recurrent network architecture that can retain the memory of the past states in a temporal sequence via gating nodes that capture lags between long-time scale events. In this deep learning approach, Tiwary and co-workers ingeniously connect the loss function with the path entropy and show that the LSTM method can accurately predict equilibrium distributions and kinetics for an alanine dipeptide and experimental single-molecular FRET data. As the recurrent neural network approach was originally developed for one-dimensional natural language processing, we expect that this LSTM approach alone may perform optimally on one-dimensional data. Nevertheless, the LSTM architecture can be incorporated into a larger framework to perform complex multidimensional tasks. For example, LSTM lies at the core of AlphaStar, 92 which processes complex inputs combined with other network architectures (e.g., transformer, 93 ResNet, 94 etc.). We believe that the work of Tiwary and co-workers 91 has great potential to be extended to handle the multidimensional MD trajectories of functional conformational changes in the future.

In this Perspective, we focused on the application of MSMs to study functional conformational changes of complex biomolecules. We introduced a state-of-the-art protocol that is tailor-made for localized functional conformational changes (see Figure 1 for the summary of the protocol). In this protocol, we highlight two challenges and recommend a series of recently developed machine learning algorithms to address them. For the first challenge, which consists of properly identifying a subset of structural features that describe the slow dynamics of the functional conformational changes, we recommend several automatic feature selection methods including Spectral-OASIS, 39 feature importance selection, 40 and AMINO. 57 The chosen features can then be subject to dimensionality reduction methods such as TICA 58 or deep learning based VAMPNets 41 or SRVs 42 to obtain CVs for subsequent microstate clustering. For the second challenge, which consists of improving the interpretation of the biophysical mechanisms, we recommend qMSMs that can produce models containing a handful of states. 43 In addition to the above two challenges, which are more specific to functional conformational changes, we note that other difficulties exist for building MSMs to study conformational dynamics. For example, the choice of clustering algorithms and distance metrics are important for the quality of MSM construction, and those issues have been extensively reviewed elsewhere. 95−97 Most of the algorithms that we recommended in this Perspective for feature selection and dimensionality reduction are based on the variational principle of the conformational dynamics, 29 in which the best models should theoretically yield the slowest time scales due to the variational bound. However, in practice, the slowest dynamic modes identified by these algorithms could correspond to irrelevant processes. For example, Husic and Noé9 8 demonstrated that the slowest dynamic mode for the folding of the Villin headpiece obtained based on the VAMP-2 score corresponds to a transition to a rare helical misfolded state, which was further examined manually by the authors and asserted to be an artifact. 98 Therefore, we believe that it remains important to evaluate and confirm the relevance of the slowest dynamic modes obtained from these automatic algorithms. In addition, VAMPNets and other deep learning algorithms could theoretically be applied to perform feature selection and dimensionality reduction at the same time. However, we expect that it will be difficult for these algorithms to achieve these two aims simultaneously when studying the localized, but often complex, functional conformational changes. We thus recommend performing feature selection first (e.g., using Spectral-oASIS 39 ) and inputting only the selected features to these deep learning algorithms.

We demonstrated that the GME-based methods, such as qMSMs, hold great promise for studying functional conformational changes, as they can be built from affordable lengths of MD simulations while only containing a few states to facilitate the understanding of biological mechanisms. In addition to qMSMs, we expect that two previously developed methods, hidden Markov models 99,100 and core-set MSMs, 101,102 could serve as alternative approaches to efficiently generate MSMs with a small number of states. Nevertheless, the hidden JACS Au pubs.acs.org/jacsau Perspective Markov model adopts a soft partitioning scheme that allows overlaps between metastable states, and this could lead to ambiguity when interpreting the biological mechanisms. In addition, the core-set MSM only focuses on the core regions of each metastable state instead of a full partitioning of the conformational space. Even though it is not trivial to correctly identify these core regions, several recent algorithms have been developed to circumvent this issue. 102 Despite all these methodological advancements to automatically construct MSMs, we are also wary of the pitfalls of blind applications of these machine learning algorithms and believe that physical intuition remains invaluable. Nevertheless, we are optimistic that MSMs will be widely applied to elucidate functional conformational changes in the future. 

Dynamic personalities of proteins

Global Dynamics of Proteins: Bridging Between Structure and Function

Protein Ensembles: How Does Nature Harness Thermodynamic Fluctuations for Life? The Diverse Functional Roles of Conformational Ensembles in the Cell

SARS-CoV-2 simulations go exascale to predict dramatic spike opening and cryptic pockets across the proteome

Millisecond dynamics of RNA polymerase II translocation at atomic resolution

Src kinase conformational activation: Thermodynamics, pathways, and mechanisms

Hierarchical Biomolecular Dynamics: Picosecond Hydrogen Bonding Regulates Microsecond Conformational Transitions

Computer Simulations of Intrinsically Disordered Proteins

Single-molecule spectroscopy of protein folding dynamics-expanding scope and timescales

The nature of protein folding pathways

Markov state models of biomolecular conformational dynamics

State Models: From an Art to a Science

Markov models of molecular kinetics: Generation and validation

Application of Molecular-Dynamics Based Markov State Models to Functional Proteins

An Introduction to Markov State Models and Their Application to Long Timescale Molecular Simulation

Automatic discovery of metastable states for the construction of Markov models of macromolecular conformational dynamics

Building Markov state models along pathways to determine free energies and rates of transitions

Modeling Conformational Ensembles of Slow Functional Motions in Pin1-WW

Rapid equilibrium sampling initiated from nonequilibrium data

Coarse master equations for peptide folding dynamics

Constructing the equilibrium ensemble of folding pathways from short off-equilibrium simulations

Taming the complexity of protein folding

Complete reconstruction of an enzyme-inhibitor binding process by molecular dynamics simulations

A Role for Both Conformational Selection and Induced Fit in Ligand Binding by the LAO Protein

Hierarchical analysis of conformational dynamics in biomolecules: Transition networks of metastable states

On the Approximation Quality of Markov State Models

A Variational Approach to Modeling Slow Processes in Stochastic Dynamical Systems

Variational Approach for Learning Markov Processes from Time Series Data

Revealing Thermodynamics and Kinetics of Lipid Self-Assembly by Markov State Model Analysis

Harnessing complexity in molecular self-assembly using computer simulations

Simulating Replica Exchange: Markov State Models, Proposal Schemes, and the Infinite Swapping Limit

Molecular Simulation of ab Initio Protein Folding for a Millisecond Folder NTL9(1−39)

Markov State Model Reveals Folding and Functional Dynamics in Ultra-Long MD Trajectories

Dynamics of an Intrinsically Disordered Protein Reveal Metastable Conformations That Potentially Seed Aggregation

Constructing Markov State Models to elucidate the functional conformational changes of complex biomolecules

Elucidating molecular mechanisms of functional conformational changes of proteins via Markov state models

Machine Learning for Molecular Simulation

Rapid Calculation of Molecular Kinetics Using Compressed Sensing

Machine Learning of Biomolecular Reaction Coordinates

VAMPnets for deep learning of molecular kinetics

Nonlinear discovery of slow molecular modes using state-free reversible VAMPnets

On the advantages of exploiting memory in Markov state models for biomolecular dynamics

Bridge helix bending promotes RNA polymerase II backtracking through a critical and conserved threonine residue

A network of molecular switches controls the activation of the twocomponent response regulator NtrC

Markov State Models Reveal a Two-Step Mechanism of miRNA Loading into the Human Argonaute Protein: Selective Binding followed by Structural Re-arrangement

Dynamics of Pyrophosphate Ion Release and Its Coupled Trigger Loop Motion from Closed to Open State in RNA Polymerase II

Molecular-Dynamics -a New Approach for Searching Pathways of Conformational Transitions

Finding multiple reaction pathways via global optimization of action

Morphing Methods Predict Intermediate Structures?

Multiple-basin energy landscapes for large-amplitude conformational motions of proteins: Structure-based molecular dynamics simulations

Modeling Structural Dynamics of Biomolecular Complexes by Coarse-Grained Molecular Simulations

Finding transition pathways using the string method with swarms of trajectories

TAPS: A traveling-salesman based automated path searching method for functional conformational changes of biological macromolecules

Automatic mutual information noise omission (AMINO): generating order parameters for molecular systems

Identification of slow molecular order parameters for Markov model construction

Least-Squares Quantization in Pcm

Clustering to Minimize the Maximum Intercluster Distance

Adaptive Partitioning by Local Density-Peaks: An Efficient Density-Based Clustering Algorithm for Analyzing Molecular Dynamics Trajectories

Constructing multi-resolution Markov State Models (MSMs) to elucidate RNA hairpin folding mechanisms

Variational cross-validation of slow dynamical modes in molecular kinetics

Choice of Adaptive Sampling Strategy Impacts State Discovery, Transition Probabilities, and the Apparent Mechanism of Conformational Changes

HTMD: High-Throughput Molecular Dynamics for Molecular Discovery

Surprisal Metrics for Quantifying Perturbed Conformational Dynamics in Markov State Models

FAST Conformational Searches by Balancing Exploration/Exploitation Trade-Offs

Atomic resolution of shortrange sliding dynamics of thymine DNA glycosylase along DNA minor-groove for lesion recognition

How do antiporters exchange substrates across the cell membrane? An atomic-level description of the complete exchange cycle in NarK

Identification of almost invariant aggregates in reversible nearly uncoupled Markov chains

Fuzzy spectral clustering by PCCA plus: application to Markov state models and data classification

Quantitative comparison of alternative methods for coarse-graining biological networks

An efficient Bayesian kinetic lumping algorithm to identify metastable conformational states via Gibbs sampling

Identifying Metastable States of Folding Proteins

Hierarchical Nystrom methods for constructing Markov state models for conformational dynamics

Variational Identification of Markovian Transition States

Complete protein-protein association kinetics in atomic detail revealed by molecular dynamics simulations and Markov modelling

Elucidation of the Dynamics of Transcription Elongation by RNA Polymerase II using Kinetic Network Models

Cavity hydration dynamics in cytochrome c oxidase and functional implications

Base-flipping dynamics from an intrahelical to an extrahelical state exerted by thymine DNA glycosylase during DNA repair process

Principal component analysis of molecular dynamics: On the use of Cartesian vs. internal coordinates

Identification of simple reaction coordinates from complex dynamics

High-Resolution Markov State Models for the Dynamics of Trp-Cage Miniprotein Constructed Over Slow Folding Modes Identified by State-Free Reversible VAMPnets

Kinetic Machine Learning Unravels Ligand-Directed Conformational Change of mu Opioid Receptor

Ancestral reconstruction reveals mechanisms of ERK regulatory evolution

Improvements in Markov State Model Construction Reveal Many Non-Native Interactions in the Folding of NTL9

Markov state models and NMR uncover an overlooked allosteric loop in p53

Target search and recognition mechanisms of glycosylase AlkD revealed by scanning FRET-FCS and Markov state models

Deep learning Markov and Koopman models with physical constraints

PMLR: Proceedings of Machine Learning Research

Role of bacterial RNA polymerase gate opening dynamics in DNA loading and antibiotics inhibition elucidated by quasi-Markov State Model

Learning molecular dynamics with simple language model built upon long short-term memory neural network

Grandmaster level in StarCraft II using multi-agent reinforcement learning

Polosukhin, I. Attention Is All You Need

Deep Residual Learning for Image Recognition

Perspective: Identification of collective variables and metastable states of protein dynamics

Unsupervised Learning Methods for Molecular Simulation Data

Clustering algorithms to analyze molecular dynamics simulation trajectories for complex chemical and biological systems

Deflation reveals dynamical structure in nondominant reaction coordinates

Projected and hidden Markov models for calculating kinetics and metastable states of complex molecules

PyEMMA 2: A Software Package for Estimation, Validation, and Analysis of Markov Models

Markov state models based on milestoning

Density-based cluster algorithms for the identification of core sets