Articles in 2023

Machine learning methods in cheminformatics have made great progress in using chemical structures of molecules, but a large portion of textual information remains scarcely explored. Liu and colleagues trained MoleculeSTM, a foundation model that aligns the structure and text modalities through contrastive learning, and show its utility on the downstream tasks of structure–text retrieval, text-guided editing and molecular property prediction.

Theoretical frameworks aiming to understand deep learning rely on a so-called infinite-width limit, in which the ratio between the width of hidden layers and the training set size goes to zero. Pacelli and colleagues go beyond this restrictive framework by computing the partition function and generalization properties of fully connected, nonlinear neural networks, both with one and with multiple hidden layers, for the practically more relevant scenario in which the above ratio is finite and arbitrary.

Interest in using large language models such as ChatGPT has grown rapidly, but concerns about safe and responsible use have emerged, in part because adversarial prompts can bypass existing safeguards with so-called jailbreak attacks. Wu et al. build a dataset of various types of jailbreak attack prompt and demonstrate a simple but effective technique to counter these attacks by encapsulating users’ prompts in another standard prompt that reminds ChatGPT to respond responsibly.

Machine learning models have been widely used in the inverse design of new materials, but typically only linear properties could be targeted. Bastek and Kochmann show that video diffusion generative models can produce the nonlinear deformation and stress response of cellular materials under large-scale compression.

Virtual drug design has seen recent progress in methods that can generate new molecules with specific properties. Separately, methods have also improved in the task of computationally predicting the outcome of chemical reactions. Qiang and colleagues use the close relation of the two problems to train a model that aims at solving both tasks.

Data-driven surrogate models are used in computational physics and engineering to greatly speed up evaluations of the properties of partial differential equations, but they come with a heavy computational cost associated with training. Pestourie et al. combine a low-fidelity physics model with a generative deep neural network and demonstrate improved accuracy–cost trade-offs compared with standard deep neural networks and high-fidelity numerical solvers.

Single-cell transcriptomics has provided a powerful approach to investigate cellular properties at unprecedented resolution. Sha et al. have developed an optimal transport-based algorithm called TIGON that can connect transcriptomic snapshots from different time points to obtain collective dynamical information, including cell population growth and the underlying gene regulatory network.

A fundamental question in neuroscience is what are the constraints that shape the structural and functional organization of the brain. By bringing biological cost constraints into the optimization process of artificial neural networks, Achterberg, Akarca and colleagues uncover the joint principle underlying a large set of neuroscientific findings.

Deep learning is a powerful method to process large datasets, and shown to be useful in many scientific fields, but models are highly parameterized and there are often challenges in interpretation and generalization. David Gleich and colleagues develop a method rooted in computational topology, starting with a graph-based topological representation of the data, to help assess and diagnose predictions from deep learning and other complex prediction methods.

Continual learning is an innate ability in biological intelligence to accommodate real-world changes, but it remains challenging for artificial intelligence. Wang, Zhang and colleagues model key mechanisms of a biological learning system, in particular active forgetting and parallel modularity, to incorporate neuro-inspired adaptability to improve continual learning in artificial intelligence systems.

scBERT, a pretrained neural network for single-cell sequencing tasks, was published last year in Nature Machine Intelligence. To test the reusability of the method, Khan et al. use the code to assess the generalizablility of transformer architectures on single-cell genomics tasks.

Prediction of high-level visual representations in the human brain may benefit from multimodal sources in network training and the incorporation of complex datasets. Wang and colleagues show that language pretraining and a large, diverse dataset together build better models of higher-level visual cortex compared to earlier models.

Graph neural networks have proved useful in modelling proteins and their ligand interactions, but it is not clear whether the patterns they identify have biological relevance or whether interactions are merely memorized. Mastropietro et al. use a Shapley value-based method to identify important edges in protein interaction graphs, enabling explanatory analysis of the model mechanisms.

Halide perovskites are promising materials for light-emitting devices, given their narrowband emission and solution processability. However, detailed information on device degradation during operation is required to improve their stability, and this is challenging to obtain. Ji et al. propose a self-supervised deep learning method to capture multi-dimensional images of such devices in their operating regime faster than allowed by conventional imaging techniques.

The reconstruction of dynamic, spatial fields from sparse sensor data is an important challenge in various fields of science and technology. Santos et al. introduce the Senseiver, a deep learning framework that reconstructs spatial fields from few observations using attention layers to encode and decode sparse data, enabling efficient inference.

Geometric deep learning has become a powerful tool in virtual drug design, but it is not always obvious when a model makes incorrect predictions. Luo and colleagues improve the accuracy of their deep learning model using uncertainty calibration and Bayesian optimization in an active learning cycle.

Human and animal motion planning works at various timescales to allow the completion of complex tasks. Inspired by this natural strategy, Yuan and colleagues present a hierarchical motion planning approach for robotics, using deep reinforcement learning and predictive proprioception.

Organisms show complex behaviour resulting from a trade-off between obtaining information (explore) and using current information (exploit). Biswas et al. observe a mode-switching strategy modulated by sensory salience in a diverse range of organisms, including electric fish and humans, and argue that the observed heuristic could inform the design of active-sensing behaviours in robotics.

Prime editors are innovative genome-editing tools, but selecting guide RNAs with high efficiency remains challenging and requires costly experimental efforts. Liu and colleagues develop a method to design prime-editing guide RNAs based on transfer learning for in silico prediction of editing efficacy.

Learning causal relationships between variables in large datasets is an outstanding challenge in various scientific applications. Lagemann et al. introduce a deep neural network approach combining convolutional and graph models intended for causal learning in high-dimensional biomedical problems.