Data engineering
Molecular simulations to investigate the impact of N6-methylation in RNA recognition: Improving accuracy and precision of binding free energy prediction
Abstract N6-Methyladenosine (m6A) is a prevalent RNA post-transcriptional modification that plays crucial roles in RNA stability, structural dynamics, and interactions with proteins. The YT521-B (YTH) family of proteins, which are notable m6A readers, functions through its highly conserved YTH domain. Recent structural investigations and molecular dynamics (MD) simulations have shed light on the mechanism of recognition of m6A by the YTHDC1 protein. Despite advancements, using MD to predict the stabilization induced by m6A on the free energy of binding between RNA and YTH proteins remains challenging due to inaccuracy of the employed force field and limited sampling. For instance, simulations often fail to sufficiently capture the hydration dynamics of the binding pocket. This study addresses these challenges through an innovative methodology that integrates metadynamics, alchemical simulations, and force-field refinement. Importantly, our research identifies hydration of the binding pocket as giving only a minor contribution to the binding free energy and emphasizes the critical importance of precisely tuning force-field parameters to experimental data. By employing a fitting strategy built on alchemical calculations, we refine the m6A partial charge parameters, thereby enabling the simultaneous reproduction of N6 methylation on both the protein binding free energy and the thermodynamic stability of nine RNA duplexes. Our findings underscore the sensitivity of binding free energies to partial charges, highlighting the necessity for thorough parametrization and validation against experimental observations across a range of structural contexts. Autori Valerio Piomponi, Miroslav Krepl, Jiri Sponer, Giovanni Bussi Rivista The Journal of Physical Chemistry B, Vol 128, Issue 37 Data di pubblicazione 06/09/2024 Consulta la pubblicazione
Detach-ROCKET: Sequential feature selection for time series classification with random convolutional kernels
Abstract: Time Series Classification (TSC) is essential in fields like medicine, environmental science, and finance, enabling tasks such as disease diagnosis, anomaly detection, and stock price analysis. While machine learning models like Recurrent Neural Networks and InceptionTime are successful in numerous applications, they can face scalability issues due to computational requirements. Recently, ROCKET has emerged as an efficient alternative, achieving state-of-the-art performance and simplifying training by utilizing a large number of randomly generated features from the time series data. However, many of these features are redundant or non-informative, increasing computational load and compromising generalization. Here we introduce Sequential Feature Detachment (SFD) to identify and prune non-essential features in ROCKET-based models, such as ROCKET, MiniRocket, and MultiRocket. SFD estimates feature importance using model coefficients and can handle large feature sets without complex hyperparameter tuning. Testing on the UCR archive shows that SFD can produce models with better test accuracy using only 10% of the original features. We named these pruned models Detach-ROCKET. We also present an end-to-end procedure for determining an optimal balance between the number of features and model accuracy. On the largest binary UCR dataset, Detach-ROCKET improves test accuracy by 0.6% while reducing features by 98.9%. By enabling a significant reduction in model size without sacrificing accuracy, our methodology improves computational efficiency and contributes to model interpretability. We believe that Detach-ROCKET will be a valuable tool for researchers and practitioners working with time series data, who can find a user-friendly implementation of the model at https://github.com/gon-uri/detach_rocket. Autori: Gonzalo Uribarri, Federico Barone, Alessio Ansuini, Eric Fransén Rivista: Data Mining and Knowledge Discovery Data di pubblicazione: 20/08/2024 Consulta la pubblicazione
Competition of Mechanisms: Tracing How Language Models Handle Facts and Counterfactuals
Abstract Interpretability research aims to bridge the gap between empirical success and our scientific understanding of the inner workings of large language models (LLMs). However, most existing research focuses on analyzing a single mechanism, such as how models copy or recall factual knowledge. In this work, we propose a formulation of competition of mechanisms, which focuses on the interplay of multiple mechanisms instead of individual mechanisms and traces how one of them becomes dominant in the final prediction. We uncover how and where mechanisms compete within LLMs using two interpretability methods: logit inspection and attention modification. Our findings show traces of the mechanisms and their competition across various model components and reveal attention positions that effectively control the strength of certain mechanisms. Autori Francesco Ortu, Zhijing Jin, Diego Doimo, Mrinmaya Schaan, Alberto Cazzaniga, Bernhard Scholkopf Rivista Accepted at the Annual Meeting of the Association for Computational Linquistics (ACL), Arxiv preprint: 2402.11655 Data di pubblicazione 06/06/2024 Consulta la pubblicazione
Emergent representations in networks trained with the Forward-Forward algorithm
Abstract The Backpropagation algorithm has often been criticised for its lack of biological realism. In an attempt to find a more biologically plausible alternative, the recently introduced Forward-Forward algorithm replaces the forward and backward passes of Backpropagation with two forward passes. In this work, we show that the internal representations obtained by the Forward-Forward algorithm can organise into category-specific ensembles exhibiting high sparsity – composed of a low number of active units. This situation is reminiscent of what has been observed in cortical sensory areas, where neuronal ensembles are suggested to serve as the functional building blocks for perception and action. Interestingly, while this sparse pattern does not typically arise in models trained with standard Backpropagation, it can emerge in networks trained with Backpropagation on the same objective proposed for the Forward-Forward algorithm. These results suggest that the learning procedure proposed by Forward-Forward may be superior to Backpropagation in modelling learning in the cortex, even when a backward pass is used. Autori Niccolò Tosato, Lorenzo Basile, Emanuele Ballarin, Giuseppe de Alteriis, Alberto Cazzaniga, Alessio Ansuini Rivista Transmissions in Machine Learning Research Data di pubblicazione 28/03/2025 Consulta la pubblicazione
Enhancing Multi-Tip Artifact Detection in STM Images Using Fourier Transform and Vision Transformers
Abstract We address the issue of multi-tip artifacts in Scanning Tunneling Microscopy (STM) images by applying the fast Fourier transform (FFT) as a feature engineering method. We fine-tune various neural network architectures using a synthetic dataset, including Vision Transformers (ViT). The FFT-based preprocessing significantly improves the performance of ViT models compared to using only the grayscale channel. Ablation experiments highlight the optimal conditions for synthetic dataset generation. Unlike traditional methods that are challenging to implement for large datasets and used offline, our method enables on-the-fly classification at scale. Our findings demonstrate the efficacy of combining the Fourier transform with deep learning for enhanced artifact detection in STM images, contributing to more accurate analysis in material science research. Autori Tommaso Rodani, Alessio Ansuini, Alberto Cazzaniga Rivista ICML ’24 Workshop ML for Life and Material Science: From Theory to Industry Applications Data di pubblicazione 17/07/2024 Consulta la pubblicazione
Enhancing predictions of protein stability changes induced by single mutations using MSA-based language models
Abstract Protein language models offer a new perspective for addressing challenges in structural biology, while relying solely on sequence information. Recent studies have investigated their effectiveness in forecasting shifts in thermodynamic stability caused by single amino acid mutations, a task known for its complexity due to the sparse availability of data, constrained by experimental limitations. To tackle this problem, we introduce two key novelties: leveraging a protein language model that incorporates Multiple Sequence Alignments to capture evolutionary information, and using a recently released mega-scale dataset with rigorous data preprocessing to mitigate overfitting. Autori Francesca Cuturello, Marco Celoria, Alessio Ansuini, Alberto Cazzaniga Rivista Bioinformatics, 2024, 40 (7) Consulta la pubblicazione
Protein family annotation for the Unified Human Gastrointestinal Proteome by DPCfam clustering
Abstract Technological advances in massively parallel sequencing have led to an exponential growth in the number of known protein sequences. Much of this growth originates from metagenomic projects producing new sequences from environmental and clinical samples. The Unified Human Gastrointestinal Proteome (UHGP) catalogue is one of the most relevant metagenomic datasets with applications ranging from medicine to biology. However, the low levels of sequence annotation may impair its usability. This work aims to produce a family classification of UHGP sequences to facilitate downstream structural and functional annotation. This is achieved through the release of the DPCfam-UHGP50 dataset containing 10,778 putative protein families generated using DPCfam clustering, an unsupervised pipeline grouping sequences into single or multi-domain architectures. DPCfam-UHGP50 considerably improves family coverage at protein and residue levels compared to the manually curated repository Pfam. In the hope that DPCfam-UHGP50 will foster future discoveries in the field of metagenomics of the human gut, we release a FAIR-compliant database of our results that is easily accessible via a searchable web server and Zenodo repository. Autori Federico Barone, Elena Tea Russo, Edith Natalia Villegas Garcia, Marco Punta, Stefano Cozzini, Alessio Ansuini, Alberto Cazzaniga Rivista Scientific Data 11, 568 (2024) Data di pubblicazione 01/06/2024 Consulta la pubblicazione
Sensory–Motor Loop Adaptation in Boolean Network Robots
Abstract Recent technological advances have made it possible to produce tiny robots equipped with simple sensors and effectors. Micro-robots are particularly suitable for scenarios such as exploration of hostile environments, and emergency intervention, e.g., in areas subject to earthquakes or fires. A crucial desirable feature of such a robot is the capability of adapting to the specific environment in which it has to operate. Given the limited computational capabilities of a micro-robot, this property cannot be achieved by complicated software but it rather should come from the flexibility of simple control mechanisms, such as the sensory–motor loop. In this work, we explore the possibility of equipping simple robots controlled by Boolean networks with the capability of modulating their sensory–motor loop such that their behavior adapts to the incumbent environmental conditions. This study builds upon the cybernetic concept of homeostasis, which is the property of maintaining essential parameters inside vital ranges, and analyzes the performance of adaptive mechanisms intervening in the sensory–motor loop. In particular, we focus on the possibility of maneuvering the robot’s effectors such that both their connections to network nodes and environmental features can be adapted. As the actions the robot takes have a feedback effect to its sensors mediated by the environment, this mechanism makes it possible to tune the sensory–motor loop, which, in turn, determines the robot’s behavior. We study this general setting in simulation and assess to what extent this mechanism can sustain the homeostasis of the robot. Our results show that controllers made of random Boolean networks in critical and chaotic regimes can be tuned such that their homeostasis in different environments is kept. This outcome is a step towards the design and deployment of controllers for micro-robots able to adapt to different environments. Authors Michele Braccini, Yuri Gardinazzi, Andrea Roli, Marco Villani Journal Sensors (MDPI) Publication Date 24/04/2024
What Is the Probability That a Random Symmetric Tensor Is Close to Rank-One?
Abstract We address the general problem of estimating the probability that a real symmetric tensor is close to rank-one tensors. Using Weyl’s tube formula, we turn this question into a differential geometric one involving the study of metric invariants of the real Veronese variety. More precisely, we give an explicit formula for its reach and curvature coefficients with respect to the Bombieri–Weyl metric. These results are obtained using techniques from random matrix theory and an explicit description of the second fundamental form of the Veronese variety in terms of Gaussian orthogonal ensemble matrices. Our findings give a complete solution to the original problem. In the case of rational normal curves it leads to a simple formula describing explicitly exponential decay with respect to the degree of the tensor. Authors Alberto Cazzaniga, Antonio Lerario, Andrea Rosana Journal SIAM Journal on Applied Algebra and Geometry Publication Date 01/02/2024 Consult the publication
Redundant representations help generalization in wide neural networks
Abstract Deep neural networks (DNNs) defy the classical bias-variance trade-off; adding parameters to a DNN that interpolates its training data will typically improve its generalization performance. Explaining the mechanism behind this ‘benign overfitting’ in deep networks remains an outstanding challenge. Here, we study the last hidden layer representations of various state-of-the-art convolutional neural networks and find that if the last hidden representation is wide enough, its neurons tend to split into groups that carry identical information and differ from each other only by statistically independent noise. The number of these groups increases linearly with the width of the layer, but only if the width is above a critical value. We show that redundant neurons appear only when the training is regularized and the training error is zero. Authors Diego Doimo, Aldo Glielmo, Sebastian Goldt, Alessandro Laio Journal Journal of Statistical Mechanics: Theory and Experiment Publication Date 15/11/2023 Consult the publication
The geometry of hidden representations of large transformer models
Abstract Large transformers are powerful architectures used for self-supervised data analysis across various data types, including protein sequences, images, and text. In these models, the semantic structure of the dataset emerges from a sequence of transformations between one representation and the next. We characterize the geometric and statistical properties of these representations and how they change as we move through the layers.By analyzing the intrinsic dimension (ID) and neighbor composition, we find that the representations evolve similarly in transformers trained on protein language taskand image reconstruction tasks. In the first layers, the data manifold expands, becoming high-dimensional, and then contracts significantly in the intermediate layers. In the last part of the model, the ID remains approximately constant or forms a second shallow peak. We show that the semantic information of the dataset is better expressed at the end of the first peak, and this phenomenon can be observed across many models trained on diverse datasets.Based on our findings, we point out an explicit strategy to identify, without supervision, the layers that maximize semantic content: representations at intermediate layers corresponding to a relative minimum of the ID profile are more suitable for downstream learning tasks. Autori Lucrezia Valeriani, Diego Doimo, Francesca Cuturello, Alessandro Laio, Alessio Ansuini, Alberto Cazzaniga Rivista Advances in Neural Information Processing Systems 36 (NEURIPS 2023) Main Conference Track Data di pubblicazione 21/09/2023 Consulta la pubblicazione
Shared metadata for data-centric materials science
Abstract The expansive production of data in materials science, their widespread sharing and repurposing requires educated support and stewardship. In order to ensure that this need helps rather than hinders scientific work, the implementation of the FAIR-data principles (Findable, Accessible, Interoperable, and Reusable) must not be too narrow. Besides, the wider materials-science community ought to agree on the strategies to tackle the challenges that are specific to its data, both from computations and experiments. In this paper, we present the result of the discussions held at the workshop on “Shared Metadata and Data Formats for Big-Data Driven Materials Science”. We start from an operative definition of metadata, and the features that a FAIR-compliant metadata schema should have. We will mainly focus on computational materials-science data and propose a constructive approach for the FAIRification of the (meta)data related to ground-state and excited-states calculations, potential-energy sampling, and generalized workflows. Finally, challenges with the FAIRification of experimental (meta)data and materials-science ontologies are presented together with an outlook of how to meet them. Authors Luca M. Ghiringhelli, Carsten Baldauf, Tristan Bereau, Sandor Brockhauser, Christian Carbogno, Javad Chamanara, Stefano Cozzini, Stefano Curtarolo, Claudia Draxl, Shyam Dwaraknath, Ádám Fekete, James Kermode, Christoph T. Koch, Markus Kühbach, Alvin Noe Ladines, Patrick Lambrix, Maja-Olivia Himmer, Sergey V. Levchenko, Micael Oliveira, Adam Michalchuk, Ronald E. Miller, Berk Onat, Pasquale Pavone, Giovanni Pizzi, Benjamin Regler, Gian-Marco Rignanese, Jörg Schaarschmidt, Markus Scheidgen, Astrid Schneidewind, Tatyana Sheveleva, Chuanxun Su, Denis Usvyat, Omar Valsson, Christof Wöll & Matthias Scheffler Journal Scientific Data Publication Date 14/09/2023 Consult the publication