Data engineering
Persistent Topological Features in Large Language Models
Abstract Understanding the decision-making processes of large language models is critical given their widespread applications. To achieve this, we aim to connect a formal mathematical framework—zigzag persistence from topological data analysis —with practical and easily applicable algorithms. Zigzag persistence is particularly effective for characterizing data as it dynamically transforms across model layers. Within this framework, we introduce topological descriptors that measure how topological features, -dimensional holes, persist and evolve throughout the layers. Unlike methods that assess each layer individually and then aggregate the results, our approach directly tracks the full evolutionary path of these features. This offers a statistical perspective on how prompts are rearranged and their relative positions changed in the representation space, providing insights into the system’s operation as an integrated whole. To demonstrate the expressivity and applicability of our framework, we highlight how sensitive these descriptors are to different models and a variety of datasets. As a showcase application to a downstream task, we use zigzag persistence to establish a criterion for layer pruning, achieving results comparable to state-of-the-art methods while preserving the system-level perspective. Authors Yuri Gardinazzi, Karthik Viswanathan, Giada Panerai, Alessio Ansuini, Alberto Cazzaniga, Matteo Biagetti Journal International Conference of Machine Learning (ICML) 2025 Publication Date 01/05/2025 Consult the publication
ResiDual Transformer Alignment with Spectral Decomposition
Abstract When examined through the lens of their residual streams, a puzzling property emerges in transformer networks: residual contributions (e.g., attention heads) sometimes specialize in specific tasks or input attributes. In this paper, we analyze this phenomenon in vision transformers, focusing on the spectral geometry of residuals, and explore its implications for modality alignment in vision-language models. First, we link it to the intrinsically low-dimensional structure of visual head representations, zooming into their principal components and showing that they encode specialized roles across a wide variety of input data distributions. Then, we analyze the effect of head specialization in multimodal models, focusing on how improved alignment between text and specialized heads impacts zero-shot classification performance. This specialization-performance link consistently holds across diverse pre-training data, network sizes, and objectives, demonstrating a powerful new mechanism for boosting zero-shot classification through targeted alignment. Ultimately, we translate these insights into actionable terms by introducing ResiDual, a technique for spectral alignment of the residual stream. Much like panning for gold, it lets the noise from irrelevant unit principal components (i.e., attributes) wash away to amplify task-relevant ones. Remarkably, this dual perspective on modality alignment yields fine-tuning level performance on different data distributions while modelling an extremely interpretable and parameter-efficient transformation, as we extensively show on 70 pre-trained network-dataset combinations (7 models, 10 datasets). Authors Lorenzo Basile, Valentino Maiorca, Luca Bartolussi, Emanuele Rodolà, Francesco Locatello Journal Transmissions in Machine Learning Research Publication Date 12/04/2025 Consult the publication
Activation Patching for Interpretable Steering in Music Generation
Abstract Understanding how large audio models represent music, and using that understanding to steer generation, is both challenging and underexplored. Inspired by mechanistic interpretability in language models, where direction vectors in transformer residual streams are key to model analysis and control, we investigate similar techniques in the audio domain. This paper presents the first study of latent direction vectors in large audio models and their use for continuous control of musical attributes in text-to-music generation. Focusing on binary concepts like tempo (fast vs. slow) and timbre (bright vs. dark), we compute steering vectors using the difference-in-means method on curated prompt sets. These vectors, scaled by a coefficient and injected into intermediate activations, allow fine-grained modulation of specific musical traits while preserving overall audio quality. We analyze the effect of steering strength, compare injection strategies, and identify layers with the greatest influence. Our findings highlight the promise of direction-based steering as a more mechanistic and interpretable approach to controllable music generation. Authors Simone Facchiano, Giorgio Strano, Donato Crisostomi, Irene Tallini, Tommaso Mencattini, Fabio Galasso, Emanuele Rodolà Journal Preprint Publication Date 06/04/2025 Consult the publication
Interpreting and Steering Protein Language Models through Sparse Autoencoders
Abstract The rapid advancements in transformer-based language models have revolutionized natural language processing, yet understanding the internal mechanisms of these models remains a significant challenge. This paper explores the application of sparse autoencoders (SAE) to interpret the internal representations of protein language models, specifically focusing on the ESM-2 8M parameter model. By performing a statistical analysis on each latent component’s relevance to distinct protein annotations, we identify potential interpretations linked to various protein characteristics, including transmembrane regions, binding sites, and specialized motifs. We then leverage these insights to guide sequence generation, shortlisting the relevant latent components that can steer the model towards desired targets such as zinc finger domains. This work contributes to the emerging field of mechanistic interpretability in biological sequence models, offering new perspectives on model steering for sequence design. Authors Edith Natialia Villegas Garcia, Alessio Ansuini Journal Workshop on Generative and Experimental Perspectives for Biomolecular Design @ ICLR 2025 Publication Date 13/02/2025 Consult the publication
Striped Twisted State in the Orientational Epitaxy on Quasicrystals
Abstract The optimal “twisted” geometry of a crystalline layer on a crystal has long been known, but that on a quasicrystal is still unknown and open. We predict analytically that the layer equilibrium configuration will generally exhibit a nonzero misfit angle. The theory perfectly agrees with numerical optimization of a colloid monolayer on a quasiperiodic decagonal optical lattice. Strikingly different from crystal-on-crystal epitaxy, the structure of the novel emerging twisted state exhibits an unexpected stripe pattern. Its high anisotropy should reflect on the tribomechanical properties of this unconventional interface. Authors Nicola Manini, Mario Forzanini, Sebastiano Pagano, Marco Bellagente, Martino Colombo, Dario Bertazioli, Tommaso Salvalaggio, Andrea Vanossi, Davide Vanossi, Emanuele Panizon, Erio Tosatti, Giuseppe E. Santoro Journal Physical Review Letters Publication Date 12/02/2025 Consult the publication
Towards the FAIRification of Scanning Tunneling Microscopy Images
Abstract In this paper, we describe the data management practices and services developed for making FAIR compliant a scientific archive of Scanning Tunneling Microscopy (STM) images. As a first step, we extracted the instrument metadata of each image of the dataset to create a structured database. We then enriched these metadata with information on the structure and composition of the surface by means of a pipeline that leverages human annotation, machine learning techniques, and instrument metadata filtering. To visually explore both images and metadata, as well as to improve the accessibility and usability of the dataset, we developed “STM explorer” as a web service integrated within the Trieste Advanced Data services (TriDAS) website. On top of these data services and tools, we propose an implementation of the W3C PROV standard to describe provenance metadata of STM images. Authors Tommaso Rodani, Elda Osmenaj, Alberto Cazzaniga, Mirco Panighel, Cristina Africh, Stefano Cozzini Journal Data Intelligence Publication Date 07/02/2025 Consult the publication
The Geometry of Tokens in Internal Representations of Large Language Models
Abstract We investigate the relationship between the geometry of token embeddings and their role in the next token prediction within transformer models. An important aspect of this connection uses the notion of empirical measure, which encodes the distribution of token point clouds across transformer layers and drives the evolution of token representations in the mean-field interacting picture. We use metrics such as intrinsic dimension, neighborhood overlap, and cosine similarity to observationally probe these empirical measures across layers. To validate our approach, we compare these metrics to a dataset where the tokens are shuffled, which disrupts the syntactic and semantic structure. Our findings reveal a correlation between the geometric properties of token embeddings and the crossentropy loss of next token predictions, implying that prompts with higher loss values have tokens represented in higher-dimensional spaces. Authors Karthik Viswanathan, Yuri Gardinazzi, Giada Panerai, Alberto Cazzaniga, Matteo Biagetti Journal Preprint Publication Date 17/01/2025 Consult the publication
Intrinsic Dimension Correlation: uncovering nonlinear connections in multimodal representations
Abstract To gain insight into the mechanisms behind machine learning methods, it is crucial to establish connections among the features describing data points. However, these correlations often exhibit a high-dimensional and strongly nonlinear nature, which makes them challenging to detect using standard methods. This paper exploits the entanglement between intrinsic dimensionality and correlation to propose a metric that quantifies the (potentially nonlinear) correlation between high-dimensional manifolds. We first validate our method on synthetic data in controlled environments, showcasing its advantages and drawbacks compared to existing techniques. Subsequently, we extend our analysis to large-scale applications in neural network representations. Specifically, we focus on latent representations of multimodal data, uncovering clear correlations between paired visual and textual embeddings, whereas existing methods struggle significantly in detecting similarity. Our results indicate the presence of highly nonlinear correlation patterns between latent manifolds. Authors Lorenzo Basile, Santiago Acevedo, Luca Bortolussi, Fabio Anselmi, Alex Rodriguez Journal International Conferenence of Learning Representations (ICLR) 2025 Publication Date 22/01/2025 Consult the publication
Counterfactual rewards promote collective transport using individually controlled swarm microrobots
Abstract Swarm robots offer fascinating opportunities to perform complex tasks beyond the capabilities of individual machines. Just as a swarm of ants collectively moves large objects, similar functions can emerge within a group of robots through individual strategies based on local sensing. However, realizing collective functions with individually controlled microrobots is particularly challenging because of their micrometer size, large number of degrees of freedom, strong thermal noise relative to the propulsion speed, and complex physical coupling between neighboring microrobots. Here, we implemented multiagent reinforcement learning (MARL) to generate a control strategy for up to 200 microrobots whose motions are individually controlled by laser spots. During the learning process, we used so-called counterfactual rewards that automatically assign credit to the individual microrobots, which allows fast and unbiased training. With the help of this efficient reward scheme, swarm microrobots learn to collectively transport a large cargo object to an arbitrary position and orientation, similar to ant swarms. We show that this flexible and versatile swarm robotic system is robust to variations in group size, the presence of malfunctioning units, and environmental noise. In addition, we let the robot swarms manipulate multiple objects simultaneously in a demonstration experiment, highlighting the benefits of distributed control and independent microrobot motion. Control strategies such as ours can potentially enable complex and automated assembly of mobile micromachines, programmable drug delivery capsules, and other advanced lab-on-a-chip applications. Authors Veit-Lorenz Heuthe, Emanuele Panizon, Hongri Gu, and Clemens Bechinger Journal Science Robotics Publication Date 18/12/2024 Consult the publication
Living in Digital Ecosystems: Are We Aware of This?
Abstract In this paper we would like to discuss and evaluate the social and economic impact of contemporary digital ecosystems. We extend the definition of digital ecosystem following the idea of the circular economy based on enhanced and efficient use/reuse of resources and products. We consider an Extended or Circular Digital Ecosystem, a Digital Ecosystem in which the final customers can create value by sharing their data and digital contents through the digital ecosystem services themselves. In this way the data consumers also become data producers. This new concept of a digital ecosystem is closely integrated with the role and actions of the people involved in it, as both users, producers, and managers. A key point of our discussion is the new and very powerful AI algorithms, which are the current focus for research and beyond. We argue that the focus should not only on the algorithms themselves but also be on the data used to train and/or feed such algorithms. We exploit the point of view of data mangers and data curators to address this issue. We make a comparison between the management of scientific research data that is moving towards correct management following the FAIR principles and the data produced every day on the internet by sharing information on social media or any other platform. Since 2009, following the idea of open science, the research community started to focus on the quality of the shared research data. In 2016, the FAIR (Findable, Accessible, Interoperable and Reusable) principles gave the guidelines to share data that are machine interoperable, i.e. that can also be correctly used and interpreted by machines and algorithms. For example, generative AI based on Large Language Models (LLMs), such as the well-known Chat GPT, was trained on a large corpus of text (about 45 TB) to understand and generate text consistently, which consisted mainly of any type of text obtained from web crawling tools. The users of digital ecosystems should be aware that all available data are currently used to train AI algorithms and the results that these algorithms produce depend on the quality of the data used to train them. It is widely demonstrated that database classification by human intervention is prone to biases and errors. Algorithms trained on those datasets will suffer from these biases, resulting in dangerous or inappropriate content. Collecting all these insights, we plead that it is crucial to consider the technical experience of data managers and curators in the wider context of data sharing and management in society, given the immense amount of data produced and available on the network. We also argue that FAIR principles for research data should also be considered in the wider societal context, to avoid potential dangers related to new developments in Artificial Intelligence, which, in our belief and experience, are not harmful in themselves. Our general goal is then to raise awareness on the users of digital ecosystems that the correct sharing of their information can improve the quality of the data available on the network and that the potential dangers of data misuse can be reduced. Authors Stefano Cozzini and Mariarita De Luca Journal Digital Environments and Human Relations Publication Date 10/12/2024 Consult the publication
The Representation Landscape of Few-Shot Learning and Fine-Tuning in Large Language Models
Abstract In-context learning (ICL) and supervised fine-tuning (SFT) are two common strategies for improving the performance of modern large language models (LLMs) on specific tasks. Despite their different natures, these strategies often lead to comparable performance gains. However, little is known about whether they induce similar representations inside LLMs. We approach this problem by analyzing the probability landscape of their hidden representations in the two cases. More specifically, we compare how LLMs solve the same question-answering task, finding that ICL and SFT create very different internal structures, in both cases undergoing a sharp transition in the middle of the network. In the first half of the network, ICL shapes interpretable representations hierarchically organized according to their semantic content. In contrast, the probability landscape obtained with SFT is fuzzier and semantically mixed. In the second half of the model, the fine-tuned representations develop probability modes that better encode the identity of answers, while the landscape of ICL representations is characterized by less defined peaks. Our approach reveals the diverse computational strategies developed inside LLMs to solve the same task across different conditions, allowing us to make a step towards designing optimal methods to extract information from language models. Authors Diego Doimo, Alessandro Serra, Alessio Ansuini, Alberto Cazzaniga Journal Neural Information Processing Systems 37 (NeurIPS) 2024 Publication Date 10/12/2024 Consult the publication
A Comprehensive Analysis of Process Energy Consumption on Multi-Socket Systems with GPUs
Abstract: Robustly estimating energy consumption in High-Performance Computing (HPC) is essential for assessing the energy footprint of modern workloads, particularly in fields such as Artificial Intelligence (AI) research, development, and deployment. The extensive use of supercomputers for AI training has heightened concerns about energy consumption and carbon emissions. Existing energy estimation tools often assume exclusive use of computing nodes, a premise that becomes problematic with the advent of supercomputers integrating microservices, as seen in initiatives like Acceleration as a Service (XaaS) and cloud computing. This work investigates the impact of executed instructions on overall power consumption, providing insights into the comprehensive behaviour of HPC systems. We introduce two novel mathematical models to estimate a process’s energy consumption based on the total node energy, process usage, and a normalised vector of the probability distribution of instruction types for CPU and GPU processes. Our approach enables energy accounting for specific processes without the need for isolation. Our models demonstrate high accuracy, predicting CPU power consumption with a mere 1.9% error. For GPU predictions, the models achieve a central relative error of 9.7%, showing a clear tendency to fit the test data accurately. These results pave the way for new tools to measure and account for energy consumption in shared supercomputing environments. Authors: Luis G. Leon-Vega, Niccolò Tosato, Stefano Cozzini Journal: 11th Latin American High Performance Computing Conference, CARLA 2024 Publication date: 30/09/2024 Consult the publication