Iñigo Parra

I am a PhD student in Linguistics at UC Berkeley working on interpretability and on how deep learning can help us understand language and human cognition.

I am also currently a research scientist intern at Snowflake, where I work on reinforcement learning, agent behavioral analysis, and model routing.

Latest publications

Language, Not Code Abstraction: Natural-language Training Shapes Brain-aligned Geometry

Parra, I. NeurReps 2026 / PMLR

TL;DR: I trained 25 GPT-2-small models on English stories, Python, C, assembly, and machine code to test whether the abstract structure of code can substitute for natural language in brain alignment. Code training leaves alignment near the level of untrained models, with no ordering by abstraction level, while English training beats code on two of four neural benchmarks. On Pereira-243, English models also converge on a reproducible geometry whose fit-free correspondence with the brain is twice that of code.

Read the paper

Encoding score on Pereira-243 at each transformer block and averaged over blocks: TinyStories models score well above Python, C, assembly and binary models, which sit near the random-weight reference.

Neural Correlates of Language Models Are Specific to Human Language

Parra, I. NeurIPS 2025 / PMLR

TL;DR: This study investigates the neural correlates between language models and human brain activity, demonstrating that the language representations underlying modern language models align specifically with human language processing patterns rather than general sequence processing. I also show shared geometrical properties of LM and brain representations.

Read the paper

Method overview: language-model embeddings predict brain responses through ridge regression, and the geometry of both spaces is compared with Gromov–Wasserstein distance and CKA.

Interpretable Sparse Features for Probing Self-Supervised Speech Models

Parra, I. AACL 2025

TL;DR: I proposed a novel application of SAEs to understand self-supervised speech models through interpretable sparse feature extraction. The method revealed linguistically meaningful patterns in SAE features, offering new insights into what these models learn about phonetic and phonological structure.

Read the paper

UMAP projection of sparse autoencoder codes, colored by k-means cluster.

Adaptive Compute Efficient Learning via Conceptual-Criticality

Parra, I.; Bharathi, M.; Kumar, M.; Kumar, P.; Mishra, P. AAAI 2025

TL;DR: We introduced a conceptual-criticality framework for adaptive compute allocation in neural networks, enabling models to dynamically adjust computational resources based on input complexity. This approach achieves significant efficiency gains while maintaining or improving model performance.

Read the paper

Architecture diagram: a dynamic threshold selection module decides how the input is routed through a stack of layers.

Morphological Typology in BPE Subword Productivity and Language Modeling

Parra, I. NeurIPS 2024

TL;DR: This project explores how morphological typology affects BPE tokenization efficiency and language model performance across diverse languages. I demonstrated that morphology has a major effect on tokenization, and that this also affects training.

Read the paper

Loss and perplexity curves over training steps for English, Spanish, French, Basque, Hungarian, and Finnish.

Noise Be Gone: Does Speech Enhancement Distort Linguistic Nuances?

Parra, I. ACL 2024

TL;DR: I investigated whether speech enhancement techniques preserve or distort fine-grained linguistic features. My findings revealed that noise reduction improves overall intelligibility, and that enhancement methods do not significantly alter phonetic distinctions critical for downstream linguistic analysis.

Read the paper

Token processing diagram: each token is copied, one copy is degraded and then enhanced, and the final dataset keeps both the original and the enhanced versions.

UnMASKed: Quantifying Gender Biases in Language Models through Linguistically Informed Job Market Queries

Parra, I. EACL 2024

TL;DR: Using linguistically informed job market queries, I developed a methodology to quantify gender biases in masked language models. The analysis revealed systematic biases in occupational associations and provides insights into how language models perpetuate societal stereotypes.

Read the paper

Summary of the approach: masked pronoun and linguistic-unit queries are given to multilingual and monolingual BERT and DistilBERT models, followed by quantitative and qualitative analysis.