Daniel
Scalena

Chain-of-thought (CoT) reasoning is the dominant paradigm for inference-time scaling in language models, yet the causal influence of individual steps on the final answer poorly understood. We estimate each step's causal importance via early exit and use this measure to study how answers form across the reasoning traces of several model families. Across diverse tasks, we find that reasoning typically crosses a commitment boundary – a sharp transition from transient intermediate guesses to a stable, high-confidence answer. This transition often happens in a single step, well before the model's reasoning block ends, and is followed by epiphenomenal CoT steps that leave the final answer probability unaltered. Using attention probes, we show that answer-formation stages can be linearly decoded from intermediate reasoning steps with high accuracy and generalize robustly to unseen reasoning tasks. We exploit this signal to early-exit reasoning blocks at the commitment boundary, reducing the length of CoTs up to 55% on average with negligible impact on model performance.

@misc{scalena2026commitmentboundary,
  title={Beyond the Commitment Boundary: Probing Epiphenomenal Chain-of-Thought in Large Reasoning Models},
  author={Daniel Scalena* and Sara Candussio* and Luca Bortolussi and Elisabetta Fersini and Malvina Nissim and Gabriele Sarti},
  year={2026},
  eprint={2606.13603},
  archivePrefix={arXiv},
  primaryClass={cs.LG},
  url={https://arxiv.org/abs/2606.13603}
}

With the rise of reasoning language models and test-time scaling methods as a paradigm for improving model performance, substantial computation is often required to generate multiple candidate sequences from the same prompt. This enables exploration of different reasoning paths toward the correct solution, however, allocates the same compute budget for each prompt. Grounded on the assumption that different prompts carry different degrees of complexity, and thus different computation needs, we propose EAGer, a training-free generation method that leverages model uncertainty through token-wise entropy distribution to reduce redundant computation and concurrently improve overall performance. EAGer allows branching to multiple reasoning paths only in the presence of high-entropy tokens, and then reallocates the saved compute budget to the instances where exploration of alternative paths is most needed. We find that across multiple open-source models on complex reasoning benchmarks such as AIME 2025, EAGer can reallocate the budget without accessing target labels, achieving the best efficiency-performance trade-off in terms of reasoning length and Pass@k. When target labels are accessible, EAGer generates up to 65

@misc{scalena2025eager,
  title={EAGer: Entropy-Aware GEneRation for Adaptive Inference-Time Scaling},
  author={Daniel Scalena and Leonidas Zotos and Elisabetta Fersini and Malvina Nissim and Ahmet Üstün},
  year={2026},
  eprint={2510.11170},
  archivePrefix={arXiv},
  primaryClass={cs.LG},
  url={https://openreview.net/forum?id=G0Mw5MaPcR},
  note={Proceedings of the 43rd International Conference on Machine Learning, (ICML 2026)}
}

High-quality machine translation systems based on large language models (LLMs) have simplified the production of personalized translations reflecting specific stylistic constraints. However, these systems still struggle in settings where stylistic requirements are less explicit and might be harder to convey via prompting. We explore various strategies for personalizing LLM-generated translations in low-resource settings, focusing on the challenging literary translation domain. We explore prompting strategies and inference-time interventions for steering model generations towards a personalized style, and propose a contrastive framework exploiting latent concepts extracted from sparse autoencoders to identify salient personalization properties. Our results show that steering achieves strong personalization while preserving translation quality. We further examine the impact of steering on LLM representations, finding model layers with a relevant impact for personalization are impacted similarly by multi-shot prompting and our steering method, suggesting similar mechanism at play.

@misc{scalena2025steeringlargelanguagemodels,
  title={Steering Large Language Models for Machine Translation Personalization},
  author={Daniel Scalena* and Gabriele Sarti* and Arianna Bisazza and Elisabetta Fersini and Malvina Nissim},
  year={2026},
  eprint={2505.16612},
  archivePrefix={arXiv},
  primaryClass={cs.CL},
  url={https://aclanthology.org/2026.eacl-long.217/},
  note={Proceedings of the 19th Conference of the European Chapter of the Association for CL, (EACL 2026)}
}

Activation steering methods were shown to be effective in conditioning language model generation by additively intervening over models' intermediate representations. However, the evaluation of these techniques has so far been limited to single conditioning properties and synthetic settings. In this work, we conduct a comprehensive evaluation of various activation steering strategies, highlighting the property-dependent nature of optimal parameters to ensure a robust effect throughout generation. To address this issue, we propose Dynamic Activation Composition, an information-theoretic approach to modulate the steering intensity of one or more properties throughout generation. Our experiments on multi-property steering show that our method successfully maintains high conditioning while minimizing the impact of conditioning on generation fluency.

@inproceedings{scalena-etal-2024-multi,
  title = "Multi-property Steering of Large Language Models with Dynamic Activation Composition",
  author = "Scalena, Daniel  and Sarti, Gabriele  and Nissim, Malvina",
  editor = "Belinkov, Yonatan  and Kim, Najoung  and Jumelet, Jaap  and Mohebbi, Hosein  and Mueller, Aaron  and Chen, Hanjie",
  booktitle = "Proceedings of the 7th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP",
  month = nov,
  year = "2024",
  address = "Miami, Florida, US",
  publisher = "Association for Computational Linguistics",
  url = "https://aclanthology.org/2024.blackboxnlp-1.34",
  pages = "577--603"
}