Sai Surya Duvvuri

Sai Surya Duvvuri

PhD Student, Computer Science

The University of Texas at Austin

Hi, I'm Sai! I am a fifth-year PhD student in Computer Science at UT Austin, advised by Prof. Inderjit S. Dhillon. My research goal is building data-efficient LLMs through (a) efficient architectures for long-context understanding and reasoning, and (b) optimization algorithms which gets the best out of each batch. My work usually has a linear algebraic flavour, utilizing theoretical insights to build algorithms with strong empirical performance — with some ideas finding their way into Google, Meta, and Microsoft.

Before my PhD, I spent two years at Microsoft Research collaborating with Neeraj Kayal, Ankit Garg, and Venkata N. Padmanabhan — where I got hooked on linear algebra and machine learning. I completed my B.Tech in CS from IIT Kharagpur. I have been fortunate to intern at Google Ads, Google DeepMind, Meta (FAIR), and IBM Research, where I met amazing collaborators including Rohan Anil, Manzil Zaheer, Cho-Jui Hsieh, and Abhijit Mishra.

Blog Posts

LUCID: Attention with Preconditioned Representations February 2026

A deep-dive into how preconditioning the attention matrix fixes attention noise in long-context LLMs.

News

2026

  • Jan Started as Student Researcher at Google, working on Diffusion and Recursive Transformers.
  • Jan Two papers from my time at Meta (FAIR) accepted to ICLR 2026: The Art of Scaling RL Compute for LLMs Oral and Test-Time Training for Long-Context LLMs!
  • Jan Three papers under review at ICML 2026.

2025

  • May Started as Visiting Researcher at Meta (FAIR), working on novel attention mechanisms for thinking LLMs.
  • May LASER: Attention with Exponential Transformation accepted to ICML 2025!
  • Jan LoRA Done RITE accepted as Oral at ICLR 2025 — work from my time at Google!

2024

  • May Started as Student Researcher at Google, working on novel attention mechanisms.
  • Jan CASPR: Combining Axes Preconditioners through Kronecker Approximation accepted to ICLR 2024!

2023

  • Sep Two papers accepted to NeurIPS 2023: SONew and Block Low-Rank Preconditioner with Shared Basis!

2021

  • Aug Started PhD in Computer Science at The University of Texas at Austin.

2019

  • May Received the Best B.Tech Thesis Award at IIT Kharagpur.

Preprints

LUCID
Sai Surya Duvvuri*, Nirmal Patel*, Nilesh Gupta, Inderjit S. Dhillon
Under review at ICML 2026
IHA
Interleaved Head Attention
Sai Surya Duvvuri*, Chanakya Ekbote*, Rachit Bansal, Rishabh Tiwari, Devvrit Khatri, David Brandfonbrener, Paul Liang, Inderjit S. Dhillon, Manzil Zaheer
Under review at ICML 2026
Adaptive Reg
Adaptive Regularization through Coupled Kronecker Factoring
Sai Surya Duvvuri, Cho-Jui Hsieh, Inderjit S. Dhillon
Under review at ICML 2026
Fast and Simplex
Aurko Roy, Timothy Chou, Sai Surya Duvvuri, Sijia Chen, Jiecao Yu, Xiaodong Wang, Manzil Zaheer, Rohan Anil
Preprint

Publications

2026

Scaling RL
Devvrit Khatri, Lovish Madaan, Rishabh Tiwari, Rachit Bansal, Sai Surya Duvvuri, Manzil Zaheer, Inderjit S. Dhillon, David Brandfonbrener, Rishabh Agarwal
ICLR 2026 Oral
TTT
Rachit Bansal, Aston Zhang, Rishabh Tiwari, Lovish Madaan, Sai Surya Duvvuri, Devvrit Khatri, David Brandfonbrener, David Alvarez-Melis, Prajjwal Bhargava, Mihir Sanjay Kale, Samy Jelassi
ICLR 2026

2025

LoRA RITE
Jui-Nan Yen, Si Si, Zhao Meng, Felix Yu, Sai Surya Duvvuri, Inderjit S. Dhillon, Cho-Jui Hsieh, Sanjiv Kumar
ICLR 2025 Oral
LASER
Sai Surya Duvvuri, Inderjit S. Dhillon
ICML 2025

2024

CASPR
Sai Surya Duvvuri, Fnu Devvrit, Rohan Anil, Cho-Jui Hsieh, Inderjit S. Dhillon
ICLR 2024

2023

SONew
Fnu Devvrit*, Sai Surya Duvvuri*, Rohan Anil, Vineet Gupta, Cho-Jui Hsieh, Inderjit S. Dhillon
NeurIPS 2023
Block LR
Jui-Nan Yen, Sai Surya Duvvuri, Inderjit S. Dhillon, Cho-Jui Hsieh
NeurIPS 2023

2020

iBox
Sachin Ashok, Sai Surya Duvvuri, Nagarajan Natarajan, Venkata N. Padmanabhan, Sundararajan Sellamanickam, Johannes Gehrke
ACM HotNets 2020

2019

Text Simplification
Sai Surya, Abhijit Mishra, Anirban Laha, Parag Jain, Karthik Sankaranarayanan
ACL 2019