The following work received a spotlight at NeurIPS:

John Yang, Kilian Lieret, Carlos E. Jimenez, Alexander Wettig, Kabir Khandpur, Yanzhe Zhang, Binyuan Hui, Ofir Press, Ludwig Schmidt, Diyi Yang

Despite recent progress in Language Models (LMs) for software engineering, collecting training data remains a significant pain point. Existing datasets are small, with at most 1,000s of training instances from 11 or fewer GitHub repositories. The procedures to curate such datasets are often complex, necessitating hundreds of hours of human labor; companion execution environments also take up several terabytes of storage, severely limiting their scalability and usability. To address this pain point, we introduce SWE-smith, a novel pipeline for generating software engineering training data at scale. Given any Python codebase, SWE-smith constructs a corresponding execution environment, then automatically synthesizes 100s to 1,000s of task instances that break existing test(s) in the codebase. Using SWE-smith, we create a dataset of 50k instances sourced from 128 GitHub repositories, an order of magnitude larger than all previous works. We train SWE-agent-LM-32B, achieving 40.2% Pass@1 resolve rate on the SWE-bench Verified benchmark, state of the art among open source models. We open source SWE-smith (collection procedure, task instances, trajectories, models) to lower the barrier of entry for research in LM systems for automated software engineering. All assets available at swesmith.com.

https://arxiv.org/abs/2504.21798

Last month many members of the RSE Group and other Princeton colleagues attended USRSE’25, the third annual conference from US-RSE. Hosted this year in Philadelphia, the conference theme was “Code, Practices, and People.” Princeton University (authors in bold) contributions included:

1. Accelerating Research: Strategies from the Field – Jen Rosiere Reynolds, Lance Parsons, Gail Rosenbaum, Joost Wagenaar, and Sarah Stevens (BoF)
2. Sustainable Models of RSE Support: The Prospects of Centralization in Institutional Research – Eric Manning, Lori Bougher, Colin Swaney, and Sangyoon Park (BoF)
3. Undate: computing with uncertain and partially-unknown dates – Rebecca S. Koeser (notebook)
4. Integrating ATR Software with University HPC Infrastructure: balancing diverse compute needs – Christine Roughan and Rebecca S. Koeser (paper)
5. INnovative Training Enabled by a Research Software Engineering Community of Trainers (INTERSECT) – Jeffrey Carver and Ian Cosden (poster)
6. Building Scientific Python Packages – Henry Schreiner (poster)
7. Community Code Review in the Digital Humanities – Julia Damerow, Rebecca S. Koeser, Jeffrey C. Carver, and Malte Vogl (poster)
8. Surveying the Digital Humanities Research Software Engineering Landscape – Rebecca S. Koeser and Julia Damerow (poster)
9. Ten Simple Rules for Catalyzing Collaborations and Building Bridges between Research Software Engineers and Software Engineering Researchers – Nasir Eisty, Jeffrey Carver, Johanna Cohoon, Ian Cosden, Carole Goble, and Samuel Grayson (poster)
10. Developing a Machine Learning-Augmented Solver for the Hydrologic Model ParFlow – Georgios Artavanis, Laura Condon, Andrew Bennett, and Reed Maxwell (talk)
11. Everything, All at Once, Yesterday: Creating Research Software with Humanities Faculty – Jeri Wieringa and Mary Naydan (talk)
12. What happened to Curt’s arm? – Curt Hillegas (RAM)
13. Agile Foundations for RSEs: Building an AI Assistant with Agile – Tisha Charles and David Luet (workshop)

Additionally, Princeton University Professor Reed Maxwell delivered the first keynote address on Accelerating Continental-Scale Groundwater Simulation With a Fusion of Machine Learning, Integrated Hydrologic Models and Community Platforms. His keynote highlighted three of his lab’s software projects centered around hydrologic data, simulations, and visualizations, and he noted contributions to those projects from five current and past RSE Group members (Vineet Bansal, Calla Chenault, Georgios Artavanis, Amy Defnet, and Bill Hasling). Professor Maxwell stated that not only RSE contributions to software, but additionally that “RSEs enable digital education and outreach content.”

All in all, it was inspiring to convene with RSEs from all over the country. We already look forward to next year’s conference to be hosted in the San Francisco Bay Area!

John Yang, Kilian Lieret, Joyce Yang, Carlos E. Jimenez, Ofir Press, Ludwig Schmidt, Diyi Yang

Current benchmarks for coding evaluate language models (LMs) on concrete, well-specified tasks such as fixing specific bugs or writing targeted tests. However, human programmers do not spend all day incessantly addressing isolated tasks. Instead, real-world software development is grounded in the pursuit of high-level goals, like improving user retention or reducing costs. Evaluating whether LMs can also iteratively develop code to better accomplish open-ended objectives without any explicit guidance remains an open challenge. To address this, we introduce CodeClash, a benchmark where LMs compete in multi-round tournaments to build the best codebase for achieving a competitive objective. Each round proceeds in two phases: agents edit their code, then their codebases compete head-to-head in a code arena that determines winners based on objectives like score maximization, resource acquisition, or survival. Whether it’s writing notes, scrutinizing documentation, analyzing competition logs, or creating test suites, models must decide for themselves how to improve their codebases both absolutely and against their opponents. We run 1680 tournaments (25,200 rounds total) to evaluate 8 LMs across 6 arenas. Our results reveal that while models exhibit diverse development styles, they share fundamental limitations in strategic reasoning. Models also struggle with long-term codebase maintenance, as repositories become progressively messy and redundant. These limitations are stark: top models lose every round against expert human programmers. We open-source CodeClash to advance the study of autonomous, goal-oriented code development.

https://arxiv.org/abs/2511.00839

By Noam Razin, Zixuan Wang, Hubert Strauss, Stanley Wei, Jason D. Lee, Sanjeev Arora

The success of Reinforcement Learning from Human Feedback (RLHF) critically depends on the quality of the reward model. However, while this quality is primarily evaluated through accuracy, it remains unclear whether accuracy fully captures what makes a reward model an effective teacher. We address this question from an optimization perspective. First, we prove that regardless of how accurate a reward model is, if it induces low reward variance, then the RLHF objective suffers from a flat landscape. Consequently, even a perfectly accurate reward model can lead to extremely slow optimization, underperforming less accurate models that induce higher reward variance. We additionally show that a reward model that works well for one language model can induce low reward variance, and thus a flat objective landscape, for another. These results establish a fundamental limitation of evaluating reward models solely based on accuracy or independently of the language model they guide. Experiments using models of up to 8B parameters corroborate our theory, demonstrating the interplay between reward variance, accuracy, and reward maximization rate. Overall, our findings highlight that beyond accuracy, a reward model needs to induce sufficient variance for efficient~optimization.

Read the paper: https://arxiv.org/abs/2503.15477

By Luis Linares-Otoya, Jaden D. Shirkey, Bhuwan Khatri Chhetri, Amira Mira, Abhishek Biswas, Samuel L. Neff, Maria V. Linares-Otoya, Ye Chen, Julio V. Campos-Florian, Mayar L. Ganoza-Yupanqui, Philip D. Jeffrey, Frederick M. Hughson & Mohamed S. Donia

Considerable advances have been made in characterizing bioactive molecules secreted by bacteria, yet the regulatory elements controlling their production remain largely understudied. Here we identify and characterize the N-acyl-cyclolysine (ACL) system—a cell-density-dependent chemical signalling system specific to and widespread in the phylum Bacteroidota (formerly Bacteroidetes)—and show that it regulates the expression of co-localized operons encoding diverse secreted molecules. Using genetic and biochemical analyses, combined with structural studies of a key biosynthetic enzyme, AclA, we elucidate the molecular structure of various ACLs and their complete biosynthetic pathway involving l-lysine acylation and ATP-dependent cyclization. Furthermore, we find that secreted ACLs are sensed by a dedicated transcription factor, AclR, resulting in the expression of associated operons and the autoinduction of ACL biosynthesis. Moreover, we show that different Bacteroidota strains produce structurally diverse ACLs and encode transcription factors with varying ligand specificities. Finally, we find that the acl circuit is widely distributed and transcribed in human gut and oral microbiome samples, with clear evidence for an active role in regulating associated operons under host colonization conditions. Understanding the function of the ACL system in different contexts has the potential to reveal details about the biology, ecology and chemistry of the Bacteroidota and how members of this phylum interact with their environments and hosts.

Read the paper: https://www.nature.com/articles/s41586-025-09418-9

By Jiayu Zhang, Elizabeth C. Chavez, Melina Winkler, Jianche Liu, Sebastian Carver, Aaron E. Lin, Abhishek Biswas, Tomokazu Tamura, Anna Tseng, Danyang Wang, Aaron Benhamou, Aoife K. O’ Connell, Mao Matsuo, Jack E. Norton, Devin Kenney, Britt Adamson, Ralph E. Kleiner, Benjamin Burwitz, Nicholas A. Crossland, Florian Douam & Alexander Ploss

The live-attenuated yellow fever 17D vaccine strain differs genetically only minimally from its virulent parent. However, it remains unclear which sequence differences lead to virulence or attenuation. Here we demonstrate, using SHAPE-MaP, that these mutations do not induce global RNA structure changes and show that protein sequence mutations are mostly responsible for the phenotypic differences between 17D and virulent YFV. Using a highly modular, combinatorial genetic approach, we identified key mutations in the envelope (E) and non-structural 2A (NS2A) proteins that increase 17D’s ability to spread and enhance host antiviral responses. Introducing these mutations into infectious clones of virulent YFV genomes results in viral attenuation in vitro and in two mouse models. Collectively, our results define the genetic basis for 17D attenuation and highlight a potentially general approach for creating live-attenuated vaccines by introducing mutations resulting in similar phenotypic changes in other pathogenic viruses.

Read the paper: https://www.nature.com/articles/s41564-025-02047-y

By Sofia A. Quinodoz, Lifei Jiang, Aya A. Abu-Alfa, Troy J. Comi, Hongbo Zhao, Qiwei Yu, Lennard W. Wiesner, Jordy F. Botello, Anita Donlic, Elizabeth Soehalim, Prashant Bhat, Christiane Zorbas, Ludivine Wacheul, Andrej Košmrlj, Denis L. J. Lafontaine, Sebastian Klinge & Clifford P. Brangwynne

Biomolecular condensates are key features of intracellular compartmentalization. As the most prominent nuclear condensate in eukaryotes, the nucleolus is a multiphase liquid-like structure in which ribosomal RNAs (rRNAs) are transcribed and processed, undergoing multiple maturation steps to form the small (SSU) and large (LSU) ribosomal subunits. However, how rRNA processing is coupled to the layered organization of the nucleolus is poorly understood owing to a lack of tools to precisely monitor and perturb nucleolar rRNA processing dynamics. Here we developed two complementary approaches to spatiotemporally map rRNA processing and engineer de novo nucleoli. Using sequencing in parallel with imaging, we found that rRNA processing steps are spatially segregated, with sequential maturation of rRNA required for its outward movement through nucleolar phases. By generating synthetic nucleoli in cells using an engineered rDNA plasmid system, we show that defects in SSU processing can alter the ordering of nucleolar phases, resulting in inside-out nucleoli and preventing rRNA outflux, while LSU precursors are necessary to build the outermost layer of the nucleolus. These findings demonstrate how rRNA is both a scaffold and substrate for the nucleolus, with rRNA acting as a programmable blueprint for the multiphase architecture that facilitates assembly of an essential molecular machine.

Read the paper: https://www.nature.com/articles/s41586-025-09207-4

By Naman Agarwal, Xinyi Chen, Evan Dogariu, Devan Shah, Hubert Strauss, Vlad Feinberg, Daniel Suo, Peter Bartlett, Elad Hazan

We address the challenge of efficient auto-regressive generation in sequence prediction models by introducing FutureFill, a general-purpose fast generation method for any sequence prediction algorithm based on convolutional operators. FutureFill reduces generation time from quadratic to quasilinear in the context length. Moreover, when generating from a prompt, it requires a prefill cache whose size grows only with the number of tokens to be generated, often much smaller than the caches required by standard convolutional or attention based models. We validate our theoretical claims with experiments on synthetic tasks and demonstrate substantial efficiency gains when generating from a deep convolutional sequence prediction model.

Read the paper: https://arxiv.org/abs/2410.03766

We’re excited to announce the publication of a new paper, Ten Simple Rules for Catalyzing Collaborations and Building Bridges between Research Software Engineers (RSEs) and Software Engineering Researchers (SERs), authored by Nasir Eisty, Jeffrey Carver, Johanna Cohoon, Ian Cosden, Carole Goble, and Samuel Grayson.

Published in IEEE Computing in Science & Engineering (CiSE), this work emerged from discussions at a Dagstuhl Seminar and addresses a critical but often overlooked opportunity in the research software ecosystem: fostering collaboration between RSEs and SERs.

While both communities share a passion for improving software in research, they often operate in distinct environments, with different vocabularies, incentives, and expectations. This paper offers ten actionable rules designed to bridge those gaps, encouraging meaningful, sustained partnerships that combine practical experience with theoretical insight.

By working together, RSEs and SERs can drive innovation in tools, practices, and infrastructure, ultimately advancing the quality and impact of scientific research.

Read the preprint: https://arxiv.org/abs/2506.03012

Published version: https://ieeexplore.ieee.org/document/11003859