publications

2026

1. 

   FrontierSmith: Synthesizing Open-Ended Coding Problems at Scale

   Runyuan He^*, Qiuyang Mang^*, Shang Zhou, Kaiyuan Liu, Hanchen Li, Huanzhi Mao, and 11 more authors

   NeurIPS 2026 Spotlight (0.95%)

   A system that synthesizes open-ended coding problems, test cases, and verifiers for training and evaluating language models.

   Abs arXiv Bib

   Accepted as a Spotlight at NeurIPS 2026 (0.95%).

   Open-ended coding remains a weak spot for LLMs, largely because open-ended training problems are scarce. FrontierSmith is an automated system that synthesizes such problems by evolving competitive programming tasks through modifying goals, restricting outputs, and generalizing inputs. The system employs a quantitative idea divergence metric to select problems that elicit genuinely diverse approaches, and generates corresponding test cases and verifiers. Models trained on FrontierSmith data show significant improvements on FrontierCS and ALE-bench benchmarks.

   @inproceedings{he2026frontiersmith,
     title = {FrontierSmith: Synthesizing Open-Ended Coding Problems at Scale},
     author = {He, Runyuan and Mang, Qiuyang and Zhou, Shang and Liu, Kaiyuan and Li, Hanchen and Mao, Huanzhi and Zhang, Qizheng and Li, Zerui and Peng, Bo and Cheng, Lufeng and Fu, Tianfu and Wang, Yichuan and Chai, Wenhao and Shang, Jingbo and Dimakis, Alex and Gonzalez, Joseph E and Cheung, Alvin},
     booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
     year = {2026},
   }

2. 

   Combee: Scaling Prompt Learning for Self-Improving Language Model Agents

   Hanchen Li^*, Runyuan He^*, Qizheng Zhang, Changxiu Ji, Qiuyang Mang, Xiaokun Chen, and 8 more authors

   COLM 2026

   A framework for parallel prompt learning that helps language model agents improve from their executions.

   Abs arXiv Bib

   We present Combee, a framework for scaling prompt learning for self-improving language model agents. Existing prompt learning methods struggle in parallel settings, experiencing quality degradation when learning from multiple agent executions simultaneously. Combee leverages parallel scans and an augmented shuffle mechanism along with dynamic batch size control to maintain quality while processing many agents concurrently, achieving up to 17x speedup over previous methods with comparable or better accuracy.

   @inproceedings{li2026combee,
     title = {Combee: Scaling Prompt Learning for Self-Improving Language Model Agents},
     author = {Li, Hanchen and He, Runyuan and Zhang, Qizheng and Ji, Changxiu and Mang, Qiuyang and Chen, Xiaokun and Agrawal, Lakshya A and Liao, Wei-Liang and Yang, Eric and Cheung, Alvin and Zou, James and Olukotun, Kunle and Stoica, Ion and Gonzalez, Joseph E},
     booktitle = {Conference on Language Modeling (COLM)},
     year = {2026},
   }

3. 

   Continuum: Efficient and Robust Multi-Turn LLM Agent Scheduling with KV Cache Time-to-Live

   Hanchen Li^*, Runyuan He^*, Qiuyang Mang^*, Qizheng Zhang, Huanzhi Mao, Xiaokun Chen, and 4 more authors

   ICLR 2026 LLA Workshop

   A serving system that schedules multi-turn LLM agents using time-to-live policies for KV caches.

   Abs arXiv Bib

   We present Continuum, a serving system to optimize job completion time for multi-turn agent workloads by introducing time-to-live mechanism for KV cache retaining. For LLM request that generates a tool call, Continuum selectively pins the KV cache in GPU memory with a time-to-live value determined by considering both the reload cost and ordering preserve benefit of retaining KV cache.

   @inproceedings{li2025continuum,
     title = {Continuum: Efficient and Robust Multi-Turn LLM Agent Scheduling with KV Cache Time-to-Live},
     author = {Li, Hanchen and He, Runyuan and Mang, Qiuyang and Zhang, Qizheng and Mao, Huanzhi and Chen, Xiaokun and Zhou, Hangrui and Cheung, Alvin and Gonzalez, Joseph and Stoica, Ion},
     booktitle = {ICLR LLA Workshop},
     year = {2026},
   }

4. 

   Automated Discovery of Test Oracles for Database Management Systems Using LLMs

   Qiuyang Mang, Runyuan He, Suyang Zhong, Xiaoxuan Liu, Huanchen Zhang, and Alvin Cheung

   SIGMOD 2026

   A database testing framework that uses LLMs and formal verification to discover sound test oracles.

   Abs DOI arXiv Bib

   This paper explores the use of large language models (LLMs) to automate the discovery and instantiation of test oracles, addressing a long-standing bottleneck towards fully automated DBMS testing. We introduce Argus, a novel framework built upon the core concept of the Constrained Abstract Query. Argus uses LLMs to generate pairs of SQL skeletons that are asserted to be semantically equivalent, then formally proven using a SQL equivalence solver to ensure soundness.

   @article{mang2025argus,
     title = {Automated Discovery of Test Oracles for Database Management Systems Using LLMs},
     author = {Mang, Qiuyang and He, Runyuan and Zhong, Suyang and Liu, Xiaoxuan and Zhang, Huanchen and Cheung, Alvin},
     journal = {Proceedings of the ACM on Management of Data (SIGMOD)},
     volume = {4},
     number = {3},
     pages = {Article 140},
     year = {2026},
     doi = {10.1145/3802017},
   }

5. 

   FrontierCS: Evolving Challenges for Evolving Intelligence

   Qiuyang Mang^*, Wenhao Chai^*, Zhifei Li^*, Huanzhi Mao^*, Shang Zhou^*, Alexander Du^*, and 18 more authors

   ICML 2026

   A benchmark of open-ended computer science problems for evaluating frontier language models.

   Abs arXiv Bib

   We introduce FrontierCS, a challenging benchmark to test the CS research knowledge of frontier LLMs. FrontierCS includes 1,092 problems hand-written by domain experts, with 71.2% contributed by Turing Award and Best Paper Award winners.

   @inproceedings{mang2025frontiercs,
     title = {FrontierCS: Evolving Challenges for Evolving Intelligence},
     author = {Mang, Qiuyang and Chai, Wenhao and Li, Zhifei and Mao, Huanzhi and Zhou, Shang and Du, Alexander and Li, Hanchen and Liu, Shu and Chen, Edwin and Wang, Yichuan and Chu, Xieting and Cheng, Zerui and Xu, Yuan and Xia, Tian and Wang, Zirui and Shi, Tianneng and Yao, Jianzhu and Zhao, Yilong and Zhang, Qizheng and Ruan, Charlie and Shen, Zeyu and Liu, Kaiyuan and He, Runyuan and others},
     booktitle = {International Conference on Machine Learning (ICML)},
     year = {2026},
   }

6. 

   Playing Psychic: Using Thought Trees to Predict Reasoning Models Accuracy on Coding Tasks

   Jiaxin Fang, Runyuan He, Sahil Bhatia, Neel Gajare, and Alvin Cheung

   COLM 2026 ER Workshop

   A study of how reasoning trace structure can predict correctness on coding tasks.

   Abs arXiv Bib

   We investigate how the structure of a reasoning trace, not just its contents, is a strong predictor of correctness in reasoning models on coding tasks. We propose using structured thought-trees to represent reasoning processes and train a lightweight classifier to identify trace correctness. By flagging and retrying structurally anomalous traces, we demonstrate consistent performance improvements across coding benchmarks.

   @inproceedings{fang2026playing,
     title = {Playing Psychic: Using Thought Trees to Predict Reasoning Models Accuracy on Coding Tasks},
     author = {Fang, Jiaxin and He, Runyuan and Bhatia, Sahil and Gajare, Neel and Cheung, Alvin},
     booktitle = {COLM ER Workshop},
     year = {2026},
   }

7. 

   PLOP: Cost-Based Placement of Semantic Operators in Hybrid Query Plans

   Qiuyang Mang^*, Yufan Xiang^*, Hangrui Zhou, Runyuan He, Jiaxiang Yu, Hanchen Li, and 2 more authors

   Preprint 2026

   A cost-based optimizer for placing LLM-powered semantic operators in hybrid database query plans.

   Abs arXiv Bib

   We present PLOP, an optimizer for hybrid database query plans that combine semantic operators powered by LLMs with traditional relational operators. The core challenge is determining where to place each semantic operator relative to relational operators: earlier placement reduces downstream data volume but increases LLM calls, while later placement minimizes LLM invocations but increases relational processing costs. PLOP employs dynamic programming to find optimal placements, achieving up to 1.5x speedup and 4.29x cost reduction while maintaining high output quality.

   @article{mang2026plop,
     title = {PLOP: Cost-Based Placement of Semantic Operators in Hybrid Query Plans},
     author = {Mang, Qiuyang and Xiang, Yufan and Zhou, Hangrui and He, Runyuan and Yu, Jiaxiang and Li, Hanchen and Parameswaran, Aditya and Cheung, Alvin},
     journal = {arXiv preprint arXiv:2604.09944},
     year = {2026},
   }

2025

1. 

   In-depth Analysis of Graph-based RAG in a Unified Framework

   Yingli Zhou, Yaodong Su, Youran Sun, Shu Wang, Taotao Wang, Runyuan He, and 5 more authors

   VLDB 2025

   A unified framework for comparing graph-based retrieval-augmented generation methods.

   Abs DOI arXiv Bib

   This paper presents a comprehensive analysis of Graph-based Retrieval-Augmented Generation (RAG) systems within a unified framework.

   @article{zhou2025depth,
     title = {In-depth Analysis of Graph-based RAG in a Unified Framework},
     author = {Zhou, Yingli and Su, Yaodong and Sun, Youran and Wang, Shu and Wang, Taotao and He, Runyuan and Zhang, Yongwei and Liang, Sicong and Liu, Xilin and Ma, Yuchi and Fang, Yixiang},
     journal = {Proceedings of the VLDB Endowment (PVLDB)},
     volume = {18},
     number = {13},
     year = {2025},
     doi = {10.14778/3773731.3773738},
   }