Model Architecture
Co-designing algorithms and infrastructure for capable models that remain efficient at scale.
~/qian-dong $
Qian Dong 董骞
>Ph.D. @ Tsinghua_
Ph.D. in Computer Science and Technology from Tsinghua University (THUIR). I work on algo & infra co-designed model architectures for scalable parameters, context, and especially intelligence.
// 01 about
I received my Ph.D. in Computer Science and Technology from Tsinghua University in June 2026, where I was a member of THUIR. I was fortunate to be advised by Prof. Shaoping Ma, Prof. Yiqun Liu, and Prof. Qingyao Ai. My research focuses on efficient, scalable model architectures and the systems that make them practical.
// 02 research
Efficient Attention
Sparse attention, index reuse, and architecture-level techniques for reducing inference cost.
Scalable Context
Extending useful context while balancing model quality, memory, throughput, and latency.
Information Retrieval
Ranking and retrieval-augmented generation that connect models with the right information.
// 03 news
- 2026.06
I received my Ph.D. in Computer Science and Technology from Tsinghua University.
- 2026.03
IndexCache was released, accelerating sparse attention via cross-layer index reuse.
- 2026.02
The GLM-5 Technical Report was released. I am one of the core contributors to model architecture.
- 2025.12
GLM-4.7 was released. See our blog for details.
- 2025.11
SelfRACG was accepted to EMNLP 2025, enabling LLMs to self-express retrieval queries for better code generation.
- 2025.09
GLM-4.6 was released. See our blog for details.
- 2025.08
The GLM-4.5 Technical Report was released. I contributed to sparse-attention adaptation during post-training.
- 2025.07
Qilin was accepted to SIGIR 2025, introducing a multimodal IR dataset with app-level user sessions.
- 2025.04
DecoupledRAG was accepted to WWW 2025, decoupling context and knowledge via cross-attention for efficient RAG.
- 2024.07
RLCF was accepted to SIGIR 2024, aligning LLMs for information retrieval through unsupervised contrastive feedback.
- 2023.10
I³Retriever was accepted to CIKM 2023, incorporating implicit query-document interaction into retrievers.
- 2023.07
T²Ranking was accepted to SIGIR 2023 as a large-scale Chinese passage-ranking benchmark.
- 2022.07
KERM was accepted to SIGIR 2022, incorporating explicit knowledge into pre-trained models for passage re-ranking.
- 2022.02
DGRe was published in Data Science and Engineering.
- 2021.07
R-FORMER was accepted to SIGIR 2021.
- 2021.04
LGRe was accepted to DASFAA 2021.
// 04 publications
Full publication list on Google Scholar
Peer-reviewed work across model architecture, retrieval, RAG, and language models.
selected papers · primary author (10)
-
SelfRACG: Enabling LLMs to Self-Express and Retrieve for Code Generation
-
Qilin: A Multimodal Information Retrieval Dataset with APP-level User Sessions
-
DecoupledRAG: An Efficient and Effective Retrieval Augmented Generation Framework via Cross Attention
-
Unsupervised Large Language Model Alignment for Information Retrieval via Contrastive Feedback
-
T²Ranking: A Large-scale Chinese Benchmark for Passage Ranking
-
I³Retriever: Incorporating Implicit Interaction in Pre-trained Language Models for Passage Retrieval
-
Incorporating Explicit Knowledge in Pre-trained Language Models for Passage Re-ranking
-
Disentangled Graph Recurrent Network for Document Ranking
-
Legal Judgment Prediction via Relational Learning
-
Latent Graph Recurrent Network for Document Ranking
selected papers · co-author (9)
-
IndexCache: Accelerating Sparse Attention via Cross-Layer Index Reuse
-
CalibraEval: Calibrating Prediction Distribution to Mitigate Selection Bias in LLMs-as-Judges
-
LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods
-
DELTA: Pre-train a Discriminative Encoder for Legal Case Retrieval via Structural Word Alignment
-
BLADE: Enhancing Black-box Large Language Models with Small Domain-Specific Models
-
SAILER: Structure-aware Pre-trained Language Model for Legal Case Retrieval
-
Layout-aware Webpage Quality Assessment
-
Incorporating Social-Aware User Preference for Video Recommendation
-
Emotion Recognition Based on Multi-View Body Gestures
// 05 education
-
2022 — 2026
Tsinghua University Ph.D.
Computer Science and Technology · THUIR
-
2019 — 2022
Chinese Academy of Sciences M.Eng.
Institute of Software
-
2015 — 2019
South China University of Technology SCUT B.Eng.
Software Engineering
// 06 honors & awards
// 07 beyond research
Away from models and systems, I enjoy exploring craft beer — from crisp wheat beers and hop-forward IPAs to Belgian witbiers and saisons. 🍻