Model Architecture
Co-designing algorithms and infrastructure for capable models that remain efficient at scale.
~/qian-dong $
Qian Dong 董骞
>Research Engineer @ Tencent HY_
Research Engineer on the Scaling Team at Tencent HY, working on model architecture. I received my Ph.D. in Computer Science and Technology from Tsinghua University.
// 01 about
I am a Research Engineer on the Scaling Team at Tencent HY, where I work on model architecture. I received my Ph.D. in Computer Science and Technology from Tsinghua University in June 2026. I was fortunate to be advised by Prof. Shaoping Ma, Prof. Yiqun Liu, and Prof. Qingyao Ai. My research focuses on efficient, scalable model architectures and the systems that make them practical.
// 02 research
Efficient Attention
Sparse attention, linear attention, and architecture-level techniques for reducing inference cost.
Scalable Context
Extending useful context while balancing model quality, memory, throughput, and latency.
Low-Precision Training
Developing stable low-precision methods that reduce training cost while preserving model quality at scale.
// 03 news
- 2026.08
Hy4 preview was released. As one of its model-architecture contributors, I helped bring our research IndexCache into production in Hy4, enabling a cost-efficient 1M-token context.
- 2026.06
I received my Ph.D. in Computer Science and Technology from Tsinghua University.
- 2026.03
IndexCache was released, accelerating sparse attention via cross-layer index reuse.
- 2026.02
The GLM-5 Technical Report was released. I am one of the core contributors to model architecture.
- 2025.12
GLM-4.7 was released. See our blog for details.
- 2025.09
GLM-4.6 was released. See our blog for details.
- 2025.08
The GLM-4.5 Technical Report was released. I contributed to sparse-attention adaptation during post-training.
// 04 publications
Full publication list on Google Scholar
Peer-reviewed work across model architecture, retrieval, RAG, and language models.
selected papers · primary author (10)
-
SelfRACG: Enabling LLMs to Self-Express and Retrieve for Code Generation
-
Qilin: A Multimodal Information Retrieval Dataset with APP-level User Sessions
-
DecoupledRAG: An Efficient and Effective Retrieval Augmented Generation Framework via Cross Attention
-
Unsupervised Large Language Model Alignment for Information Retrieval via Contrastive Feedback
-
T²Ranking: A Large-scale Chinese Benchmark for Passage Ranking
-
I³Retriever: Incorporating Implicit Interaction in Pre-trained Language Models for Passage Retrieval
-
Incorporating Explicit Knowledge in Pre-trained Language Models for Passage Re-ranking
-
Disentangled Graph Recurrent Network for Document Ranking
-
Legal Judgment Prediction via Relational Learning
-
Latent Graph Recurrent Network for Document Ranking
selected papers · co-author (9)
-
IndexCache: Accelerating Sparse Attention via Cross-Layer Index Reuse
-
CalibraEval: Calibrating Prediction Distribution to Mitigate Selection Bias in LLMs-as-Judges
-
LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods
-
DELTA: Pre-train a Discriminative Encoder for Legal Case Retrieval via Structural Word Alignment
-
BLADE: Enhancing Black-box Large Language Models with Small Domain-Specific Models
-
SAILER: Structure-aware Pre-trained Language Model for Legal Case Retrieval
-
Layout-aware Webpage Quality Assessment
-
Incorporating Social-Aware User Preference for Video Recommendation
-
Emotion Recognition Based on Multi-View Body Gestures
// 05 beyond research
Away from models and systems, I enjoy exploring craft beer — from crisp wheat beers and hop-forward IPAs to Belgian witbiers and saisons. 🍻