Model Architecture
Co-designing algorithms and infrastructure for capable models that remain efficient at scale.
RESEARCH ENGINEER · TENCENT HY
Qian Dong董骞
Efficient models.
Scalable intelligence.
I work on model architecture at Tencent HY's Pretraining Team, exploring how to make capable models more efficient at scale. Ph.D. in Computer Science, Tsinghua University.
01 / ABOUT
A little about me
From research to real-world systems.
I am a Research Engineer on the Pretraining Team at Tencent HY, where I work on model architecture. I received my Ph.D. in Computer Science and Technology from Tsinghua University in June 2026. I was fortunate to be advised by Prof. Shaoping Ma, Prof. Yiqun Liu, and Prof. Qingyao Ai. My research focuses on efficient, scalable model architectures and the systems that make them practical.
02 / RESEARCH
Research interests
Making intelligence more efficient.
Efficient Attention
Sparse attention, linear attention, and architecture-level techniques for reducing inference cost.
Scalable Context
Extending useful context while balancing model quality, memory, throughput, and latency.
Low-Precision Training
Developing stable low-precision methods that reduce training cost while preserving model quality at scale.
03 / NEWS
Latest updates
Recent work and milestones.
Hy4 preview was released. As one of its model-architecture contributors, I helped bring our research IndexCache into production in Hy4, enabling a cost-efficient 1M-token context.
I received my Ph.D. in Computer Science and Technology from Tsinghua University.
IndexCache was released, accelerating sparse attention via cross-layer index reuse.
The GLM-5 Technical Report was released. I am one of the core contributors to model architecture.
Earlier updates 2025
GLM-4.7 was released. See our blog for details.
GLM-4.6 was released. See our blog for details.
The GLM-4.5 Technical Report was released. I contributed to sparse-attention adaptation during post-training.
04 / PUBLICATIONS
Selected publications
Ideas, experiments, and findings.
Work on model architecture, information retrieval, and language models.
IndexCache: Accelerating Sparse Attention via Cross-Layer Index Reuse
Accelerating sparse attention through cross-layer index reuse. Brought into production in Hy4.
Efficient attentionSelfRACG: Enabling LLMs to Self-Express and Retrieve for Code Generation
Enabling language models to self-express and retrieve for code generation.
Retrieval-augmented generationPrimary author 10 papers
-
SelfRACG: Enabling LLMs to Self-Express and Retrieve for Code Generation
-
Qilin: A Multimodal Information Retrieval Dataset with APP-level User Sessions
-
DecoupledRAG: An Efficient and Effective Retrieval Augmented Generation Framework via Cross Attention
-
Unsupervised Large Language Model Alignment for Information Retrieval via Contrastive Feedback
-
T²Ranking: A Large-scale Chinese Benchmark for Passage Ranking
-
I³Retriever: Incorporating Implicit Interaction in Pre-trained Language Models for Passage Retrieval
-
Incorporating Explicit Knowledge in Pre-trained Language Models for Passage Re-ranking
-
Disentangled Graph Recurrent Network for Document Ranking
-
Legal Judgment Prediction via Relational Learning
-
Latent Graph Recurrent Network for Document Ranking
Co-author 9 papers
-
IndexCache: Accelerating Sparse Attention via Cross-Layer Index Reuse
-
CalibraEval: Calibrating Prediction Distribution to Mitigate Selection Bias in LLMs-as-Judges
-
LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods
-
DELTA: Pre-train a Discriminative Encoder for Legal Case Retrieval via Structural Word Alignment
-
BLADE: Enhancing Black-box Large Language Models with Small Domain-Specific Models
-
SAILER: Structure-aware Pre-trained Language Model for Legal Case Retrieval
-
Layout-aware Webpage Quality Assessment
-
Incorporating Social-Aware User Preference for Video Recommendation
-
Emotion Recognition Based on Multi-View Body Gestures
05 / OFF THE CLOCK
Beyond research
A little curiosity outside of work.
Away from models and systems, I enjoy exploring craft beer — from crisp wheat beers and hop-forward IPAs to Belgian witbiers and saisons.