I am a Ph.D. student at Shanghai Jiao Tong University, expected to graduate in June 2027. I received my B.Eng. degree from the University of Electronic Science and Technology of China in June 2022.
My research focuses on LLM Pre-training, Scaling Laws, Data Strategy, and Model Efficiency. I am particularly interested in understanding how language models learn from data at the token level during pre-training, and in turning these insights into more efficient training strategies. I also work on model pruning, multimodal reasoning, and AI for healthcare. My work has been published at NeurIPS, ICML, ICLR, IEEE JSTSP, ACL, EMNLP, COLING, ICASSP.
I am currently a research intern with the scaling ladder construction at Hunyuan Team, Tencent. Previously, I was a research intern at Shanghai AI Laboratory and Dots Studio, Xiaohongshu Inc.
I am actively seeking for full-time opportunities. Please feel free to reach out regarding potential openings!
π₯ News
- 2026.11: π The interactive
Project Page of Stepwise Token Learning is live, with Code, Data, and Colab. Come and play!
- 2026.10: π Honored to serve as a NeurIPS 2026 Top Reviewer!
- 2026.09: π Our work Stepwise Token Learning is accepted by NeurIPS 2026 Spotlight (0.95%)!
- 2026.06: π Honored to serve as an ICML 2026 Gold Reviewer!
- 2026.05: π Our work NTK-Selector is accepted by ICML 2026!
- 2026.04: π Our two works GuessBench and CrossOmni are accepted by ACL 2026 Main!
- 2026.04: π Our work AnchorNet is accepted by IEEE JSTSP!
- 2025.11: π Our two works DSVD and D2R are accepted by EMNLP 2025 Main and EMNLP 2025 Findings, respectively!
- 2025.07: π Our work MedOmniKB is accepted by ACL 2025 Main!
- 2025.05: π Our work DiSF is accepted by ICLR 2025 Oral!
- 2025.05: π Our work LIAR is accepted by IEEE JSTSP!
- 2025.03: π Our work AuscMLLM is accepted by ICASSP 2025 Oral!
- 2024.05: π Our two works P-pruning and CE-VDG are accepted by COLING 2024!
- 2024.04: π Our work MSG-BART is accepted by ICASSP 2024!
πΌ Internships
- Tencent Β· Hunyuan2026.8 - Present
- Xiaohongshu Inc. Β· Dots2025.12 - 2026.8
- Shanghai Artificial Intelligence Laboratory Β· Frontier2023.12 - 2025.10
π Selected Publications
* Equal contribution.
2026

Small Models Know Which Data Scales
Pingjie Wang, Zechen Hu, Peiru Yang, Zixuan Gu, Qi Fu Xianmeng Zhang, Yu Wang.
Under Review
We propose PAIR, which uses paired lightweight language models to reconstruct a target-aligned training distribution, and show that data selected by a 19M scorer yields a 3.9Γ data-efficiency speedup that persists from 1e18 to 1e21 FLOPs.

Smooth Scaling Laws Hide Stepwise Token Learning
Pingjie Wang*, Zechen Hu*, Peiru Yang*, Fu Guo, Debing Zhang
NeurIPS 2026 Spotlight (0.95%)
We show that smooth scaling laws emerge from localized token-level learning events and use the learning-time spectrum to explain and improve training dynamics.

Mining Useful General Data for Low-Resource Domain Adaptation
Pingjie Wang*, Hongcheng Liu*, Yusheng Liao, Ziqing Fan, Yaxin Du, Shuo Tang, Yanfeng Wang, Yu Wang
ICML 2026
We introduce NTK-Selector, a Jacobian-free NTK approximation that mines general-domain samples useful for adapting LLMs to low-resource domains.

When Seeing Is not Enough: Revealing the Limits of Active Reasoning in MLLMs
Hongcheng Liu*, Pingjie Wang*, Yuhao Wang, Siqu Ou, Yanfeng Wang, Yu Wang
ACL 2026 Main Conference
We introduce GuessBench to evaluate active evidence acquisition in MLLMs and identify fine-grained perception and timely decision-making as key bottlenecks.
IEEE JSTSP 2026M2K-VDG: Model-Adaptive Multimodal Knowledge Anchor Enhanced Video-grounded Dialogue Generation. Hongcheng Liu, Pingjie Wang, Yu Wang, Yanfeng Wang. IEEE JSTSP 2026. [Paper]
2025


Combatting Dimensional Collapse in LLM Pre-Training Data via Diversified File Selection
Ziqing Fan, Siyuan Du, Shengchao Hu, Pingjie Wang, Li Shen, Ya Zhang, Dacheng Tao, Yanfeng Wang
ICLR 2025 Oral
We introduce DiSF, a diversified file-selection method that mitigates feature-space dimensional collapse and substantially improves LLM pre-training data efficiency.
ICASSP 2025 OralAuscMLLM: Bridging Classification and Reasoning in Heart Sound Analysis with a Multimodal Large Language Model. Zihan Zhao*, Pingjie Wang*, Liudan Zhao, et al. ICASSP 2025 Oral [Paper]

A Generalist Audio Foundation Model for Comprehensive Body Sound Auscultation
Pingjie Wang, Liudan Zhao, Zihan Zhao, Miao He, Xin Sun, Ya Zhang, Kun Sun, Yanfeng Wang, Yu Wang
arXiv 2024
We introduce AuscultaBase and AuscultaBench for scalable body-sound analysis across abnormality detection, disease diagnosis, and activity recognition.
2024

Pruning before Fine-tuning: A Retraining-free Compression Framework for Pre-trained Language Models
Pingjie Wang, Hongcheng Liu, Yanfeng Wang, Yu Wang
COLING 2024 Oral
We introduce P-pruning, which removes redundant modules before fine-tuning through module clustering and centroid selection, reducing fine-tuning cost without retraining.
-
EMNLP 2024HSDreport: Heart Sound Diagnosis with Echocardiography Reports. Zihan Zhao*, Pingjie Wang*, Liudan Zhao, et al. EMNLP 2024 Findings [Paper] -
COLING 2024CE-VDG: Counterfactual Entropy-based Bias Reduction for Video-grounded Dialogue Generation. Hongcheng Liu, Pingjie Wang, Zhiyuan Zhu, Yanfeng Wang, Yu Wang. COLING 2024. [Paper]
π Education
- 2022.09 β 2027.06, Ph.D., Shanghai Jiao Tong University, Shanghai.
- 2018.09 β 2022.06, Undergraduate, University of Electronic Science and Technology of China, Chengdu.
π§ββοΈ Academic Service
- NeurIPS 2026 Top Reviewer.
- ICML 2026 Gold Reviewer.
- Invited reviewer for NeurIPS, ICML, ICLR, ACL, EMNLP, AAAI, IEEE TPAMI, and IEEE JSTSP.