I am a Ph.D. student at Shanghai Jiao Tong University, expected to graduate in June 2027. I received my B.Eng. degree from the University of Electronic Science and Technology of China in June 2022.

My research focuses on LLM Pre-training, Scaling Laws, Data Strategy, and Model Efficiency. I am particularly interested in understanding how language models learn from data at the token level during pre-training, and in turning these insights into more efficient training strategies. I also work on model pruning, multimodal reasoning, and AI for healthcare. My work has been published at NeurIPS, ICML, ICLR, IEEE JSTSP, ACL, EMNLP, COLING, ICASSP.

I am currently a research intern with the scaling ladder construction at Hunyuan Team, Tencent. Previously, I was a research intern at Shanghai AI Laboratory and Dots Studio, Xiaohongshu Inc.

I am actively seeking for full-time opportunities. Please feel free to reach out regarding potential openings!

Email / LinkedIn / WeChat

πŸ”₯ News

  • 2026.11: πŸŽ‰ The interactive Project Page of Stepwise Token Learning is live, with Code, Data, and Colab. Come and play!
  • 2026.10: πŸŽ‰ Honored to serve as a NeurIPS 2026 Top Reviewer!
  • 2026.09: πŸŽ‰ Our work Stepwise Token Learning is accepted by NeurIPS 2026 Spotlight (0.95%)!
  • 2026.06: πŸŽ‰ Honored to serve as an ICML 2026 Gold Reviewer!
  • 2026.05: πŸŽ‰ Our work NTK-Selector is accepted by ICML 2026!
  • 2026.04: πŸŽ‰ Our two works GuessBench and CrossOmni are accepted by ACL 2026 Main!
  • 2026.04: πŸŽ‰ Our work AnchorNet is accepted by IEEE JSTSP!
  • 2025.11: πŸŽ‰ Our two works DSVD and D2R are accepted by EMNLP 2025 Main and EMNLP 2025 Findings, respectively!
  • 2025.07: πŸŽ‰ Our work MedOmniKB is accepted by ACL 2025 Main!
  • 2025.05: πŸŽ‰ Our work DiSF is accepted by ICLR 2025 Oral!
  • 2025.05: πŸŽ‰ Our work LIAR is accepted by IEEE JSTSP!
  • 2025.03: πŸŽ‰ Our work AuscMLLM is accepted by ICASSP 2025 Oral!
  • 2024.05: πŸŽ‰ Our two works P-pruning and CE-VDG are accepted by COLING 2024!
  • 2024.04: πŸŽ‰ Our work MSG-BART is accepted by ICASSP 2024!

πŸ’Ό Internships

  • Tencent Β· Hunyuan2026.8 - Present
  • Xiaohongshu Inc. Β· Dots2025.12 - 2026.8
  • Shanghai Artificial Intelligence Laboratory Β· Frontier2023.12 - 2025.10

πŸ“ Selected Publications

* Equal contribution.

2026

Under Review
PAIR-19M continued pre-training accuracy versus the candidate corpus

Small Models Know Which Data Scales

Pingjie Wang, Zechen Hu, Peiru Yang, Zixuan Gu, Qi Fu Xianmeng Zhang, Yu Wang.

Under Review

We propose PAIR, which uses paired lightweight language models to reconstruct a target-aligned training distribution, and show that data selected by a 19M scorer yields a 3.9Γ— data-efficiency speedup that persists from 1e18 to 1e21 FLOPs.

NeurIPS 2026 Spotlight
Token-level learning trajectories and learning-time spectrum

Smooth Scaling Laws Hide Stepwise Token Learning

Pingjie Wang*, Zechen Hu*, Peiru Yang*, Fu Guo, Debing Zhang

NeurIPS 2026 Spotlight (0.95%)

[Project] [Paper] [Code]

We show that smooth scaling laws emerge from localized token-level learning events and use the learning-time spectrum to explain and improve training dynamics.

ICML 2026
NTK-Selector results across low-resource domains

Mining Useful General Data for Low-Resource Domain Adaptation

Pingjie Wang*, Hongcheng Liu*, Yusheng Liao, Ziqing Fan, Yaxin Du, Shuo Tang, Yanfeng Wang, Yu Wang

ICML 2026

[Paper] [Code]

We introduce NTK-Selector, a Jacobian-free NTK approximation that mines general-domain samples useful for adapting LLMs to low-resource domains.

ACL 2026 Main
GuessBench and GuessAgent overview

When Seeing Is not Enough: Revealing the Limits of Active Reasoning in MLLMs

Hongcheng Liu*, Pingjie Wang*, Yuhao Wang, Siqu Ou, Yanfeng Wang, Yu Wang

ACL 2026 Main Conference

[Paper]

We introduce GuessBench to evaluate active evidence acquisition in MLLMs and identify fine-grained perception and timely decision-making as key bottlenecks.

  • IEEE JSTSP 2026 M2K-VDG: Model-Adaptive Multimodal Knowledge Anchor Enhanced Video-grounded Dialogue Generation. Hongcheng Liu, Pingjie Wang, Yu Wang, Yanfeng Wang. IEEE JSTSP 2026. [Paper]

2025

IEEE JSTSP 2025
LIAR reconstruction framework

Reconstruct the Pruned Model Without Retraining

Pingjie Wang, Ziqing Fan, Shengchao Hu, Zhe Chen, Yanfeng Wang, Yu Wang

IEEE JSTSP 2025

[Paper] [Code]

We introduce LIAR, a backpropagation-free linear-interpolation framework that reconstructs pruned modules across encoder- and decoder-based models.

ICLR 2025 Oral
Feature-space coverage of diversified file selection methods

Combatting Dimensional Collapse in LLM Pre-Training Data via Diversified File Selection

Ziqing Fan, Siyuan Du, Shengchao Hu, Pingjie Wang, Li Shen, Ya Zhang, Dacheng Tao, Yanfeng Wang

ICLR 2025 Oral

[Paper] [Code]

We introduce DiSF, a diversified file-selection method that mitigates feature-space dimensional collapse and substantially improves LLM pre-training data efficiency.

  • ICASSP 2025 Oral AuscMLLM: Bridging Classification and Reasoning in Heart Sound Analysis with a Multimodal Large Language Model. Zihan Zhao*, Pingjie Wang*, Liudan Zhao, et al. ICASSP 2025 Oral [Paper]
arXiv 2024
AuscultaBase evaluation tasks and clinical applications

A Generalist Audio Foundation Model for Comprehensive Body Sound Auscultation

Pingjie Wang, Liudan Zhao, Zihan Zhao, Miao He, Xin Sun, Ya Zhang, Kun Sun, Yanfeng Wang, Yu Wang

arXiv 2024

[Paper] [Code]

We introduce AuscultaBase and AuscultaBench for scalable body-sound analysis across abnormality detection, disease diagnosis, and activity recognition.

2024

COLING 2024 Oral
P-pruning framework

Pruning before Fine-tuning: A Retraining-free Compression Framework for Pre-trained Language Models

Pingjie Wang, Hongcheng Liu, Yanfeng Wang, Yu Wang

COLING 2024 Oral

[Paper] [Code]

We introduce P-pruning, which removes redundant modules before fine-tuning through module clustering and centroid selection, reducing fine-tuning cost without retraining.

  • EMNLP 2024 HSDreport: Heart Sound Diagnosis with Echocardiography Reports. Zihan Zhao*, Pingjie Wang*, Liudan Zhao, et al. EMNLP 2024 Findings [Paper]

  • COLING 2024 CE-VDG: Counterfactual Entropy-based Bias Reduction for Video-grounded Dialogue Generation. Hongcheng Liu, Pingjie Wang, Zhiyuan Zhu, Yanfeng Wang, Yu Wang. COLING 2024. [Paper]

πŸŽ“ Education

  • 2022.09 – 2027.06, Ph.D., Shanghai Jiao Tong University, Shanghai.
  • 2018.09 – 2022.06, Undergraduate, University of Electronic Science and Technology of China, Chengdu.

πŸ§‘β€βš–οΈ Academic Service

  • NeurIPS 2026 Top Reviewer.
  • ICML 2026 Gold Reviewer.
  • Invited reviewer for NeurIPS, ICML, ICLR, ACL, EMNLP, AAAI, IEEE TPAMI, and IEEE JSTSP.