Can Models Learn Skill Composition from Examples?
Haoyu Zhao, Simran Kaur, Dingli Yu, Anirudh Goyal, Sanjeev Arora Conference on Neural Information Processing Systems ( NeurIPS 2024 )
CV
Education
- Ph.D. Student, Princeton University, 2026 (Expected)
- Computer Science Department
- B.E. in Computer Science and Technology, Tsinghua University, 2020
- Institute for Interdisciplinary Information Sciences
- GPA: 3.86, Rank: 5/46
Experiences
- Undergraduate Research Assistant Sept. 2019 - Dec. 2019
- Tsinghua University
- Supervisor: Professor Yang Yuan
- Research Intern Sept. 2019 - Dec. 2019
- Microsoft Research Asia
- Supervisor: Professor Wei Chen
- Research Intern Feb. 2019 - July 2019
- Duke University
- Supervisor: Professor Rong Ge
- Research Intern July 2018 - Nov. 2018
- Microsoft Research Asia
- Supervisor: Professor Wei Chen
- Undergraduate Research Assistant May 2018 - Jan. 2019
- Tsinghua University
- Supervisor: Professor Jian Li
Awards
- Yao Award (Bronze Medal) Sept. 2019
Teaching
- Teaching Assistant for Economics and Computing (COS 445) at Princeton Spring 2022
- Teaching Assistant for Advanced Algorithm Design (COS 521) at Princeton Fall 2022
Service
Reviewer of ICML, NeurIPS, ICLR, and AISTATS since 2020.
- Highlighted Reviewer of ICLR 2022 April 2022
Publications
Keeping LLMs Aligned After Fine-tuning: The Crucial Role of Prompt Templates
Kaifeng Lyu*, Haoyu Zhao*, Xingran Gu*, Dingli Yu, Anirudh Goyal, Sanjeev AroraConference on Neural Information Processing Systems ( NeurIPS 2024 )Adversarial Attacks on Combinatorial Multi-Armed Bandits
Rishab Balasubramanian, Jiawei Li, Prasad Tadepalli, Huazheng Wang, Qingyun Wu, Haoyu Zhao (alpha-beta)International Conference on Machine Learning ( ICML 2024 )Do Transformers Parse while Predicting the Masked Word?
Haoyu Zhao*, Abhishek Panigrahi*, Rong Ge, Sanjeev AroraConference on Empirical Methods in Natural Language Processing ( EMNLP 2023 )Faster Rates for Compressed Federated Learning with Client-Variance Reduction
Haoyu Zhao, Konstantin Burlachenko, Zhize Li, Peter RichtárikSIAM Journal on Mathematics of Data Science ( SIMODS )Task-Specific Skill Localization in Fine-tuned Language Models
Abhishek Panigrahi*, Nikunj Saunshi*, Haoyu Zhao, Sanjeev AroraInternational Conference on Machine Learning ( ICML 2023 )Coresets for Vertical Federated Learning: Regularized Linear Regression and K-Means Clustering
Lingxiao Huang, Zhize Li, Jialin Sun, Haoyu Zhao (alpha-beta)Conference on Neural Information Processing Systems ( NeurIPS 2022 )SoteriaFL: A Unified Framework for Private Federated Learning with Communication Compression
Zhize Li, Haoyu Zhao, Boyue Li, Yuejie ChiConference on Neural Information Processing Systems ( NeurIPS 2022 )BEER: Fast O(1/T) Rate for Decentralized Nonconvex Optimization with Communication Compression
Haoyu Zhao, Boyue Li, Zhize Li, Peter Richtárik, Yuejie ChiConference on Neural Information Processing Systems ( NeurIPS 2022 )Combinatorial Semi-Bandit in the Non-Stationary Environment
Wei Chen, Liwei Wang, Haoyu Zhao, Kai Zheng (alpha-beta)The Conference on Uncertainty in Artificial Intelligence ( UAI 2021 )Combinatorial Pure Exploration of Dueling Bandit
Wei Chen, Yihan Du, Longbo Huang, Haoyu Zhao (alpha-beta)International Conference on Machine Learning ( ICML 2020 )Online Second Price Auction with Semi-bandit Feedback Under the Non-Stationary Setting
Haoyu Zhao, Wei ChenAAAI Conference on Artificial Intelligence ( AAAI 2020 )Gradient Method for Continuous Influence Maximization with Budget-Saving Considerations
Wei Chen, Weizhong Zhang, Haoyu Zhao (alpha-beta)AAAI Conference on Artificial Intelligence ( AAAI 2020 )Stochastic One-Sided Full-Information Bandit
Haoyu Zhao, Wei ChenThe European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases ( ECML/PKDD 2019 )An FPTAS for Stochastic Unbounded Min-Knapsack Problem
Zhihao Jiang, Haoyu Zhao (alpha-beta)International Frontiers of Algorithmics Workshop ( FAW 2019 )
Talks
Video presentation at NeurIPS 2022
Conference proceedings talk at Conference on Neural Information Processing Systems (NeurIPS 2022), New Orleans, United States
Video presentation at NeurIPS 2022
Conference proceedings talk at Conference on Neural Information Processing Systems (NeurIPS 2022), New Orleans, United States
Oral presentation at AAAI 2020
Conference proceedings talk at AAAI Conference on Artificial Intelligence (AAAI 2020), New York, United States
Oral presentation at ECML/PKDD 2019
Conference proceedings talk at The European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML/PKDD 2019), Wurzburg, Germany
Contests
- Meritorious Winner of MCM/ICM Jan. 2018
- Gold Medal(11th place) of Chinese Mathematical Olympiad (CMO) Dec. 2015
- Second Prize in Chinese Physics Tournament Oct. 2015
- "The Harvard-MIT Mathematics Tournament" (HMMT 2015) Feb. 2015
- The 22nd place (Geometry, Individual Round)
- the 50th place (Overall, Individual Round)
- the 3rd place (Team Round)
- Silver Medal of Chinese Mathematical Olympiad Dec. 2014
Scholarships and Certificates
- Baidu Scholarship for Research and Innovation Oct. 2019
- Scholarship for Senior Sept. 2019
- Xuetang Scholarship Sept. 2019
- Mitsubishi UFJ Foundation Sept. 2018
- Scholarship for Junior Sept. 2018
- Xuetang Scholarship Sept. 2018
- ZhengGeru Scholarship Sept. 2017
- Scholarship for Sophomore Sept. 2017
- Xuetang Scholarship Sept. 2017
- Scholarship for Freshman Sept. 2016
- Xuetang Scholarship Sept. 2016
Languages
- Chinese
- Native
- English
- Toefl
- 107, 29 (Reading) + 27 (Listening) + 24 (Speaking) + 27 (Writing).
- GRE
- 326, 157 (Verbal) + 169 (Quantitative) + 4.0 (Writing).
- Toefl