Yihao Zhang张益豪
应用数学博士三年级
北京大学数学科学学院
现于腾讯元宝安全中心任职
我是张益豪（Yihao Zhang），北京大学数学科学学院应用数学博士三年级学生。我目前在 Meng Sun 教授的指导下开展研究，是其课题组成员。
我于 2024 年在北京大学获得学士学位（数据科学与大数据技术、数学）。2023 年 10 月至 2024 年 5 月，我在新加坡管理大学担任访问研究助理，导师是 Jun Sun。
我的研究方向包括：
- 大模型智能体安全
- 面向大模型智能体的形式化方法
- 大模型越狱攻击
- 提示注入
- 大模型可解释性
- 表征工程
- AI 安全，验证 AI 中的鲁棒性 / 公平性 / 可信赖性问题
- 自动交互式定理证明，AI4ITP
我已在多个国际会议上发表多篇论文，Google Scholar 总引用量 
如需访问我 2024 年之前的个人主页，请点击 这里。
💼 工作经历
腾讯元宝安全中心在职
Tencent Yuanbao Security Center
字节跳动 Flow 安全
ByteDance Flow Security
🔥 新闻
📝 代表性论文
(*: 共同一作; ${}^\dagger$: 通讯作者)
AgentWorm: Self-Propagating Attacks Across LLM Agent Ecosystems (NDSS 2027)
Yihao Zhang, Zeming Wei, Xiaokun Luan, Jiangrong Wu, Chengcan Wu, Zhixin Zhang, Haolin Wu, Huanran Chen, Jun Sun, Meng Sun
We present AgentWorm, a self-propagating attack that compromises persistent LLM agents and autonomously spreads across agent ecosystems. The work studies the full infection loop, its transferability, and practical defenses for deployed agent systems.
On the Duality Between Sharpness-Aware Minimization and Adversarial Training (ICML 2024)
Yihao Zhang*, Hangzhou He*, Jingyu Zhu*, Huanran Chen, Yifei Wang, Zeming Wei${}^\dagger$
Adversarial Training (AT), which adversarially perturb the input samples during training, has been acknowledged as one of the most effective defenses against adversarial attacks, yet suffers from a fundamental tradeoff that inevitably decreases clean accuracy. Instead of perturbing the samples, Sharpness-Aware Minimization (SAM) perturbs the model weights during training to find a more flat loss landscape and improve generalization. However, as SAM is designed for better clean accuracy, its effectiveness in enhancing adversarial robustness remains unexplored. In this work, considering the duality between SAM and AT, we investigate the adversarial robustness derived from SAM. Intriguingly, we find that using SAM alone can improve adversarial robustness. To understand this unexpected property of SAM, we first provide empirical and theoretical insights into how SAM can implicitly learn more robust features, and conduct comprehensive experiments to show that SAM can improve adversarial robustness notably without sacrificing any clean accuracy, shedding light on the potential of SAM to be a substitute for AT when accuracy comes at a higher priority. Code is available at this https URL.
Adversarial Representation Engineering: A General Model Editing Framework for Large Language Models (NeurIPS 2024)
Yihao Zhang, Zeming Wei, Jun Sun${}^\dagger$, Meng Sun${}^\dagger$
Since the development of Large Language Models (LLMs) has achieved remarkable success, understanding and controlling their internal complex mechanisms has become an urgent problem. Recent research has attempted to interpret their behaviors through the lens of inner representation. However, developing practical and efficient methods for applying these representations for general and flexible model editing remains challenging. In this work, we explore how to use representation engineering methods to guide the editing of LLMs by deploying a representation sensor as an oracle. We first identify the importance of a robust and reliable sensor during editing, then propose an Adversarial Representation Engineering (ARE) framework to provide a unified and interpretable approach for conceptual model editing without compromising baseline performance. Experiments on multiple model editing paradigms demonstrate the effectiveness of ARE in various settings. Code and data are available at this https URL.
其他论文
(*: 共同一作; ${}^\dagger$: 通讯作者)
Symbolic-Neural Soft-Logic Reasoning: Towards Robust and Verifiable Thinking Chains via Cooperative Evolution (Preprint, 2026)
On Mutation Testing of In-Context Learning Systems (Journal of Systems Architecture, 2026)
Automata-Based Steering of Large Language Models for Diverse Structured Generation (ICFEM 2025 · Best Paper)
Robust and Efficient Watermarking of Large Language Models Using Error Correction Codes (PETS 2025)
When Thinking LLMs Lie: Unveiling the Strategic Deception in Representations of Reasoning Models (Preprint, 2025)
Mitigating Fine-tuning Risks in LLMs via Safety-Aware Probing Optimization (Preprint, 2025)
Towards Formal Modeling and Verification of the Stellar Consensus Protocol in Z3 (QRS Companion 2025)
MedTiny Code Generation for Enhancing RegLang Smart Contract Reliability (SAC 2025)
Component Composition in MedTiny: Multi-Level Constructs and Operational Semantics (SEKE 2025)
Exploring the robustness of in-context learning with noisy labels (ICASSP 2025, ICLR 2024 R2-FM Workshop)
DocVerify: Constraint-Based Verification of LLM Outputs Against Document Knowledge (ICECCS 2026)
MedTiny: Enhanced Mediator Modeling Language for Scalable Parallel Algorithms (QRS 2023)
Sharpness-aware minimization alone can improve adversarial robustness (ICML 2023 AdvML-Frontiers Workshop)
Weighted automata extraction and explanation of recurrent neural networks for natural language tasks (JLAMP, Vol 136)
🎖 荣誉奖励
- 北京大学校长奖学金，2025
- 怀新学士（荣誉学士学位），2024
- 入选北京大学数学科学学院拔尖计划（研究生）。
- 北京大学奖学金，2023
- 全国大学生数学竞赛（北京赛区）二等奖，2023
📖 教育经历
- 2023.10 - 2024.05，研究助理，新加坡管理大学计算机与信息系统学院。
- 2020.09 - 2024.06，本科生，北京大学数学科学学院。
- 2024.09 - 2028.06（预期），博士，北京大学数学科学学院。
💬 报告
- 2024.09，在烟台大学展示我们的工作 On the Duality Between Sharpness-Aware Minimization and Adversarial Training。
- 2024.04，在华为内部分享 Adversarial Representation Engineering。
- 2023.06，在 SEKE 2023 上展示论文 “Using Z3 for Formal Modeling and Verification of FNN Global Robustness”。
💻 项目
- 2026-2029, Safety Assurance of LLM-Driven Agents, Key Member, NSFC General Program.
- 2025-2027, LLM Feature Reconstruction & Manipulation, Key Member, Industry Collaboration.
- 2023-2025, Study on the interpretability of large language model architecture and algorithm, Program Director.
- 2022-2025, Trustworthy guarantee of deep learning system, Member.
👨🏫 导师
🔗 链接
- 🧑🎓 合作者: Zeming Wei, Xiaokun Luan, Xiyue Zhang, Huanran Chen, Jiangrong Wu