The Role of AI in Scientific Peer Review

NeurIPS 2025 Social

About

This social event will explore the role of Artificial Intelligence (AI) in addressing the current challenges and shaping the future of scientific peer review. We will examine how AI can be applied across the entire scholarly publishing process, from authoring to reviewing, editing, and even readership. The event will foster critical discussion on the ethical implications, potential benefits, and practical implementation of AI in this critical scientific process. Our goal is to bring together researchers, practitioners, and stakeholders from diverse fields in an interactive format to build community and explore actionable solutions for a more efficient, fair, and transparent peer review system.

Event Schedule

Join us for the Social event at NeurIPS 2025. The social will feature a series of short, insightful talks from leading experts to set the stage for a broader community discussion.

Duration: 2 hours

Date: Wednesday - December 03

Time: 7:00 pm - 9:00 pm

Location: Ballroom Sec 6 CDEF

19:00 - 19:05
Welcome and Introduction · Isabelle Guyon · Hieu Khuong
19:05 - 19:20
Invited Talk 1: AI meets Peer Review: The Good, The Bad, and The Ugly · Nihar B. Shah (Carnegie Mellon University, USA) We discuss three facets of AI in Peer Review: (1) The Good: What AI can do in peer review that human reviewers do not; (2) The Bad: How fraudsters can game vulnerabilities in the use of AI in the review process; (3) The Ugly: Autonomous AI scientists have great promise, but also suffer from critical methodological pitfalls.
19:20 - 19:35
Invited Talk 2: Can LLM feedback enhance review quality? A study of 20K reviews at ICLR 2025 · Nitya Thakkar (Stanford University, USA) This talk presents the first large-scale randomized controlled trial where an LLM provided feedback to over 20,000 ICLR 2025 reviewers. Results show 27% of reviewers updated their reviews based on LLM suggestions, leading to longer, more informative reviews without biasing final decisions.
19:35 - 19:50
Invited Talk 3: From AI-Generated AI Science to AI-Assisted Reviewing · Yutaro Yamada (Sakana AI, Japan) Building on AI-Scientist-v2, this talk will discuss efforts to scale this into an ‘AI-AI Conference’ where AI systems generate and review research. It will also highlight work on AI-assisted peer review, including graph-based approaches for structured reviews.
19:50 - 20:05
Invited Talk 4: Running Pilots to Responsibly Scale the Use of AI in Scientific Peer Review · Joelle Pineau (McGill University, Canada) A discussion on strategies for responsibly implementing and scaling AI tools in the scientific peer review process through carefully designed pilot programs.
20:05 - 20:55
Panel and Round-Table Discussion · Moderated by Chris Bregler Discussion with our speakers above and panelists including Chris Bregler (Google DeepMind), Thomas Dietterich (Oregon State U.), Andrew McCallum (Univ. of Massachusetts Amherst), Nathan Srebro (TTIC), Markus Wulfmeier (Google DeepMind), and James Zou (Stanford University), followed by breakout discussions.
20:55 - 21:00
Closing Remarks · Isabelle Guyon

Organizers

We gratefully acknowledge the efforts of the Organizing Team.

  • Isabelle Guyon (Google DeepMind, USA & Univertité Paris-Saclay, France)
  • Hieu Khuong (Université Paris-Saclay, France)
  • Kent Rachmat (Université Paris-Saclay, France)
  • Ihsan Ullah (ChaLearn, USA)
  • Zhen (Zach) Xu (University of Chicago, USA)

For inquiries, please contact us at: ai-reviewer@chalearn.org

Acknowledgments

We gratefully acknowledge our speakers and panelists (see schedule above) for their valuable contributions to this event. We also extend our special thanks to the advisors who provided valuable feedback to shape the proposal.

  • Joaquin Vanchoren (Google Deepmind & Eindhoven University of Technology, Netherlands)
  • Amir Globerson (Google & Tel Aviv University, Israel)