Menu

Gautam Kamath

About Gautam Kamath

Posts by Gautam Kamath:

Announcing the ICML 2026 Workshops and Affinity Workshops

By ICML 2026 Workshop Chairs Gergely Neu and Courtney Paquette. 

We are thrilled to announce the accepted workshops at ICML 2026. This year’s cohort reflects the remarkable breadth and energy of the machine learning community, spanning foundational theory, applications, and the pressing societal questions our field continues to grapple with. Whether you are a seasoned researcher, a newcomer to the conference, or somewhere in between, we hope you will find workshops that challenge and inspire you. ICML 2026 will feature 44 workshops (which will run during the post-conference workshop days, Friday July 10 and Saturday July 11),  as well as 4 affinity workshops (which will run concurrently to the main conference, from Tuesday July 7 to Thursday July 9). 

In this post, we’ll first give some insight into the workshop selection process, and then share the full list of accepted workshops and affinity workshops. 

Selection Process

The process was especially competitive this year, due to an unprecedented volume of submissions. Up from about 150 proposals for ICML 2025, we received 247 proposals (not counting a handful of desk rejected and withdrawn submissions). Nearly all submissions were of very high quality, which made it very challenging to select the 44 workshops that we will be able to host. Our selection procedure was driven by the following principles.

Our objective throughout the selection process was to make sure that we end up with a program that covers a diverse range of topics that excites the ICML community. We had outstanding material to work with, which forced us to make some hard decisions to make sure that our goals are met within the strict space constraints imposed by the venue. To deal with this challenge, we designed a two-phase process. In the first phase, we arrived at a shortlist of outstanding proposals. In the second phase, this was refined down to the final list. For consistency, the selection process was conducted entirely by the two workshop chairs, with no external reviewers. As a main principle, each proposal was evaluated according to the criteria laid out in the Call for Workshops, but the massive volume of excellent submissions required us to additionally make the final decisions based on soft criteria and subjective comparisons of proposals.

In the first round, each of the submissions was read by one of the two workshop chairs to verify they meet the standards we expect at ICML. Within this round, we had to eliminate several workshops that had any sort of shortcoming: e.g., not having a (nearly) fully confirmed lineup of speakers or a feasible schedule, not having a well designed organization committee with clearly specified roles, not having a clear explanation of the relevance of the proposed speakers to the topic of the workshop, or not having sufficient detail on the background of the organizers to help us verify that they can indeed run their proposed event successfully. We also ruled out workshops with topics that are not relevant for the ICML audience (or whose relevance was not argued clearly enough in the proposal). After this initial round, we ended up with about 70 proposals. We are confident that all these 70 proposals would have resulted in great ICML workshops.

In an ideal world, the process could have stopped there, but unfortunately we still had to make significant cuts to arrive at the final list. This is where we took a closer look at the topics of the workshops, and clustered together proposals on similar topics to get a better side-by-side comparison within each area. We converged on the final decisions after reading all of the surviving proposals one more time, followed by several video calls where we discussed the merits of each potential workshop. Inevitably, we had to make some subjective decisions at this stage, reflecting our personal taste as scientists. We also had to consider some more complex soft criteria, such as whether to admit second (or third, etc.) installments of previous ICML workshops run by one group of organizers or to give a new group a chance to run a workshop on the same topic instead (answers varied on a case-by-case basis).

During the review process, we had the privilege of getting a first-hand account of the topics that get the AI/ML community excited in 2026. It is no surprise that a significant part of the submissions were concerned with aspects of generative AI and large language models. Perhaps more surprisingly, some specific topics within this area were especially popular, most notably with some variation of “agentic AI” appearing in the title of no less than 60 submissions! Many of these proposals had significant overlap not only in their titles, self-described scopes and objectives, but also their invited speakers. Despite the massive interest, we could only accept a handful of these workshops to leave space for other topics as well. And indeed, besides a handful of exceedingly popular areas, the distribution of the topics observed in the submissions was extremely heavy-tailed: these ranged from philosophy, ethics, and governance through statistical hypothesis testing and game theory, all the way to wireless networking and finance.

The diversity of the originally received proposals is clearly reflected in the final program, which stands as a testament to the lively creativity of our field today. We hope that each member of the ICML audience will find inspiration, insight, or possibly other unexpected values within the program.

The selection of affinity workshops was independent of the workshop selection process, and done in consultation with the Inclusion & Accessibility chairs. Seven affinity workshop proposals were submitted, and four were selected for inclusion in the program. The proposals not selected were incomplete in some way, e.g., having unconfirmed speakers (or no speakers at all), or not having enough content to fill an entire workshop. 

Workshops

Links to individual workshops are provided where available, several workshop submission servers can be found on OpenReview. Details may be updated, please email Gautam Kamath if there is information that is incorrect or missing. 

Affinity Workshops

Announcing the ICML 2026 Tutorials

By ICML 2026 Tutorial Chairs Claire Vernade and Adam White.

Tutorials are a critical part of the conference experience. They provide graduate students the opportunity to learn in depth from topic experts. They provide practitioners and theoreticians the ability to learn foundational background and the dark-arts of how to get things working in practice. With our growing community, tutorials can often provide a much needed smaller community to connect with.

With this rapid growth comes challenges. A huge diversity of topics. A large number of proposed tutorial submissions. At the same time we want to channel the community’s inputs, while relying on non-artificial evaluation of submitted proposals. In this post, we outline the review process we used for this year’s tutorials and formally announce the selected tutorials.

This year’s process

This year we used a three-pronged approach. We wanted some invited tutorials, suggestions from the community, and a rigorous review process. In late December we reached out and confirmed three invited tutorial presenters: 

  • Is numerical optimization theory irrelevant to machine learning practice in 2026?

After we confirmed these speakers, we reached out to the ICML community to solicit nominations for both topic suggestions and members of the community to deliver tutorials. We received 118 suggestions from a health diversity of folks from undergraduates all the way to industry professionals:

From these initial seeds, we created a call for proposals. We distilled the community suggestions into six suggested topics (as well as “Other,” of course):

  • Beyond-Transformer sequence models (state-space, S4, etc.)
  • Deep Learning and Deep RL Theory and Applications
  • Diffusion models – quantitative & theoretical understanding
  • LLM post-training and test-time training
  • Safety, machine unlearning, watermarking, and fingerprinting
  • Theorem proving and Lean

We also reach out to 14 individuals, personally inviting them to submit a proposal, as they were nominated by the community in phase two. 

The call resulted in 52 submissions! All proposals were evaluated in the same way, even if they were invited. Both chairs independently reviewed and read every single proposal and made a short list for each of the 7 topic areas (including “Other”). In the final calibration, both chairs agreed almost unanimously on all topic areas. The main criteria used were: 1) quality of proposal, 2) established topic expertise by presenters, and 3) teaching experience. The latter did not exclude industry researchers but required folks to explain their teaching experience. 

In the end, we selected an additional seven tutorials for ICML 2026 (making a total of ten) and we are thrilled to announce them below. All tutorials will take place on the first day of the conference, Monday July 6. 

Retrospective on PAT x ICML 2026 AI Paper Assistant Program

By Google Researchers Rajesh Jayaram and Vincent Cohen-Addad, and ICML 2026 Program Chairs Alekh Agarwal, Miroslav Dudik, Sharon Li, Martin Jaggi.

Following the success of the STOC Experimental AI Feedback program, we launched a similar experiment using Google Research’s Paper Assistant Tool (PAT) before the ICML 2026 deadline. Our goal was to provide authors with actionable, substantive feedback prior to the review process, allowing them to improve their submissions before the deadline. The PAT feedback was totally separate from the peer review process, and was not visible to reviewers, area chairs, program chairs, or anyone outside of the authors. Now that the program has concluded and the feedback from our author survey is in, we want to share some of the results.

The program ran from January 14th to January 26th, providing feedback for approximately 4,500 papers. Papers sent to the system received feedback within ~30 minutes on average, with some edge cases (primarily due to large PDFs causing failures in the AI models). While we initially set conservative eligibility criteria to ensure a smooth rollout, we were able  to quickly relax these requirements. Ultimately we offered a voucher to any author with an OpenReview account older than one month. We believe this afforded a maximal and equitable offering to the community, without allowing for misuse of the tool. We were quite pleased that we could make the program as accessible as possible. 

Highlights

After concluding the program, we released a feedback survey to participating authors. We received 869 responses over the week it was open, and the results demonstrate a high level of satisfaction with the tool. In particular, the tool was able to frequently identify issues, and make suggestions which allowed the authors to make significant changes to their papers before the submission deadline. 

  • High Retention & Satisfaction: 92.1% of respondents stated they would use the tool again. Furthermore, 73.3% rated the feedback as “Very” or “Mostly” helpful. Only 1.6% found the tool to not be useful at all. 
  • Delivering Deep Feedback: The most encouraging results concern the depth of the feedback. The tool successfully identified suggestions beyond surface-level edits and typos:
    • 35.4% of authors of papers containing theory reported the tool identified significant theory gaps that took more than an hour to fix.
    • 31% of authors of papers with experimental results said the feedback prompted them to run new experiments.
  • Clarity and Education: 87.3% of authors felt the tool improved the clarity and readability of their papers. Additionally, 84.5% saw clear educational value in the tool.

Author Quotes on PAT:

The PAT feedback was invaluable in improving my paper’s technical rigor before submission. It helped me identify and fix critical issues including:
– Mathematical contradictions in objective functions
– Inconsistencies in formal problem definitions
– Unsupported claims in experimental sections
– Missing theoretical justifications
The structured, segment-by-segment analysis made it possible to address issues systematically under tight deadline pressure.
— Apoorv Varshney, Department of Mathematics and Computing, Dr. B. R. Ambedkar National Institute of Technology Jalandhar
Compared to most LLMs, PAT’s performance is already quite excellent. PAT discovered that one of my theorem proofs was not rigorous and provided some possible solutions.
— Peihan Wu, Shanghai Normal University
The system was very helpful and also rather “objective”. I look forward to seeing the “human” revisions and comparing them.
— Paolo Stefano Giudici, Professor of Statistics, University of Pavia
[PAT] gives a comprehensive review, allowing authors to address issues that could potentially be questioned by reviewers.
— Enver Menadjiev, M.S.+Ph.D. student, Statistical AI Lab (SAILab) at Kim Jaechul Graduate School of AI at KAIST
Outside feedback is so important when you’re about to submit. I [iterated] for two rounds with [competitor] before, which I paid for. PAT had deeper feedback.
— Domenic Rosati, Dalhousie University

How it worked

The pipeline used for this experiment was scaffolding built on top of state-of-the-art Gemini-based models. To handle the complexity of technical papers, PAT segments the document into logical categorical sections, giving separate feedback for each section, with a high level summary at the beginning. 

Looking Ahead

The PAT x ICML 2026 experiment shows that AI feedback can drive improvements in scientific work, enabling authors to fix theoretical gaps and conduct new experiments in advance of an expert-led peer review process. 

With the continued exponential growth in submissions to machine learning conferences,  AI driven feedback has the benefit both of improving the quality of authors’ papers, and therefore the likelihood of acceptance and reproducibility, as well as easing the increasing burden on human reviewers. 

The PAT pipeline is still an experimental program in development. We plan to incorporate learnings from the ICML collaboration as we prepare for the next iteration of the Paper Assistant Tool. 

Thank you to the thousands of authors who participated in the program, provided feedback, and helped shape the future of AI-assisted research. We look forward to our continued collaboration to make our conference even better!

On Violations of LLM Review Policies

By ICML 2026 Program Chairs Alekh Agarwal, Miroslav Dudik, Sharon Li, Martin Jaggi, Scientific Integrity Chair Nihar B. Shah, and Communications Chairs Katherine Gorman and Gautam Kamath.

AI has increasingly become a valuable part of researchers’ workflows. Unfortunately, AI has the potential to hurt the integrity of peer review if improperly used. Conferences must adapt, creating rules and policies to handle the new normal, and taking disciplinary action against those who break the rules and violate the trust that we all place in the review process. 

ICML is actively working to adapt. This year, we desk-rejected 497 papers (~2% of all submissions), corresponding to submissions of the 398 reciprocal reviewers who violated the rules regarding LLM usage that they had previously explicitly agreed to. 

ICML 2026 has two policies regarding LLM use in reviewing:

  • Policy A (Conservative): No LLM use allowed.
  • Policy B (Permissive): LLMs allowed to help understand the paper and related works, and polish reviews.

This two-policy framework was formed based on community preferences and feedback — indeed, the community is divided on the best way to use LLMs in peer review, with issues such as author consent colliding with preferred reviewer workflows. Further details on the policy are available here. 

After a selection process, in which reviewers got to choose which policy they would like to operate under, they were assigned to either Policy A or Policy B. In the end, based on author demands and reviewer signups, the only reviewers who were assigned to Policy A (no LLMs) were those who explicitly selected “Policy A” or “I am okay with either [Policy] A or B.” To be clear, no reviewer who strongly preferred Policy B was assigned to Policy A. 

795 reviews (~1% of all reviews) written by 506 unique reviewers who were assigned Policy A (no LLMs) were detected to have used LLMs in their review. Again, recall that these are reviewers who explicitly agreed to not use LLMs in their reviews. The method used is described below, and generic AI-text detectors were not used. Every flagged instance was manually verified by a human, in order to avoid false positives. 

If the designated Reciprocal Reviewer for a submission produced such a review, their submission was rejected. In total, this resulted in 497 rejections, corresponding to submissions of 398 reciprocal reviewers. All Policy A (no LLMs) reviews that were detected to be LLM generated were removed from the system. If more than half of the reviews submitted by a Policy A reviewer were detected to be LLM generated, then all of their reviews were deleted, and the reviewer themselves was removed from the reviewer pool. 51 Policy A reviewers were detected to have used LLMs in more than half of their reviews, which is about 10% of the total of 506 detected reviewers. 

To be clear, we are not making a judgment call about the quality of flagged reviews or the reviewers’ intentions. This is simply a statement that the reviewer used an LLM at some point when composing the review, which is unfortunately a violation of the policy they agreed to abide by.

We regret the disruption this will cause in the peer review process. We have been in direct  communication with SACs and ACs impacted, and offered support where we can. The reviews that violated policy have been removed, and ACs may need to find new reviewers. Some submissions which already received a full set of reviews have been desk rejected. And some reviewers whose submissions have been desk rejected may become unresponsive. 

Technical Approach 

At a high level, the LLM detection involved watermarking submission PDFs with hidden LLM instructions, which would subtly influence any review produced via an LLM. Note that this is not a difficult measure to circumvent, particularly if it is known publicly (which was the case for almost the entire review period). Indeed, it may only catch some of the most egregious and careless uses of LLMs in reviewing, where the reviewer is inputting the PDF to the LLM and then directly copy-pasting the output from the LLM. Action was only taken for reviews that were explicitly written by reviewers who agreed to not use LLMs (Policy A). Despite all these caveats, 795 reviews (~1% of all reviews) were found to be violations of the policy. 

The method used was based on recent work by Rao, Kumar, Lakkaraju, and Shah. First, we created a dictionary of 170,000 phrases. For each paper, we sampled two phrases randomly from this dictionary. The probability with which a given pair of phrases is picked is thus smaller than one in ten billion. We watermarked the PDF of each paper submitted with instructions, visible only to an LLM, instructing it to include the two selected phrases in the review. (A human reading the PDF would not directly see this watermark.) 

This technique will not always be successful in detecting LLM generated reviews. The reviewer could discover the watermark and remove it or work around it. The review text could be modified. The LLM may simply ignore the hidden instructions. In experiments shortly before the submission deadline, frontier LLMs generally (though not always) followed the injected instructions. Success rates were over 80% for most models, potentially depending on the method chosen by some LLMs to read the PDF. 

Reviewers may be understandably concerned that their reviews could be wrongly flagged. As described in the paper that outlines the approach we used, false positive rates are strongly controlled. Every flagged instance was also manually inspected by a human to ensure that the review wasn’t simply mentioning that the watermark was present. 

Reviewers can fall short of our expectations in many ways, with or without AI involved. This initiative focused only on one particular action (breaking previously agreed-upon rules for LLM usage) and still identified it for ~1% of all reviews. 

We hope that by taking strong action against violations of agreed-upon policy we will remind the community that as our field changes rapidly the thing we must protect most actively is our trust in each other. If we cannot adapt our systems in a setting based in trust, we will find that they soon become outdated and meaningless.

Update (3/26):

    • We have calculated the probability of incorrectly flagging even a single submitted review, where the reviewer was required to follow Policy A. This probability is 0.0001 (the family-wise error rate).

    • We want to express our gratitude to those of you who have reached out, acknowledging the policy violation and accepting responsibility. We recognize that this was not an easy situation, and your willingness to accept the outcome with integrity reflects the professionalism that makes our community strong.

    • The original version of this post stated that 497 papers, co-authored by 506 reciprocal reviewers, were desk rejected. These 497 papers actually correspond to submissions of 398 reciprocal reviewers, the remaining 108 flagged reviewers were not reciprocal reviewers of active submissions. The text above has been edited to reflect this. 

ICML Experimental Program using Google’s Paper Assistant Tool (PAT)

By Google Researchers Rajesh Jayaram and Vincent Cohen-Addad, and ICML 2026 Program Chairs Alekh Agarwal, Miroslav Dudik, Sharon Li, Martin Jaggi. 

ICML is excited to announce that we will be providing an experimental tool, in partnership with Google, called the Paper Assistant Tool (PAT), to support authors in improving their submissions to ICML 2026.

This program offers authors a unique opportunity to receive automated, private, and actionable feedback on their manuscripts before the final deadline. In a pilot at the Annual ACM Symposium on Theory of Computing (see blog post), 94% of participants found the pre-submission feedback generated by an AI assistant to be helpful, and 85% reported that the feedback resulted in improved clarity of their paper. The feedback the authors will receive from PAT through the ICML experiment will not be used in the review process. Reviewers, area chairs, and program committee members will not have access to the PAT feedback.

What is PAT?

PAT is a specialized Gemini-based tool powered by Google’s Gemini models, utilizing a reasoning-focused pipeline. It is similar to those that have achieved high-level performance in mathematical competitions.

While the program at STOC focused heavily on theoretical correctness, the ICML experiment  has been specifically tuned to address the needs of the machine learning community. The model is designed to help authors identify issues that human reviewers might flag, including (but not limited to) experimental and methodological rigor, narrative clarity in English, and technical correctness.

The goal is to help authors improve the quality of their papers, not to replace human peer review. By fixing clarity issues and potential technical gaps before the work is submitted to the conference, we hope to give authors actionable feedback before their paper enters the review process.

Logistics and Eligibility

The program is entirely optional and operates completely outside the official review process. The program will run for a limited window between January 14 and January 22, 2026.  This deadline may be extended subject to the determination of the program organizers. Eligible submissions uploaded within the window will typically receive the feedback within 24 hours, but there might be additional delays if the request volume is too high. 

To manage resources fairly, each eligible author is granted one virtual “voucher” to have a single paper run through the AI feedback system. We define an eligible author as someone who has authored at least one published paper at one of the following venues *hosted on OpenReview*: ICLR, NeurIPS, ICML, CVPR, or AISTATS.

To redeem this voucher, authors will select a button “Ready for LLM Feedback” on their paper submission form to flag the manuscript for AI review. This feature becomes active only after a PDF has been successfully uploaded to the OpenReview server. The author who checks the button redeems their voucher for the paper, and the version of the document at the time of the edit is then sent to the pipeline for automated feedback. If an ineligible author or an author who has already used their voucher attempts to select the “Ready for LLM Feedback” button, an error message will appear and the paper will not be sent out for review. 

The technical staff will be able to provide limited, best-effort support on the program, such as answering questions or checking for failed paper delivery. Please direct such questions, as well as any feedback you may have on the program, to icml-paperassistant@google.com.  

Privacy and Data Safety

We recognize the sensitivity of unpublished research. Trust is the cornerstone of this experiment, and we have implemented strict protocols to ensure author safety:

  1. Strict Separation from Peer Review: The AI Feedback is entirely independent of the official ICML review process. It is visible only to the authors. Reviewers, Area Chairs, and Program Chairs will have no access to this feedback.
  2. Stateless Inference (No Training): Submissions will not be used to train, fine-tune, or improve Google’s models. The model operates in a stateless “inference-only” mode; it processes the text to generate feedback and retains no memory of the specific content for future learning.
  3. Data Destruction: To minimize data exposure, Google will employ a strict deletion policy. All PDFs and feedback submitted to Google are stored in a restricted access environment and are scheduled for permanent deletion shortly after the feedback is delivered and the program is completed.
  4. Restricted Access: Google access to the data (submission PDFs and generated feedback) during the retention window is strictly limited to the primary program organizers and a minimal number of essential technical staff (listed at the bottom of this post) required for system maintenance. Google organizers and staff will only inspect the data in the event of technical difficulties with a submission (and if necessary to resolve those difficulties). 

Transparency: The AI feedback will be clearly flagged with a disclaimer stating it is AI-generated. It is framed strictly as “feedback” rather than a “review” to manage author expectations. It will be posted as a comment on the submission page that is only visible to authors.

Caveats and Disclaimers

Like all LLMs,  the models used by the PAT pipeline are not infallible. Authors should treat the generated feedback with the same critical eye they would apply to a human review.

  • The model may occasionally flag correct statements as errors or miss actual flaws. It is the author’s responsibility to verify the validity of the feedback.
  • Note that the model may make suggestions for the paper that you disagree with. This is not necessarily a bug, and may even be expected. Think of these suggestions as potential points of criticism from a reviewer. By considering why you disagree with the suggestion, you may be able to add justification to the paper which preempts these reviewers’ comments, thereby increasing the strength of the paper.

Outcomes

Our primary objective is to empower authors to elevate the quality of their submissions by acting as a high-precision filter. This tool is designed to help authors catch nuanced errors—in proofs, experimental setups, or reasoning—that human reviewers might miss or lack the time to detail.

To achieve this, the program utilizes models that are more fit for purpose than standard generalist LLMs. We are leveraging a reasoning-focused pipeline related to the Gemini models that achieved Gold-medal performance at the International Mathematical Olympiad (IMO). By designing a pipeline specialized for ML conference structures, we can offer distinct advantages over generic tools. In the post-mortem for the STOC experiment, authors consistently confirmed that these confidential models provided quality meaningfully superior to public alternatives.

We believe this tool represents a unique opportunity to pioneer the responsible use of AI to elevate the standard of our own scientific submissions. We look forward to seeing the results of this experiment, which will be shared with the broader community via the ICML blog once the experiment is over.

Google Program Organizers: Rajesh Jayaram Vincent Cohen-Addad 

Google Sponsors: Vahab Mirrokni

Google Technical Staff: Rajesh Jayaram Vincent Cohen-Addad Drew Tyler Jieming Mao Jon Schneider

Contact (for feedback and best-effort assistance): icml-paperassistant@google.com 

Introducing ICML 2026 policy for self-ranking in reviews

By ICML 2026 Integrity Chair Weijie Su and Associate Integrity Chair Buxin Su. 

In recent years, AI conferences like ICML have seen an exponentially increasing number of submissions. In contrast, the pool of high-quality reviewers has grown much more slowly, raising widespread concerns about the quality of peer review. When review scores are noisy, they can encourage a cycle in which authors submit more borderline papers in the hope that variance in review scores will work in their favor. To assist the program committee in making better decisions, ICML 2026 will implement self-ranking to identify low-quality reviews. This approach leverages authors’ insights to scrutinize review outcomes with minimal additional effort, rather than increasing the burden on reviewers.

We introduce the following policies. Immediately after the full paper submission deadline, authors with multiple submissions will be asked to rank their papers according to their perceived quality and significance. Once pre-rebuttal review scores are available, we will compute a projected score for each submission by adjusting the pre-rebuttal scores to align with the self-rankings using the Isotonic Mechanism [1,2,3,4]. Notably, a paper that is ranked highest (top-1) by all authors is guaranteed to receive a projected score at least as high as their other submissions (see details in Final policy design). We will then use the absolute value of the discrepancy between projected scores and pre-rebuttal scores as a signal: submissions with large absolute discrepancies are those for which authors strongly disagree with the reviewers. These submissions will be flagged to Area Chairs, who can decide either to recruit emergency reviewers or to re-examine the reviews themselves. This method is intended as a Pareto improvement to review quality, encouraging Area Chairs to devote extra attention to submissions that may have received lower-quality reviews.

Where does this framework come from?

The starting point is a simple observation: authors usually have the best knowledge of their own papers but are often reluctant to truthfully share their relative preferences. This idea was formalized in the Isotonic Mechanism proposed in [1,2], a computationally efficient approach to enhancing the accuracy of noisy review scores by incorporating authors’ self-rankings of their submissions. Under this mechanism, authors with multiple submissions are asked to rank their papers in descending order of perceived quality, and the pre-rebuttal scores are then calibrated to be consistent with these rankings. Papers [1,2] show that authors are incentivized to report their rankings truthfully, because truthful reporting maximizes their expected utility. Building on this game-theoretic foundation, we hypothesize that self-rankings are informative: authors have a unique understanding of their work’s conceptual depth and long-term promise. Over the last three years at ICML, the Integrity Chairs and Associate Integrity Chair conducted pilot studies to empirically examine this idea [3,4], and for ICML 2026, the resulting evidence directly informed the final policy design.

Pilot studies

The policy is based on three pilot studies conducted around ICML over the last three years, beginning with ICML 2023. All pilot studies were anonymous, were GDPR-compliant, and were conducted after the full paper submission deadline.

In the ICML 2023 pilot study, authors with multiple submissions were asked to rank their papers according to perceived quality and significance. This produced a rich dataset of self-rankings from 1,342 authors, covering 2,592 submissions to ICML 2023, along with their official review scores and acceptance decisions.

We first compared the outputs of the Isotonic Mechanism (the projected scores) to the pre-rebuttal review scores in terms of how accurately they reflect submission quality [3]. To do so, we needed a proxy for ground-truth submission quality. Since submissions typically receive multiple reviews, we used the average of the remaining scores as a proxy for the ground truth “expected review score” of a submission when evaluating the performance of an estimator applied to a single randomly selected review score. We showed that the Isotonic Mechanism substantially reduces proxy estimation errors, specifically mean squared error and mean absolute error. Moreover, the results suggest that the improvement becomes more substantial as the number of submissions of an author increases. 

This dependence suggests that more author-provided rankings can lead to greater error reduction through the Isotonic Mechanism. Because the change in proxy estimation error is an unbiased estimator of the change in ground-truth estimation error, we could produce confidence intervals for the decrease in ground-truth squared error. We observed substantial decreases, statistically significant at the 99% confidence level.

Different strategies implement different ways of aggregating the rankings provided by multiple authors of the same submission. The simple-averaging strategy runs the Isotonic Mechanism separately for each author who provides a ranking, and the submission’s projected score is computed as the simple average of the resulting modified scores. The greedy strategy starts by running the mechanism for the author who provides the longest ranking, then removes that author and all of their submissions from further consideration; this process is repeated for the remaining authors and their submissions until each submission has exactly one modified score. The multi-owner strategy partitions all submissions into disjoint blocks such that, within each block, every submission shares a common set of authors, and then applies the simple-averaging strategy within each block.

We also validated our hypothesis that self-rankings capture authors’ unique understanding of conceptual depth and long-term promise [4]. Our analysis of the ICML 2023 data reveals that authors’ self-rankings are a powerful predictor of a paper’s future impact, measured by citations accumulated over 16 months. Submissions ranked highest by their authors received, on average, twice as many citations as those they ranked lowest, a trend that held for both accepted and rejected submissions, suggesting that self-rankings can not only denoise review scores but also capture a distinct dimension of impact. 

The predictive power was particularly striking for identifying high-impact work: of the 22 papers in our dataset that garnered over 150 citations, 17 were ranked highest by their authors. For comparison, we demonstrate that these self-rankings are a more accurate predictor of future citations than the review scores. Stratified by review scores, higher-ranked papers consistently received more citations even when controlling for similar scores. These findings were highly statistically significant and remained robust after controlling for potential confounding factors or using other metrics of impact, such as GitHub stars.

Final policy design

In ICML 2026, immediately after the full paper submission deadline, authors will receive a request to rank their submissions according to quality and significance. We will compute projected scores using the Isotonic Mechanism (as detailed below) and, once pre-rebuttal scores are available, calculate their discrepancies.

The Isotonic Mechanism operates as follows. Consider an author who submits n papers to a conference. The mechanism requires the author to rank these submissions in descending order of perceived quality, denoted by π. Given the (average) raw review scores y for the n submissions, the Isotonic Mechanism outputs projected scores r by minimizing the Euclidean distance between y and r, subject to the constraint that r is consistent with the author’s ranking. Formally, this convex optimization problem is equivalent to isotonic regression. For example, letting y = (8,7,4,3) and (π(1), π(2), π(3), π(4)) = (1,3,2,4), the projected scores are (8,5.5,5.5,3). To synthesize rankings provided by multiple authors of the same submission, we use the simple-averaging strategy which resulted in the greatest decrease in error in our previous studies. The simple-averaging strategy begins by running the isotonic mechanism separately for each author who provides a ranking. The projected score for the submission is calculated as the simple average of these modified scores. For additional details, please see [3].

The absolute value of the discrepancy will be used to categorize submissions into a small number of buckets. We will notify Area Chairs and Senior Area Chairs which bucket the submission falls into, without revealing the sign of the discrepancy. Based on the bucket, Area Chairs can then choose to recruit emergency reviewers or to re-examine the reviews themselves.

To help Area Chairs and Senior Area Chairs visualize these signals, OpenReview will display the discrepancy category as an additional field in the meta-review in the SAC and AC consoles. In addition, the CSV exports containing submission data for SACs and ACs will include the discrepancy category as an additional column.

Participation in self-ranking is optional. For authors who do not respond, and for those with a single submission, the process for recruiting emergency reviewers or for AC re-examination will proceed as in previous years. Self-ranking is intended only to provide an additional signal and to serve as side information that can help ACs target their efforts. Moreover, by design of the Isotonic Mechanism, trivial or uninformative self-rankings, for example declaring all of one’s submissions as equally good, leave the projected scores identical to the original review scores, resulting in zero discrepancy.

The ICML 2026 policy is grounded in a clear commitment to respecting authors’ expectations of a fair process and strengthening the community’s trust in review outcomes. Self-ranking enables authors to provide truthful, adaptive feedback, and the isotonic mechanism turns that input into a more informative signal for identifying submissions that require additional attention. We therefore expect self-ranking, used as targeted side information, to make the review process more transparent, more reliable, and ultimately more collaborative.

[1] Su, W. (2021). You Are the Best Reviewer of Your Own Papers: An Owner-Assisted Scoring Mechanism. Advances in Neural Information Processing Systems, 34, 27929-27939.
[2] Su, W. (2025). You Are the Best Reviewer of Your Own Papers: The Isotonic Mechanism. Operations Research.
[3] Su, B., Zhang, J., Collina, N., Yan, Y., Li, D., Cho, K., Fan, J., Roth, A., & Su, W. (2025). The ICML 2023 Ranking Experiment: Examining Author Self-Assessment in ML/AI Peer Review. Journal of the American Statistical Association.
[4] Su, B., Collina, N., Wen, G., Li, D., Cho, K., Fan, J., Zhao, B., & Su, W. (2025). How to Find Fantastic AI Papers: Self-Rankings as a Powerful Predictor of Scientific Impact Beyond Peer Review. 

What’s New in ICML 2026 Peer Review

By ICML 2026 Program Chairs Alekh Agarwal, Miroslav Dudik, Sharon Li, Martin Jaggi.

As ICML 2026 opens for submissions, we would like to review several new elements that we are introducing this year to improve the reviewing process and to adapt to the changing landscape of LLM use in research.

Thinly sliced contributions

There is a rising concern in our community about a large volume of submissions that do not necessarily advance the field, while burdening the peer-review system. The scale of this issue is exacerbated by the use of generative AI tools. We are taking aim at what we call “thinly sliced contributions”, by which we mean submission of a large number of papers with small variations on the same theme.

To combat this phenomenon, we are strengthening our concurrent submission policy. The policy states that all papers submitted to ICML 2026 by the same author must be viewed as prior work, and if they are on a related topic (which would be normally included in the related work section of the paper), they must cite each other and discuss each other in the the body of the paper (anonymized PDFs of the cited submissions must be provided in the supplementary material). Violations of this policy are viewed as abuse of the peer-review system and may result in the desk rejection of the violating submissions or of all the submissions by the submitting authors.

Cascading desk rejections for peer-review abuse

Violations of integrity of peer-review process through prompt injection, collusion, as well as other forms of abuse like submission of low-quality AI-generated content (AI slop) and submission of thinly sliced contributions may result in penalties as outlined in Peer-review Ethics. In particular, these violations may result in desk rejection of all papers by any of the authors of an offending paper. This places an additional responsibility on group leaders. If group leaders are co-authors on papers that abuse the peer-review system, all of the papers where they are listed as an author might be desk rejected regardless of whether they were aware of the specific instance of peer-review abuse. We urge group leaders to be vigilant about the papers they submit and work closely with all of the contributing authors to avoid cascading desk rejections that would impact all work they are submitting, and all authors they are submitting with. 

Reciprocity

As in the past years, we expect paper authors to give back to the community for the service that they receive by having their papers reviewed by their peers. As such, each paper must nominate a qualified reciprocal reviewer. Authors that submit four or more papers must be involved as a reviewer, area chair, senior area chair or in other organizing roles at ICML (details of this policy, including exceptions, are spelled out in the Call for Papers). Moreover, if any reviewer or area chair fails in their duties, all of their papers might be desk rejected (details in Peer-review Ethics).

Supporting authors and reviewers

The policies discussed so far seek to improve accountability of authors and reviewers. We also have several policies that seek to meet various author and reviewer needs:

  • Our flexible policy for LLMs in reviewing allows a broad (but not completely unconstrained) use of LLMs in reviewing, as long as authors consent to that form of review.
  • Top 25% of reviewers will receive free ICML registration.
  • We are integrating author-provided self-rankings (for authors with multiple submissions) to help flag papers that may need more attention from area chairs (specifically, we flag papers with a large discrepancy between reviewer scores and self-rankings). More details on this will be given in a forthcoming blog post.
  • Authors of accepted papers do not need to attend the conference in person. Their papers will still appear in the proceedings, but without a poster or a talk at the conference.
  • Finally, we are planning to make an advanced reasoning LLM system available to ICML authors. Authors will be able to submit their paper drafts and receive feedback (up to a few days before the submission deadline). More details on this forthcoming. Stay tuned!

We hope that these efforts will continue to improve the quality of the conference and the ease of collaboration in our community.