Two weeks ago, we looked at ICLR 2026 and found a hidden cow, a TODO to "think clear," and a handful of ChatGPT citation leftovers. 1 in 26 papers had something in them that shouldn't have been there.
Now that ICLR 2027 submissions are on OpenReview, we wanted to rerun the analysis on this year's papers. We ran 1,022 of them through Reviewer3's submission screener, which flags unfinished work, leftover LLM output, and hidden prompts.
Here's what we found:
- 1 in 25 papers still had draft notes, template text, or AI leftovers.
- 0 papers had hidden prompts, down from 0.2% in last year's sample. ICLR now prints a notice at the bottom of every submission's first page, warning reviewers that if they let an LLM make things up, their own papers could be desk-rejected.
- 0.1% of papers had visible ChatGPT leftovers, down from 0.3% last year.
Below you can find a list of some of the most memorable things we found.
1. "Author verification pending"
New this year, ICLR requires every paper to include an AI use statement explaining how generative AI was used. One paper's statement ends like this:
Author verification pending: Human authors must verify the final text, references, and this disclosure before submission.
The disclosure says an LLM only helped polish the language, and then lets us know that no human has verified that.
Try it yourself. Open the paper on OpenReview, press Ctrl+F (or Cmd+F on a Mac), and paste:
Human authors must verify
It wasn't the only AI use statement that didn't quite get finished:
- One kept the template's fill-in-the-blank: "we used generative AI tools for [design or provide feedback on research methodology or experiments]" (PDF, search
[design). - Five kept the template's reminder, "(This section is required and does not count toward the page limit.)", right under the heading (one, two, three, search
This section is required). - 30 papers didn't have an AI use statement at all.
896 of the 1,022 submissions had an AI disclosure statement we could parse, so we ingested them and sorted them by what the authors say they used AI for. Here are the top AI use categories:
| The authors say AI was used to... | Papers |
|---|---|
| Polish, edit or translate the writing | 97% |
| Write or debug code | 58% |
| Search the literature | 39% |
| Draft substantial text, not just polish it | 26% |
| Shape ideas, hypotheses or methodology | 25% |
| Help with proofs or theory | 16% |
| Run experiments or analyses | 15% |
| Nothing at all | 0.8% |
62 papers (7%) say AI wrote their code, ran their experiments, and drafted their text.
The most commonly named tools were ChatGPT, Claude (including Claude Code), and OpenAI's Codex.
None of this breaks ICLR's rules, which ask authors to disclose their AI use, not avoid it. But it's a useful baseline: in 2027, only 0.8% of machine learning papers say they used no AI at all.
2. The wrong PDF
Have you ever attached the wrong file to an email? Or the wrong paper to a submission?
The OpenReview page says the paper is called Confounder or Mediator? Nuisance Probing Needs a Null Derived from the Design. But the PDF says this:
Somewhere out there, a paper about nuisance probing is still waiting to be uploaded.
Image created with ChatGPT
3. Figures, in name only
LaTeX has a draft mode that skips loading images and prints a box with the file path instead. It's handy while you're writing, since it makes compiling faster. In one submission, it's also all you get.
Try it yourself. Open the PDF and search:
hierarchy_kde
4. Wrong conference, wrong year
Every ICLR submission has to use the ICLR 2027 template. A few papers didn't quite make the switch:
- Two papers were written in the NeurIPS template, and their first pages still say so: "Submitted to 37th Conference on Neural Information Processing Systems (NeurIPS 2023). Do not distribute." (PDF), and the same line for NeurIPS 2025 (PDF). Search
NeurIPS. - Three still have last year's header, "Under review as a conference paper at ICLR 2026" (one, two). Search
ICLR 2026. - And one paper is feeling very confident. Every page says "Published as a conference paper at ICLR 2027," the header for accepted papers, before a single review has been written (PDF, search
Published as a conference).
5. Notes to self
The best draft note: "until tonight." If you've ever finished an analysis the night before a deadline, this one might feel familiar. In the appendix of a paper on KV-cache allocation, the authors explain how they validated a fitted band:
…until tonight its only check was the three rows it is displayed against…
We'd love to know which night that was.
Try it yourself. Open the PDF and search:
until tonight
Here are some other notes that made it in, each quoted verbatim:
- "Pin exact release/commit used in experiments." Inside a reference entry, right where the release and commit should be (PDF, search
Pin exact). - "CVPR 2026; update URL/pages when CVF proceedings appear." Also in a reference list (PDF, search
update URL). - "Copy ann extended recall vs candidate gap.pdf into paper appendix artifacts/ ." Next to the figure it was presumably about (PDF, search
Copy ann). - "The full original inline review notes are preserved in ORIGINAL_INLINE_COMMENTS.txt and ORIGINAL_APPENDIX_COMMENTS.txt in this Overleaf project." Unfortunately, they're not in this PDF (PDF, search
Overleaf project). - "The result is a clean Paper-1 foundation." (PDF, search
Paper-1).
6. ChatGPT receipts
Last year, we found a bunch of ChatGPT's internal citation markers and several links ending in ?utm_source=chatgpt.com. This year, across more than a thousand papers, we found just one. Progress!
Try it yourself. Open the PDF and search:
chatgpt.com
The rest of the leftovers
Here are some other leftovers, each quoted verbatim from a real PDF:
| Paper's text | What happened | Search query |
|---|---|---|
| "The model name was optimized using the ADAM optimizer" | The model was never named (PDF) | The model name |
| "[Open code placeholder]" | At the end of the abstract (PDF) | placeholder |
| "publicly available [or: institutionally provided] scientific simulation data" | They never picked one (PDF) | institutionally provided |
| "Example material, pending replacement with the final implementation and experimental records." | Three times, in a benchmark paper (PDF) | Example material |
| "Acknowledgments: TBA" | Still to be announced (PDF) | TBA |
| "Pending" | Two cells of Table 4 (PDF) | Pending |
| "Figure 5: How to read the shape scores… (placeholder)" | A figure caption (PDF) | (placeholder) |
| "Natural Science Foundation of Inner Mongolia under Grant and…" | The grant number never arrived (PDF) | under Grant and |
| "Use unnumbered third level headings for the acknowledgments." | ICLR template text, in four papers (one, two) | unnumbered third level |
| "You may include other additional sections here." | Back again from last year (PDF) | additional sections here |
Tips for screening papers
Now that ICLR 2027 reviews are underway, here are some tips on how to screen a paper before getting into the review:
- Search the PDF for common leftovers, like
TODO,??,placeholder,TBA,Pending, andchatgpt.com. - Check that the PDF's title matches the OpenReview page. It apparently doesn't always!
- Read the AI use statement, and make sure it was actually filled in (and not "author verification pending").
- Copy the text into an editor to inspect for hidden prompts and anything that shouldn't be there.
- Or run it through Reviewer3 and let us read every page for you.
The full screening results are live at reviewer3.com/live/iclr-2027.