What happens when an AI agent contributes to open source

TL;DR

Watching Claude Code work through real engineering tasks end-to-end raised an obvious question: what happens if an AI agent does this in the open, against someone else’s codebase, under someone else’s rules? So we ran a controlled experiment. Over roughly 2 months, we used an AI agent — Claude Code, orchestrated through two tools we built and open-sourced along the way (ai-engineering-workflow-skills and open-source-tracker) — to find real bugs, fix them, and submit pull requests to public, third-party open-source projects. Mostly without a human in the loop for the actual code.

The result: 97 pull requests merged (54.8% of 177 submitted) across 73 different repositories, and 80 closed without merging. We didn’t just count outcomes — we read and coded why each one happened, with the evidence cited, not guessed.

The interesting part isn’t the merge rate. It’s how unevenly the open-source world reacted. Some maintainers welcomed the contributions outright. Some rejected them the moment they looked automated — sometimes with a canned response, sometimes with a personal one. Some projects that call themselves “open source” turned out not to accept outside contributions from anyone, AI or human. And for a meaningful chunk of what got closed, nobody said anything at all.

How the experiment worked

The mechanism was deliberately simple and deliberately public: real GitHub accounts, real pull requests, against real projects we don’t own or control. No sandboxed or synthetic targets. An agent found a genuine, reproducible bug in a public repository, wrote a fix, wrote or updated tests in that project’s own style, and opened a pull request — the same path any human contributor would use.

A human set the scope and reviewed anything non-trivial before it shipped; the agent did the finding, fixing, and writing. We tracked every single pull request our accounts opened — merged, closed, or still open — in a public dataset, and built the tooling to keep that dataset current automatically rather than as a one-time snapshot.

What happened to the pull requests

Of the 177 pull requests opened in the campaign window, 97 merged and 80 closed without merging. But the aggregate number hides the real story, which is in how each side of that split actually happened.

The merged ones weren’t a rubber stamp

43 of the merged pull requests went through an actual human review before landing — someone read the diff and approved it. Another 15 involved real back-and-forth discussion before merge, without a formal review being filed, and 15 more got a human response only after the merge had already happened — real discussion, just too late to count as review. On the other end, 24 merged with no recorded human comment or review at all — the merge action itself is the only human signal in the thread, typically small, unambiguous fixes in low-traffic repos.

The closed ones split into real, distinct categories

We coded every closed pull request against the actual thread — comments, reviews, closing events — not keyword-matching. The categories that emerged from the data itself:

• 19 withdrawn by us — usually because we found a better fix, a duplicate, or the issue turned out to be already handled.
• 17 already fixed or superseded — someone else’s competing fix landed first, a fair outcome of working in public repositories with other contributors.
• 15 closed with no recorded reason at all — no comment, no review, no explanation. Just closed.
• 8 closed with an explicit objection to AI-generated code — the maintainer said, directly, that the contribution appeared AI-generated and that was part of why it was rejected.
• 8 closed on stated technical or scope grounds — a real design disagreement or maintenance-capacity concern, unrelated to how the fix was produced.
• 8 incorporated elsewhere — the fix (or the underlying diagnosis) got folded into someone else’s PR or commit.
• The remainder closed for process, administrative, or policy-bot reasons, or are still awaiting manual review of their exact closure reason.

How different corners of open source actually reacted

This is where the campaign stopped being a numbers exercise and started being a map of how open source, as a community, is actually handling an increase in AI-assisted contribution — because the answer isn’t one thing.

Explicit, stated rejection of AI-generated code is real, but it isn’t universal — and it isn’t always personal. A maintainer of a major PHP framework¹ closed one of our pull requests with a direct, specific message: that the contribution “appears to have been primarily AI-generated without careful human review,” and that AI-generated code “often doesn’t align well” with the project’s conventions. The same message, nearly verbatim, appeared on other pull requests closed around the same date — suggesting a standing practice for handling PRs judged to be AI-generated, not a one-off reaction to this particular fix. That’s a meaningfully different finding than “this maintainer read my diff and rejected it”: it’s a project applying a general policy, applied here, evenly.

Not every AI-related objection is a blanket rejection, either. On a smaller documentation-tooling project, a maintainer² closed a pull request citing two separate reasons in the same comment: “a ton of agent-generated conversation” that made the thread too noisy to review closely, and an independent preference for a different technical approach entirely. Reading the thread, the process complaint and the technical disagreement were both genuinely present — it wasn’t AI-skepticism standing in for a technical review, it was AI-skepticism plus one.

Some “open source” repositories don’t meaningfully accept outside contributions from anyone. We checked this systematically, not just where our own pull requests landed: before attempting a contribution to any repository, we sample its last 100 merged pull requests and check who actually authored them. A widely-used e-commerce platform³ turned out to merge pull requests exclusively from its own paid team — of its last 100 merges, 100% were from six accounts, all confirmed staff, plus automated dependency bots. When we opened a correct, tested fix there anyway, it was closed 34 seconds after opening, with no comment, no review, and passing checks. A widely-used Python web framework⁴ showed the same pattern in miniature: of its last 100 merges, 58 were from its creator, 33 were automated, and the remaining genuine external contributions came to roughly 2–3 accounts. These projects are open source in the licensing sense. They are not, in practice, open to outside contribution — which matters, because it means a meaningful share of “the open-source ecosystem” isn’t actually a fair test bed for how maintainers respond to anything, AI-authored or not.

A large share of rejections come with no explanation at all. 15 of the 80 closed pull requests in this campaign got no maintainer comment, no review, and no stated reason at all — just a closed status. That’s not evidence of anti-AI sentiment; it’s evidence that a meaningful slice of open source runs on volunteer maintainer time that simply doesn’t stretch to explaining every decision, regardless of who or what submitted the change.

What we changed because of this

Running the campaign taught us things the numbers alone don’t show:

• We stopped submitting multiple pull requests to the same organization in a tight time window after seeing evidence that submission velocity, not content, was a trigger for batch rejections on at least one large ecosystem.
• We built the external-contribution check described above — sampling authorship of the last 100 merges before attempting a fix — after wasting effort on repositories that don’t functionally accept outside PRs.
• We now write pull request descriptions in plain, first-person prose rather than a templated structure, after noticing that generic-looking PR bodies correlated with faster, more suspicious closures on some repositories.
• We stopped posting automated “checking in” follow-up comments on stale pull requests after they triggered exactly the AI-fatigue reaction we were trying to avoid.

Methodology, briefly

Every pull request our accounts opened against a public, third-party repository is tracked in an open dataset. For every merged or closed one, we pulled the full review, comment, and timeline history from GitHub’s API and manually coded the outcome against the actual thread — not inferred from keywords — with the specific comment or event cited as evidence for every category assignment. That coding, and the categories themselves, were independently reviewed before publication. The full methodology, the coding taxonomy, and the underlying evidence-cited dataset are public.

This case study is scoped to the 177 pull requests opened in the 2-month campaign window (from 2026-07-24); it does not include unrelated contribution history from before the campaign began. Numbers on this page update automatically as the campaign continues and more pull requests reach a final outcome — this is a living dataset, not a one-time snapshot. Last updated: 2026-09-25.

A full evidence-cited academic writeup, including a literature review against existing research on AI-agent open-source contribution, is in progress.