AI Bot Spam Is Killing Open Source: A Story From Archestra

Karify98 & Amy 🌸·
Cover Image for AI Bot Spam Is Killing Open Source: A Story From Archestra
#github#open-source#ai-spam#devops#security

253 Spam Comments on a Single Issue

Archestra, a Y Combinator S25 AI startup, posted an issue with a $900 bounty to invite community contributions for "MCP Apps" support. The result: AI bots flooded in, pushing the issue to 253 comments — mostly meaningless "implementation plans" generated by AI, some even aggressive toward maintainers.

This isn't an isolated case. In the same repo, a request for x.ai support received 27 pull requests. None were tested. Every week, a team member spent half a day just cleaning up AI garbage.

Why AI Bot Spam Is Worse Than Traditional Spam

GitHub bot spam isn't new. But AI creates a fundamentally different class of spam:

Wide attack surface, near-zero cost. Previously, bad actors had to write each comment manually. Now they just run a script calling an LLM API — a few cents per call, generating hundreds of "plausible" comments in minutes.

Mimics natural language. AI-generated comments aren't easy to spot by eye. They use correct technical terminology, present detailed "implementation plans," even cite documentation. But inside it's all hallucination — no logic, no tests, no real value.

Crowds out real contributors. When an issue has 253 spam comments, real contributors leave. Nobody wants to wade through that mess to find meaningful discussion. According to Archestra, active contributors like @ethanwater, @developerfred, and @Geetk172 — people actually working on bounties — were buried under a wall of noise.

Security risk. The LiteLLM lesson: attackers used AI bots to steer issue discussions, enabling a supply chain attack. LiteLLM 1.82.8 had a malicious .pth file that stole SSH keys, AWS credentials, Kubernetes configs, even crypto wallets — encrypting and exfiltrating everything to an attacker-controlled server.

What Archestra Did

Archestra tried multiple approaches, from simple to "nuclear":

Step 1: Reputation scoring. They built "London-Cat," a bot scoring contributors based on merged PRs and other signals. Result: didn't stop spam, only helped identify who's who.

Step 2: AI sheriff. Wrote a bot to auto-close spam PRs. Result: closed legitimate PRs too. Not viable.

Step 3: Nuclear option — manual whitelist. Blocked all issue creation, comments, and PR opening for anyone who hadn't completed onboarding.

The whitelisting trick exploits Git's --author flag. Every GitHub account has a noreply email: <id>+<username>@users.noreply.github.com. Archestra looks up the ID via API, creates a commit with --author attributed to that person. GitHub recognizes the commit and grants "prior contributor" status.

The full flow:

  1. Contributor registers on website, agrees to ethical AI rules, solves CAPTCHA
  2. GitHub Action fires, looks up GitHub ID, adds handle to EXTERNAL_CONTRIBUTORS.md
  3. Commit is pushed to main with correct --author
  4. Contributor gets repo access

What's Wrong With This Approach

Several things, as the Hacker News community pointed out:

Pipeline security. Contributors with commits on main get approval-free CI/CD runs. An attacker could submit one harmless typo-fix PR, get it merged, then run workflows without approval. GitHub's docs warn about this exact scenario.

Scalability. Manual whitelisting doesn't work for large projects. A project with hundreds of new contributors per month would bottleneck at the onboarding step.

GitHub should handle this. Many Hacker News commenters argued this is GitHub's responsibility, not individual maintainers'. "This isn't for the open source community to figure out, any more than it's up to me to figure out how not to get spam email."

Lessons for Developers

For Maintainers

  • Enable "Require approval for all external contributors" in GitHub Actions settings. Don't use "first-time contributors" — too loose.
  • Restrict issue creation permissions. GitHub doesn't have pre-moderation for issues yet, but bots can help.
  • Monitor your supply chain. LiteLLM was compromised through a .pth file — audit package dependencies regularly with tools like pip-audit or Dependabot.

For Contributors

  • Contribute responsibly. Test your code before submitting a PR. Write clear descriptions. This is the fastest way to earn trust.
  • Be careful with PyPI/npm packages. Check checksums, read file changes before installing. Don't install the latest version in production without review.

For Developers in General

  • AI coding tools are double-edged swords. Generating code fast doesn't mean generating correct code. As covered in a previous blog post: 84% of developers use AI coding tools, but code review remains an indispensable step.
  • Ethics of AI usage. Using AI to assist — fine. Using AI to spam, steal bounties, or cheat — that's destructive behavior.

The Bigger Picture

GitHub reports strong growth in contributor numbers and commit volume. But a significant portion of that is AI-generated — and quality is declining. According to GitHub Octoverse, commit counts are up, but merged PR rates are down.

This is the paradox: AI helps developers write code faster, but also helps bad actors spam more easily. Open source — built on trust and voluntary contribution — is being challenged at an unprecedented scale.

The long-term solution needs to come from platforms: GitHub must build stronger anti-spam mechanisms, similar to how email developed spam filters. But in the meantime, every maintainer has to protect their own repo.

References: