Live audit · public repository

A gap the rulebook never flagged — found by reading activity, not documentation

Five pull requests on a 60,000-star, YC-backed repository — since closed unmerged — all used a "Fixes #N" auto-close keyword while touching only a test file. Nothing in the team's own AGENTS.md says whether that's an intentional pattern. This is what that audit looks like.

mem0ai/mem0— memory-layer infra, YC + Peak XV + Basis Set, ~60K★, pushed daily

What the repo already gets right

This isn't a case of a team with nothing written down. mem0's AGENTS.md is unusually thorough: build/lint/test commands per package, CI/CD workflow architecture, provider-pattern conventions, a full PR template, explicit "do not" rules. A lazy audit would pretend every team's docs are empty. This one isn't.

The harder, narrower question a rulebook actually has to answer isn't does documentation exist — it's does the written rulebook match what's happening in the repo right now.

The gap

Five PRs, same external contributor, identical undocumented shape — all since closed unmerged; the linked issues remain open.

PRTitleDiffClaims
#6004fix(mem0): add test for issue #5903test_issue_5903.py (+7/-0)Fixes #5903
#6021fix(mem0): add test for issue #5915test_issue_5915.py (+7/-0)Fixes #5915
#6022fix(mem0): add test for issue #5901test_issue_5901.py (+7/-0)Fixes #5901
#6038fix(mem0): add test for issue #5871test_issue_5871.py (+7/-0)Fixes #5871
#6039fix(mem0): add test for issue #5869test_issue_5869.py (+7/-0)Fixes #5869

Checked in detail — #6021 → issue #5915

mem0/memory/main.py — AsyncMemory.reset()

The linked bug is real and specific: AsyncMemory.reset() hardcodes delete_col() and never checks for the faster VectorStoreFactory.reset() path that the sync Memory.reset() already uses.

The PR's diff was exactly one new 7-line test file. No line in mem0/ — the actual source — changed. The PR body read "Fixes #5915," GitHub's auto-close keyword — had it merged as-is, GitHub would have closed issue #5915 as resolved while the async/sync mismatch stayed exactly as broken. It was closed unmerged instead, and the issue remains open — so this particular case never played out. The pattern across all five PRs is the finding regardless: nothing in the public rulebook tells a contributor, reviewer, or AI agent whether a test-only "Fixes #N" PR is a sanctioned reproduce-first step or a mistake waiting to auto-close a real bug.

Not a false verdict

This might be a sanctioned "reproduce-first" contribution pattern the maintainers already know about — that's a legitimate possibility, not ruled out here. What's certain either way: nothing in the public rulebook resolves that ambiguity before a PR like this reaches a reviewer.

What a compiled rule looks like

Not a vague instruction — a checkable procedure an agent can actually run.

---
name: pr-autoclose-verification
description: Flag PRs that claim "Fixes #N" but whose diff doesn't touch the module the issue describes
---

When a PR body contains a GitHub auto-close keyword (Fixes/Closes/Resolves #N):
1. Look up the linked issue and identify which source path(s) it concerns.
2. Check the PR's diff against those paths.
3. If the diff touches ONLY test files, and no source file in the issue's
   described module, flag it: "This PR uses an auto-close keyword but doesn't
   appear to touch the source location the issue describes — confirm this is
   intentional before merge, since merging will auto-close the issue whether
   or not the underlying behavior changed."
4. Do not auto-block. Surface for a human decision.

The same audit, run on your repo

This wasn't a hypothetical — every PR and line above is public and independently checkable in five minutes. I'll run the same check against your team's actual CLAUDE.md, AGENTS.md, or SKILL.md files: read what your recent PRs, reviews, and incidents actually show, and tell you exactly where the written rulebook and current reality have drifted apart.

Request the free audit No cost · no pitch attached · real findings either way