How PR Lens keeps exactly one comment per pull request

PR Lens keeps one comment on a pull request with four mechanisms stacked in order: a hidden HTML marker in the body that says the comment is a PR Lens comment, an author check that says this App wrote it, a Redis pointer to its id, and a Redis lease held around the only dangerous operation, which is creating the first one. A reconcile pass afterwards collapses any duplicate that still slipped through onto the oldest comment. None of this is exotic. It is the same problem every pull request bot has, and most of them solve three quarters of it, which is why so many repositories have a column of stale bot comments in them.

This post is the whole mechanism, written up because the failure modes are more interesting than the happy path. If you want the product rather than the plumbing, what is PR Lens is the page for that.

Why bots end up with five comments on one pull request

The naive version is three lines: list the comments, find yours, create if it is missing. It works on your laptop and fails in production for reasons that all look like the same reason.

Two pushes land within a second of each other, so two runs start. Both list the comments. Neither finds one, because neither has posted yet. Both create. Now the pull request has two, and every subsequent run picks one of them arbitrarily and updates it, so the repository has one comment that keeps changing and one that is frozen at whatever the first push said.

A webhook retry does the same thing with one push instead of two. So does a workflow step that failed after the create and got retried. So does a run that spent 40 seconds rendering between deciding to create and actually creating.

The hidden marker

Every PR Lens comment carries a sticky marker in its body:

<!-- pr-lens:sticky -->

HTML comments render as nothing, survive an edit, and are cheap to search for. That is the whole trick, and it is what almost every sticky comment implementation uses.

PR Lens writes a second, larger marker at the very end of the body, holding the state the comment needs to redraw itself:

<!-- pr-lens:state {"v":1,"graphKey":"graphs/owner/repo/pr/12/<sha>.json","headSha":"...","options":{...}} -->

That is why a checkbox in the comment can change what the diagrams show without re-running any analysis. The graph document is already in storage, the marker says which key it lives at, and a toggle just redraws from it.

One detail there is load bearing. The decoder reads the last occurrence of the state prefix in the body, never the first. Model-derived text appears earlier in a PR Lens comment, and a summary containing a perfectly valid looking marker of its own would otherwise choose which stored graph a re-render reads. The composer writes the real marker last, so position is what identifies it.

The checkbox reader is paranoid in the same direction. A line only counts as an option when it is a Markdown checkbox that also carries the option's marker. Matching the marker first would let text quoted elsewhere in the body claim the option and silence the real checkbox.

The marker is public, so it cannot be the only test

Anyone can copy <!-- pr-lens:sticky --> into a comment of their own. A human quoting the comment in a reply carries it too.

So ownership is two conditions, not one: the body carries the marker, and the comment's author is a bot whose login is this App's own login. Only comments passing both are adopted, edited or deleted. A marker tells you which comment is yours; only the author tells you whether you wrote it.

The pointer, and why the pointer is not enough

Listing every comment on a pull request costs an API call per hundred comments, so the id of the comment is cached in Redis under a key derived from the repository id and the pull request number, with a 60 day expiry.

The pointer records an id. It does not record authorship, and it outlives the identity that earned it. The same Redis carries across an App identity change while the bot login underneath it changes, and editing on the pointer alone therefore PATCHes whatever now sits at that id. That is not a hypothetical: it is how a run under a new App identity came to rewrite the previous bot's comment.

The fix is dull and worth stating plainly. Re-read the comment at the cached id, apply the same ownership test the marker search uses, and if it fails, leave it alone and fall back to the search. A cache of an id is a hint about where to look, never a licence to write.

Creating exactly once: the lease

Creation is the only step that can produce a duplicate, so it is the only step that takes a lock.

A caller sets a Redis key with NX and a 60 second expiry, holding a random token as the value. Getting the key means you own creation for this pull request. Not getting it means someone else does, and you wait.

Three things make the lock hold up:

The whole guarded section runs under the lease, including the pre-create lookup. A lookup that ran outside it could itself outlive an unattended lock, and then the create behind it happens with no protection at all.

A heartbeat extends the key every 20 seconds while the work runs, and a failed extension is treated as lost ownership rather than ignored. Before posting, the caller asks the lease whether it still holds it and aborts if not. A GitHub call slower than the lock would otherwise let the next caller post as well.

Release is a compare and delete in one Lua script: delete the key only if its value is still my token. A separate GET then DEL can delete the lock a different owner acquired after this one's expiry ran out, which turns your lock into a machine for producing the exact bug it was there to prevent.

A caller that never gets the lock waits, re-checking the pointer and the comment list as it goes, for up to four minutes. Past that it raises a retryable error rather than creating anyway. A retry is cheap. A second comment on somebody's pull request is not.

Converging when a duplicate slips through

Locks have expiries, so they can be lost. The invariant is therefore repaired rather than merely defended.

Straight after creating, the run lists its own sticky comments again, sorted by id. If there is more than one, the oldest survives, the rest are deleted, and the surviving comment is updated with the new body. A pull request that briefly held two PR Lens comments ends the run holding one, instead of holding two forever.

Latest commit wins

Sticky is only half the promise. The other half is that the comment shows the current commit.

Analysis is slow enough that a push can land while it runs. Two checks handle that. The run compares its commit against GitHub's current head before doing expensive work, and again before publishing. GitHub is the authority on what latest means; webhook delivery order is not. An older analysis that finishes last is marked superseded and discarded rather than written over a newer render.

There is a matching rule on the publish side: publishing never proceeds on an unconfirmed head. If GitHub does not answer, the run raises a retryable error and finalizes once it can ask again. A run that cannot tell whether it is stale assumes it might be.

The rules, if you are building one of these

Six things, in the order they bit us:

1. Search by marker, but confirm by author. The marker is public.
2. Read the last marker in the body, not the first. Everything before it may be text somebody else wrote.
3. Treat a cached comment id as a hint. Re-read and re-check before you PATCH.
4. Lock the create, not just the decision to create, and hold the lock across the lookup that precedes it.
5. Release the lock with a compare and delete, never a bare delete.
6. Reconcile afterwards. Assume the lock will be lost eventually and make the duplicate self-healing.

The visible result is one comment that is always current, which is what installing the GitHub App gets you, and it is also what makes the animated SVGs inside it addressable at all. Those have their own set of constraints, which I wrote up in animated SVG in GitHub comments. Where the stored graph behind the marker actually lives is in what PR Lens does with your code.