PR Security Guardrails: Observe Before You Enforce

Most PR security guardrails fail the same way. Someone writes a strict policy, turns it on across every repository on a Tuesday, and by Thursday the security team is fielding angry messages about blocked merges on code nobody touched in a year. The policy gets relaxed, then disabled, then forgotten. The next attempt starts with less goodwill than the first.

The problem is rarely the policy itself. It is the rollout. A guardrail that blocks a pull request is a production change to how engineering ships code, and it deserves the same care as any other production change: measure first, announce second, enforce last.

This post walks through a rollout pattern that works: start every guardrail in observe mode, read what it would have done, promote it to warn, and only then let it block. It covers what to measure at each stage, which policies are worth enforcing first, and the SCM settings that decide whether "block" actually means anything.

What a PR security guardrail actually is

A PR security guardrail is a policy evaluated against every pull request, reported back as a status check on the pull request itself. The developer sees a green check or a red X in the same place they see their tests. No separate dashboard, no nightly report, no email a week later.

That placement is the whole point. A finding raised at pull request time costs the author a few minutes, because the code is still in their head and the change is still small. The same finding raised after merge becomes a ticket, a context switch, and a negotiation about priority.

Guardrails usually cover a handful of risk classes:

• Dependency vulnerabilities the pull request adds or upgrades into.
• Dependency hygiene: compromised packages, unmaintained packages, unapproved licenses, versions published too recently to trust.
• Source code weaknesses the branch introduces, compared with the default branch.
• Secrets committed anywhere in the pull request's history.
• Infrastructure as code misconfigurations, and increasingly, changes to AI agent files.

Each of these can take one of three actions when it matches, and the three actions are the rollout plan.

Observe, warn, block: three actions, three stages

• Observe records the match and changes nothing on the pull request. The merge is allowed and the developer sees nothing. You see everything.
• Warn surfaces the match to the developer on the pull request, but the merge is still allowed.
• Block fails the status check. If your source control enforces that check, the pull request cannot merge until the match is resolved.

Treat these as stages, not settings. A new guardrail starts in observe. It moves to warn when you trust its signal. It moves to block when you trust its signal and the people it affects know it is coming.

Stage 1: observe and measure

Observe mode answers one question: if this guardrail were blocking today, what would it have blocked? Run it long enough to see a normal cycle of work, usually one to two weeks, and then read the results as an engineer would, not as a dashboard would.

What to measure in observe mode

• Match volume per week. How many pull requests would this have stopped? If the answer is a third of all merges, the policy is not ready, whatever it says on paper.
• Concentration. Are matches spread across the estate, or is one repository, one team, or one dependency responsible for most of them? A single legacy service tripping a vulnerability rule is a remediation project, not a guardrail problem.
• True positive rate. Open twenty matches at random. How many would a reasonable security engineer agree should have been stopped? Anything that fails this test needs a narrower rule, a scope change, or a different action.
• Actionability. For each match, could the author have fixed it in the same pull request? A block with no way forward teaches developers to ignore the guardrail.

That last point is the one most rollouts miss. If a pull request is blocked on a critical vulnerability that has no fixed version upstream, the author has three options: wait, find a workaround, or ask for an exception. None of them are good. Pair every vulnerability rule with a "fix version available" condition, so the guardrail only fires when there is a concrete upgrade to make.

Stage 2: warn and tell people

Warn mode is where the guardrail meets developers for the first time. It is also the stage where you learn whether the message on the pull request is clear enough to act on.

Before promoting to warn:

1. Announce it. A short note to engineering leads: what the guardrail checks, why it exists, when it will start blocking, and where to ask questions.
2. Check the message. Read the pull request output for a real match. Does it name the dependency, the version, the finding and the fix? Could a developer who has never heard of your security program act on it?
3. Set an exception path. Decide who can approve an exception and how it is recorded. A guardrail with no exception path gets bypassed at the SCM level instead, and then you have no record at all.

Watch the warn stage for the same metrics as observe, plus one more: how often do warned pull requests merge without the warning being addressed? A high rate means either the signal is weak or developers do not believe it will ever block. Both are worth knowing before you enforce.

Stage 3: block, and make it hold

Block is the stage where the guardrail becomes a control. Two things decide whether it works.

Scope the block narrowly first

Do not promote a guardrail from warn to block everywhere at once. Scope it to the services where the risk is highest and the owners are ready. A common pattern:

• Block on production, internet-accessible, tier 1 services.
• Warn everywhere else.

This is only possible if the guardrail can see where the code runs. A policy that knows a repository deploys to an internet-facing production service can be strict there and lenient on an internal tool, with the same rule. A policy that sees only the repository has to treat every repository the same, which usually means treating them all leniently.

Enforce the check in your SCM

A failed status check does not stop a merge on its own. Your source control decides that. For block to hold:

• On GitHub, add the check as a required status check in branch protection rules or a ruleset.
• On GitLab, require external status checks to pass before merge.
• On Bitbucket and Azure DevOps, use branch restrictions or branch policies that require the build status.

Check this before you announce the block date. The worst possible outcome is a guardrail that reports "blocked" while the merge button stays green, because it trains everyone to ignore the red X.

Which PR security guardrails to enforce first

Not every policy deserves to block. Start with the ones where a match is almost always a real problem and the fix is almost always available.

1. Compromised or malicious packages. There is no legitimate reason to merge a known-malicious package. Block early, block everywhere.
2. Actively exploited vulnerabilities with a fix available. A CVE in the CISA Known Exploited Vulnerabilities catalog with an upgrade path is the clearest case for a block.
3. Secrets in the pull request history. A committed credential is exposed the moment it is pushed. The guardrail's job is to make sure it is seen and rotated, not merged and forgotten.
4. Minimum dependency age. A short cooldown on brand-new package versions, measured in days, blocks a class of supply-chain attack that relies on a malicious release being picked up before anyone notices.
5. Critical vulnerabilities with a fix available, scoped to production services.

Leave license policy, unmaintained dependencies and most code-quality-adjacent rules at warn for longer. They matter, but a false positive costs more developer trust than it saves.

A rollout checklist

• Every new guardrail is created in observe mode.
• Vulnerability rules are paired with "fix version available".
• Observe runs for at least one full cycle of normal work.
• Twenty random matches are reviewed by hand before promotion.
• The pull request message names the finding and the fix.
• Engineering leads are told what is changing and when.
• An exception path exists and is recorded.
• Block is scoped to the highest-risk services first.
• The status check is required in branch protection before block goes live.
• Merged-with-unresolved-violations is reviewed weekly.

How Heeler handles guardrail rollout

Heeler runs PR Guardrails as a native status check on GitHub, GitLab, Bitbucket and Azure DevOps. Every guardrail takes one of the three actions above, and the product is built around moving between them.

• A recommended set, off by default. New Heeler tenants start with a curated set of recommended guardrails already installed, inactive and in Observe mode. Nothing blocks a pull request until an administrator turns it on. The recommended set covers compromised dependencies, known actively exploited vulnerabilities, minimum dependency age, risky and unmaintained dependencies, and unapproved licenses.
• Measurement built in. The Pull Requests tab lists every evaluated pull request and its outcome. The Violations tab lists every individual match, so a single dependency tripping guardrails across many repositories shows up as one pattern. A violation whose pull request merged anyway is marked "merged unresolved".
• Scope from runtime context. A guardrail can be scoped by repository, branch, and service runtime context such as tier, application and internet accessibility, because the Context Engine knows where each repository's code is deployed.
• Suggestions for gaps. Heeler suggests guardrails you do not have yet, such as blocking actively exploited vulnerabilities when nothing does, and flags when guardrails are configured but none are enabled.

When a guardrail blocks, the pull request can often be fixed from the pull request itself: where an auto-fix exists, the check carries a fix action, so the author does not have to work out the upgrade by hand.

FAQ

How long should a guardrail stay in observe mode?

Long enough to see one normal cycle of work, usually one to two weeks. The goal is to see the match volume, where matches concentrate, and how many are real before anyone is affected.

Why pair vulnerability guardrails with "fix version available"?

A block with no available fix leaves the developer with nothing to do but wait or ask for an exception. Gating only when an upgrade exists keeps every block actionable.

Does a failed status check stop a merge?

Only if your source control requires it. Add the check as a required status check in branch protection, or the equivalent setting on GitLab, Bitbucket or Azure DevOps, before a guardrail moves to block.

Should every guardrail eventually block?

No. Block the policies where a match is almost always real and fixable, such as malicious packages and exploited vulnerabilities with a fix. Keep lower-confidence policies at warn and review them.

What is the difference between warn and observe?

Observe records a match silently for the security team. Warn shows the match to the developer on the pull request. Neither prevents the merge.

If you want to see how guardrails behave on your own repositories before anything is enforced, start with the recommended set in observe mode and read what it would have caught. Get a demo