Skip to content
NanoFlags
Start free

Reference

AI change review

When approval is requested for a change, NanoFlags checks what it does and shows approvers a risk level with reasons. With the AI review on, approvers also get a plain-language summary and suggested scenario checks. The review is advisory only: it never approves, blocks or publishes a change.

What it does

  • A review runs when approval is requested, and again when the draft changes while the change waits (after two quiet minutes). Editors can also ask for a review of the current draft from the change page.
  • Each draft is reviewed once: the review is kept with the change and its draft checksum, so a draft that was already reviewed is never reviewed again.
  • The risk level is low, medium or high. The AI can raise it, never lower it below what the automatic checks found.
  • Approvals, scenario requirements and publishing work exactly as before. Nothing reads the review to decide anything.

Automatic checks

These rules run on every change, without AI, also when the AI review is off or its monthly limit is reached. Severities are higher in production environments (protected, or named production, prod or live).

CheckRisk
A rule now matches every contextMedium / High
A rule that singled out a group (a segment or an exclusion) is removedMedium / High
Another rule is removedMedium
A rollout grows by 50 points or more, or to 100%Medium / High
The default value changesLow / Medium
The flag is turned on or offLow / Medium
A served value does not match the flag type or its JSON schemaHigh
The flag type changesHigh
A segment the flag uses changes / is removedMedium / High
The flag becomes visible to browser and mobile keysMedium
Rules change while an experiment runs on the flagMedium
5 or more services evaluated the flag in the last 7 days (usage insights)Medium
A risky edit on a flag that no scenario check coversMedium
An AI agent opened the change or proposed edits in the draft (the agent is named; the AI summary says so too)Medium
Shadow evaluation measured that 10% or more of sampled production evaluations would change valueMedium
Shadow evaluation measured fewer changes, or draft rules that matched no sampled evaluationLow

AI summary and suggestions

With the AI review on (Team plan and above), the review also contains:

  • a short summary of what the change does for the people affected, in English and Arabic (each reader sees their language);
  • the main risks, explained per flag;
  • up to three suggested scenario checks: an evaluation context and the value the flag must return. They are never added automatically. Choose Add scenario on the ones that make sense; they then run like any other scenario, including in CI.
AI text can be wrong. It is labelled as written by AI; compare it with the change before relying on it.

Where approvers see it

  • Change page: the Change review panel, with the risk badge, the checks, the summary and the suggestions. A review of an earlier draft is marked as such.
  • Slack: the approval message of the Slack app gets one compact line with the risk and the summary, in the channel's language. It is updated in place when the review is ready.

Turning it on

The AI review is off by default. An owner or admin turns it on under Settings → Organization → AI change review; the setting explains what is sent. Turning it on or off is recorded in the audit log (organization.ai_review_enabled, organization.ai_review_disabled).

What is sent

Only configuration and counts, and only when the AI review is on:

  • the published and draft configuration of the flags in the change: flag keys and types, rules (attribute names, operators and the values typed into rules), the segments they use, JSON schemas;
  • the change title and reason, the environment name, and the results of the automatic checks;
  • counts from usage insights: how many services evaluated each flag in the last 7 days, and how many scenario checks cover it.

Never sent: evaluation contexts, diagnostics samples, scenario contexts, or anything else about your end users. Do not type personal data into rules. The request is processed by our AI sub-processor only to write the review.

Limits

PlanAI reviews per month
Explore, SoloAutomatic checks only
Team200
Scale1,000
EnterpriseBy contract

A review counts when the AI writes it; a failed call is given back. Once the limit is reached, changes still get the automatic checks until the next calendar month (UTC). See Plans and limits.