Reference
AI change review
On this page
What it does
- A review runs when approval is requested, and again when the draft changes while the change waits (after two quiet minutes). Editors can also ask for a review of the current draft from the change page.
- Each draft is reviewed once: the review is kept with the change and its draft checksum, so a draft that was already reviewed is never reviewed again.
- The risk level is low, medium or high. The AI can raise it, never lower it below what the automatic checks found.
- Approvals, scenario requirements and publishing work exactly as before. Nothing reads the review to decide anything.
Automatic checks
These rules run on every change, without AI, also when the AI review is off or its monthly limit is reached. Severities are higher in production environments (protected, or named production, prod or live).
| Check | Risk |
|---|---|
| A rule now matches every context | Medium / High |
| A rule that singled out a group (a segment or an exclusion) is removed | Medium / High |
| Another rule is removed | Medium |
| A rollout grows by 50 points or more, or to 100% | Medium / High |
| The default value changes | Low / Medium |
| The flag is turned on or off | Low / Medium |
| A served value does not match the flag type or its JSON schema | High |
| The flag type changes | High |
| A segment the flag uses changes / is removed | Medium / High |
| The flag becomes visible to browser and mobile keys | Medium |
| Rules change while an experiment runs on the flag | Medium |
| 5 or more services evaluated the flag in the last 7 days (usage insights) | Medium |
| A risky edit on a flag that no scenario check covers | Medium |
| An AI agent opened the change or proposed edits in the draft (the agent is named; the AI summary says so too) | Medium |
| Shadow evaluation measured that 10% or more of sampled production evaluations would change value | Medium |
| Shadow evaluation measured fewer changes, or draft rules that matched no sampled evaluation | Low |
AI summary and suggestions
With the AI review on (Team plan and above), the review also contains:
- a short summary of what the change does for the people affected, in English and Arabic (each reader sees their language);
- the main risks, explained per flag;
- up to three suggested scenario checks: an evaluation context and the value the flag must return. They are never added automatically. Choose Add scenario on the ones that make sense; they then run like any other scenario, including in CI.
Where approvers see it
- Change page: the Change review panel, with the risk badge, the checks, the summary and the suggestions. A review of an earlier draft is marked as such.
- Slack: the approval message of the Slack app gets one compact line with the risk and the summary, in the channel's language. It is updated in place when the review is ready.
Turning it on
The AI review is off by default. An owner or admin turns it on under Settings → Organization → AI change review; the setting explains what is sent. Turning it on or off is recorded in the audit log (organization.ai_review_enabled, organization.ai_review_disabled).
What is sent
Only configuration and counts, and only when the AI review is on:
- the published and draft configuration of the flags in the change: flag keys and types, rules (attribute names, operators and the values typed into rules), the segments they use, JSON schemas;
- the change title and reason, the environment name, and the results of the automatic checks;
- counts from usage insights: how many services evaluated each flag in the last 7 days, and how many scenario checks cover it.
Never sent: evaluation contexts, diagnostics samples, scenario contexts, or anything else about your end users. Do not type personal data into rules. The request is processed by our AI sub-processor only to write the review.
Limits
| Plan | AI reviews per month |
|---|---|
| Explore, Solo | Automatic checks only |
| Team | 200 |
| Scale | 1,000 |
| Enterprise | By contract |
A review counts when the AI writes it; a failed call is given back. Once the limit is reached, changes still get the automatic checks until the next calendar month (UTC). See Plans and limits.