Guide

AI Content That Actually Passes Review: A Working System (Not a Tool List)

By Istar editors
At a glance

AI first drafts fail review for detectable reasons. How Istar’s upstream writing step and refine–validate–audit–repair–revalidate loop catches them, including errors two models missed, and a pre-publish checklist you can copy.

As of August 2026, an AI first draft is not a publishable guide. It fails in a small set of ways a process can catch: a fact that is no longer true, a rule that contradicts a later paragraph, a number with no source, a sentence that sounds like a product launch, or a “best tools” dump that never makes a decision. Those are not mysteries of taste. They are checks.

This site writes practical English guides for freelancers and small teams. As of 23 August 2026 the repo holds 37 Markdown files under src/content/guides/, 36 of them draft: true. Writing is the upstream step. The quality loop in docs/agent-pipeline/ — described in How to Run a Multi-Model Content Pipeline — is refine → validate → independent audit → repair on FAIL → revalidate. Google’s helpful-content guidance still says ranking looks for people-first pages and treats scaled AI whose main job is to manipulate search as spam. E-E-A-T is a way to ask whether a page shows experience and claims a reader can check.

Three layers that catch different errors

One model reviewing its own draft can miss the errors it just made. Split the jobs.

Layer 1 — automatic validate. The site script docs/agent-pipeline/validate.mjs is a lightweight regex preflight, not Astro’s content schema and not a quality oracle. It fails a file that lacks title, description, pubDate, or draft; that sets draft: false in this pilot; that uses the legal domain-style name as a display brand; that hits part of the house list of launch-marketing stems; or that reads like a ranked-tools staple. Word count under 250 also fails. It does not catch every banned stem, and it will not notice that two prices use different denominators.

Layer 2 — refine for tone and usefulness. A second model (on this site, Claude) edits against house rules: Istar as the only display name, English, freelancers and small teams, no hype words, prices sourced or removed, a who-it’s-for section, a number the reader can run. This pass makes the piece readable. It tends to fix wording and leave logic holes.

Layer 3 — independent audit. A third model (on this site, Codex in a read-only role) is told not to rewrite. It looks for fact errors, contradictions, unsourced hard numbers, and absolute claims. Only a FAIL triggers a repair pass. The point is independence, not a longer prompt. In a side-by-side comparison on this site, Claude refined both drafts; only the independent auditor caught errors the first two models had left in.

docs/agent-pipeline/pipeline-batch.sh is the measured path after a draft exists: Claude refine → validate.mjs → Codex audit → Codex repair only on FAIL → validate again. A human still decides whether draft: true becomes a public page.

What the independent pass actually caught

These are not hypotheticals. They are from this site’s own comparison notes.

On an AI coding-assistant guide, the auditor flagged four problems the write and refine passes had left: payback math that ignored net time saved, a $20 price treated as if every assistant cost the same, absolute wording, and clumsy phrasing. The two-role draft kept a broken break-even sentence.

On a paired transcription experiment (same brief, same writer and refiner; only one draft got an independent audit), the auditor failed five items: a repeated H1; a rule that said “pay if you will publish” and later said a newsletter could stay on free plus edits; a “10+ hours a month” threshold that did not convert to money; Rev AI and Rev’s human-reviewed product treated as one offering; and an unsourced accuracy-ceiling claim. Those five never appeared in the two-role edit notes.

A later topic pack also flagged mixing per-minute with per-hour denominators in one “worth it” sentence. Layer 1 will not see that. Layer 3 is there for that class of miss. In these tests, quality depended more on an independent pass than on a dearer writer.

Who should build this, and a smaller version

Build the four-stage loop if you publish prices, comparisons, or advice a reader might spend money on, and if a wrong sentence costs more than an extra review hour. A two-person content shop, a freelancer who sells retainers on the back of guides, a site that will later put affiliate links next to recommendations.

Use a thinner version if you ship one unsourced opinion a week. Then the minimum is a validate script or checklist, a next-day read, and a rule that external facts need a date and a URL. Worked examples can stay if they are labeled.

Skip the loop for a private note, a Slack answer, or a first outline. The pipeline is slower than a single chat. On this desk it earned its keep when the auditor found contradictions; it is overhead when the piece contains no facts.

Google’s spam policies target scaled, low-value pages built mainly to manipulate search. A human edit does not, by itself, make a page safe; scale, purpose, and whether the page helps a reader still decide. The operational read is the same as E-E-A-T: show the work, name the date, do not ship a farm.

A one-page pre-publish template

Copy this into the top of a working doc. Do not publish until every line is yes or “not applicable.”

  1. Frontmatter has title, description, pubDate (YYYY-MM-DD), draft: true until a person says otherwise.
  2. Display name is Istar only. No banned marketing stems. The piece is not a “top N tools” list.
  3. Opening paragraph states a dated judgment (month and year), not a slogan.
  4. External facts have a check date and a URL. Worked examples are labeled with their inputs. Unverified rates stay out of a public page (they may sit in a draft as a blank to fill).
  5. No two rules in the piece disagree. No two prices use mixed units without a conversion.
  6. A who-it’s-for and who-it’s-not section exists.
  7. A reader can run one calculation or one checklist without opening another tab for the method.
  8. An independent pass (other model or other person) has looked only for facts and logic, and every FAIL is fixed or explicitly waived.
  9. validate.mjs (or your equivalent) exits 0.
  10. A human accepts the facts. The script does not publish.

If you want the mechanical version this site uses, start with validate.mjs and pipeline-batch.sh in docs/agent-pipeline/, then write the house rules the refiner may not ignore. In the comparisons on this desk, quality depended more on refusing to let the same model certify the draft it just wrote than on picking a dearer writer.