A CI/CD pipeline for marketing content built in Python. Brief → Write → Gate → Approve → Log → Learn. Here's how it actually works.
Every Claude Code project has a CLAUDE.md — the agent reads it on startup to learn what tools exist, what the codebase conventions are, what good output looks like. Without it the agent guesses. The harness applies the same pattern to marketing: MARKETING.md is parsed on every run into a live config object.
# Project conventions Language: TypeScript Test runner: vitest Lint: eslint + prettier # What good code looks like # Agent reads this before writing
# Brand voice Tone: direct, data-backed Four U's min: 12/16 Frameworks: Algorithmic Authorship # Agent reads this before writing
# MarketingConfig parses MARKETING.md on every run MARKETING.channels["kaicalls"] # → { discord_id, ga4_id, gsc_url } MARKETING.formats # → ["blog", "linkedin", "cold-email", ...] MARKETING.framework_map["blog"] # → ["algorithmic-authorship.md", "four-us.md"] MARKETING.learned_defaults # → { avg_word_count: 1200, hook_type: "question" } MARKETING.skill_contract("blog") # → harness/skill-contracts/blog-post.yaml # Now routes to gate policy: blog-publish # Quality scorer runs 28 rules, applies blog-publish.yaml thresholds
One command. Seven stages. Each enforced.
kai-harness run --task blog --site kaicalls --keyword "law firm answering service"
Pulls live GSC data for the keyword — position, CTR, impressions, competing URLs. GA4 session depth. Identifies intent gap between what's ranking and what searchers actually want. Outputs structured JSON.
Reads the brief, reloads MARKETING.md, resolves framework file paths from the Framework Map table, loads winning patterns from what-works.md. Everything feeds a structured prompt.
The unified quality scorer runs 28 rules across 4 categories (Algorithmic Authorship, GEO/AEO Signals, Content Structure, Four U's). Format maps to a YAML gate policy that sets auto-approve/hold/reject thresholds. Returns proposal_id, score, grade, and per-line violations with fix suggestions. All proposals stored in SQLite for audit.
On gate rejection, the revision prompt contains exact violations from the quality scorer — rule ID, line number, violation text, and fix suggestion. No vague “improve quality” instructions. Each violation is actionable.
# What the revision agent receives on failure (from quality scorer): """ Current quality score: 58/100. Fix ONLY the listed issues. ISSUES TO FIX (from quality scorer — each has rule ID, line number, and exact fix): - [AA-01] Condition Before Clause (line 14, 6 occurrences): "If you want better results, do X" → Fix: Move condition after main clause - [GEO-01] Citation Density (line 1, 1 occurrence): "3 citations per 1K words" → Fix: Add 2+ more citations with source attribution - [CS-07] AI Cliche Detected (line 42, 1 occurrence): "it's important to note" → Fix: Remove or replace with specific claim HARD CONSTRAINTS: - Keep at least 1,091 words — do NOT shorten - Do NOT remove numbers, data points, or named examples - Fix every listed issue. Change nothing else. """
Gated draft posts to the site's Discord channel with score, grade, proposal ID, and approve command. Proposal can be approved via CLI: python -m scripts.quality gate --approve <proposal_id>
📝 Content ready for approval
Site: kaicalls | Format: blog | Keyword: law firm answering service
Score: 78/100 (grade C) | Violations: 5 | Policy: blog-publish
Proposal: gp-k4m7n2x8
Approve: python -m scripts.quality gate --approve gp-k4m7n2x8
React ✅ to approve · ❌ to reject with reason
On publish: appends to content_log.json and creates a pending_checks/[id].json file that schedules the 30-day performance pull.
Cron fires at 0 2 * * *. Reads all pending check files. For each URL past check_after: pulls GSC position + CTR, GA4 session duration. Grades winner / average / underperformer.
28 rules across 4 weighted categories. Replaces the old three-script gate (four_us_score.py, banned_word_check.py, seo_lint.py) with a single scored pipeline. Per-line violations. Exact fix suggestions. SQLite audit trail.
{ "file_path": "article.md", "overall_score": 78.3, "overall_grade": "C", "categories": [ {"category": "Algorithmic Authorship", "score": 82.1, "weight": 0.35, "violation_count": 8}, {"category": "GEO/AEO Signals", "score": 65.0, "weight": 0.20, "violation_count": 3}, {"category": "Content Structure", "score": 89.4, "weight": 0.25, "violation_count": 2}, {"category": "Four U's", "score": 71.0, "weight": 0.20, "violation_count": 0} ], "top_fixes": [ {"rule_id": "AA-01", "rule_name": "Condition Before Clause", "impact": 4.2, "violation_count": 6, "suggestion": "Move conditions after the main clause"}, {"rule_id": "GEO-01", "rule_name": "Citation Density", "impact": 3.1, "suggestion": "Add 2+ citations with source attribution per 1K words"}, {"rule_id": "CS-07", "rule_name": "AI Cliche Detected", "impact": 2.8, "suggestion": "Remove 'it's important to note' — replace with specific claim"} ], "metadata": {"word_count": 1423, "sentence_count": 89, "reading_level": 7.2} }
Each format maps to a YAML gate policy. Policies define auto-approve, hold, and reject score thresholds — plus required rules that must pass and blocked terms that trigger instant rejection.
| Policy | Formats | Auto-approve | Hold | Reject | Required Rules | Blocked Terms |
|---|---|---|---|---|---|---|
| blog-publish | blog, seo | ≥85 | 60–85 | <60 | CS-07, AA-23 | “in conclusion”, “harness the power”… |
| linkedin-article | ≥80 | 55–80 | <55 | CS-07, CS-01 | “it's important to note”… | |
| cold-email | cold-email, email-lifecycle | ≥75 | 50–75 | <50 | CS-07 | “guaranteed”, “act now”… |
| press-release | press | ≥80 | 55–80 | <55 | CS-07 | “excited to announce”, “industry-leading”… |
| tiktok-script | tiktok | ≥70 | 45–70 | <45 | CS-07 | “link in bio”, “mind-blowing” |
| meta-ad | meta-ads | ≥70 | 45–70 | <45 | CS-07 | “click here”, “don't miss out”… |
| google-ad | google-ads | ≥65 | 40–65 | <40 | CS-07 | — |
| default | everything else | ≥85 | 60–85 | <60 | — | — |
id: blog-publish auto_approve_above: 85 hold_between: [60, 85] reject_below: 60 require_rules_pass: - CS-07 # no AI cliches (mandatory) - AA-23 # no back-references block_if_terms: - "it's important to note" - "in conclusion" - "in today's fast-paced" - "harness the power" - "without further ado"
# Score content → apply policy → auto-approve / hold / reject # All proposals stored in SQLite for audit proposal = await gate.propose( content=draft, file_path="<harness-draft>", policy_name="blog-publish", ) # Returns: { "proposal_id": "gp-k4m7n2x8", "status": "pending", # approved | pending | rejected "score": 78.3, "grade": "C", "reason": "Score 78 in hold range (60-85)", "violation_count": 13, "top_fixes": [...], "policy": "blog-publish" } # Human approval: # python -m scripts.quality gate --approve gp-k4m7n2x8
Every piece of content feeds data back into the system. Quality compounds over time — the harness learns what works and adjusts both writing prompts and gate thresholds.
0 2 * * * — performance_check.pyReads all pending_checks/*.json files past check_after date. Pulls GSC + GA4 per URL. Writes grade back to content_log.json. Winners trigger pattern extraction.
Fetches published URL, strips HTML, runs the 28-rule quality scorer. Stores per-rule scores in content_log.json alongside GSC/GA4 data. Enables rule-level correlation with search performance.
0 14 * * 1 (Monday) — pattern_extract.pyAnalyzes all winners. Finds shared traits: hook type, format, word count range, publish day, persona. Appends findings to knowledge/playbooks/what-works.md. Every subsequent write step reads this file before drafting.
Compares per-rule quality scores between winners (position ≤5, CTR ≥5%, session ≥90s) and losers. Surfaces which specific rules predict search success. Example: “AA-01 winners avg 0.92 vs losers 0.61 (+51%, n=7+12)”
30 14 * * 1 (Monday +30min) — harness_defaults_update.pyWhen n≥5 data points share a pattern with ≥15% lift: rewrites the ## Learned Defaults section of MARKETING.md. Also auto-updates YAML policy thresholds (auto_approve_above, reject_below) based on winner/loser score distributions. The next kai-harness run calls MARKETING.reload() and picks up the new defaults — no restart, no code change.
The write prompt now contains the learned defaults in MARKETING.learned_defaults. The agent starts from what's been statistically proven, not from theory.
Posts avg quality score trend + top correlated rules to Discord. “Avg score: 74/100 across 23 pieces. Trend: 68 → 74 (↑6 pts). Rules that correlate with winners: AA-01 +51%, GEO-01 +38%, CS-05 +22%”
Everything lives under /opt/cmo-analytics/. The quality scorer is a self-contained module. Gate policies are YAML files — add a new format by dropping a new YAML.
/opt/cmo-analytics/ ├── quality/ # Unified quality scorer module │ ├── engine.py # QualityEngine.score_content() orchestrator │ ├── gate.py # propose(), approve(), reject() — SQLite audit │ ├── gate.db # SQLite proposals table │ ├── parser.py # Markdown parser → Document object │ ├── types.py # QualityReport, CategoryScore, RuleResult, Violation │ ├── config.py # Category weights, grade boundaries, targets │ ├── rules/ │ │ ├── algorithmic_authorship.py # 15 AA rules │ │ ├── geo_signals.py # 4 GEO rules │ │ ├── content_structure.py # 7 CS rules │ │ └── four_us.py # LLM-scored Four U's (2 rules) │ └── policies/ │ ├── blog-publish.yaml # blog, seo formats │ ├── linkedin-article.yaml │ ├── cold-email.yaml │ ├── press-release.yaml │ ├── tiktok-script.yaml │ ├── meta-ad.yaml │ ├── google-ad.yaml │ └── default.yaml ├── scripts/ │ ├── kai_harness.py # CLI — calls quality.gate.propose() for gating │ ├── performance_check.py # 30d check + retro quality scoring + batch-score │ ├── pattern_extract.py # Winner analysis + rule correlation + weekly summary │ ├── harness_defaults_update.py # n≥5 patterns → MARKETING.md + policy YAML updates │ ├── content_log.py # Publish logger + pending_checks scheduler │ └── harness_discord.py # !harness Discord command router └── data/ ├── content_log.json # All published content + performance + quality_retro └── pending_checks/ # 30-day check schedulers
The harness is a content pipeline. It doesn't try to be a CMS, scheduler, or distribution tool.
/tmp/harness_draft.md. Pushing to WordPress, Webflow, or blog directory requires manual action.
ocx image generate tool. Not integrated into gate pipeline.
SITE_FACTS constants are manually maintained. No automated CRM-to-brief data syncing exists currently.
What you see when you run !harness status in Discord or kai-harness status on the CLI.
══════════════════════════════════════════════════ Kai Harness — Status ══════════════════════════════════════════════════ Tracked pieces: 0 Winners: 0 Pending 30d checks: 0 Knowledge base: ✅ MARKETING.md: ✅ Quality scorer: ✅ (unified gate) Policies: blog-publish, linkedin-article, cold-email, press-release, tiktok-script, meta-ad, google-ad, default Formats: blog, linkedin, email-lifecycle, cold-email, tiktok, meta-ads, google-ads, press, seo Sites: kaicalls, buildwithkai, abp, meetkai, connorgallic, vocalscribe