Most cold email advice tells you to test subject lines. Nobody tells you how to close the loop automatically — pause what's dying, clone what's working, reload fresh leads, and repeat. That's the loop we built, and it runs every Monday without touching it.
Here's exactly how it works, with the real numbers behind it.
The Problem With Manual A/B Testing
Running cold email campaigns manually has 3 failure modes. First, you forget to check — campaigns keep burning leads on dead copy for weeks. Second, you check but don't act fast enough — by the time you pause a loser, it's already eaten 200 contacts. Third, you act but don't redistribute — the fresh leads sit paused in losing campaigns instead of going to work.
The fix is a weekly automated loop that handles all 3.
The Setup: 5 Campaigns, Same Audience, Different Hooks
The audience was 8,831 verified party vendor leads — bounce house rental companies, tent suppliers, catering outfits, DJs. The goal was to find which cold email hook makes them reply.
5 campaigns launched simultaneously. Each campaign received roughly 1,766 leads. All 5 used the same follow-up sequence. The only variable was the first email — subject line and opening hook.
| Campaign | Subject Line | Hook | Contacted | Replies | Rate |
|---|---|---|---|---|---|
| Test E — Who Answers | {{firstName}} - who answers? | Question about missed calls while on a job | 110 | 9 | 8.2% 🏆 |
| Test C — $800/week | $800/week | Pain point dollar figure | 74 | 1 | 1.4% |
| Test B — Curiosity | idea for {{companyName}} | Vague value proposition | 65 | 0 | 0% |
| Test D — Free Listing | free listing for {{companyName}} | Free value offer | 71 | 0 | 0% |
The winner was the email that asked a single question and nothing else. No pitch. No value prop. One line:
"When you're setting up at an event and a new customer calls — who answers?"
That's the entire first email. No product mention. No CTA beyond the implied one. The question did the work.
Why One Question Beats a Pitch
Party vendors spend most of their day physically on-site. They can't answer their phone during setup. They know this is a problem — they just haven't thought about it in exactly those terms.
The question makes them answer it in their head. The mental engagement happens before they even hit reply. A pitch about an AI call answering service triggers sales resistance. A question about their actual workday triggers recognition.
The Instantly.ai copywriting framework calls this "resonating variables." The highest-converting copy makes prospects think "damn, that's me." A dollar figure ($800/week) is compelling in theory. A moment they live every Saturday is compelling by default.
The Automation Loop: How It Works
The weekly optimizer runs every Monday at 1pm ET. The loop has 5 steps.
The Code Behind It
The optimizer runs as a Python script against the Instantly v2 API. The key pieces are a paginated lead stat collector, a campaign creator with templated sequences, and a lead splitter/uploader.
# Core logic: rank campaigns, pause losers, create variations
ranked = sorted(active.items(),
key=lambda x: x[1]['replies'] / max(x[1]['contacted'], 1),
reverse=True)
winner_id, winner_stats = ranked[0]
winner_rate = winner_stats['replies'] / winner_stats['contacted']
losers = [(cid, s) for cid, s in active.items()
if cid != winner_id
and s['contacted'] >= MIN_CONTACTS
and s['replies'] / s['contacted'] < LOSER_MAX_RATE
and campaigns[cid]['status'] == 1]
for cid, s in losers:
pause_campaign(cid)
fresh_leads.extend(get_fresh_leads(cid))The script stores a library of subject line and body variations. Each week it seeds the random picker with the week number — so the same variations don't repeat, but the selection is deterministic if you need to debug it.
What the Data Shows So Far
Across all campaigns running in the same period — KaiCalls home service tests, law firm outreach, party vendor splits — 3 patterns held consistently.
- Questions outperform statements. "Who answers?" at 8.2% beat "$800/week" at 1.4% beat every feature-led subject line at 0%. The more the opening sounds like a real person thinking out loud, the better it performs.
- Longer subject lines underperform. "idea for {{companyName}}" — 8 words — got 0 replies. "{{firstName}} - who answers?" — 4 words — got 9. Shorter creates more cognitive open loops.
- Pain point specificity matters less than situational specificity. "$800/week" is a specific pain point. "Setting up at an event" is a specific situation. The situation outperforms the number because it creates a visual, not just a concern.
The Compounding Effect
The loop compounds in 2 ways. First, every week the winning hook gets tested against more variations — the system gets better at finding angles that work for this specific audience. Second, no leads get wasted. Paused campaigns don't mean dead leads. The 5,299 uncontacted leads from the first generation of losers immediately fed the next generation of tests.
Total leads in the system: 8,831. Total wasted: 0.
The optimizer runs fully automated via an OpenClaw cron job. Every Monday it posts the results to a Discord channel — winner announced, losers named, new campaigns live. No manual work required until a human needs to approve an action or respond to a reply.
What to Test Next
The next generation of variations runs off the "who answers?" hook with different framings. The 4 active campaigns test: busy season timing, a specific Saturday afternoon scenario, a one-question format with no follow-up copy, and the same hook combined with a free listing offer.
The hypothesis is that the hook works because it's situationally specific. The next test is whether adding any additional context in email 1 helps or hurts. Every variation keeps the 4-step follow-up sequence identical — bump on day 2, case study on day 5, breakup on day 8.
Results in two weeks. The optimizer will report them automatically.