Decide whether an experiment should ship, stop, or keep running. — Claude Skill
En Claude-skill för Claude Code av ElasticFlow✓ — kör /ab-test-analysis i Claude·Uppdaterad 12 juni 2026·vmanual@2026-06-12
Reads experiment results, sample size, conversion changes, guardrail metrics, and business context to recommend a clear ship, stop, or continue decision.
- Explains experiment results in plain language instead of only reporting a p-value or dashboard screenshot.
- Checks primary metric, sample size, segment differences, and guardrail metrics before recommending a decision.
- Separates meaningful lift from noise, novelty effects, broken tracking, or mixed segment behavior.
- Returns a decision memo with evidence, risk, next test idea, and what a human should confirm.
A growth marketer screenshots the experiment dashboard, says the test is up, and debates confidence in a meeting.
Run /ab-test-analysis with the result table and context. The skill returns a decision, evidence, risks, and follow-up test.
För vem
Turn experiment results into clear launch, stop, or continue decisions.
Visa skills för denna rollUnderstand experiment impact on user behavior, product risk, and next iteration.
Visa skills för denna rollSpot tracking, sample, and guardrail issues before stakeholders trust the readout.
Visa skills för denna rollVad det gör
Turn Optimizely, Amplitude, or GA results into a decision memo.
Check whether a conversion lift came with revenue, support, speed, or retention risk.
Find tracking, segment, sample size, or timing problems before trusting the result.
Så fungerar det
Share the experiment goal, variants, dates, traffic, sample size, and metric results.
Add guardrail metrics such as churn, revenue, refund rate, support tickets, or page speed if available.
The skill interprets lift, confidence, practical significance, and business risk.
It recommends ship, stop, keep running, or re-run with a cleaner design.
Indatametoder
Hypothesis, variants, dates, traffic split, audience, and success metric.
Exempel
Experiment: Pricing page CTA copy. Variant A: Start free trial. Variant B: Build my plan. Dates: June 1-14. Results: - A: 24,100 visitors, 1,084 trials, 4.5% conversion. - B: 23,900 visitors, 1,267 trials, 5.3% conversion. - Confidence shown in tool: 96%. Guardrails: - Paid conversion after trial: A 18.4%, B 17.9%. - Support questions about pricing increased 11% for B. Need: ship, stop, or continue, and what to tell leadership.
Ship to 50% first, not 100%. Variant B improves trial start rate from 4.5% to 5.3%, but paid conversion is slightly lower and pricing questions increased.
The lift is likely real for trial starts. The business impact is not proven until trial quality and support load are watched for one more week.
Pricing confusion may be rising. Add a pricing FAQ link near the CTA before full rollout.
The new CTA increases trial starts by about 18% relative, but we will roll out gradually while monitoring paid conversion and pricing support tickets.
Confirm attribution window, whether paid conversion is mature enough, and whether support ticket tagging is consistent.
Förbättrade mätvärden
Fungerar med
Compare result tables and write the decision memo.
Use experiment results, variants, confidence, and traffic allocation.
Check product behavior, activation, retention, and segment impact.
Use traffic, conversion, and acquisition context.
Fungerar överallt
Paste the notes, exports, screenshots, or summaries you already have. The skill works without a connected system.
Connect the relevant support, analytics, CRM, or data tool when you want fresher source evidence.
Vill du använda A/B Test Analysis?
Välj hur du vill komma igång.
Installera och kör denna skill lokalt på din dator.
Öppna en terminal på din dator och klistra in detta kommando:
Besök GitHub-repot och följ installationsinstruktionerna i README.
Starta Claude Code och skriv kommandot:
A/B Test Analysis
Command: /ab-test-analysis
When to use it
Reads experiment results, sample size, conversion changes, guardrail metrics, and business context to recommend a clear ship, stop, or continue decision.
What the skill produces
- Explains experiment results in plain language instead of only reporting a p-value or dashboard screenshot.
- Checks primary metric, sample size, segment differences, and guardrail metrics before recommending a decision.
- Separates meaningful lift from noise, novelty effects, broken tracking, or mixed segment behavior.
- Returns a decision memo with evidence, risk, next test idea, and what a human should confirm.
Inputs to provide
- Experiment setup: Hypothesis, variants, dates, traffic split, audience, and success metric.
- Result table: Visitors, conversions, conversion rate, revenue, confidence, or exported dashboard numbers.
- Guardrails and context: Support volume, refunds, page speed, churn, revenue per user, or segment constraints.
Recommended flow
- Share the experiment goal, variants, dates, traffic, sample size, and metric results.
- Add guardrail metrics such as churn, revenue, refund rate, support tickets, or page speed if available.
- The skill interprets lift, confidence, practical significance, and business risk.
- It recommends ship, stop, keep running, or re-run with a cleaner design.
Useful result example
Decision
Ship to 50% first, not 100%. Variant B improves trial start rate from 4.5% to 5.3%, but paid conversion is slightly lower and pricing questions increased.
Why
The lift is likely real for trial starts. The business impact is not proven until trial quality and support load are watched for one more week.
Guardrail risk
Pricing confusion may be rising. Add a pricing FAQ link near the CTA before full rollout.
Leadership wording
The new CTA increases trial starts by about 18% relative, but we will roll out gradually while monitoring paid conversion and pricing support tickets.
Human review
Confirm attribution window, whether paid conversion is mature enough, and whether support ticket tagging is consistent.
Guardrails
- Keep user-provided numbers, dates, tool names, commands, IDs, URLs, and rules intact.
- Do not invent a source, metric, owner, decision, or risk that is not present in the supplied material.
- Clearly mark what a human must confirm before publishing, changing a tool, or making a business decision.
Referensdokument
A/B Test Analysis
ElasticFlow editorial instructions for presenting /ab-test-analysis in the catalogue.
Purpose
Reads experiment results, sample size, conversion changes, guardrail metrics, and business context to recommend a clear ship, stop, or continue decision.
Non-technical presentation
Explain the business problem, what the user provides, what the AI returns, and what a human still needs to confirm. Avoid implementation detail unless the user supplied it.
Catalogue Presentation Method
Every skill should read clearly for a business owner: current painful workflow, better workflow, concrete example, and review checklist.
The page must answer four questions: when to use it, what to provide, what the AI returns, and which human decision remains.