Decide whether an experiment should ship, stop, or keep running. — Claude Skill
Uno skill Claude per Claude Code di ElasticFlow✓ — esegui /ab-test-analysis in Claude·Aggiornato il 12 giu 2026·vmanual@2026-06-12
Reads experiment results, sample size, conversion changes, guardrail metrics, and business context to recommend a clear ship, stop, or continue decision.
- Explains experiment results in plain language instead of only reporting a p-value or dashboard screenshot.
- Checks primary metric, sample size, segment differences, and guardrail metrics before recommending a decision.
- Separates meaningful lift from noise, novelty effects, broken tracking, or mixed segment behavior.
- Returns a decision memo with evidence, risk, next test idea, and what a human should confirm.
A growth marketer screenshots the experiment dashboard, says the test is up, and debates confidence in a meeting.
Run /ab-test-analysis with the result table and context. The skill returns a decision, evidence, risks, and follow-up test.
Per chi
Turn experiment results into clear launch, stop, or continue decisions.
Vedi gli skill per questo ruoloUnderstand experiment impact on user behavior, product risk, and next iteration.
Vedi gli skill per questo ruoloSpot tracking, sample, and guardrail issues before stakeholders trust the readout.
Vedi gli skill per questo ruoloCosa fa
Turn Optimizely, Amplitude, or GA results into a decision memo.
Check whether a conversion lift came with revenue, support, speed, or retention risk.
Find tracking, segment, sample size, or timing problems before trusting the result.
Come funziona
Share the experiment goal, variants, dates, traffic, sample size, and metric results.
Add guardrail metrics such as churn, revenue, refund rate, support tickets, or page speed if available.
The skill interprets lift, confidence, practical significance, and business risk.
It recommends ship, stop, keep running, or re-run with a cleaner design.
Opzioni di input
Hypothesis, variants, dates, traffic split, audience, and success metric.
Esempio
Experiment: Pricing page CTA copy. Variant A: Start free trial. Variant B: Build my plan. Dates: June 1-14. Results: - A: 24,100 visitors, 1,084 trials, 4.5% conversion. - B: 23,900 visitors, 1,267 trials, 5.3% conversion. - Confidence shown in tool: 96%. Guardrails: - Paid conversion after trial: A 18.4%, B 17.9%. - Support questions about pricing increased 11% for B. Need: ship, stop, or continue, and what to tell leadership.
Ship to 50% first, not 100%. Variant B improves trial start rate from 4.5% to 5.3%, but paid conversion is slightly lower and pricing questions increased.
The lift is likely real for trial starts. The business impact is not proven until trial quality and support load are watched for one more week.
Pricing confusion may be rising. Add a pricing FAQ link near the CTA before full rollout.
The new CTA increases trial starts by about 18% relative, but we will roll out gradually while monitoring paid conversion and pricing support tickets.
Confirm attribution window, whether paid conversion is mature enough, and whether support ticket tagging is consistent.
Metriche migliorate
Funziona con
Compare result tables and write the decision memo.
Use experiment results, variants, confidence, and traffic allocation.
Check product behavior, activation, retention, and segment impact.
Use traffic, conversion, and acquisition context.
Funziona ovunque
Paste the notes, exports, screenshots, or summaries you already have. The skill works without a connected system.
Connect the relevant support, analytics, CRM, or data tool when you want fresher source evidence.
Vuoi usare A/B Test Analysis?
Scegli come iniziare.
Installa ed esegui questo skill localmente sul tuo computer.
Apri un terminale sul tuo computer e incolla questo comando:
Visita il repository GitHub e segui le istruzioni di installazione nel README.
Avvia Claude Code, poi digita il comando:
A/B Test Analysis
Command: /ab-test-analysis
When to use it
Reads experiment results, sample size, conversion changes, guardrail metrics, and business context to recommend a clear ship, stop, or continue decision.
What the skill produces
- Explains experiment results in plain language instead of only reporting a p-value or dashboard screenshot.
- Checks primary metric, sample size, segment differences, and guardrail metrics before recommending a decision.
- Separates meaningful lift from noise, novelty effects, broken tracking, or mixed segment behavior.
- Returns a decision memo with evidence, risk, next test idea, and what a human should confirm.
Inputs to provide
- Experiment setup: Hypothesis, variants, dates, traffic split, audience, and success metric.
- Result table: Visitors, conversions, conversion rate, revenue, confidence, or exported dashboard numbers.
- Guardrails and context: Support volume, refunds, page speed, churn, revenue per user, or segment constraints.
Recommended flow
- Share the experiment goal, variants, dates, traffic, sample size, and metric results.
- Add guardrail metrics such as churn, revenue, refund rate, support tickets, or page speed if available.
- The skill interprets lift, confidence, practical significance, and business risk.
- It recommends ship, stop, keep running, or re-run with a cleaner design.
Useful result example
Decision
Ship to 50% first, not 100%. Variant B improves trial start rate from 4.5% to 5.3%, but paid conversion is slightly lower and pricing questions increased.
Why
The lift is likely real for trial starts. The business impact is not proven until trial quality and support load are watched for one more week.
Guardrail risk
Pricing confusion may be rising. Add a pricing FAQ link near the CTA before full rollout.
Leadership wording
The new CTA increases trial starts by about 18% relative, but we will roll out gradually while monitoring paid conversion and pricing support tickets.
Human review
Confirm attribution window, whether paid conversion is mature enough, and whether support ticket tagging is consistent.
Guardrails
- Keep user-provided numbers, dates, tool names, commands, IDs, URLs, and rules intact.
- Do not invent a source, metric, owner, decision, or risk that is not present in the supplied material.
- Clearly mark what a human must confirm before publishing, changing a tool, or making a business decision.
Documenti di riferimento
A/B Test Analysis
ElasticFlow editorial instructions for presenting /ab-test-analysis in the catalogue.
Purpose
Reads experiment results, sample size, conversion changes, guardrail metrics, and business context to recommend a clear ship, stop, or continue decision.
Non-technical presentation
Explain the business problem, what the user provides, what the AI returns, and what a human still needs to confirm. Avoid implementation detail unless the user supplied it.
Catalogue Presentation Method
Every skill should read clearly for a business owner: current painful workflow, better workflow, concrete example, and review checklist.
The page must answer four questions: when to use it, what to provide, what the AI returns, and which human decision remains.