Decide whether an experiment should ship, stop, or keep running. — Claude Skill
Um Skill Claude para Claude Code por ElasticFlow✓ — executar /ab-test-analysis no Claude·Atualizado em 12 de jun. de 2026·vmanual@2026-06-12
Reads experiment results, sample size, conversion changes, guardrail metrics, and business context to recommend a clear ship, stop, or continue decision.
- Explains experiment results in plain language instead of only reporting a p-value or dashboard screenshot.
- Checks primary metric, sample size, segment differences, and guardrail metrics before recommending a decision.
- Separates meaningful lift from noise, novelty effects, broken tracking, or mixed segment behavior.
- Returns a decision memo with evidence, risk, next test idea, and what a human should confirm.
A growth marketer screenshots the experiment dashboard, says the test is up, and debates confidence in a meeting.
Run /ab-test-analysis with the result table and context. The skill returns a decision, evidence, risks, and follow-up test.
Para quem é
Turn experiment results into clear launch, stop, or continue decisions.
Ver skills para esta funçãoUnderstand experiment impact on user behavior, product risk, and next iteration.
Ver skills para esta funçãoSpot tracking, sample, and guardrail issues before stakeholders trust the readout.
Ver skills para esta funçãoO que faz
Turn Optimizely, Amplitude, or GA results into a decision memo.
Check whether a conversion lift came with revenue, support, speed, or retention risk.
Find tracking, segment, sample size, or timing problems before trusting the result.
Como funciona
Share the experiment goal, variants, dates, traffic, sample size, and metric results.
Add guardrail metrics such as churn, revenue, refund rate, support tickets, or page speed if available.
The skill interprets lift, confidence, practical significance, and business risk.
It recommends ship, stop, keep running, or re-run with a cleaner design.
Opções de entrada
Hypothesis, variants, dates, traffic split, audience, and success metric.
Exemplo
Experiment: Pricing page CTA copy. Variant A: Start free trial. Variant B: Build my plan. Dates: June 1-14. Results: - A: 24,100 visitors, 1,084 trials, 4.5% conversion. - B: 23,900 visitors, 1,267 trials, 5.3% conversion. - Confidence shown in tool: 96%. Guardrails: - Paid conversion after trial: A 18.4%, B 17.9%. - Support questions about pricing increased 11% for B. Need: ship, stop, or continue, and what to tell leadership.
Ship to 50% first, not 100%. Variant B improves trial start rate from 4.5% to 5.3%, but paid conversion is slightly lower and pricing questions increased.
The lift is likely real for trial starts. The business impact is not proven until trial quality and support load are watched for one more week.
Pricing confusion may be rising. Add a pricing FAQ link near the CTA before full rollout.
The new CTA increases trial starts by about 18% relative, but we will roll out gradually while monitoring paid conversion and pricing support tickets.
Confirm attribution window, whether paid conversion is mature enough, and whether support ticket tagging is consistent.
Métricas que melhora
Funciona com
Compare result tables and write the decision memo.
Use experiment results, variants, confidence, and traffic allocation.
Check product behavior, activation, retention, and segment impact.
Use traffic, conversion, and acquisition context.
Em qualquer lugar
Paste the notes, exports, screenshots, or summaries you already have. The skill works without a connected system.
Connect the relevant support, analytics, CRM, or data tool when you want fresher source evidence.
Quer usar A/B Test Analysis?
Escolha como começar.
Instale e execute este skill localmente no seu computador.
Abra um terminal no seu computador e cole este comando:
Visite o repositório do GitHub e siga as instruções de instalação no README.
Inicie o Claude Code, depois escreva o comando:
A/B Test Analysis
Command: /ab-test-analysis
When to use it
Reads experiment results, sample size, conversion changes, guardrail metrics, and business context to recommend a clear ship, stop, or continue decision.
What the skill produces
- Explains experiment results in plain language instead of only reporting a p-value or dashboard screenshot.
- Checks primary metric, sample size, segment differences, and guardrail metrics before recommending a decision.
- Separates meaningful lift from noise, novelty effects, broken tracking, or mixed segment behavior.
- Returns a decision memo with evidence, risk, next test idea, and what a human should confirm.
Inputs to provide
- Experiment setup: Hypothesis, variants, dates, traffic split, audience, and success metric.
- Result table: Visitors, conversions, conversion rate, revenue, confidence, or exported dashboard numbers.
- Guardrails and context: Support volume, refunds, page speed, churn, revenue per user, or segment constraints.
Recommended flow
- Share the experiment goal, variants, dates, traffic, sample size, and metric results.
- Add guardrail metrics such as churn, revenue, refund rate, support tickets, or page speed if available.
- The skill interprets lift, confidence, practical significance, and business risk.
- It recommends ship, stop, keep running, or re-run with a cleaner design.
Useful result example
Decision
Ship to 50% first, not 100%. Variant B improves trial start rate from 4.5% to 5.3%, but paid conversion is slightly lower and pricing questions increased.
Why
The lift is likely real for trial starts. The business impact is not proven until trial quality and support load are watched for one more week.
Guardrail risk
Pricing confusion may be rising. Add a pricing FAQ link near the CTA before full rollout.
Leadership wording
The new CTA increases trial starts by about 18% relative, but we will roll out gradually while monitoring paid conversion and pricing support tickets.
Human review
Confirm attribution window, whether paid conversion is mature enough, and whether support ticket tagging is consistent.
Guardrails
- Keep user-provided numbers, dates, tool names, commands, IDs, URLs, and rules intact.
- Do not invent a source, metric, owner, decision, or risk that is not present in the supplied material.
- Clearly mark what a human must confirm before publishing, changing a tool, or making a business decision.
Documentos de referência
A/B Test Analysis
ElasticFlow editorial instructions for presenting /ab-test-analysis in the catalogue.
Purpose
Reads experiment results, sample size, conversion changes, guardrail metrics, and business context to recommend a clear ship, stop, or continue decision.
Non-technical presentation
Explain the business problem, what the user provides, what the AI returns, and what a human still needs to confirm. Avoid implementation detail unless the user supplied it.
Catalogue Presentation Method
Every skill should read clearly for a business owner: current painful workflow, better workflow, concrete example, and review checklist.
The page must answer four questions: when to use it, what to provide, what the AI returns, and which human decision remains.