Decide whether an experiment should ship, stop, or keep running. — Claude Skill
Un Skill de Claude para Claude Code por ElasticFlow✓ — ejecutar /ab-test-analysis en Claude·Actualizado el 12 jun 2026·vmanual@2026-06-12
Reads experiment results, sample size, conversion changes, guardrail metrics, and business context to recommend a clear ship, stop, or continue decision.
- Explains experiment results in plain language instead of only reporting a p-value or dashboard screenshot.
- Checks primary metric, sample size, segment differences, and guardrail metrics before recommending a decision.
- Separates meaningful lift from noise, novelty effects, broken tracking, or mixed segment behavior.
- Returns a decision memo with evidence, risk, next test idea, and what a human should confirm.
A growth marketer screenshots the experiment dashboard, says the test is up, and debates confidence in a meeting.
Run /ab-test-analysis with the result table and context. The skill returns a decision, evidence, risks, and follow-up test.
Para quién es
Turn experiment results into clear launch, stop, or continue decisions.
Ver skills para este rolUnderstand experiment impact on user behavior, product risk, and next iteration.
Ver skills para este rolSpot tracking, sample, and guardrail issues before stakeholders trust the readout.
Ver skills para este rolQué hace
Turn Optimizely, Amplitude, or GA results into a decision memo.
Check whether a conversion lift came with revenue, support, speed, or retention risk.
Find tracking, segment, sample size, or timing problems before trusting the result.
Cómo funciona
Share the experiment goal, variants, dates, traffic, sample size, and metric results.
Add guardrail metrics such as churn, revenue, refund rate, support tickets, or page speed if available.
The skill interprets lift, confidence, practical significance, and business risk.
It recommends ship, stop, keep running, or re-run with a cleaner design.
Opciones de entrada
Hypothesis, variants, dates, traffic split, audience, and success metric.
Ejemplo
Experiment: Pricing page CTA copy. Variant A: Start free trial. Variant B: Build my plan. Dates: June 1-14. Results: - A: 24,100 visitors, 1,084 trials, 4.5% conversion. - B: 23,900 visitors, 1,267 trials, 5.3% conversion. - Confidence shown in tool: 96%. Guardrails: - Paid conversion after trial: A 18.4%, B 17.9%. - Support questions about pricing increased 11% for B. Need: ship, stop, or continue, and what to tell leadership.
Ship to 50% first, not 100%. Variant B improves trial start rate from 4.5% to 5.3%, but paid conversion is slightly lower and pricing questions increased.
The lift is likely real for trial starts. The business impact is not proven until trial quality and support load are watched for one more week.
Pricing confusion may be rising. Add a pricing FAQ link near the CTA before full rollout.
The new CTA increases trial starts by about 18% relative, but we will roll out gradually while monitoring paid conversion and pricing support tickets.
Confirm attribution window, whether paid conversion is mature enough, and whether support ticket tagging is consistent.
Métricas que mejora
Funciona con
Compare result tables and write the decision memo.
Use experiment results, variants, confidence, and traffic allocation.
Check product behavior, activation, retention, and segment impact.
Use traffic, conversion, and acquisition context.
En cualquier lugar
Paste the notes, exports, screenshots, or summaries you already have. The skill works without a connected system.
Connect the relevant support, analytics, CRM, or data tool when you want fresher source evidence.
¿Quieres usar A/B Test Analysis?
Elige cómo empezar.
Instala y ejecuta este skill localmente en tu computadora.
Abre una terminal en tu computadora y pega este comando:
Visita el repositorio de GitHub y sigue las instrucciones de instalación del README.
Inicia Claude Code, luego escribe el comando:
A/B Test Analysis
Command: /ab-test-analysis
When to use it
Reads experiment results, sample size, conversion changes, guardrail metrics, and business context to recommend a clear ship, stop, or continue decision.
What the skill produces
- Explains experiment results in plain language instead of only reporting a p-value or dashboard screenshot.
- Checks primary metric, sample size, segment differences, and guardrail metrics before recommending a decision.
- Separates meaningful lift from noise, novelty effects, broken tracking, or mixed segment behavior.
- Returns a decision memo with evidence, risk, next test idea, and what a human should confirm.
Inputs to provide
- Experiment setup: Hypothesis, variants, dates, traffic split, audience, and success metric.
- Result table: Visitors, conversions, conversion rate, revenue, confidence, or exported dashboard numbers.
- Guardrails and context: Support volume, refunds, page speed, churn, revenue per user, or segment constraints.
Recommended flow
- Share the experiment goal, variants, dates, traffic, sample size, and metric results.
- Add guardrail metrics such as churn, revenue, refund rate, support tickets, or page speed if available.
- The skill interprets lift, confidence, practical significance, and business risk.
- It recommends ship, stop, keep running, or re-run with a cleaner design.
Useful result example
Decision
Ship to 50% first, not 100%. Variant B improves trial start rate from 4.5% to 5.3%, but paid conversion is slightly lower and pricing questions increased.
Why
The lift is likely real for trial starts. The business impact is not proven until trial quality and support load are watched for one more week.
Guardrail risk
Pricing confusion may be rising. Add a pricing FAQ link near the CTA before full rollout.
Leadership wording
The new CTA increases trial starts by about 18% relative, but we will roll out gradually while monitoring paid conversion and pricing support tickets.
Human review
Confirm attribution window, whether paid conversion is mature enough, and whether support ticket tagging is consistent.
Guardrails
- Keep user-provided numbers, dates, tool names, commands, IDs, URLs, and rules intact.
- Do not invent a source, metric, owner, decision, or risk that is not present in the supplied material.
- Clearly mark what a human must confirm before publishing, changing a tool, or making a business decision.
Documentos de referencia
A/B Test Analysis
ElasticFlow editorial instructions for presenting /ab-test-analysis in the catalogue.
Purpose
Reads experiment results, sample size, conversion changes, guardrail metrics, and business context to recommend a clear ship, stop, or continue decision.
Non-technical presentation
Explain the business problem, what the user provides, what the AI returns, and what a human still needs to confirm. Avoid implementation detail unless the user supplied it.
Catalogue Presentation Method
Every skill should read clearly for a business owner: current painful workflow, better workflow, concrete example, and review checklist.
The page must answer four questions: when to use it, what to provide, what the AI returns, and which human decision remains.