Decide whether an experiment should ship, stop, or keep running. — Claude Skill
Claude Code向けClaudeスキル · 提供：ElasticFlow✓ · 実行：/ab-test-analysis（Claude内）·更新日：2026年6月12日·vmanual@2026-06-12
Reads experiment results, sample size, conversion changes, guardrail metrics, and business context to recommend a clear ship, stop, or continue decision.
- Explains experiment results in plain language instead of only reporting a p-value or dashboard screenshot.
- Checks primary metric, sample size, segment differences, and guardrail metrics before recommending a decision.
- Separates meaningful lift from noise, novelty effects, broken tracking, or mixed segment behavior.
- Returns a decision memo with evidence, risk, next test idea, and what a human should confirm.
A growth marketer screenshots the experiment dashboard, says the test is up, and debates confidence in a meeting.
Run /ab-test-analysis with the result table and context. The skill returns a decision, evidence, risks, and follow-up test.
対象ユーザー
Turn experiment results into clear launch, stop, or continue decisions.
この役職のスキルを見るUnderstand experiment impact on user behavior, product risk, and next iteration.
この役職のスキルを見るSpot tracking, sample, and guardrail issues before stakeholders trust the readout.
この役職のスキルを見る機能
Turn Optimizely, Amplitude, or GA results into a decision memo.
Check whether a conversion lift came with revenue, support, speed, or retention risk.
Find tracking, segment, sample size, or timing problems before trusting the result.
仕組み
Share the experiment goal, variants, dates, traffic, sample size, and metric results.
Add guardrail metrics such as churn, revenue, refund rate, support tickets, or page speed if available.
The skill interprets lift, confidence, practical significance, and business risk.
It recommends ship, stop, keep running, or re-run with a cleaner design.
入力オプション
Hypothesis, variants, dates, traffic split, audience, and success metric.
例
Experiment: Pricing page CTA copy. Variant A: Start free trial. Variant B: Build my plan. Dates: June 1-14. Results: - A: 24,100 visitors, 1,084 trials, 4.5% conversion. - B: 23,900 visitors, 1,267 trials, 5.3% conversion. - Confidence shown in tool: 96%. Guardrails: - Paid conversion after trial: A 18.4%, B 17.9%. - Support questions about pricing increased 11% for B. Need: ship, stop, or continue, and what to tell leadership.
Ship to 50% first, not 100%. Variant B improves trial start rate from 4.5% to 5.3%, but paid conversion is slightly lower and pricing questions increased.
The lift is likely real for trial starts. The business impact is not proven until trial quality and support load are watched for one more week.
Pricing confusion may be rising. Add a pricing FAQ link near the CTA before full rollout.
The new CTA increases trial starts by about 18% relative, but we will roll out gradually while monitoring paid conversion and pricing support tickets.
Confirm attribution window, whether paid conversion is mature enough, and whether support ticket tagging is consistent.
改善される指標
対応ツール
どこでも使える
Paste the notes, exports, screenshots, or summaries you already have. The skill works without a connected system.
Connect the relevant support, analytics, CRM, or data tool when you want fresher source evidence.
A/B Test Analysisを使ってみますか？
始め方を選択してください。
A/B Test Analysis
Command: /ab-test-analysis
When to use it
Reads experiment results, sample size, conversion changes, guardrail metrics, and business context to recommend a clear ship, stop, or continue decision.
What the skill produces
- Explains experiment results in plain language instead of only reporting a p-value or dashboard screenshot.
- Checks primary metric, sample size, segment differences, and guardrail metrics before recommending a decision.
- Separates meaningful lift from noise, novelty effects, broken tracking, or mixed segment behavior.
- Returns a decision memo with evidence, risk, next test idea, and what a human should confirm.
Inputs to provide
- Experiment setup: Hypothesis, variants, dates, traffic split, audience, and success metric.
- Result table: Visitors, conversions, conversion rate, revenue, confidence, or exported dashboard numbers.
- Guardrails and context: Support volume, refunds, page speed, churn, revenue per user, or segment constraints.
Recommended flow
- Share the experiment goal, variants, dates, traffic, sample size, and metric results.
- Add guardrail metrics such as churn, revenue, refund rate, support tickets, or page speed if available.
- The skill interprets lift, confidence, practical significance, and business risk.
- It recommends ship, stop, keep running, or re-run with a cleaner design.
Useful result example
Decision
Ship to 50% first, not 100%. Variant B improves trial start rate from 4.5% to 5.3%, but paid conversion is slightly lower and pricing questions increased.
Why
The lift is likely real for trial starts. The business impact is not proven until trial quality and support load are watched for one more week.
Guardrail risk
Pricing confusion may be rising. Add a pricing FAQ link near the CTA before full rollout.
Leadership wording
The new CTA increases trial starts by about 18% relative, but we will roll out gradually while monitoring paid conversion and pricing support tickets.
Human review
Confirm attribution window, whether paid conversion is mature enough, and whether support ticket tagging is consistent.
Guardrails
- Keep user-provided numbers, dates, tool names, commands, IDs, URLs, and rules intact.
- Do not invent a source, metric, owner, decision, or risk that is not present in the supplied material.
- Clearly mark what a human must confirm before publishing, changing a tool, or making a business decision.
参照ドキュメント
A/B Test Analysis
ElasticFlow editorial instructions for presenting /ab-test-analysis in the catalogue.
Purpose
Reads experiment results, sample size, conversion changes, guardrail metrics, and business context to recommend a clear ship, stop, or continue decision.
Non-technical presentation
Explain the business problem, what the user provides, what the AI returns, and what a human still needs to confirm. Avoid implementation detail unless the user supplied it.
Catalogue Presentation Method
Every skill should read clearly for a business owner: current painful workflow, better workflow, concrete example, and review checklist.
The page must answer four questions: when to use it, what to provide, what the AI returns, and which human decision remains.