技能 产品商业 封闭测试项目执行指南

封闭测试项目执行指南

v20261108
beta-program
本技能提供在正式发布前运行封闭测试项目的结构化指南。它定义了用户群体、退出标准和每周节奏,以便与设计合作伙伴验证功能。输出包括决策备忘录,根据预先约定的成功标准决定批准、延长、转向或终止功能。
获取技能
73 次下载
概览

Closed Beta Program Playbook

Overview

A structured playbook for running a closed beta that produces clear learning, retained design partners, and an unambiguous decision on whether to proceed to general availability. Most betas fail not because the product is wrong, but because the program is run informally: recruitment is opportunistic, success criteria are implicit, communication is sporadic, and exit gates are never defined. This skill replaces all of that with concrete templates and decision rules.

A good closed beta runs 4-8 weeks, recruits 10-30 participants from three concentric cohorts ("friends, family, fanatics"), publishes a weekly cadence the team commits to, and exits on pre-agreed criteria rather than the calendar. The outputs are a beta program plan, a recruitment script, a weekly cadence template, and an exit memo that either greenlights GA, extends the beta, or kills the feature.

When to Use

  • Pre-GA validation -- A feature has cleared internal alpha but needs external validation under real workloads before a public launch.
  • High-risk feature -- Significant blast radius, regulatory exposure, or pricing changes that warrant a controlled rollout to a small group first.
  • Design partner program -- Recruiting a small set of customers whose feedback shapes v1 scope and pricing.
  • Enterprise rollout -- New module that needs reference customers and case studies before broad sales enablement.

When NOT to Use

  • Continuous trunk-based delivery where every change ships to all users behind feature flags (use launch-playbook/ and a progressive rollout instead).
  • Pure usability testing with 5-7 participants (use a research protocol, not a beta program).
  • Internal-only dogfooding (run a 2-week alpha with engineering and customer-facing staff).

Clarify First

Before building the beta plan, settle the inputs below. When one is missing or vague and would change the result, ask about it; when it wouldn't, use a sensible default and say which one you used.

  • Beta hypothesis & headline outcome: what you are trying to learn and the one result that matters (drives the whole plan and which exit gates you set).
  • Exit gates: the quantitative + qualitative success criteria, agreed before recruiting (resolve to Greenlight / Extend / Pivot / Kill — the decision rule of the program).
  • Target cohort & size: Friends / Family / Fanatics mix and participant count (drives the recruitment script and how you weight feedback).
  • Duration: the 4-8 week window (drives the weekly cadence and when the exit memo is due).

Ask about the two or three inputs that matter most for this request, not the whole list. If the user wants a result straight away, produce it and state your assumptions at the top.

Workflow

  1. Frame the beta hypothesis, target cohort and headline outcome (from create-prd/).
  2. Set exit gates in assets/beta-program-plan.md and get PM, Eng and sponsor sign-off before recruiting. Gates set afterwards drift toward whatever the beta happened to show.
  3. Recruit via assets/recruitment-script.md. Defaults [RECOMMENDED], to adjust for your context:
  • Over-recruit about 2x: roughly half of recruited participants go quiet by week 2, so 2x keeps the active cohort at target. A 6-account enterprise beta with contracted participants needs far less.
  • Fanatics at 30% or more: friends and colleagues are polite; cold-recruited users with active pain are the ones who report real friction.
  • Weight Fanatic feedback about 2x when tallying, for the same reason; say so in the exit memo.
  1. Stand up the comms channel and publish the cadence on day 1; run it weekly (assets/weekly-cadence-template.md).
  2. Decide at the pre-agreed date with assets/exit-memo.md: Greenlight, Extend, Pivot or Kill, against the gates as written.

Done when the exit memo records each gate's result against its pre-agreed threshold and a decision signed by the same people who signed the gates.

Framework tables, troubleshooting and success criteria: references/beta-framework.md.

Anti-Patterns

Recruiting only friends

Mistake: The cohort is 30 employees and 5 advisor friends, with no Fanatics cold-recruited from the ICP.
Why it happens: Friends say yes within two days, and cold recruiting is slow.
Instead: Sequence Friends, then Family, then Fanatics, and keep Fanatics at 30% of the cohort or more. Screen them for active pain in the problem area.

Recruitment that leads with the feature

Mistake: The outreach email opens "We're launching an exciting new AI assistant, want to try it?"
Why it happens: The team is proud of what it built and leads with it.
Instead: Name the participant's problem in the first paragraph, then the ask (hours per week, duration) and the payoff. Target people who described this pain in discovery, starting from assets/recruitment-script.md.

Building one-offs for a single participant

Mistake: The largest beta customer asks for a custom export format, and engineering builds it in week 3 for that customer alone.
Why it happens: The participant is a paying account, and keeping them happy feels like protecting the beta.
Instead: Log the request in the customer-feedback-triage/ queue with the rest of the beta input. Commit to it for v1.1 only if the broader signal supports it, and track sales-promised customizations separately.

Nine more, each with a bad and a good example: references/red-flags.md.

Files

FilePurpose
assets/beta-program-plan.mdPlan template: gates, cohort sizing, timeline
assets/recruitment-script.mdOutreach templates for the three cohorts, screening questions
assets/weekly-cadence-template.mdMonday digest, office hours, review, Friday snapshot
assets/exit-memo.mdExit decision memo (Greenlight / Extend / Pivot / Kill)
examples/shared-dashboards-closed-beta.mdWorked beta

References

  • references/beta-framework.md -- Full framework (cohort, Kano, success gates, cadence, exit gates), the 8-step workflow, troubleshooting matrix, and success criteria. Read this when building the plan, setting gates, running the cadence, or diagnosing problems.
  • references/beta-program-guide.md -- Practical reference guide: why betas fail, cohort sequencing, Kano deep dive, exit-gate worked examples, common failure modes. Read this for narrative depth and worked examples.
  • references/red-flags.md -- Bad-vs-good examples for the beta plan, recruitment script, cadence, and exit memo. Read this to review beta artifacts before opening recruitment.
  • Kano, Noriaki. "Attractive Quality and Must-Be Quality." Journal of the Japanese Society for Quality Control, 1984.

Scope & Limitations

In Scope:

  • Closed beta planning, recruitment, cadence, and exit-gate decisions
  • Three-cohort sequencing (Friends, Family, Fanatics)
  • Kano scoping for beta feature set
  • Weekly communication templates and exit memo
  • Beta-to-GA handoff to launch-playbook/

Out of Scope:

  • Public open beta or waitlist management (different mechanics; treat as a soft launch via launch-playbook/)
  • A/B testing and statistical experiment design (see discovery/brainstorm-experiments/)
  • Pricing experiments (run a dedicated pricing test, not piggybacked on beta)
  • Marketing campaign execution for the beta (light recruitment only; full GTM lives in launch-playbook/)

Important Caveats:

  • A 30-person closed beta is not statistically representative. Treat outcome metrics as directional, not conclusive.
  • Friends-cohort feedback is biased toward "this is great" -- weight Fanatics-cohort feedback 2x in the exit decision.
  • If your beta participants are also paying customers, any service disruption is a real incident. Run beta features behind a feature flag with a documented rollback path (see delivery-manager/).

Integration Points

IntegrationDirectionWhat Flows
create-prd/Receives fromPRD defines the hypothesis and headline outcome that beta gates measure
discovery/identify-assumptions/Receives fromRiskiest assumptions become the beta's primary learning goals
discovery/brainstorm-experiments/ComplementaryBeta is one experiment type; brainstorm-experiments covers smaller/faster alternatives
daci-framework/UsesDACI driver owns the exit-gate decision
launch-playbook/Feeds intoGreenlit beta hands testimonials, case studies, and known issues to launch
release-notes/Feeds intoBeta release notes preview the GA changelog
summarize-meeting/ComplementaryOffice hours and 1:1 notes become structured weekly summaries
senior-pm/Reports toExec sponsor receives the exit memo and signs the GA decision
信息
分类 产品商业
名称 beta-program
版本 v20261108
大小 32.8KB
更新时间 2026-11-19
语言