---
name: manage-experiment-backlog
description: >-
  Keeps a Google Ads testing program organized: harvests test ideas from account data (search terms, RSA asset
  performance, auction insights), forces each idea into a falsifiable hypothesis, scores it on impact ×
  confidence ÷ effort, ranks the queue against the account's current bottleneck, checks live-experiment
  capacity, and schedules the top 1-3 into the calendar, then logs every finished test's outcome so nothing
  gets re-tested out of ignorance. Use it at weekly/monthly review time, whenever the user says "what should we
  test next", when a test just finished and needs recording, or when the idea list has gone stale. Do NOT use
  it to actually build and run a single experiment, that is run-a-campaign-experiment, which this skill feeds
  and receives results from. Performance diagnosis itself belongs to run-a-weekly-performance-review /
  run-a-monthly-performance-review; this skill consumes their findings as test candidates.
---
# Manage the Experiment Backlog

## Purpose

A testing program lives or dies on process, not ideas, ideas are cheap and everyone has too many. This skill
maintains the pipeline around `run-a-campaign-experiment`: capture candidate tests with real hypotheses, score
and rank them, respect account capacity, schedule launches so they don't collide, and archive results into a
learning log that compounds. The output is always a current, ranked backlog plus a short list of tests actually
booked into the calendar.

## When to run

- During or right after a weekly/monthly performance review, to sweep new observations into the backlog.
- The user asks what to test next, or complains the test list is a mess / has gone stale.
- An experiment just concluded and its result needs recording (target: within 48 hours of close-out).
- Start of a planning period (month or sprint) when the next 1-3 tests must be picked.
- Monthly, as a standing full-backlog review: re-score, archive, report program health.

## When NOT to run

- Designing, launching, or reading out one specific test → `run-a-campaign-experiment`.
- Finding out WHY performance dropped this week → `run-a-weekly-performance-review` (its findings come back here
  as candidates).
- Deep strategic goal-setting and quarter planning → `run-a-quarterly-business-review`.
- Ad-copy iteration cycles that never need a formal split → `rsa-testing-with-the-iteration-loop`.

## Prerequisites

- A durable backlog store the user can see. Default: a Google Sheet maintained via `sheets_write_values`
  (create it with `sheets_create_spreadsheet` if none exists). An existing user spreadsheet or project tool is
  fine, ask before creating a new one.
- The user's statement of the account's current bottleneck (tracking gaps, unit economics, on-site conversion,
  traffic volume, or creative), this drives ranking and cannot be inferred from Ads data alone.
- Recent review notes or at least API access to pull fresh signals (step 2).
- Results of any recently finished experiments, if the close-out wasn't already logged.

## Procedure

1. **Open or create the backlog.** Read the existing sheet if one is on record. If starting fresh, write a header
   row via `sheets_write_values`: ID, date added, source, campaign(s), bottleneck area, hypothesis, impact,
   confidence, effort, priority score, status, result, learning. Statuses used throughout: New → Ready →
   Scheduled → Running → Done / Dropped.

2. **Harvest candidates from account data.** Pull each signal source and note anything that implies a testable
   change (do not filter at this stage):
   - `gads_get_search_terms_report`, query clusters that suggest structure, match-type, or targeting tests;
   - `gads_get_rsa_asset_performance`, asset patterns that suggest a creative-direction or messaging test;
   - `gads_get_auction_insights`, competitor pressure shifts that suggest bid-strategy or target tests;
   - `gads_list_experiments`, recently finished tests whose outcomes imply an obvious follow-up.
   **HUMAN STEP (outside VigilDog):** collect stakeholder/business input (upcoming offers, pricing moves, site
   changes) and any landing-page analytics observations, these routinely produce the highest-impact candidates.

3. **Force every candidate into a hypothesis.** Each new row must state: the change, the metric expected to move,
   the estimated size of the move, and the reason. Reject vague entries, "try a new landing page" is not a
   backlog item until it says which page, which metric, how much, and why. Write accepted rows to the sheet via
   `sheets_write_values` with status New.

4. **Deduplicate.** Scan existing rows and the learning log (step 10) before adding. A near-duplicate of an open
   item → merge the evidence into the existing row. A repeat of an already-answered question → cite the prior
   result to the user instead of re-adding.

5. **Score each new or changed item (1-3 each):**
   - *Impact*, 3 if it directly attacks the stated bottleneck metric on a high-spend campaign; 2 if it helps a
     secondary metric or supporting campaign; 1 if marginal.
   - *Confidence*, 3 for a proven pattern backed by this account's data; 2 for a reasonable, partially-supported
     hypothesis; 1 for a speculative punt.
   - *Effort*, 1 for a config-only change doable in under an hour; 2 for up to a day of setup; 3 for multi-day
     work or external dependencies (new page builds, feed work).
   Priority = (Impact × Confidence) / Effort. Bands: ≥4.5 high (book next slot), 2.0-4.4 medium (queue this
   cycle), <2.0 low (park, revisit monthly). Write scores back with `sheets_write_values` and sort the sheet
   view by score.

6. **Apply the upstream-first filter.** Order of repair: measurement/tracking problems first, then unit-economics
   questions, then on-site conversion, then traffic acquisition, then creative. A high-scoring creative test gets
   deferred while the account's tracking is broken, a test measured through a broken pipe answers nothing.
   Exception: a very cheap, high-score downstream test may run in the background without becoming the focus.
   Confirm the active bottleneck with the user; it is their call, not a computed value.

7. **Check capacity before selecting.** `gads_list_experiments` for anything running or scheduled. Constraints:
   Google caps concurrent custom experiments per account (five); never more than one experiment per campaign;
   practical recommendation is 1-2 live tests at a time so monitoring stays honest. Subtract what's running from
   what can be booked.

8. **Select 1-3 and gate them.** Take the top-ranked, bottleneck-aligned items and apply the formal-test gate
   from `run-a-campaign-experiment`: expected effect >10%, outcome genuinely uncertain, target campaign at ~100+
   conversions/month. Items failing the gate get re-routed: "just do it and monitor" for confident/low-impact
   changes, "judgment call" for low-volume campaigns, mark those rows accordingly rather than deleting them.
   Update selected rows to Ready via `sheets_write_values`.

9. **Schedule.** Assign start dates observing: one experiment per campaign; stagger launches across campaigns by
   3-5 days so day-1 delivery checks don't overlap; avoid launch windows inside known seasonal spikes; align with
   the user's planning cycle where possible. Record planned dates in the sheet. The launch itself, experiment
   creation, treatment change, scheduling via `gads_create_experiment` / `gads_schedule_experiment`, is executed
   through `run-a-campaign-experiment`, with every write previewed first and applied only after user approval.
   Mark launched rows Running. **HUMAN STEP (outside VigilDog):** tell stakeholders what is being tested, how long
   it runs, and that the test campaigns are frozen until close-out.

10. **Log every conclusion within 48 hours.** When a test ends (winner promoted via `gads_promote_experiment` or
    retired via `gads_end_experiment` inside `run-a-campaign-experiment`, both preview-then-approve writes),
    append a results row with `sheets_write_values`: outcome (treatment won / control won / inconclusive), the
    per-arm numbers, and the one-line transferable learning. Then queue the mandatory follow-up item:
    - clear winner → "replicate on similar campaigns" candidate;
    - inconclusive → "re-test longer or at higher volume" candidate;
    - surprising loser → "investigate why" candidate;
    - partial win → refined-hypothesis candidate.

11. **Monthly maintenance pass (30-45 min per account).** Re-score every active row, impact shifts as
    bottlenecks get resolved; confidence shifts as new data lands. Archive: Done rows with logged results,
    obsolete rows (metric fixed, strategy changed), duplicates. Then compute and report program health:
    active items (healthy range roughly 10-25), tests completed this month (aim 1-3), share of completed tests
    with a clear winner (a 30-50% win rate is normal, much higher means the tests are too safe), median days
    from idea to launch for high-priority items (target under 30), and count of items idle past 90 days (keep
    under 5). **HUMAN STEP (outside VigilDog):** walk stakeholders through last month's results and the next
    month's top three, and capture their new candidates.

## Decision rules

- Priority score = (Impact × Confidence) / Effort on 1-3 scales; ≥4.5 books the next slot, 2.0-4.4 queues,
  <2.0 parks.
- Ranking is score-order *within* the active bottleneck area; upstream areas always outrank downstream ones
  regardless of score, except cheap background wins (Effort 1, score ≥4.5) which may run without becoming the
  focus.
- Formal-experiment gate (all three or it doesn't get an experiment slot): >10% expected effect, genuine
  uncertainty, ~100+ conversions/month in the target campaign.
- Capacity: hard cap five concurrent custom experiments per account (Google), one per campaign, 1-2 recommended;
  stagger cross-campaign launches by 3-5 days.
- Results discipline: log within 48 hours; every completed test spawns exactly one follow-up backlog item; check
  the learning log before admitting any new candidate.
- A test whose win rate reads 100% over a quarter is a red flag, not a triumph, the program is only running
  sure things and learning nothing.

## Common failure modes

- **Idea graveyard.** Items pile up, nothing launches. Fix: the selection step is mandatory every cycle, 1-3
  picked, dated, and handed to `run-a-campaign-experiment`, even if the picks are modest.
- **Vague entries.** "Test audiences" rows can't be scored or scheduled. Fix: hypothesis format enforced at
  intake (step 3); reject or rewrite on the spot.
- **Re-testing known answers.** Happens when results live in someone's head. Fix: the learning log is checked at
  dedupe time, and close-out logging is a hard 48-hour rule.
- **Testing what's interesting instead of what's blocking.** Creative tests while conversion tracking is broken.
  Fix: the upstream-first filter runs before selection every time.
- **Collision launches.** Two experiments touching the same campaign, or five launched the same Monday. Fix:
  `gads_list_experiments` capacity check plus the 3-5 day stagger rule.
- **Stakeholder surprise.** Someone edits a frozen test campaign mid-flight because nobody told them. Fix: the
  communication step in scheduling is part of the procedure, not a courtesy.
- **Scores feel arbitrary at first.** Calibrate by retro-scoring a handful of past tests whose outcomes are
  already known, then keep the same yardstick.

## Related skills

- Downstream: `run-a-campaign-experiment` executes each selected test and returns its result here.
- Upstream: `run-a-weekly-performance-review` and `run-a-monthly-performance-review` generate most candidates;
  `analyze-auction-insights` and `analyze-search-term-reports` are rich single-source feeders.
- Adjacent: `scale-bids-and-budgets` (winners often route into a scaling program),
  `rsa-testing-with-the-iteration-loop` (creative iteration that bypasses formal experiment slots).