---
name: run-a-campaign-experiment
description: >-
  End-to-end driver for one Google Ads custom experiment: form a testable hypothesis, size the sample and
  duration, create and schedule a 50/50 split against a base campaign, apply exactly one treatment change,
  watch both arms for delivery problems, then promote the winner or retire the test with the results written
  down. Reach for it when the user wants to trial a bid-strategy switch, a tCPA/tROAS change, an audience
  addition, or a campaign-level landing-page swap on a campaign with roughly 100+ conversions a month and the
  outcome is genuinely uncertain. Do not use it to decide WHICH test to run next or to keep a testing pipeline
  organized, that is manage-experiment-backlog. Skip it entirely for RSA asset rotation (Google rotates assets
  automatically; see rsa-testing-with-the-iteration-loop), for full manual-to-Smart-Bidding migrations
  (migrate-from-manual-to-smart-bidding), and for budget/target scaling programs (scale-bids-and-budgets).
---
# Run a Campaign Experiment

## Purpose

Turn a risky campaign change into a controlled trial: half of traffic keeps the current setup, half gets the
proposed change, and the decision is made from measured deltas rather than opinion. This skill covers the full
lifecycle, design, build, launch, monitoring, and close-out, using VigilDog tools for every step the Google Ads
API supports. A badly run test is worse than no test, so the procedure enforces the three things that make
results trustworthy: one variable, a pre-committed duration, and guardrail metrics checked before declaring a
winner.

## When to run

- A bid-strategy change is proposed (e.g. Maximize Conversions → Target CPA) and the impact is not predictable.
- A meaningful tCPA or tROAS adjustment is on the table and stakeholders disagree about the outcome.
- An audience segment is being added or removed at campaign level and its incremental effect is unknown.
- Campaign-level final-URL / landing-page routing is changing and CVR impact must be isolated.
- Any change expected to move the primary KPI by more than ~10% where the direction or size is uncertain, on a
  campaign converting ~100+ times per month.

## When NOT to run

- Choosing and prioritizing among many candidate tests → `manage-experiment-backlog`.
- Testing RSA headlines/descriptions → `rsa-testing-with-the-iteration-loop` (asset rotation is automatic; a
  campaign experiment adds nothing).
- Full migration off manual bidding → `migrate-from-manual-to-smart-bidding` (it wraps the experiment method in
  migration-specific baselining and rollback rules).
- Deciding how far to push budgets/targets on an already-working campaign → `scale-bids-and-budgets`.
- Landing-page variant testing where the page itself is the unit under test → `run-a-landing-page-a-b-test`.
- Low-stakes changes (<10% expected effect) or changes you are already confident in, apply them directly and
  monitor; an experiment only adds delay.

## Prerequisites

- Working conversion tracking on the target campaign (if in doubt, verify first, a test measured with broken
  tracking is noise).
- ~100+ conversions/month in the campaign being tested; below that, significance will take months.
- A written hypothesis, one primary metric, and 1-2 guardrail metrics agreed with the user before any build step.
- No other experiment currently running on the same campaign.
- User awareness that the campaign must be left untouched for the whole test window.

## Procedure

1. **Baseline read.** Use `gads_run_gaql_query` on the target campaign for the last 90 days segmented by month:
   conversions, cost, conversion value, CPA/ROAS, and click-to-conversion lag (query `segments.days_to_conversion`
   or compare conversions by date vs. by conversion date). Confirm the ~100 conversions/month floor. If volume is
   short, stop and tell the user an experiment cannot reach significance in reasonable time, recommend a
   best-judgment direct change with monitoring instead.

2. **Gate the decision.** Apply the Decision rules below (impact / uncertainty / volume). If any gate fails,
   report which one and end here rather than building a doomed test.

3. **Design on paper first.** Write out with the user: hypothesis in the form "changing X should move metric Y by
   ~Z% because...", the primary metric, success threshold, guardrails with floors (e.g. conversion volume must
   hold ≥90% of control), and the computed duration (see Decision rules). Get explicit sign-off on the design, this is the pre-commitment that prevents ending early.

4. **Check the campaign is clean.** `gads_list_experiments`, confirm no experiment is active or scheduled on this
   campaign. Google allows several concurrent experiments per account, but never stack two on one campaign.

5. **Policy check.** Before the session's first Google Ads write, call `gads_policy_guardrail` and honor its
   output.

6. **Create the experiment.** `gads_create_experiment` against the base campaign: name it
   `<campaign> | <variable under test> | <start date>`, 50/50 traffic split, start and end dates from the design.
   Preview first (validate_only default); apply only after the user approves the previewed payload.

7. **Apply the single treatment change to the experiment arm.** Depending on the variable:
   - bid strategy or target value → `gads_set_campaign_bidding_strategy` on the treatment campaign;
   - final URLs, network, or other campaign settings → `gads_update_campaign` on the treatment campaign.
   Preview first, apply only after user approval. Change exactly one thing; if the user asks for a second change,
   refuse and explain it destroys attribution of the result.

8. **Schedule and launch.** `gads_schedule_experiment` with the agreed dates. Preview first, apply only after user
   approval. Then confirm via `gads_list_experiments` that status shows the experiment as scheduled/running.

9. **Day-1 delivery check.** `gads_run_gaql_query` pulling impressions, clicks, and cost for both the base
   campaign and the treatment campaign since launch. Both arms should be serving and the traffic share should sit
   near the configured split. No traffic in one arm within 24-48h means a setup or policy problem, investigate
   before letting the clock run.

10. **Days 2-3.** Re-check delivery and look for disapprovals or serving restrictions on the treatment arm
    (`gads_run_gaql_query` on ad approval status, or `gads_get_disapproved_ads` if available in session). Fix-able
    issue → pause, fix, restart the window; do not let a partially-serving arm burn test days.

11. **Weekly watch, hands off otherwise.** Once a week, `gads_run_gaql_query` for the primary and guardrail
    metrics per arm. The only permitted mid-test actions: document external events (promos, seasonality,
    site changes) and check the early-stop rule (Decision rules). Never adjust budgets, bids, ads, or targeting in
    either arm mid-flight, and decline user requests to do so with a reminder of the pre-commitment.

12. **Close-out read.** After the end date, wait additional days equal to the measured conversion lag from step 1
    so late conversions land. Then pull final per-arm numbers with `gads_run_gaql_query`.
    **HUMAN STEP (outside VigilDog):** read the statistical-significance callout in the Google Ads Experiments UI, the API does not expose Google's significance computation. As a cross-check, a two-proportion comparison can
    be computed from the raw arm counts, but treat the UI readout as the deciding source.

13. **Act on the outcome** (preview first, apply only after user approval, for every write):
    - Treatment wins on the primary metric with guardrails intact → `gads_promote_experiment` to fold the change
      into the base campaign.
    - Control wins, or treatment wins the primary but breaches a guardrail the user won't trade →
      `gads_end_experiment`, keeping the original settings.
    - Not significant → decide with the user using the tie-break rules below, then promote or end accordingly.

14. **Write it down.** Record hypothesis, dates, per-arm results, significance, decision, and the transferable
    lesson. Hand the record to `manage-experiment-backlog` so the learning log stays current and follow-up tests
    get queued.

15. **Post-apply stability check.** 7 and 14 days after promoting a winner, `gads_run_gaql_query` on the base
    campaign to confirm the production numbers match what the treatment arm showed. A promoted change that
    regresses in production goes back through this skill or gets reverted via `gads_update_campaign` /
    `gads_set_campaign_bidding_strategy` (preview → approval → commit).

## Decision rules

- **Run an experiment only if all three hold:** expected effect >10% on the primary KPI; outcome genuinely
  uncertain (not predictable from existing data); campaign volume ≥ ~100 conversions/month. Otherwise implement
  directly and monitor, or make a best-judgment call at low volume.
- **Sample size per arm (rule of thumb at 95% confidence):** detecting a ~20% effect needs on the order of
  100-150 conversions per arm; ~15% needs roughly 200-300; ~10% needs roughly 400-500. Smaller expected effects
  than 10% are usually not worth a formal test.
- **Duration:** `days = (per-arm conversions needed × 2) / daily conversion rate`, plus the campaign's conversion
  lag, plus ~7 extra days if the treatment changes Smart Bidding (learning period). Floor: 14 days regardless of
  volume, to cover at least two full weekly cycles.
- **Traffic split:** 50/50 unless the user articulates a specific risk reason for an uneven split; uneven splits
  lengthen the required duration on the smaller arm.
- **Early stop is allowed only when:** one arm is >30% worse on the primary metric AND the difference is already
  statistically significant; or a technical/policy failure blocks delivery; or an external event invalidates the
  comparison. "Looks worse this week" is never grounds, short streaks are expected noise.
- **Reading the result:** significant + better on primary + guardrails hold → promote. Significant + worse → end,
  keep control. Significant on primary but guardrail breached → present the trade-off to the user; no default.
  Not significant → prefer the simpler configuration; if one arm trended consistently better and the test was
  close to significance, offer a 1-2 week extension before deciding.

## Common failure modes

- **Calling it early.** The most common way tests lie. Mitigate by getting duration sign-off in step 3 and
  refusing mid-test verdicts except under the early-stop rule.
- **Mid-test edits.** Someone tweaks a budget or ad in one arm and the comparison is dead. State the freeze
  window to the user at launch; check `gads_get_change_history`-style reads if contamination is suspected.
- **Two variables in the treatment.** Result becomes uninterpretable. Enforce one write in step 7, period.
- **Testing a tiny effect.** A 5% hoped-for lift on a 200-conversion/month campaign needs months; the gate in
  step 2 exists to catch this before any build work.
- **Ignoring conversion lag.** Reading results on the end date undercounts the treatment arm's late conversions
  and biases toward control. Always add the lag window from the baseline read.
- **No record kept.** An undocumented test will be re-run by someone else in six months. Step 14 is part of the
  procedure, not optional hygiene.

## Related skills

- Before: `manage-experiment-backlog` (chooses which test earns a slot), weekly/monthly review skills that
  surface the underlying performance question.
- After: `manage-experiment-backlog` (log the result, queue follow-ups and scale-outs).
- Adjacent: `migrate-from-manual-to-smart-bidding`, `scale-bids-and-budgets`, `run-a-landing-page-a-b-test`,
  `rsa-testing-with-the-iteration-loop`.