The Case for Revenue Experiment Design Before Execution
A clinic can split enquiries or change a process without creating a comparison that can answer the intended revenue question. Care Journey defines the hypothesis, single change, unit of assignment, real exposure, guardrails, invalidation conditions and decision rule before execution. The clinic receives a practical experiment specification—or an early finding that another method is more suitable—before resources are committed.
Why the Experiment Must Be Designed Before Execution
Suppose a clinic proposes assigning alternate enquiries to an existing booking process and a revised one. On paper, each enquiry has a label. In practice, the same reception team handles both groups, sees the revised prompt in a shared system and may carry the new behaviour into every conversation. The label describes assignment, but it does not preserve the intended contrast. Before anyone chooses a tool or starts a run, the design has to name the unit that truly receives the change and how exposure could cross that boundary.
(Care Journey lists revenue experiment design among its one-time Revenue Development options for UAE clinics). That authority supports a design service with explicit inputs, checks, limits and exits. It does not authorize implementation, publish a standard test volume or duration, guarantee sufficient data, declare causality or promise a revenue result.
Pass every proposal through the designability gate
| Design Field | Question That Must Be Answerable | Common Reason to Revise |
|---|---|---|
| Decision | What clinic choice could this evidence inform? | The request asks only whether a metric might move |
| Hypothesis | Which specified change is expected to affect which defined response, and why? | Several interventions or outcomes are bundled together |
| Comparator | What experience remains meaningfully different from the proposed change? | The comparison changes at the same time or cannot be reproduced |
| Assignment | What entity receives the condition, and can that assignment remain stable? | An individual identifier is used although delivery happens by team, location or shared system |
| Exposure | Which assigned units can actually encounter the change, and how is that known symmetrically? | Only the changed condition reveals who was exposed |
| Measures | What primary measure informs the decision, and which guardrails protect against unacceptable trade-offs? | A convenient dashboard event substitutes for the actual decision |
| Validity | Which failures would make later movement unusable for this decision? | Missingness, leakage or concurrent changes are left for post-hoc judgement |
| Authority | Who separately owns ethics, privacy, operations, implementation and analysis? | A completed specification is mistaken for permission to execute |
The gate issues one design-only exit. DESIGN_SPEC_COMPLETE means every required field is explicit enough for the appropriate owners to assess separately. DESIGN_REVISION_REQUIRED means a material field can plausibly be repaired. ALTERNATIVE_METHOD_REQUIRED means the intended comparison cannot remain coherent or the proposed experiment is not the right way to support the decision. None of those exits says that an intervention has started or that a result exists.
(The Institute for Healthcare Improvement describes testing a change through a prediction, a plan for collecting data and learning from a cycle). That improvement method does not provide a universal revenue-experiment recipe or statistical threshold. Here it supports specifying what the proposal expects to learn and how the relevant observation would be collected before operational activity begins.
Draw assignment, exposure and spillover on the same map
| Map Layer | What to Record | Stress Question |
|---|---|---|
| Delivery Unit | The individual, team, location, system or time block through which the change is actually applied | Could the operator deliver different conditions without carrying one into the other? |
| Assignment Unit | The entity assigned to the proposed condition or comparator | Is the assignment stable, identifiable and compatible with delivery? |
| Exposure Unit | The entity that can encounter the changed experience | Can exposure be identified for both conditions without using treatment-only knowledge? |
| Observation Unit | The entity and event represented in the analysis data | Does repeated or linked observation require a different analysis structure? |
| Shared Pathways | People, locations, devices, systems, calendars, referral paths or time windows connecting conditions | Where could the change spill across the intended contrast? |
| Decision Population | The population to which the clinic hopes to apply the eventual decision | Would the observed units support that decision, even if the mechanics worked? |
(NIH Research Methods Resources explains that group- or cluster-randomized trials assign groups or clusters rather than individuals). A marketing or operational design is not automatically a clinical trial, and this page does not prescribe randomization. The source makes the structural point concrete: when an intervention is delivered through a group, pretending each person is independently assigned can misdescribe both spillover and analysis.
- Start with delivery rather than the available identifier. Describe how the proposed change would reach people in the real clinic system.
- Mark assignment separately. Name the smallest entity that could reliably receive a condition without requiring staff to remember hidden case-by-case rules they cannot sustain.
- Trace exposure. Identify who could encounter the changed experience and what evidence could establish exposure under either condition using the same logic.
- Trace observation. Specify the event and entity represented by each record, including repeated contacts, linked journeys or delayed outcomes that could violate an assumed independence.
- Circle every shared pathway. Teams, locations, platforms, calendars, assets and referral processes can move the intervention across the intended boundary.
- Compare the map with the decision population. If the observed units differ materially from the population or process the clinic hopes to change, revise the claim before designing further.
What Revenue Experiment Design Covers
An experiment specification is fragile when it describes what would count as desirable movement but leaves data loss, unstable assignment, spillover or concurrent changes to later judgement. Validity must take precedence. The design should state which conditions would prevent a later estimate from informing the intended clinic decision, even if the displayed direction looks encouraging.
| Predeclared Check | What Could Fail | Design Response |
|---|---|---|
| Assignment Integrity | Units switch, duplicate or cannot be linked reliably to their assigned condition | Specify an integrity check and an invalidation boundary |
| Exposure Comparability | The proposed rule finds affected observations differently across conditions | Redesign the exposure rule or the question |
| Missingness | Loss of events or outcomes differs in a way that could distort the comparison | Name acceptable evidence of completeness and the consequence of failure |
| Contamination | Staff, systems or assets carry the changed experience into the comparator | Change the assignment unit, isolate delivery or select another method |
| Concurrent Change | A material operational, measurement or channel change alters the comparison | Define what must remain fixed and which change would invalidate interpretation |
| Guardrail Breach | A protective measure moves outside the clinic's predeclared acceptable boundary | Ensure the decision rule cannot ignore the guardrail because the primary measure moved favourably |
(Research on trustworthy experimentation under telemetry loss shows that missing telemetry can bias estimates and reduce statistical power). The paper concerns large software systems and does not provide a failure rate for clinic revenue experiments. It supports the design principle that data completeness is part of validity, not a housekeeping issue to investigate only when the result is inconvenient.
The final decision rule should combine the primary measure, its practical meaning, guardrails and every material validity condition. (The American Statistical Association states that a p-value does not measure the size or importance of an effect and does not by itself provide a good measure of evidence). A threshold can be one component of a qualified analysis plan; it cannot supply the clinic decision, repair an invalid design or promise causality on its own. The rule should also state what happens when the evidence is precise but operationally trivial, practically meaningful but too uncertain, or directionally favourable while a guardrail or validity condition fails. Writing those combinations down does not predetermine a business answer; it prevents a single convenient statistic from becoming the answer by default.
- This service covers the design of one revenue-related experiment and the conditions needed for an interpretable comparison.
- Tool configuration, traffic allocation, live execution, causal analysis and revenue guarantees remain separate.
- A sound design permits an informed implementation decision; it is not proof that the change will increase revenue.
Challenge the proposal before implementation can harden it
Name what receives the change, what can be exposed, what becomes an observation, what could invalidate the comparison and which clinic decision the evidence is meant to inform.
The proposed clinic decision, falsifiable hypothesis, specified change, comparator, assignment unit, exposure rule, primary measure, guardrails, data path, validity conditions and responsible owners must form one coherent specification. A plausible hypothesis alone is not enough when delivery or observation cannot preserve the intended comparison.
A change may actually be delivered through a shared team, location, system, asset or time block. If those shared structures carry the new behaviour across individually labelled enquiries, the individual label does not describe an isolated condition. The design must match assignment to real delivery and account for spillover.
Assignment records which condition a unit is allocated to. Exposure describes whether and how an eligible unit could encounter the changed experience. When exposure is selective, the design needs a symmetric rule for identifying comparable opportunities under both conditions rather than filtering later with information available only for the changed condition.
Unstable assignment, asymmetric exposure, missing data, contamination or material concurrent changes can make movement unusable for the intended decision. Declaring those conditions first prevents an encouraging direction from quietly lowering the evidence standard after the fact.
No. A qualified analysis may use a p-value, but it does not measure effect size or practical importance and cannot replace the primary measure, guardrails, validity checks or clinic decision context. The design must state how those elements work together without asserting a universal threshold.
No. DESIGN_SPEC_COMPLETE means the design fields are explicit enough for separate feasibility, ethics, privacy, operational, implementation and analysis decisions. The work may instead exit as DESIGN_REVISION_REQUIRED or ALTERNATIVE_METHOD_REQUIRED. None of these design exits launches a test, interprets a result or guarantees causality or revenue impact.
Is Revenue Experiment Design the Right Next Step?
Share the revenue question, proposed change, people or events affected, available measurement and operational constraints with Care Journey. We will assess whether the question supports a coherent experiment, identify the safeguards the design needs, and recommend the most practical next step.
The useful output is not a prediction. It is an inspectable exit: a coherent specification ready for separate assessment, a focused revision question, or a documented reason to use another method. Each protects the clinic from spending execution effort on a comparison that could never support the decision it was supposed to answer.