What an alternative hypothesis is
An alternative hypothesis is a statement about a population or process that contrasts with the null hypothesis in a statistical test. In a marketing experiment, it might state that two experiences have different conversion rates or that a new experience increases the rate. It describes the effect being evaluated, not a conclusion already established by the data.
The statement should identify the outcome, population, comparison, and direction where applicable. A vague belief that a redesign is better does not define a testable statistical claim. Better could mean more purchases, faster completion, or fewer support requests, and those outcomes can move differently.
Write the hypothesis before inspecting results. This helps the team choose an appropriate design and prevents a surprising pattern from being presented afterward as if it were the original prediction.
Connect the alternative with the null
The null and alternative define the competing parameter values considered by the test. For a simple two-sided comparison of conversion rates, the null can state that the difference is zero, while the alternative states that the difference is not zero. The test evaluates evidence against the null under its assumptions.
For a directional improvement claim, the alternative may state that the new rate exceeds the existing rate, with the null covering no improvement. The exact formulation should match the procedure being used. Do not describe a one-sided test while interpreting its result as if both directions had been tested symmetrically.
Keep the statistical statement separate from the business explanation. A proposed mechanism, such as clearer pricing reducing confusion, motivates the experiment but is not automatically proven if the conversion-rate test produces a significant result.
Choose direction before seeing the result
A two-sided alternative considers differences in either direction. A one-sided alternative focuses on a specified direction. The choice affects the test and should follow the decision problem, not a desire to obtain a smaller p-value after looking at the data.
A team testing a checkout change may care about both improvement and harm, making a two-sided comparison useful. Another carefully defined decision may justify a directional procedure. In either case, specify guardrails for outcomes that matter even if they are not the primary hypothesis.
Changing direction after seeing the observed effect undermines the stated error-control interpretation unless the analysis accounts for that selection. If an unexpected pattern creates a new hypothesis, label it exploratory and consider a new test. Honest separation of planned and exploratory analysis preserves the value of both.
Work through an experiment example
Suppose a team compares an existing inquiry form with a shorter version among eligible visitors. The primary outcome is a qualified inquiry within a defined observation window. A two-sided alternative states that the population qualified-inquiry rates differ between versions; the null states that their difference is zero.
If the observed rates are 4% and 4.6%, the estimated difference is 0.6 percentage points, or a 15% relative increase from 4%. Those numbers alone do not establish statistical evidence. The sample sizes, design, uncertainty, and test assumptions still matter.
The team should also inspect inquiry quality and technical failures. More submissions may not mean more qualified demand. A statistical result about the specified outcome supports that outcome under the design; it does not automatically validate every business claim associated with the shorter form.
Interpret evidence without claiming proof
Rejecting a null hypothesis under a valid procedure provides evidence against it at the chosen criterion. It does not prove the alternative with certainty, establish the proposed mechanism, or tell the team that the effect is commercially important. Report the estimated effect and uncertainty alongside the test result.
Failing to reject the null does not prove no effect. The experiment may be too small, noisy, or poorly aligned with the relevant outcome. Statistical power concerns the ability to detect a specified effect under stated assumptions, so planning should consider the effect size that matters to the decision.
A p-value is not the probability that the alternative is true. Avoid translating it into a confidence percentage about a business story. Use the procedure's actual interpretation and explain the limits in plain language.
Keep the analysis aligned with the decision
Specify the primary hypothesis, analysis population, observation window, stopping rule, and treatment of multiple comparisons before launch. Repeatedly checking results and stopping when a preferred threshold appears can change error rates unless the method is designed for that process.
If several outcomes or segments are explored, distinguish those findings from the primary planned analysis and use appropriate methods where confirmatory claims are intended. A favorable result in one small subgroup should not silently replace the original overall question.
Finally, connect the estimated effect with implementation cost, customer impact, and uncertainty. A statistically detectable improvement may be too small to justify a costly change. Conversely, an inconclusive result can still guide the next research step. The hypothesis structures learning; the business decision requires the full evidence and relevant constraints.
Frequently asked questions
Is the alternative hypothesis always that the new version is better?
No. It can state any specified departure from the null, including a difference in either direction or a decrease in an outcome. The formulation depends on the question and statistical procedure. Define what better means in measurable terms before testing, and do not assume that the desired business outcome determines what the data will show.
Can the alternative be changed after collecting data?
A new hypothesis can be developed from observed patterns, but it should be labeled exploratory rather than presented as the original planned claim. Selecting a direction or outcome because it looks favorable changes the analysis context. Use appropriate methods or a new confirmatory study before making claims that rely on a prespecified test interpretation.
Does a nonsignificant result mean the null is true?
No. It means the procedure did not provide sufficient evidence to reject the null under its criterion and assumptions. The data may still be compatible with effects that matter. Examine the effect estimate, uncertainty, sample size, and design. Demonstrating practical equivalence requires a suitable framework and predefined margins, not simply a large p-value.
What is the difference between a hypothesis and a prediction?
A statistical hypothesis states something about a population parameter or process. A prediction describes an expected observation or outcome, sometimes more informally. The team may predict that clearer wording improves completion, then formulate a precise alternative about the relevant completion-rate difference. Keeping the distinction clear helps connect a practical idea with a valid analysis.
How does practical significance differ from statistical significance?
Statistical significance concerns evidence under a testing procedure, while practical significance concerns whether the effect is meaningful for the decision. A tiny effect can be statistically detectable with enough information yet have little business value. Evaluate magnitude, uncertainty, costs, and customer consequences together rather than treating a threshold crossing as an automatic instruction to launch.
Related terms
- Null hypothesis: the reference claim evaluated by the test, contrasted with the alternative under a defined statistical framework.
- P-value: a measure calculated under the null model, which should not be interpreted as the probability that the alternative is true.
- Statistical power: the probability of detecting a specified effect under stated assumptions, relevant when planning a useful experiment.
- Type I error: rejection of a true null hypothesis, whose control depends on the procedure and analysis choices.
- Practical significance: the real decision value of an effect, distinct from whether it crosses a statistical threshold.
- A/B testing: controlled comparison of experiences, where clear null and alternative hypotheses help organize the analysis.