Six inquiries beat four. That doesn't settle your landing page test.

TL;DR

• In the invented contractor landing page test below, six inquiries from 50 assigned visitors beat four from 50. That is a 50% relative lift, a four-percentage-point difference and only two additional inquiries. The result provides weak evidence of a difference.
• Ask how visitors were assigned, what counted as an inquiry and whether the decision date was fixed in advance. A test with broken tracking needs repair before a performance verdict.
• Choosing a page because its price, service area or phone number is correct is a valid operating decision. Record that reason separately from a claim that the page produces more customers.

What is inside the 50% lift?

Your agency sends a screenshot: the new page converts 50% better. You have a repair crew to keep working, and the screenshot seems to settle which page gets the next month's advertising.

Start with the counts. Every number in this example is invented for teaching; these are not Lonta client results or an industry benchmark. Imagine 100 different eligible visitors assigned at random, 50 to each page, during the same period. Each person is counted once. The outcome is whether that person makes one valid new inquiry within the same defined follow-up window.

The new page's rate is 1.5 times the original rate: 12% divided by 8%. Subtract one and the relative lift is 50%. Subtract the two rates directly and the absolute difference is four percentage points.

Both descriptions are mathematically correct. Neither tells you how stable the difference is. Reclassifying just one inquiry from the new page to the original would leave five versus five. That sensitivity is a reason to inspect the individual records, not permission to move them to achieve a preferred answer.

How much evidence do ten inquiries provide?

For this simplified example, an exact comparison can enumerate how ten inquiry outcomes might fall between two groups of 50 if the pages had equal inquiry probabilities. Our independently calculated two-sided Fisher test gives a p-value of approximately 0.741.

That number is not a 74.1% probability that the pages are equal. It says that, under the equal-probability model and fixed totals, allocations at least as incompatible with equality as the observed table are common. The example does not provide strong evidence to reject equal inquiry probabilities.

The NIST description of Fisher's exact test, checked October 17, 2026, explains the small-table method and its conditional probability basis. The figures here are our calculation from the invented table, not a result published by NIST or a recreation of Google Ads' experiment reporting.

You do not need to calculate that test during a dispatch meeting. Ask whoever presents a winner to include the raw table, the uncertainty and the agreed decision rule. A large relative percentage with very few outcomes should invite a discussion, not an automatic rollout.

An inconclusive result also does not establish that the pages perform equally. The new page could be better, worse or similar. This sample has not separated those possibilities well enough for a confident performance claim.

Was it a comparison of the pages or of two different weeks?

A random split during the same period addresses a different question from replacing the page on Monday and comparing the next week with the previous week. The second comparison can include a change in weather, campaigns, prices, technician availability and who answered the phone.

Before reading the percentages, write down these conditions:

This is a proposed review sheet, not a claim that every website tool supports persistent person-level assignment. Use the actual unit your implementation can measure, then ask the analyst to choose an appropriate method. The exact calculation above assumes independent people assigned once. Repeated sessions, households contacting from several devices or adaptive traffic allocation can violate that simple setup.

What if the new page brings more inquiries but fewer usable jobs?

Keep the inquiry comparison and the service outcome comparison separate. A page promising availability you cannot provide can generate calls that consume office time and never reach a technician. A page stating the service area clearly might receive fewer inquiries while reducing those mismatches.

For the ten invented inquiries, the next sheet should list service requested, service-area fit, contact outcome and appointment status. Keep unknown statuses open. Do not label an unanswered new inquiry unqualified to improve the result.

The first-20 inquiry review provides a small manual log for this part. It will not make ten observations statistically sufficient. It makes the meaning of those observations inspectable.

Avoid switching the official success measure after seeing which column favors the new page. If qualified inquiries were always the goal, define qualification before running the comparison. If you discover a better measure halfway through, document the discovery and treat it as preparation for a later test.

Must a small shop keep paying until a test becomes conclusive?

No fixed number of weeks guarantees a usable answer. The required volume depends on the baseline rate, the difference that would change your decision and the error tolerance. A small market may not supply that volume within a sensible budget.

Set a spending boundary and a review date before starting. If you reach them without an informative result, record the test as inconclusive. You may retain the original page, choose the easier page to maintain or select the page with clearer customer information. State that operating reason without upgrading it into measured conversion improvement.

A broken call button, incorrect offer or false service promise can justify immediate correction. You do not need to keep a factual mistake live for the sake of experimental purity. Log when it changed so the next report does not quietly combine two versions.

For the agreement to make with an agency, use the marketing test stop-rule guide. Other account checks live in the Google Ads owner guides.

What this doesn't cover

• The calculation assumes a single fixed comparison with independent assignment and binary inquiry outcomes. It does not reproduce a platform's experiment model.
• Looking repeatedly and stopping at the first attractive result changes the statistical problem. This article supplies no sequential-testing rule or universal sample-size threshold.
• Inquiry rate does not establish additional sold jobs, collected revenue or profit. Those outcomes need their own definitions, observation time and cost accounting.

Do this in your account

1. Ask for visitors and valid inquiries in both groups, including the records behind the inquiry counts.
2. Write down the assignment method and the unit counted. Check repeat visitors and cross-device limitations.
3. Confirm one outcome definition and an equal follow-up window. Record exclusions before comparing the rates.
4. List every offer, traffic and tracking change made during the test.
5. Request uncertainty alongside relative lift, using a method appropriate for the actual design.
6. At the agreed review date, record a performance conclusion or an inconclusive result. Keep an operating decision distinct from statistical evidence.

FAQ

Is a 50% conversion-rate improvement always important? It may be commercially important if it is real and produces usable jobs at an acceptable cost. Four inquiries becoming six is a 50% relative improvement, but the small underlying count leaves considerable uncertainty.

Can I test a page by changing it next month? You can compare the observed periods, but other changes may explain the difference. Describe it as a before-and-after comparison rather than a randomized page experiment.

Should I stop testing if I cannot get enough traffic? Use the spending and time boundaries agreed in advance. An inconclusive test can be closed; you can make a practical page choice while preserving the uncertainty about performance.

Sources

• Fisher Exact Test, NIST Dataplot Reference Manual. Read October 17, 2026. Method reference only; the example and exact calculation are original teaching material.

Lonta AI manages advertising for home-service businesses. Bring the counts behind your next test to a free strategy review.