Key takeaways
- Test with at least 25 scripted calls across emergencies, routine bookings, price questions, existing customers and edge cases before any real customer hears it.
- Addresses and callback numbers fail most often; require a read-back and check every test transcript against what you said.
- Include accents, background noise and a Spanish-speaking caller; nearly one in five US residents spoke a language other than English at home in 2019[1].
- Test the failure paths on purpose: the on-call tech not answering, a caller demanding a person, a question it should not answer.
- Keep testing after launch; adoption is still early in construction, so few peers will have done this for you[2].
Why a demo is not a test
Every AI receptionist sounds good in a sales demo, because the demo uses a clean line, a cooperative caller and a problem the system was built around. Your callers are standing in a flooded basement on a cellphone with a dog barking, or driving, or upset. The only way to know how the system handles your calls is to make those calls yourself, on your configuration, before you forward a single real customer to it.
You will be doing this earlier than most of the trade. In the Census Bureau's October 2026 business survey, 19.8% of construction businesses said employees used generative AI for work tasks, against 26.8% of all businesses[2]. There is no industry playbook yet, so build your own.
Source: U.S. Census Bureau, 2026[2]
Show the data
| Item | Value |
|---|---|
| Construction businesses | 19.8% |
| All US businesses | 26.8% |
Set up the test properly
Configure it as you would for real
Load your real service area, fee wording, triage tiers, on-call ladder and booking rules. Testing a half-configured system tells you nothing.
Use a test schedule and test on-call numbers
Point bookings at a test calendar and the on-call ladder at phones held by people who know they are part of the test, so nobody gets a fake 2 am emergency by surprise.
Recruit three or four callers
Use people the system has never heard: a family member, a tech, someone with a strong regional or non-native accent, a Spanish speaker if you serve Spanish-speaking customers.
Write the scripts down
Each tester gets a card with the scenario, the details to give and one twist. Keep a copy of what each caller was told to say, so you can check the record against it.
Score every call the same way
Use one sheet with the same pass/fail checks for every call. Decide the pass criteria before you start, not after you hear the results.
The test matrix
| Scenario | What to check | Pass looks like |
|---|---|---|
| Burst pipe at night, caller panicking | Shut-off guidance, urgency tag, escalation speed | Marked urgent, on-call tech rung within your target time, caller told what happens next |
| Water heater leaking from the bottom | Triage tier, safety wording | Correct tier from your list; no invented repair advice |
| Sewage backing up into a tub | Triage tier, health wording | Dispatch or first slot per your rules |
| Slow drain, wants Tuesday | Booking into the schedule | Correct slot, address, phone and job type in the calendar |
| "What do you charge to come out?" | Fee wording | Your approved wording only; no repair price guesses |
| Caller in a ZIP you don't serve | Service-area check | Polite decline with your approved next step |
| Existing customer moving an appointment | Lookup and reschedule | Finds the booking and moves it correctly |
| Caller asks for a real person | Hand-off path | Transfers or takes a message as configured, without arguing |
| Spanish-speaking caller | Language switch and summary | Switches language; your summary is in English and correct |
| On-call tech doesn't answer | No-answer step | Backup rung; caller told honestly what happens next |
Run each core scenario at least twice with different callers. Then add the edge cases below. Twenty-five calls is a sensible minimum; fifty is better if you have the time.
Addresses, numbers and names
Most damaging mistakes are small: one wrong digit in a house number or a phone number. Almost every caller will be on a cellphone, since 98% of US adults own one[3], and cellphone audio drops syllables. Test the details that cause wasted truck rolls.
- Street names that sound like other words, such as Mill and Hill, or Fifth and Fist.
- House numbers with repeating or teen digits: 1515, 16 versus 60, 4040.
- Apartment or unit numbers, gate codes and "the house behind the main house".
- A caller who gives the cross street instead of the number.
- A callback number that differs from the number they are calling from.
- Names that are hard to spell, and a caller who spells theirs fast.
Pass rule: the system must read the address and callback number back to the caller on every booking and every urgent call. If it doesn't, fix the configuration before going any further.
Accents, languages and noise
The Census Bureau counted 67.8 million people who spoke a language other than English at home in 2019, with Spanish the most common[1]. Many of them speak English very well; among Spanish speakers in the 2018-2022 survey, 61.0% said they did[4]. So you need two kinds of test: English with a range of accents, and full conversations in the other languages your customers use.
- Strong regional accents from your area, including fast talkers.
- Non-native English speakers who switch words mid-sentence.
- A full call in Spanish, if you serve Spanish speakers, describing a real problem.
- Background noise: running water, a shop vac, a TV, a car on speaker.
- A caller who is crying or shouting, and one who barely speaks.
Edge cases that reveal the rules
Edge cases show whether the system knows the limits of its instructions. A good one says it doesn't know and routes the call; a bad one guesses confidently.
- "Can you just tell me if it's the flapper or the fill valve?" It should not diagnose; it should book or take a request.
- "My landlord said you'd bill him." It should capture landlord details per your policy, not promise billing terms.
- "Are you a robot?" It should answer honestly.
- "I want to speak to the owner about my last bill." Complaint routing, not a booking.
- A sales call from a vendor, and a robocall. It should not book these or page the on-call tech.
- A caller who gives two different addresses during the call.
- An emergency that matches none of your examples, such as water coming through a light fixture.
Test the hand-off and the failure paths
The most important tests are about what happens when things go wrong. Have the on-call tester ignore the first ring, then the second. Check how long the system waits, whether it tries the backup, and what it tells the caller. Have a caller insist on a person three times. Turn off the test calendar connection and see whether the system falls back to taking a request rather than promising a slot it cannot hold.
Check the record of each call too: the summary, the urgency tag, the transcript and the recording if you keep one. Your dispatcher should be able to act on the summary without listening to the call.
A simple scoring sheet
Score every test call on the same eight checks, each pass or fail. Totals matter less than patterns: one check failing across several calls points to a configuration problem you can fix in an afternoon.
- Answered promptly, in your business name, with the right greeting.
- Correct triage tier for the scenario.
- Address correct, including unit number, after read-back.
- Callback number correct after read-back.
- Problem described accurately in the summary, in words your tech would understand.
- Only approved wording used for fees, service area and policies.
- Escalation or booking completed as your rules say, within your time target.
- Caller told clearly what happens next, and the call ended politely.
Set a bar before you start. A reasonable one is no failures on checks two, three, four and seven in the emergency scenarios, because those are the ones that send a tech to the wrong place or leave water running.
Compliance checks
Listen to the greeting as a caller would. If you record calls, the greeting should handle consent; federal law allows one-party consent but some states require every party's consent[5]. If the system will make outbound calls, such as reminders, remember that the FCC ruled in 2024 that AI-generated voices count as artificial under the TCPA[6], so those calls need prior consent. Test that opt-out requests are honored.
Example: what a first test round finds
Example (fictional): A three-truck shop runs 30 test calls over two evenings. Twenty-six pass. Two failures are addresses: "Fifteen" heard as "fifty" on a speakerphone, caught because the caller corrected the read-back once and not the second time. One failure is a fee answer that added "plus parts" to the approved wording. One is a slow no-answer step: the backup tech was rung after four minutes instead of two. The owner fixes the fee wording and timer, asks for unit numbers to be read back digit by digit, and reruns the eight affected scenarios. All pass on the second round, and the shop goes live on nights only for the first two weeks.
Keep testing after go-live
Re-run a short set of ten test calls every time you change a rule, a fee or the on-call ladder, and once a month regardless. Review a sample of real transcripts weekly for the first month and fortnightly after that. Track wrong addresses, missed escalations, and any statement you did not approve. If one of those appears twice, pause and fix it before it reaches a third customer.
Frequently asked questions
How many test calls should I make before going live?
At least 25 scripted calls covering emergencies, bookings, price questions, existing customers and edge cases, with several different callers. Re-test after every rule change.
What is the most common AI receptionist mistake for plumbers?
Wrong details on noisy cellphone lines, especially house numbers and callback numbers. Require a read-back on every booking and urgent call.
Should I test in Spanish?
If you serve Spanish-speaking customers, yes. Spanish was the most common non-English language spoken at home in 2019[1]. Have a fluent speaker run real scenarios and check the English summary.
Can I go live gradually?
Yes. Many shops start with nights or overflow only, review every call for two weeks, then expand once the results hold.
References
U.S. Census Bureau (2022). Nearly 68 Million People Spoke a Language Other Than English at Home in 2019 (opens in a new tab) โ
U.S. Census Bureau (2026). Business Trends and Outlook Survey (BTOS), response estimates by sector (opens in a new tab) โ
Pew Research Center (2026). Mobile Fact Sheet (opens in a new tab) โ
U.S. Census Bureau (2023). Most Americans Speak Only English at Home or Speak English 'Very Well' (opens in a new tab) โ
Legal Information Institute, Cornell Law School (U.S. Code) (2026). 18 U.S. Code ยง 2511 - Interception and disclosure of wire, oral, or electronic communications prohibited (opens in a new tab) โ
Federal Communications Commission (2024). FCC Makes AI-Generated Voices in Robocalls Illegal (news release) (opens in a new tab) โ