Key takeaways
- Pilot one problem with one scope, such as after-hours calls, and write the success criteria before day one.
- Measure a baseline week first; without it, nobody can say whether the pilot helped.
- Most construction firms are still deciding: 17.1% use AI now and 19.4% expect to within six months[1].
- End with a written go, extend or stop decision against the criteria you set, not a feeling.
Why a pilot, not a purchase
Most plumbing owners are still at the trying stage with AI, and that is sensible. In the Census Bureau's October 2026 survey, 17.1% of construction businesses were using AI in a business function and 19.4% expected to within six months[1]. Across all US businesses, 26.2% said they did not know whether they would use AI in the next six months[2]. A short, structured pilot turns that uncertainty into evidence from your own calls, your own customers and your own staff.
Source: U.S. Census Bureau, 2026[1]
Show the data
| Item | Value |
|---|---|
| Used AI in the last two weeks | 17.1% |
| Expect to use AI in the next six months | 19.4% |
The point of a pilot is not to prove the tool works. It is to find out, cheaply and safely, whether it works for your shop, and what it would take to rely on it. A pilot that ends with a clear no is a success if it cost little and taught you something.
Thirty days is long enough to see several hundred calls or tasks, at least four weekends and probably one bad-weather spike, and short enough that nobody loses patience. If your volume is low, extend to six weeks rather than drawing conclusions from a handful of examples.
Pick one problem and one scope
Write the problem in one sentence before you look at products. Good problem statements are specific and measurable: after-hours calls go to voicemail and we lose some of those jobs; our CSR spends an hour a day on reschedule calls; estimates go out two days late because the service manager writes them at night.
Then limit the scope so a failure cannot hurt you badly:
- Phones: nights and weekends only, or overflow when the office line is busy, not the whole line.
- Booking: requests only at first, confirmed by the office, rather than direct booking.
- Office writing: one task, such as customer emails, for one person.
- Estimates: wording only, on one estimator's jobs, with normal approval.
- Customers: residential only, excluding commercial and property-manager accounts at first.
Name an owner for the pilot, usually the office or operations manager, who reviews results every day and has authority to switch it off.
Measure the baseline first
Spend the week before the pilot measuring how things work today. For a phone pilot, count the calls in the pilot hours, how many reached a person, how many went to voicemail, how many voicemails led to a booked job, and how long callbacks took. Callbacks are worth measuring closely: only 19% of US adults generally answer calls from unknown numbers[3], so many shops find a large share of their after-hours callbacks never connect.
For an office-writing or estimate pilot, time a sample of tasks and keep copies of the output, so you can compare quality as well as speed. Write the baseline down and share it with the team; it becomes the yardstick at the end.
Agree the terms with the vendor
A pilot needs a vendor willing to be measured. Before signing anything, get these points in writing, ideally in a short pilot agreement rather than a long-term contract:
- The exact start and end dates, and what happens on the end date if you do nothing.
- Which features are switched on for the pilot, and confirmation that each one is live today rather than planned.
- Access to every recording, transcript and summary, exportable in a usable format during and after the pilot.
- Who on the vendor side fixes problems, and how fast they respond during nights and weekends.
- What happens to your data if you stop: how long it is kept, and how to have it deleted.
- That any pricing discussion comes after the results, based on the scope you actually decide to keep.
If a vendor will not agree to a time-limited trial on part of your call volume, with your own measurements, you have learned something important before spending anything.
The 30-day plan
Week 0: setup and baseline
Measure the baseline. Write rules, approved answers and the urgent path. Make at least twenty scripted test calls or test tasks, and fix every failure before going live.
Week 1: live, with full review
Switch on for the pilot scope only. The pilot owner reviews every call or output daily and logs each error by type. Fix rules the same day.
Week 2: steady state
Keep reviewing every urgent call and a daily sample of the rest. Hold a 15-minute check-in with the team on what callers and staff are saying.
Week 3: stretch test
Leave the setup alone and watch it handle a full week, including a weekend. Note anything the rules did not anticipate.
Week 4: measure and decide
Compare every metric with the baseline, read the error log, talk to staff and decide: go, extend or stop.
Metrics to track
| Metric | Definition | Example target |
|---|---|---|
| Answer rate | Calls in scope answered, divided by calls received | Above baseline, near all calls |
| Completed calls | Calls ending in a booking, urgent transfer or clean handover | Most calls in scope |
| Urgent calls reached | Urgent calls where the on-call tech was reached, and how fast | Every urgent call, within minutes |
| Record accuracy | Summaries with correct address, ZIP, phone and outcome | No errors on hard facts |
| Booked jobs | Jobs booked from pilot-scope calls | Above baseline |
| Requests for a person | Calls where the caller asked for a human | Falling over the month |
| Staff time | Minutes per day spent on callbacks or corrections | Below baseline |
| Complaints | Complaints or reviews about the experience | None serious |
Guardrails during the pilot
- Recording notice: federal law requires at least one party's consent to record, and some states require all parties[4]. Put the notice in the greeting from day one.
- Kill switch: know exactly how to send calls back to your old setup in under five minutes, and test it.
- Urgent review: read every call marked urgent, and a sample of those marked routine, to catch missed emergencies.
- No silent changes: log every rule change with the date and reason.
- Tell your team: staff should know what is being tested and how to report problems, so they do not find out from customers.
Brief the on-call technicians in particular. They will receive calls and messages in a new format during the pilot, often at night, and they are the first to notice if urgent calls are being described badly or routed to the wrong person. Ask them to text the pilot owner the same night if anything looks wrong, and thank them for every report.
Go, extend or stop
Write the decision rules when you write the scope, so the result is not decided by whoever is most enthusiastic in week four. A simple version:
- Go: every urgent call handled correctly, hard-fact accuracy at target, at least two other metrics clearly better than baseline, no serious complaints.
- Extend: no safety failures, but results are mixed or volume was too low to judge. Run two to four more weeks with specific fixes.
- Stop: any missed emergency that was not fixed by a rule change, repeated wrong addresses, or staff time higher than before.
A go decision should still widen scope in steps: from nights to overflow, from requests to direct booking for one job type. Each step deserves a short check of its own.
Example: a pilot that went to extend
Example: a five-tech shop pilots AI answering for nights and weekends. The baseline week shows 41 after-hours calls, 33 to voicemail and 9 booked jobs. In the pilot's final week, 44 calls are answered, 2 callers hang up during the greeting, 14 jobs are booked or sent to the on-call tech, and every urgent call reaches a tech. But three summaries had a wrong unit number. The owner chooses extend: two more weeks with a stricter read-back for apartments. After those weeks the errors stop, and the shop decides go.
Common pilot mistakes
- No baseline, so the end-of-month debate is opinion against opinion.
- Scope too wide, so one bad day affects the whole business.
- Nobody owning daily review, so problems pile up unseen.
- Changing several rules at once, so you cannot tell what helped.
- Judging on the vendor's dashboard rather than your own records and recordings.
- Letting the pilot drift past 30 days without a decision.
Frequently asked questions
How long should an AI pilot last?
Thirty days suits most plumbing shops, with a baseline week before it. Extend to six weeks if your call or task volume is low.
What should a plumbing company pilot first?
Usually after-hours or overflow call handling, because the problem is clear and easy to measure. Office writing is a lower-risk alternative.
Who should run the pilot?
One named owner, often the office or operations manager, who reviews results daily and can switch the tool off.
What if results are mixed?
Extend with specific fixes and a short deadline. Do not widen the scope until the safety and accuracy targets are met.
References
U.S. Census Bureau (2026). Business Trends and Outlook Survey (BTOS), response estimates by sector (opens in a new tab) โ
U.S. Census Bureau (2026). Business Trends and Outlook Survey (BTOS), national response estimates (opens in a new tab) โ
Pew Research Center (2020). Most Americans don't answer cellphone calls from unknown numbers (opens in a new tab) โ
Legal Information Institute, Cornell Law School (2026). 18 U.S. Code ยง 2511 - Interception and disclosure of wire, oral, or electronic communications prohibited (opens in a new tab) โ