Key takeaways
- Audit what reaches your techs, not answer speed: twenty messages a month, each checked against its recording.
- Treat three fields as critical: callback number, service address including the unit[2], and urgency. One wrong critical field fails the message.
- Twenty messages is a screen, not a verdict. Two errors in twenty fit a true error rate anywhere from about 3% to 30%[3].
- Take findings back with timestamps, ask for a cause per critical error, fix your own instructions too, and re-audit in thirty days.
Audit the message, not the answer time
Most answering service reports lead with the figure that is easiest to measure: how quickly calls were picked up. The figure that costs a plumbing company money rarely appears on any report: how often the message that reached your office or your on-call tech was right. A call answered in two rings and logged with one wrong digit in the callback number is a lost job, and nothing on the monthly summary will show it.
Message errors are easy to miss because each one looks like bad luck. The customer “didn't pick up.” The address “must have been a typo.” The tech drove to the right street and the wrong building. Put twenty messages side by side with their recordings and those one-offs often turn into a pattern you can fix with one phone call to the provider.
The audit below takes about two hours a month. It works for any answering service, and just as well for your own CSRs on the overflow line.
Pull a sample that covers the risky hours
Take twenty messages from the last thirty days
Not the ones you remember. Ask for the full message log and pick by rule, so the sample is not built from complaints.
Weight the sample toward nights and weekends
A useful mix: eight overnight, four weekend daytime, four the service marked urgent and four ordinary business-hours overflow. The errors that matter most hide in hours when nobody in your office is awake to catch them.
Request the recording for each message
If the provider cannot produce recordings, that is your first finding. You cannot audit what you cannot hear.
Pull your own record of what happened next
The job ticket, the on-call log and whether the customer was reached. The message is half the story; the outcome is the other half.
Score each message against its recording
Listen first, then read the message. Score each field right or wrong. Partial credit feels fair and hides exactly the problems you are looking for.
| Field | Points | Counts as right when |
|---|---|---|
| Callback number (critical) | 3 | Every digit matches what the caller said, and it is the number they asked to be called on |
| Service address and unit (critical) | 3 | Street number, street, town and any apartment, suite or unit number |
| Urgency (critical) | 4 | Matches your written rules: active water, no water, sewage backing up, gas or a CO alarm treated as urgent; routine work not woken |
| Action taken | 3 | The on-call tech was reached when the rules say so, and was not woken when they don't |
| Caller name | 2 | Spelled correctly, or flagged as unclear |
| Problem in the caller's words | 2 | The detail kept: “water coming through the kitchen ceiling,” not “leak” |
| What the caller was told | 2 | Callback time and any fee or policy line match your script |
| Script and tone | 1 | Your greeting used, and no promises you have not approved |
Urgency gets the most points because it is where a message error turns into damage. Water damage and freezing claims averaged $15,400 per homeowners claim over 2019 to 2023[1]. Carbon monoxide is worse: CDC counts more than 400 deaths and more than 100,000 emergency department visits a year from unintentional poisoning not linked to fires[6]. A CO alarm call logged as “water heater question, call in AM” is the worst error this audit can find, and one is enough to act on. Your rules for waking the on-call tech are the answer key for this column, and gas calls should match what your team should say when a caller smells gas.
Numbers and addresses fail in predictable ways: transposed digits, a missing apartment number, Maple Court written as Maple Street, a town that shares a street name with the next one. The unit is not optional detail; postal addressing standards treat it as a required part of any address that has one[2]. Listen for whether agents read the number and address back to the caller and spell names letter by letter. If the recordings show they don't, that single habit is usually the cheapest fix on offer, and how to take a phone message has read-back wording you can hand the provider.
What twenty messages can and cannot tell you
Twenty messages is enough to catch a habit and too few to grade a provider to the decimal point. NIST's statistics handbook recommends the Wilson interval for estimating a proportion and warns that the familiar quick formula is not accurate enough when samples are small[3]. Applied to an ordinary audit result, the chart shows the problem: two wrong messages out of twenty is consistent with a true error rate anywhere from about 3% to 30%.
So read the sample as a screen. Zero critical errors in twenty does not prove the service is flawless; the same calculation still allows a true rate of up to about 16%[3]. One or more critical errors on urgent calls is a finding on its own, whatever the percentage, because each one can cost so much. And when the screen shows a problem, pull twenty more before deciding anything drastic. Forty messages narrow the range, and you will have more recordings to put in front of the provider.
17 of 20
messages correct in every field
2
callback numbers with a wrong digit, both overnight
1
burst supply line logged as routine and not dispatched
0
apartment or suite numbers missed
Example results, invented for illustration. Three failed messages, two from the same overnight shift.
Read the results for patterns
- By hour. Errors bunched between midnight and 6 am usually point to thin overnight staffing or a less experienced shift.
- By field. Repeated number errors point to read-back. Repeated urgency errors point to your instructions, which may be vaguer than you think.
- By call type. If burst pipes are handled well and sewer backups are not, the triage questions for that one problem need rewriting.
- By outcome. Line up failed messages against jobs you lost. A wrong callback number usually ends the job outright: the customer waits for a call that never comes and rings someone else.
- By agent. If the provider can tell you who took each failed message, two failures from one person is a coaching conversation, not a contract problem.
Even a correct message starts at a disadvantage, which is why accuracy matters so much. Only 19% of adults say they generally answer calls from unknown numbers[7], so your callback often has one chance to land, and it needs the right number to have even that.
Add three test calls between audits
Real messages show how the service handles the calls it happens to get. Test calls show how it handles the ones you fear most. Once a month, have someone outside the office place three calls from a mobile number the service will not recognize, and score them on the same sheet.
- A 2 am burst pipe with water spreading. The on-call tech should be reached under your rules, and the caller told what happens next and roughly when.
- A caller outside your service area. The service should say so plainly and not take a booking you cannot keep.
- A price question on work you never quote by phone. The answer should match your script word for word, with no improvised figure.
Tell the on-call tech in advance which night the test falls on, so nobody drives out to a made-up address, and call the service afterwards to cancel the job. Test calls are also the fairest way to compare two providers, or a provider against your own CSRs, because every candidate gets the same caller with the same problem at the same hour.
Take the findings back to the provider
Specific evidence gets action; general complaints get apologies. Telephone answering is a specialist industry, 1,035 US establishments employing 41,582 people in 2023[8], and your account is one of many on the provider's board. Send findings to your named account manager rather than a general support inbox, and make them easy to act on.
- Send the scorecard with a timestamp for every failed message, so they can pull the same recordings.
- Ask for a cause for each critical error: the agent, the script, training, or your own instructions.
- Agree the fix in writing, with a date: a read-back rule, a script change or a new triage question.
- Fix your side too. Many urgency errors trace back to an instruction sheet that says “emergencies” without defining one; the answering service instruction sheet shows how to write it.
- Re-audit in thirty days with a fresh twenty. Two months running with critical errors on urgent calls is a reason to reread your contract's exit terms and compare alternatives using questions to ask when choosing a plumbing answering service.
Example email: “We audited twenty of October's messages against the recordings. Three failed. On 14 October at 2:12 am and 22 October at 1:48 am the callback numbers had a wrong digit. On 19 October at 11:05 pm a burst supply line was logged as routine and the on-call tech was not called. Please tell us the cause of each by Friday and how you will prevent it. We will audit November's messages the same way.”
- System
- Your team
01System
Pull twenty messages
Chosen by rule, weighted to nights, weekends and urgent calls.
02Your team
Match each to its recording
Listen first, then read what reached your office.
03Your team
Score the critical fields
Number, address and unit, urgency. One wrong field fails the message.
04Your team
Send findings with times
Ask for a cause and a dated fix for every critical error.
05Your team
Re-audit in thirty days
A fresh twenty. The trend matters more than any single month.
Example cycle. About two hours a month for a small office.
Use the same audit on any answering setup
Nothing here is specific to a live answering service. Run the same scorecard on your own CSRs' overflow calls, on a night dispatcher, or on an SI Employee. With an SI Employee, every call already has a recording, transcript and summary in your account, so you can score every urgent call instead of a sample. The comparison is identical: what the caller said against what reached your team.
Monitoring AI call quality week by week adapts this scorecard to a weekly rhythm, and the after-hours plumbing example on assistcall.ai shows what those records look like on a night shift. Commercial work fails differently, usually through the wrong supervisor paged or a missing PO number; the commercial contractor example scenario adds those fields to the checklist.
Frequently asked questions
How often should we audit our answering service?
Monthly while you are new to a provider or fixing a problem, then quarterly once two audits in a row show no critical errors. Audit again after any change: a new script, a new season, new on-call rules.
What if the provider won't share recordings?
Ask why, in writing. Without recordings you can only check messages against what customers tell you later, which catches far fewer errors. Recording access is worth putting in the contract before you sign; in-house CSR vs answering service covers other terms worth weighing.
Is twenty messages enough for a busy shop?
Enough to catch habits, not to measure a precise rate[3]. Busier shops can score twenty urgent calls plus twenty routine ones each month.
Should we tell the provider we are auditing?
Yes. The sample comes from calls already taken, so warning them cannot change it, and knowing you audit every month tells them accuracy is what you are paying for.
What error rate should we accept?
There is no published industry benchmark to lean on. A workable standard: no critical errors on urgent calls, and a falling count of other errors month to month.
References
Insurance Information Institute (2019-2023). Facts + Statistics: Homeowners and renters insurance (opens in a new tab) ↑
United States Postal Service (2026). Publication 28, Postal Addressing Standards: Secondary Address Unit Designators (opens in a new tab) ↑
National Institute of Standards and Technology (2026). 7.2.4.1. Confidence intervals (NIST/SEMATECH e-Handbook of Statistical Methods) (opens in a new tab) ↑
Legal Information Institute, Cornell Law School (U.S. Code) (2026). 18 U.S. Code § 2511 - Interception and disclosure of wire, oral, or electronic communications prohibited (opens in a new tab) ↑
The Florida Senate (2025). Florida Statutes 934.03 Interception and disclosure of wire, oral, or electronic communications prohibited (opens in a new tab) ↑
Centers for Disease Control and Prevention (2026). Carbon Monoxide Poisoning: About (opens in a new tab) ↑
Pew Research Center (2020). Most Americans don't answer cellphone calls from unknown numbers (opens in a new tab) ↑
U.S. Census Bureau (2023). County Business Patterns 2023, U.S. national file (cbp23us) (opens in a new tab) ↑