Test protocol
How we will test
By the BotsRanked Test Team · Last updated: Oct 31, 2026
This is our test plan for AI phone receptionists: what we'll call, how we'll score it and how we'll publish it. We haven't made any test calls yet. Accounts get opened once our test phone line is set up. If the plan changes, the change log at the bottom will say what changed and when.
This plan may change before the first round. Every change goes in the change log below.
Why calls, not demos
A demo is set up by the vendor to sound its best. Business owners on Reddit say the same thing: don't trust demos, test with real callers. So we'll set each product up ourselves and call it like a customer would.
Accounts we'll test on
- Our own setup. We plan to use each vendor's free trial or free plan, load our own fake business into it and call it. This is the fairer test, because we control the facts.
- Vendor demo lines. Some vendors publish a number you can call to hear their AI. Where that's the only way in, we'll call it with the same scripts and label the result "demo line."
Please read this part. A product tested on a trial or a demo line may behave differently on a paid plan. Trials can have fewer minutes or features, and demo lines are tuned by the vendor. Every result will show which kind of account it came from and the date.
Trials run out, usually after 3 to 14 days. In later months we'll retest through a free plan or demo line where one exists. If none does, the last result stays up with its original date.
Vendors with no free trial or demo line are listed with pricing only and marked not tested. Ruby, Sameday and Slang.ai show no free trial and no demo line on their sites.
The test setup
- One fake business per industry. For example "Test Plumbing Co" or "Test Dental." Each has a one-page facts sheet: hours, services, prices, service area, emergency rules and who to transfer to.
- Same facts in every product. We load that sheet into each vendor's setup screen and use only settings any customer could use. No help from the vendor's team.
- Not every vendor in every industry. Trials have limits (AI-Receptionist.com's trial includes 30 minutes of live calls). Each industry page will list which vendors we tested for it.
- Same caller for everyone. Calls come from one US test phone line, with a fixed voice and pace, so every product hears the same thing.
The scripts
Each industry gets five scripted calls. The scenario stays the same every month; names, dates and small details change each round so nobody can tune for an exact script.
- New customer booking. The caller wants an appointment or a visit and gives name, number and reason.
- Price or service question. The caller asks something the facts sheet answers, such as a fee, hours or service area.
- Urgent call. Plumbers: a burst pipe at night. HVAC: no heat in winter. Dental: a bad toothache. Law firms: a new client with a deadline this week. Real estate: a buyer who wants a showing today.
- "Can I talk to a person?" The caller asks for a human early and repeats it if needed.
- Hard caller. Talks over the receptionist, corrects a detail halfway through and calls with background noise.
At least one call per vendor goes out after business hours. Each script runs twice per vendor per round, as far as the trial allows.
How we'll score
Each call is scored out of 100:
- Getting the lead or booking right: 25. Name, number, reason and time are captured and correct.
- Facts: 20. Answers match the facts sheet. Anything made up (a price, a promise, an opening) makes this 0 for that call.
- Urgent calls: 15. Spots the emergency and follows the business's rule, such as transfer, urgent alert or priority callback.
- Speed: 10. How fast it answers and how long it pauses before replying.
- Human handoff: 10. Transfers or takes a clear message when asked. Loops count against it.
- Handling the conversation: 10. Copes with interruptions, corrections and noise without losing the thread.
- Owner follow-up: 10. The call summary, text or email reaches the business, and it's correct.
On a demo line we can't check facts or see the follow-up, so those calls are scored on the other five parts (70 points) and scaled to 100.
Our test team listens to every call and checks the transcript against the scoresheet before a score goes up. A vendor's industry score is the average of its calls for that industry. Its overall score is the average across the industries we tested it in.
Recordings and the test-call notice
- Every call opens with: "Hi, this is a test call from BotsRanked, and it's being recorded."
- Every call is recorded. We publish the audio and a transcript on the vendor's page.
- Only made-up business and caller details are used. No real customer data is ever on a call.
- For services with human staff, like Ruby, staff names are removed from transcripts.
Monthly retests
- A new round every month, with the same scenarios and rubric.
- Every result shows its test date and account type. Old results stay in the score history.
- Every price shows the date we checked it and links to the vendor's page.
What vendors can and can't do
- They can't pay for a score, a rank, a review or a place in our lineup.
- They can't see the exact scripts before a round.
- They can send corrections to facts, like a price change.
Corrections
If we get a fact wrong, email us (the address is on our about page). We check it against the vendor's own page or our recording, fix it, and add a dated note to the page and to the change log below. We don't remove a recording because a vendor dislikes the result.
Test log
- Oct 31, 2026: Lineup of 14 vendors chosen. Prices, free trials and demo lines checked on each vendor's site. No test calls made yet.