Governance and control
Testing before every release
How every change to your agents is tested against your industry, against the behaviour that already works, and by your own team, before any customer sees it.
On this page
Every change to your agents is tested before it reaches your customers. A new workflow, a new escalation rule, an updated policy document, a newer language model (the AI component that understands the conversation and composes the reply): each is tested in a separate environment against conversations from your industry, checked against everything that already works, and tried by your own team before it goes live. This is what makes it safe to improve your agents continuously, and what lets your organisation approve a change on evidence rather than on trust.
What is tested before every release
| Area | What the tests check | Example |
|---|---|---|
| Answers | Every answer comes from your systems and approved documents, and figures match the record | The instalment amount given is the amount in the payment system |
| Limits | The agent takes only the actions, and offers only the values, your rules permit | No rebooking option is offered that the fare does not allow |
| Scope | Topics out of scope are declined and routed to a person, including when a question is rephrased | Investment advice requested three different ways, in Arabic and English |
| Decision boundaries | Decisions that stay with your team are not attempted | A policyholder pressing for a view on liability |
| Escalation | Each trigger fires when it should, and the handover reaches the right destination with a brief | A frustrated caller is passed to the duty team |
| Language | Gulf, Egyptian and Levantine Arabic, English, and switching between Arabic and English within a sentence | "Can I change the booking to الجمعة?" |
| Conversation design | The conversation follows the order your industry needs | Checking a caller is safe before taking a motor claim |
| Integrations | Lookups and write-backs to your systems work correctly | The claim created in the test claims system has every required field |
| Approved responses and tone | Scripted wording is exact, and the tone matches what you approved | A regulatory disclosure is read word for word |
Industry test sets
Telonic builds a test set for each industry: conversations drawn from the real patterns of that industry, each scored against what a correct outcome means there. In property, a correct answer about the next instalment on a construction-linked plan (one where payments fall due as building work reaches agreed stages) depends on the milestone the project has reached. In travel, a correct rebooking is one the fare rules permit. In insurance, a correct first notice of loss (the first report of a claim) asks the right questions in the right order and makes no comment on liability.
Each test set grows with every deployment in that industry, as new workflows and edge cases are added. What is shared between deployments is industry knowledge and test scenarios, never customer data.
Regression suites before every release
A regression suite is a set of tests re-run on every change, to confirm that nothing that worked before has stopped working. Before every release, the regression suite runs against your industry's definitions of success, as well as the specific tests for the change itself.
A change that adds a new escalation rule is tested for the new rule, and for every other rule, limit and answer that could be affected. A release goes ahead only when it passes. See How changes go live.
Simulated conversations
Conversation simulation tests the agent against realistic scenarios before go-live (the point at which the agent starts handling real customers) and before significant changes. Simulated customers hold full conversations with the agent in the test environment: an angry guest whose room is not ready, a caller who switches between Arabic and English mid-sentence, a policyholder who asks an out-of-scope question in four different ways.
Scenarios are written with your team, from the situations your people know best. Conversation simulation is configured during implementation.
Test and live environments
Every deployment has a test environment, separate from the live one. Changes are made and tested there, connected to test versions of your systems where you have them, so no real customer receives a message and no real record is changed. Only an approved change moves to the live environment.
Your team tests before your customers do
Before go-live, and before any significant change, your team tries the agent in the test environment. Your people listen to the voice, read the replies, test the questions they know customers ask, and check the handovers arrive where they should. For a voice deployment, both your telephony team and ours test calls, transfers and the recording consent step using test numbers.
Nothing goes live until your team has approved it.
New models and providers
Telonic adopts a new language model or provider for your deployment only when it performs at least as well as the current one on your industry's test set. Every change of model or provider is approved by your organisation before release. Fallback providers, used if a provider has an outage, are tested in the same way, in advance. See Models and providers.
The technical detail
| Question | Answer |
|---|---|
| What is a release? | Any change to an agent's configuration, workflows, rules, knowledge sources or model, moved from the test environment to the live one |
| What runs before every release? | The regression suite against your industry's definitions of success, the tests specific to the change, and, where configured, simulated conversations |
| Is the test environment connected to our systems? | To test versions of your systems where you have them. Where you do not, lookups and actions are tested against sample records agreed with your IT team |
| What do we receive before approving? | A summary of what changed, what was tested, and the results |
| Are Telonic's own platform changes tested the same way? | Yes. Changes to the platform are reviewed and tested before release, including against the industry test sets |
In practice
Qamar Stays, a hotel group in Riyadh and Jeddah, wants its agent to handle early check-in requests.
- Its front office manager agrees the rule with Telonic: early check-in is offered on flexible rates when the room is ready, and anything else goes to the front desk.
- Telonic builds the change in the test environment, connected to a test copy of the reservation system.
- The travel and hospitality test set and the regression suite run. They show the new workflow gives correct outcomes, and that existing booking changes, transfers and escalations are unaffected.
- Simulated guests ask for early check-in in Gulf Arabic, in English and in a mix of both, on flexible and non-refundable rates.
- Two members of the front office team try it on WhatsApp and by phone, and ask for one change to the wording.
- The change is made, the tests run again, and the front office manager approves the release.
What your team controls
- The scenarios your team wants tested, written with Telonic.
- Your team's own review before go-live and before significant changes.
- Approval of every release, including every change of model or provider.
Related
- How changes go liveGovernance and control
- Models and providersAgents
- Industry modelsAgents
- Secure development and testingSecurity and data protection
- The controls your team holdsGovernance and control
Product names and logos are trademarks of their owners. Their mention shows systems Telonic connects to and does not imply partnership or endorsement.