> For the complete documentation index, see [llms.txt](https://docs.borndigital.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.borndigital.ai/digital-agent/conversation-flow/regression-test.md).

# Regression test

Check in one run that the agent still sends typical customer messages to the right place – before you deploy a new version.

**Validate** › **Regression test** sends a list of test messages to the selected, trained version of the project and shows where the conversation goes after each of them. Use it after you change intents, the training set or the flow, to see that what worked before still works.

Each test case is **one message from one starting point**: the test does not run whole conversations, and it compares where the conversation goes – not the text of the answer.

<figure><img src="https://4261467870-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F6iQTvxgRZRwPS1NgIGEb%2Fuploads%2Fgit-blob-1fbf8696db986371bfc546606f0b5c6606a4d268%2Fregression-test-results.png?alt=media" alt="Regression test results: the accuracy chart, the score 0 errors, 0 fails, 1 unknowns, 0 SSI matches, 4 matches, and the results table with start state ANS_MAIN_MSG, the utterances, target and predicted states END_DONE_LOOP and ANS_MAIN_MSG, confidence and result icons"><figcaption><p>Results of five test cases</p></figcaption></figure>

## Prepare the test cases

A test case has three columns:

| Column         | Required | Content                                                                                                     |
| -------------- | -------- | ----------------------------------------------------------------------------------------------------------- |
| `start_state`  | yes      | The state where the conversation is waiting when the message comes.                                         |
| `utterance`    | yes      | The customer's message.                                                                                     |
| `target_state` | no       | The state where you expect the conversation to wait next. Without it, the result is only shown, not judged. |

The states are the names from the [Code editor](https://docs.borndigital.ai/digital-agent/advanced-functions/code-editor.md) (YAML), not the node names on the canvas:

* An [Answer node](https://docs.borndigital.ai/digital-agent/conversation-flow/nodes-explained/answer-node.md) named `ANS_MAIN` waits for the customer in the state **`ANS_MAIN_MSG`** – use it as the `start_state`, and as the `target_state` when the conversation should come back to that question.
* An End node named `END_DONE` ends in the state **`END_DONE_LOOP`**.

Example for the [tutorial agent](https://docs.borndigital.ai/digital-agent/building-new-projects.md):

```
start_state;utterance;target_state
ANS_MAIN_MSG;bye;END_DONE_LOOP
ANS_MAIN_MSG;goodbye, thank you;END_DONE_LOOP
ANS_MAIN_MSG;What are your opening hours?;ANS_MAIN_MSG
ANS_MAIN_MSG;Can I pay by card?;
```

The first row with the column names is optional; without it, the columns must be in this order.

## Run the test

The test starts as soon as you load the data:

* **Upload** a `.csv`, `.tsv` or `.txt` file with the columns separated by **semicolons** (`;`), or
* select the rows in Excel (or another spreadsheet), copy them and paste them anywhere on the page with **Ctrl+V** – the columns are then separated by tabs.

{% hint style="warning" %}
The text on the screen says the file is "tab delimited, ^ separator". Uploaded files are read with **semicolons**; tabs work when you paste the rows.
{% endhint %}

The test runs against the **version selected in the project**, which must be trained. Larger sets take a while – the rows are processed one after another. The results are not stored: download them before you leave the page.

## Read the results

* The **chart** shows, for each target state, the share of test cases that reached it (**Accuracy \[%]**).
* **Score** counts the results: **errors** (the test case could not be run), **fails**, **unknowns** (no target state), **SSI matches** and **matches**.
* The **table** shows for each test case the **Preprocessed Utterance** (after the [Setting tab](https://docs.borndigital.ai/digital-agent/conversation-flow/setting-tab.md) vocabulary and pre-processing), the **Predicted state**, the **Confidence**, the other **Candidates** with their scores, and the **Result**:

| Result        | Meaning                                                                            |
| ------------- | ---------------------------------------------------------------------------------- |
| green tick    | Match – the conversation reached the target state.                                 |
| amber tick    | SSI match – the predicted state belongs to the same group of states as the target. |
| red alert     | Fail – the conversation went elsewhere.                                            |
| question mark | Unknown – no target state was given.                                               |
| red circle    | Error – hover over it to see why, for example a state that does not exist.         |

**Download as CSV file** saves the results (tab-separated, without the candidates).

{% hint style="info" %}
Keep your test cases in a spreadsheet and add a row every time you fix a misunderstood message. Before every deployment, paste the whole sheet and check that nothing that worked before has broken.
{% endhint %}
