Fine-Tuning Intent Recognition using Generative AI
This guide provides you with step-by-step instructions on harnessing the power of generative AI within our low-code platform to optimize the prompt for intent recognition.
Discover the ins and outs of refining prompts, allowing you to create more intelligent and context-aware conversational experiences for your users. Dive into the details and unleash the full potential of generative AI for precise and accurate intent recognition.
Understanding Basics of Generative AI for Intent Recognition
Generative AI plays a pivotal role in intent recognition, offering a dynamic approach to understanding user input in conversational agents. Unlike rule-based methods, generative AI allows the system to generate responses based on learned patterns and context, providing a more flexible and context-aware interaction.
Step-by-step guide on setting Generative AI intents
Navigate to the ANS node within the flow editor. Open the modal for the ANS node by clicking on it.
Add a new intent: In the Intent section of the ANS node modal, click on the "+" button to add a new intent.
Select Generative AI: Choose "Generative AI" as the intent recognition method for your new intent.
Enter a friendly name: Input a friendly and recognizable name for your intent. This name will help you identify and manage intents effectively.
Define brief intent meaning: Provide a concise meaning for your intent in a few words. This brief description will assist the system in recognizing user input and triggering the appropriate responses.
Add intent description: Optionally, you can fill in a brief description of your intent. Keep it concise, considering the token limit, and provide additional context that aids in understanding the purpose of the intent.
❗Be mindful of the tokens you use in the intent description. Tokens are units of text that the model processes and there may be limitations of max token limits for the whole intent recognition prompt. Efficient use of tokens ensures optimal performance and cost.
While Generative AI is trained on a diverse range of general language data, recognizing specific intents may require some fine-tuning to align with your unique use case and domain. This is particularly important to ensure that the conversational agent accurately interprets and responds to user input in the context of your application.
In our low-code platform, you have two powerful options for fine-tuning Generative AI intent recognition, allowing you to tailor the model to better suit your specific needs:Method #1: Intent description: Enhancing intent understanding
Fill in the intent description to provide additional context for the Generative AI model. This option allows you to offer a concise but informative description of the intent, aiding the model in better understanding the nuances and specifics associated with each intent.
⚠️When crafting these descriptions, be mindful of token usage, ensuring that they effectively communicate the intent without exceeding any limitations. ⚠️
Method #2: Edit default super prompt: Adding context of a conversationCustomize the default super prompt used behind the scenes to add dynamic context for enhanced intent recognition. This option empowers you to inject specific details or context relevant to your application directly into the model's training prompt.
By modifying the super prompt, you can guide the model to focus on certain aspects or nuances, refining its ability to accurately recognize and respond to user intents.
Crafting meaningful intent descriptions can significantly enhance the model's understanding of user queries. Discover valuable tips and examples on effectively filling in intent descriptions, ultimately leading to more precise and context-aware interactions with your conversational agents. Let's delve into the art of providing descriptive context to elevate your intent recognition capabilities.
Provide user's utterance examples
When designing prompts for intent recognition, it's crucial to provide clear and diverse examples of user utterances.
Example:
Intent description: User asks for weather forecast [Show me the weather in London; Tell me how hot it'll be in Sydney; I wonder if it's going to rain in Seattle; Is it going to be sunny in Miami?]
Example:
Intent description: User asks you to wait [eg. A moment please; Hold on for a sec; Just a second]
Include relevant keywords
When designing prompts for intent recognition, it's crucial to sprinkle in relevant keywords related to the specific topic. These keywords help guide the model to better understand the user's intent.
Example:
Meaning: Reclamation
Intent Description: claim, damage, not working, broken, won't turn on, not functioning
Anticipate variations in user language by including synonymsor alias terms within the intent description. This expands the model's understanding and improves recognition accuracy.
Example: If the intent involves checking account balances, include synonyms like "account status," "financial overview," or other terms users might use interchangeably.
Highlight contextual boundaries
Explicitly outline contextual boundaries by specifying situations where the intent is not relevant. This ensures that the model does not mistakenly associate the intent with unrelated user input.
Example: Intent Description: "User expressing interest in product discounts during promotions. Irrelevant for questions about product warranties or post-purchase support."
Use negative definitions for exclusions
Clearly articulate what the intent is not about by providing negative definitions or exclusions. This helps the model distinguish between closely related intents and improves precision.
Example: Intent Description: "Focused on user inquiries regarding product availability and stock status. Excludes questions about product specifications or customer reviews."
Consider multifaceted intents
Acknowledge intents that may have multiple facets or subcategories. Break down complex intents into distinct components within the description, providing clarity for the model.
Example: Intent Description: "User expressing interest in various tech-related discounts, such as promotional offers, seasonal discounts, or loyalty rewards."
Method #2: Edit default super prompt: Adding context of a conversation
Behind the scenes, all Generative AI intent inputs provided through the GUI are amalgamated into a comprehensive super prompt that empowers the ANS node. This unified prompt serves as a directive for the model during intent recognition, ensuring a seamless and efficient process. To facilitate this, a standardized prompt is generated for the ANS node, instructing the model on how to handle incoming utterances.
The template for this system message is as follows:
System message
You are a useful assistant used for intent recognition. You are given a list of intents with its name, meaning and description. Please classify the sent utterance and respond only intent name from the list as a JSON object only. Carefully read all the instructions.
Super prompt instructions
Classify the utterance that you receive as the next message into one of these categories by selecting the most dominant topic from this list below:
- friendly name of intent 1: meaning of intent 1 [description of intent 1]
- friendly name of intent 2: meaning of intent 2 [description of intent 2]
[...]
If the requested values cannot be found, you should return "NOT_FOUND". Always respond with a valid JSON object in the format { 'recognized_intent': string }. Do not return anything other than JSON.
When fine-tuning your Generative AI intents, you have the option to customize the default super prompt, which serves as a crucial directive for the model during intent recognition.
To access the super prompt editing window you need:
Have at least one existing Generative AI intent set and saved.
Navigate to the Intents sections in the ANS node and click on the pencil icon✏️.
Next, you can customize the super prompt system message and instructions.
Edit instructions or add more context in the prompt. Furthermore, you can add placeholders of existing variables to fill in the prompt in a personalised, dynamically changing way. The current value of the variable will be filled into the prompt.
Use this feature to your advantage. Enhance the understanding of the situation, previous interactions or knowing personal info about the user to handle the Digital agent - user communication gracefully and to master intent recognition with contextual awareness.
While this customization allows you to add context and refine prompt, we strongly recommend limiting edits to the system message only. Especially don't temper with the "Text after list of intents" section of a prompt. This is essential for maintaining the integrity of backend processes and ensuring seamless interactions between the user and the conversational agent.
Customizing the system message within the default super prompt offers a powerful avenue to introduce context for more nuanced and accurate intent recognition. It's crucial to note, however, that digital agents lack inherent knowledge of the preceding user interactions unless explicitly provided in the prompt. Additionally, without supplementary contextual information, there may be a decrease in intent recognition precision.
Here are some customization tips:
Explicitly provide previous question information
Edit system message to add context. Explicitly state the previous question, so the generative AI is given context of conversation and can better classify the user's utterance into intent categories.
Examples:
System message
[...] Context: The user was asked "What your favourite animal is?" Please classify [...]
System message
[...] Context: The user was asked to describe for further description of product malfunction. Please classify [...]
You can also add a placeholder of existing variables to be able to fill in the conversational context dynamically. In flow, set a variable in each MSG node and populate it with a string of message text resumé. Then, use:
Example:
System message
[...] Context: The user was asked {last_question}. Please classify [...]
Provide conversation history summary
Pre-set a variable with summary of interactions or conversation history. Then, use a placeholder variable to add this as a context.
Example:
System message
[...] Context: Digital agent informed the user that {summary}. The user's utterance probably reacts to that. Please classify [...]
Provide user-specific personalised metadata
Add a placeholder for an existing, pre-defined variable with personalised metadata. These can help the generative AI to understand the user's situation better.
To illustrate, the utterance "I want to have 10 GB of mobile data" might belong to intent tariff upgrade or tariff downgrade depending on the user's current tariff parameters.
Example:
System message
[...] Context: The user account state is {balance}, with {debt} to pay. Please classify [...]
System message
[...] Context: The user currently has the [tariff_name}, with {data_amount} GB of mobile data. Please classify [...]
System message
[...] Context: User's last bought product are {product_1}, {product_2}, {product_3} Please classify [...]
Provide temporal refenrences
For complex recognition which includes temporal awareness or comparison against calendar, provide a placeholder for variables that contain time and date metadata.
Example:
System message
[...] Context: The user was asked whether she/he can pay the debt before {deadline_date}. Today is {current_date}. Please classify [...]
System message
[...] Context: Today is {day_of_week}, {date}. It's {time} right now. Please classify [...]
System message
[...] Context: The shop is open from {opening_hour} to {closing_hour}. Please classify [...]
Provide conversation context
Add contextual info and describe a previous interaction or scenario that lead the user to give the current input.
Examples:
System message
[...] Context: The previously recognized intent was on {topic}. Take into consideration that current input may or may not reflect the previous interaction. Please classify [...]
System message
[...] Context: User successfully finished a process of {scenario_type} and is asked whether he/she wants anything else. Please classify [...]
