What is the Test & Train Page?
The Test & Train page allows you to evaluate how your AI responds to customer questions and make corrections when needed. This essential tool helps you refine your AI's knowledge and behavior to ensure it accurately represents your brand and provides helpful information to your customers.
How to Test Your AI
Test Your Concierge
Type any question in the chat interface to see how your AI would respond to a customer
Use the preset example questions for quick testing:
"Tell me about your store"
"What is the shipping policy?"
"What products do you sell?"
Click "Show more" to reveal additional example questions
Review the AI's response to evaluate its accuracy and tone
Click "Reset test" to clear the conversation and start fresh
Testing Best Practices
Test questions your customers commonly ask
Try variations of the same question to ensure consistent answers
Test both simple and complex queries
Verify that product information, policies, and brand messaging are accurate
Check that the AI's tone matches your brand's personality settings
How to Train Your AI
If your AI provides incorrect or incomplete information, you have some options to improve its responses:
Add Custom FAQs
Click the "+ Add FAQs" button
Enter a customer question exactly as they might ask it
Provide the ideal answer you want your AI to give
Save the FAQ to teach your AI this specific response
Test the same question again to verify the AI now answers correctly
Provide Additional AI Instructions
Scroll to the "AI instructions" section
Enter clear guidelines for how your AI should handle certain topics or questions
Test related questions to ensure your instructions are being applied
Effective Training Strategies
Writing Effective FAQs
Use natural language that matches how customers actually ask questions
Include variations of common questions (e.g., "Do you ship internationally?" and "What countries do you ship to?")
Provide comprehensive answers that address the core question
Group related FAQs together for better context
Prioritize adding FAQs for your most common customer inquiries
Creating Clear AI Instructions
Focus on general behavior guidance rather than specific questions
Specify important information that should always be mentioned
Clarify how to handle sensitive topics or common misunderstandings
Update instructions as you notice patterns in AI responses that need improvement
Customize What Your AI Says When It Can't Answer
Your AI answers most customer questions successfully, but occasionally it encounters a question it can't handle directly. When this happens, your AI automatically apologizes and offers to escalate to your support team. You can customize this behavior through your AI Instructions to better match your brand and business needs.
Understanding Response Sources and Corrections
The test interface offers several important features to help you understand and improve your AI's responses:
Sources button: Click the "Sources" dropdown to see where your AI is getting its information from. This may include:
Links to your website pages and collections
Training documentation
Correct the AI button: If the AI provides incorrect information, click the "Correct the AI" button to:
Create a custom FAQ for this specific question
Provide the correct answer you want your AI to give
Override incorrect responses with accurate information
How to Correct Your AI
When you click "Correct the AI" button:
You'll see the original question and incorrect answer
Enter the correct answer your AI should provide
Save your correction as a custom FAQ
This correction will override any conflicting information for this specific question
Test the same question again to verify the correction is working
This direct correction method is the fastest way to fix specific inaccuracies without changing your broader AI instructions.
Troubleshooting
AI not applying new FAQs: Sometimes it takes a few seconds for new FAQs to be fully processed. Try testing again after a short wait.
Instructions seem diluted: The more detailed and clear your instructions, the better your AI can prioritize them among all the information it processes.
Competing information sources: If your AI seems confused, check if your website content contradicts your instructions or if different skill sections have conflicting guidance.
Fine-tuning is normal: Like training a new team member, your AI might need several rounds of feedback and refinement to fully align with your expectations.
Multiple training methods working together: Remember that AI personality settings, FAQs, and instructions all work together to shape your AI's responses.
Best Practices for Ongoing AI Improvement
Test your AI regularly, especially after adding new products or changing policies
Review customer conversations to identify opportunities for improvement
Add new FAQs based on actual customer interactions
Update instructions to address any recurring issues
Balance being comprehensive with being concise in your training