What Is the Test & Train Page?
The Test & Train page is where you preview exactly how your AI will respond to customers — and fix it on the spot when something is off. It's your unified testing hub for every channel your AI runs on: the chat widget on your storefront, AI email replies, and Meta DMs on Instagram, Facebook, and WhatsApp.
You test on the left, train on the right. Pick a tab, send a sample message, and see the AI's reply rendered exactly the way a customer would receive it on that channel. If the response is wrong, you can correct it, add an FAQ, or update your AI Instructions without leaving the page.
The Three Channel Tabs
Test & Train has three tabs across the top of the page. Each tab shows a simulator styled to match the real channel.
Chat — Always available. Tests the AI assistant that shoppers see in the chat widget on your storefront.
Email — Available when the Answer Emails support skill is turned on (which requires a connected helpdesk such as Gorgias, Zendesk, or Re:amaze). If the tab is greyed out, hover for a tooltip and click to jump to Support Skills to activate it.
Meta — Available when at least one Meta platform (Instagram, Facebook, or WhatsApp) is connected. You can test a connected platform even if its AI Answering is still turned off — Test & Train always shows what your AI would say (see "Test & Train Always Uses Your AI" below). If no Meta platform is connected, the tab is greyed out; click it to jump to your Social integrations.
Greyed-out tabs always tell you exactly which setting needs to change, and clicking the tab takes you straight there.
Test & Train Always Uses Your AI
Test & Train is your preview space, so it always shows what your AI would say — on every tab, for every channel — even if you haven't turned AI Answering on for that channel yet.
This matters most for Meta (Instagram, Facebook, WhatsApp) and Email. In live conversations, your AI only replies on a channel once you've switched on AI Answering for it in Integrations. But you shouldn't have to expose real customers to an untuned AI just to see how it performs. So on the Test & Train screen the AI always responds — letting you tune your FAQs, AI Instructions, and personality first, and switch the channel on in production only once you're confident.
When you open the Meta or Email tab for a channel that still has AI Answering turned off, a short banner appears above the simulator:
Test & Train always uses your AI
Even when AI answering is off, this screen shows what your AI would say. Turn it on to use it in live conversations.
The banner includes a Manage integration button that takes you straight to that channel's settings, where you can turn AI Answering on whenever you're ready. On the Meta tab, the banner updates as you switch between Instagram, Facebook, and WhatsApp — it only appears for a platform that currently has AI Answering off, and its button always points to the platform you're viewing.
A few things to keep in mind:
Your live customers are not affected. Turning a channel's AI Answering on or off only changes what happens in real conversations. Test & Train is a private preview — testing here never sends anything to a customer.
The Chat tab has no banner. Chat always uses your AI both in testing and on your storefront, so there's nothing to enable.
When you're ready to go live, use the Manage integration button to jump to Integrations and switch AI Answering on for that channel.
Testing on the Chat Tab
The Chat simulator looks like your storefront chat widget — with your AI's name, greeting, and quick-reply buttons.
Type a question in the input at the bottom of the simulator (or use the quick prompt pills on the right).
Press Enter or click Send.
Read the AI's reply. If product recommendations are returned, you'll see them in a product carousel beneath the response.
Send a follow-up message to test multi-turn conversations.
Below each AI reply, click Sources to see the website pages, FAQs, or documents the AI used. Click Correct the AI if the answer is wrong.
Click Reset test to clear the conversation and start over.
Testing on the Email Tab
The Email simulator is built for AI email replies — the kind your customers get when they email your support address and the Answer Emails skill handles it.
(Optional) Fill in the Subject line.
Write your test email in the Body field — exactly like a customer would.
Click Test Email.
The view changes to an email thread: your message appears as a gray card, the AI's reply as a white card with a purple left border, with a proper email greeting, body paragraphs, and a signature block.
Click Reply below the thread to send a follow-up and test how the AI handles a back-and-forth conversation.
As with Chat, Sources and Correct the AI are available under every AI reply.
The AI uses your tone of voice, brand description, AI name, knowledge base, FAQs, and your Email AI Instructions — so what you see in the simulator is identical to what a real customer would receive.
Testing on the Meta Tab
The Meta tab is for DMs on Instagram, Facebook, and WhatsApp. Each platform's simulator mirrors that platform's real chat styling so you can see how the response will actually look in your customer's inbox.
Choose a platform from the pills at the top of the simulator. Connected platforms appear as active pills — including ones that still have AI Answering turned off — and the first one is selected by default. Unconnected platforms appear greyed out — click them to jump to Integrations.
Write a test message in the Customer message field.
Click Test Message.
-
Read the AI's reply in the platform's native style:
Instagram — pink-purple AI bubbles on the left, gray customer bubbles on the right
Facebook Messenger — blue AI bubbles on the left, gray customer bubbles on the right
WhatsApp — green AI bubbles, customer messages with double check marks, on a beige background
Send a follow-up message to test multi-turn DMs.
Switch platforms anytime — each platform keeps its own conversation state.
When the AI Can't Answer: The Training Banner
Sometimes the AI doesn't have enough information to answer confidently. Instead of leaving you waiting, the simulator replaces the response with a clear training banner in the place where the AI's reply would normally appear:
No confident answer yet — Teach the AI so it handles this next time. Add a custom FAQ for this exact question, or a global instruction for the broader topic.
The banner has two one-click actions, both pre-filled with the original test question so you don't retype anything:
+ Add FAQ — Opens the FAQ editor. Type the answer you want the AI to give and save. The FAQ applies across every channel.
+ Add instruction — Opens the AI Instructions editor. Add a behavioral rule that covers this kind of question more broadly.
Once you've saved your FAQ or instruction, send the same test message again — the AI should now answer correctly.
This is the fastest way to close knowledge gaps the moment you find them, on whichever channel you're testing.
Training Your AI From Test & Train
The right side of the page contains the tools you use to test and train your AI, and they all work across every channel.
Quick Prompts: Starter Questions Built From Your Store
The four quick prompt pills on the right side of the page are starter questions written specifically for your store. Rep AI reads your product catalog, your collections, your brand description, and the policies in your knowledge base and on your website, then writes questions a real shopper might actually ask you — things like "What's included in the Pokémon buildable figures gift set?", "Which of your teas is the most popular?", or "Do you ship to Canada?"
Before a question ever reaches you, Rep AI answers it using your store's own information and checks the result. A question becomes a prompt only if the answer is grounded in your real content, genuinely useful, and in your brand's voice. So every prompt you see is one your AI can already answer well — clicking it is a fair test of a real answer, not a guess.
A few things worth knowing:
You always get four. If your store's information doesn't yet support four store-specific questions, the set is filled out with general starter questions such as "Tell me about your store" or "What is your return policy?" — shown in your store's language.
They're ready before you are. Your prompts are prepared as soon as your catalog finishes syncing during setup, so store-specific questions are already waiting the first time you open Test & Train.
They stay current. Prompts refresh automatically about once a day, picking up new products, new collections, and updated policies.
The same four prompts appear on every tab. Clicking one sends it to whichever tab you're on — Chat sends it to the chat simulator, Email fills in the subject and body and sends the test email, and Meta sends it to the platform you have selected.
There's nothing to set up. No configuration, no editing, no button to press. The prompts follow whatever you've taught the AI.
Your quick prompts are also a quick read on how well Rep AI knows your store. Prompts that name real products or collections confirm the AI has that part of your catalog and can talk about it accurately. Prompts that are all general shipping and returns questions mean the AI couldn't confidently ground a question in your products yet — which usually points to thin product descriptions, a brand description that hasn't been filled in, or a catalog that has only just synced. Enrich those, and your prompts get more specific on the next refresh.
Add Custom FAQs
Click + Add FAQs.
Enter the customer question exactly as a customer might phrase it.
Provide the ideal answer.
Save.
FAQs apply globally — they're used by the AI on every channel.
For deeper guidance on writing good FAQs, see Create and Manage Custom FAQs.
Channel-Specific AI Instructions
AI Instructions let you set behavioral rules — tone, what to mention, what to avoid. With the redesign, each channel has its own independent instruction set, because the way you want your AI to behave on a casual Instagram DM is often very different from a formal customer support email.
The Instructions panel automatically retitles itself based on the active tab:
Global AI Instructions — when the Chat tab is active (applies to chat and as a baseline)
Email AI Instructions — when the Email tab is active
Instagram AI Instructions — when Meta → Instagram is selected
Facebook AI Instructions — when Meta → Facebook is selected
WhatsApp AI Instructions — when Meta → WhatsApp is selected
Click + New to add an instruction for the channel that's currently active. Each channel maintains its own list — what you write under Email won't affect WhatsApp, and vice versa.
Don't forget to Save. AI Instructions don't auto-save — click Save before you leave the page or switch tabs, or your changes will be lost.
Correct the AI
Whenever you see a wrong AI response, click Correct the AI below it. You'll see the original question and the AI's incorrect answer. Enter the correct answer and save — your correction is stored as a custom FAQ and will override conflicting information for that question across every channel.
This is the fastest way to fix a specific inaccuracy without changing your broader instructions.
Testing Best Practices
Start with a quick prompt. It's the fastest way to see what Rep AI already knows about your store — and the four prompts are chosen precisely because the AI can answer them from your own catalog and policies.
Test every channel you've enabled. Chat, Email, and each Meta platform can answer differently based on their channel-specific instructions — what's right for one might be wrong for another.
Tune before you go live. You don't need to switch AI Answering on to test a channel. Preview and correct the AI in Test & Train first, then enable the channel in Integrations once you're happy.
Test variations of the same question. Customers ask the same thing in many ways. Make sure the AI answers consistently across phrasings.
Use the fallback banner as a signal. A response that triggers the training banner is a knowledge gap — fix it on the spot.
Test after every change. New product line, updated shipping policy, new support skill — open Test & Train and verify the AI picked it up.
Check Sources, not just the reply. A correct-sounding answer pulled from an outdated source will go wrong eventually. Use the Sources dropdown to confirm the AI is using the right material.
Troubleshooting
A tab is greyed out. That channel isn't fully enabled yet. Hover the tab for the exact requirement, then click it to jump to the right settings page.
The Email tab is greyed out even after enabling Answer Emails. Make sure a supported helpdesk integration is connected — Answer Emails requires both the skill toggle on and the helpdesk hooked up.
A Meta platform is connected but AI Answering is off. You can still test it in Test & Train — the AI responds, and a banner reminds you it isn't live yet. When you want it to answer real customers, use Manage integration to turn AI Answering on. Only platforms that aren't connected at all appear greyed out.
AI Instructions aren't taking effect. Confirm you clicked Save for the right channel — Instructions are scoped per tab and don't auto-save.
A new FAQ isn't being used yet. Give it a few seconds to process, then re-test the same question.
The fallback banner keeps appearing for the same question. Either your FAQ doesn't match the phrasing closely enough, or your AI Instructions are too vague. Try rewording the FAQ to match how a real customer would ask, or tighten the instruction.
Instructions feel diluted. The more specific and concrete your instructions, the more likely the AI will follow them. Avoid vague guidance.
My quick prompts are all general questions, not about my products. Rep AI only publishes a prompt it can answer confidently from your store's information, so a fully generic set means it couldn't yet ground a question in your catalog. Check that your catalog has finished syncing, fill in your brand description, and enrich your product descriptions and knowledge base. Prompts refresh about once a day.
My quick prompts are different from last time. That's expected. They're rewritten from your latest catalog, collections, and content, so the wording changes as your store changes.
I want to edit or reorder my quick prompts. Quick prompts aren't editable today — they're shaped entirely by what you've given the AI to work with. To steer them, improve your product descriptions, brand description, FAQs, and knowledge base.
My catalog reaches Rep AI through a direct API feed rather than a Shopify or SFCC sync. In that case your prompts are prepared the first time you open Test & Train rather than during setup. Open the page, then come back to it a moment later and your store-specific prompts will be there.
Tabs and platforms keep their state. Switching tabs preserves your test conversations. Use Reset test to wipe a specific channel's simulator and start fresh.
Keep Improving Over Time
Test & Train isn't a one-time setup. The merchants who get the most out of Rep AI treat it as an ongoing quality loop:
Review real customer conversations for issues, then reproduce them in Test & Train and fix them.
Whenever the fallback banner appears, add the FAQ or instruction immediately — the question is already there waiting for an answer.
After major catalog or policy changes, run a quick test pass on every enabled channel.
Periodically re-read your channel-specific instructions — what worked at launch may not match how your brand sounds today.