Overview
The ObservePoint AI Chat Agent provides instant support to help you get the most out of the platform. This FAQ guide is designed to help you navigate the agent’s capabilities and find answers to your questions quickly. ObservePoint uses Intercom's FIN AI Agent as our AI Chat Agent.
Here are some of the frequently asked questions:
What model or version of the model is the AI Chat Agent?
There isn’t a single “model name” and “version number” listed, because it’s a mix of our custom LLMs and third-party models. The AI chat agent uses custom LLMs trained specifically on customer service interactions, including retrieval, reranker, summary, escalation-detection, and customer-response-understanding models. It doesn't have traditional version numbers like software releases.
Third-party AI providers that are used to deliver the AI Product include:
AWS Bedrock (listed as Amazon Web Services, Inc.) - utilizing Anthropic’s models within the AWS Bedrock infrastructure.
OpenAI, LLC
Anthropic, PBC
Microsoft’s Azure OpenAI
Google (Vertex)
ElevenLabs
What are the human oversight or human-in-the-loop controls that apply to the AI Chat Agent?
The AI Chat agent detects when customers need human support and can escalate automatically. When the AI Chat agent doesn't know an answer, it will say so and can hand over to a human. Escalation rules are set for the AI Chat agent to automatically transfer specific requests and topics (such as whenever the customer sends images, links, or requests to talk to a certain team member) to human agents.
A Technical Writer reviews, evaluates, and improves answers based on the previous conversations. The process is based on a weekly review of already closed conversations where the AI Chat agent was involved, checking the answers provided, and improving data sources if needed.
Can I have a detailed description of the data sources used by the AI Chat Agent?
The AI Chat agent uses as its main information source the content we provide:
Your past conversations
ObservePoint Help Site
ObservePoint marketing site
What are the size and type of data used to train the AI Chat Agent?
The AI Chat Agent is trained on anonymized customer service interactions with an opt-out option available. There are 386 public articles written, reviewed and published by ObservePoint (on our helpsite https://help.observepoint.com/en/ ), and 887 website pages from our ObservePoint websites (https://www.observepoint.com/resources &. https://api-docs.observepoint.com/).
What are the top features used by the AI Chat Agent?
Analyze, Train, Test, and Deploy capabilities, all in one workspace, are among the most important features of the AI chat agent we are using.
What are the accuracy assessment measures used by the AI Chat Agent?
Some of the ways we assess accuracy for the AI Chat agent include: manual testing and rating (rate responses after the AI delivers answers), and reviewing conversations by the support team and the Technical Writer.
AI Chat Agent's resolution rate is calculated as the % of conversations the chat agent resolves, divided by the conversations the chat agent was involved in.
How is the AI Chat Agent regularly monitored to ensure it is performing as expected, and used only for the documented intended use?
Performance is monitored through real-time analytics that analyze the effectiveness of each stage of the answer generation process. The system tracks resolution rates, CSAT scores, and provides AI-powered optimization suggestions through the Analyze dashboard.
Can you provide some support documentation or user guide for the AI Chat Agent?
This is Intercom's official documentation and user guides for the AI Chat Agent ObservePoint is using: