Analytics
Analytics & Insights
From the moment you connect, Hyperfence starts tracking everything flowing through your chatbot. No extra setup, just a dashboard showing what your users are asking, how the model is responding, and what is being blocked.
Dashboard
The dashboard gives you a live view of your chatbot traffic. Four numbers are always front and center:
Token usage
How many tokens your chatbot has sent and received. Useful for keeping an eye on API costs.
Active Hyperfences
How many hyperfencing rules are currently switched on. Zero means hyperfencing is not active for your traffic.
Blocked events
Messages that were stopped before reaching your model - or responses stopped before reaching your users.
Content analysis
See semantic similarity, average similarity over time, response sentiment, and topic classification so you can track how closely answers match prompts and what users are actually asking about.
What gets tracked
All of this is captured automatically for every message - nothing to configure.
Topic classification
Prompt topic mix is tracked over time, with values showing how strongly each topic appears in recent prompts. This gives you a clear picture of what users are asking about and whether traffic is shifting into unexpected areas.
Response sentiment
Responses are classified as positive, neutral, or negative over time. If model behavior starts trending negative, you will see it clearly in the dashboard.
Semantic similarity
Measures how similar the prompt input and model output are on a scale from 0 to 1. The dashboard shows average similarity across models over time, making it easy to spot when responses drift away from what was asked.
Blocked events
Every blocked message is logged with the reason, whether that is a prompt injection attempt, a content policy violation, or a Hyperfence match, so you can see exactly what is being caught and how often.
Exporting to your own dashboards
All metrics are available in Prometheus format if you want to pull them into Grafana, Datadog, or any other monitoring tool. The full metric list is below.
request_countTotal requests per customer, provider, and model.token_countTokens consumed, labelled by direction (input/output), customer, provider, and model.response_countUpstream responses by HTTP status code.prompt_forward_delayHistogram: time from request receipt to upstream dispatch.response_processing_delayHistogram: proxy processing time for non-streaming and streaming responses.
prompt_topic_distributionWeighted counter per topic category across all prompts.response_topic_distributionSame for responses. Comparing prompt vs. response topics shows when the model is answering off-topic.semantic_similarityCosine similarity between prompt and response embeddings. Tracks response relevance over time.response_sentimentCounter of positive / neutral / negative classifications per customer, provider, and model.
prompts_blockedCounter with reason label (prompt_injection, contains_hap, hyperfence).responses_blockedCounter with reason, provider, and model labels.