Analytics
Analytics & Insights
From the moment you connect, Hyperfence analyzes everything flowing to and from your LLM. No extra setup - just a dashboard showing what your top-level topics your users are discussing, how the model is responding, and how many prompts and responses are being blocked.
Platform
The platform gives you an up-to-date view of your LLM traffic. Four numbers are always front and centre:
Token usage
How many tokens your LLM has sent and received. Useful for keeping an eye on API costs.
Active Hyperfences
How many hyperfencing rules are currently switched on. Zero means hyperfencing is not active for your traffic.
Blocked events
Number of messages that were stopped before reaching your model - or responses stopped before reaching your users.
Content analysis
See semantic similarity, average similarity over time, response sentiment, and topic classification so you can track how closely answers match prompts and what users are actually asking about.
What gets tracked
All of this is captured automatically for every message - nothing to configure.
Topic classification
Prompt topic mix is tracked over time, with values showing how strongly each topic appears in recent prompts. This gives you a clear picture of what users are asking about and whether traffic is shifting into unexpected areas.
Response sentiment
Responses are classified as positive, neutral, or negative over time. If model behavior starts trending negative, you will see it clearly in the dashboard.
Semantic similarity
Measures how similar the prompt input and model output are on a scale from 0 to 1. The dashboard shows average similarity across models over time, making it easy to spot when responses drift away from what was asked.
Blocked events
A count of blocked prompts and responses is kept along with the rule that triggered the block - prompt injection attempt, content policy violation, hyperfence match - so you can see what's being caught and how often.
Exporting to your own dashboards
For paid accounts, metrics are available in Prometheus format if you want to pull them into Grafana, Datadog, or any other monitoring tool. The full metric list is below.
request_countTotal requests per customer, provider, and model.token_countTokens consumed, labelled by direction (input/output), customer, provider, and model.response_countUpstream responses by HTTP status code.prompt_forward_delayHistogram: time from request receipt to upstream dispatch.response_processing_delayHistogram: proxy processing time for non-streaming and streaming responses.
prompt_topic_distributionWeighted counter per topic category across all prompts.response_topic_distributionSame for responses. Comparing prompt vs. response topics shows when the model is answering off-topic.semantic_similarityCosine similarity between prompt and response embeddings. Tracks response relevance over time.response_sentimentCounter of positive / neutral / negative classifications per customer, provider, and model.
prompts_blockedCounter with reason label (prompt_injection, contains_hap, hyperfence).responses_blockedCounter with reason, provider, and model labels.