Please enable JavaScript.
Coggle requires JavaScript to display documents.
LLM Security Observability - Coggle Diagram
LLM Security Observability
Prompt Injection
What to Observe?
Suspicious Prompt Patterns
Bypass Attempts
Unusual Parameter Override
Encoded Anomalies
Base64/hex-encoded segments
Invisible Unicode characters
Unusual whitespace patterns
Key Metrics
Injection_pattern_matches
Blocked_request_ratios
Policy_violation_rate
Detection Tools
OPA/Rego policies
Rule-based Heuristics
DistilBERT Classifier
Data Leakage
What to Observe?
Auto-redaction triggers
PII
Emails
Phone Numbers
SSNs (Social Security Numbers)
Names
Secret Patterns
API Keys
Password or tokens
Restricted Content
High Entropy Data Transfer
entropy >4.5 bits/character
Key Metrics
PII_match_count
Redaction_events
data_entropy_spike
request_latency_seconds
calculate_entropy
Detection Tools
spaCy NER + Regex
LLM-as-Judge
OpenAI Moderation API
OPA/Rego
Model Extraction
Types
Model‑Functionality Extraction
Prompt‑Based Data Recovery
Private Text Reconstruction:
Prompt‑Targeted Attacks
API‑Based Knowledge Distillation
Direct API Querying
Parameter & Architecture Recovery
Training‑Data Extraction
Prompt Stealing
Systematic Stealing
Defense
DoS Protection
What to Observe?
Token Consumption Spikes
Abnormal Request Bursts
Concurrent Session Surge
Key Metrics
token_usage_rate
requests_per_second
error_429_count
Detection Tools
Token counters
OpenTelemetry Latency Tracing
API Gateway rate limiter
FastAPI middleware
Auth & Access Monitor
What to Observe?
Auth Requests
Total vs. failed login attempts
Timestamps & request rates
Anomaly Scores
Rate‐anomaly z‑scores
Behavioral‐profile deviations
Geo-anomalies
Usage Surge Alert
Key Metrics
failed_auth_count
request_total
total_token_consumed
endpoint_requests_total
Detection Tools
OpenTelemetry collectors
Stream processor (Kafka/Kinesis)
Prometheus & Grafana