Please enable JavaScript.

Coggle requires JavaScript to display documents.

AI Observability - Coggle Diagram

- - - - EU AI Act Report: risk categorizations
      - Model Cards
        
        Bias Tests
        
        Accuracy metrics
        
        Usage Limitations
      - GDPR Documentation: Data lineage maps
    - - Bias/fairness results
      - Runtime Decision Tracking
        
        Prediction with input/output snapshots
        
        Data Lineage
        
        Policy Violation and override Actions
  - - - Data Type
      - Field Existence
      - Value Formats
    - - Value Range
      - Null rate Threshold
- - - - Outlier Detection
        
        Input Outlier Detection
        
        Statistical Anomaly Detection
        
        Z-Score Analysis
        
        IQR Fencing
        
        Dynamic Anomaly Detection
        
        Temporal Deviation Analysis
        
        Velocity Analysis
        
        Output Outlier Detection
        
        Multivariate Anomaly Scoring
        
        Threshold Sanity Checks
        
        Cross Feature Outlier Detection
        
        Unexpected Value Combinations
      - Data Drift Detection
    - - Data Drift Monitoring
        
        Output data drift
        
        Behavioral Drift
        
        Response Drift
        
        Performance Drift
        
        Input data drift
        
        Syntactic Drift
        
        Embedding Comparison
        
        Contextual Drift
        
        Embedding Analysis
        
        Lexical Drift
        
        Embedding Comparison
        
        Domain Knowledge Drift
        
        Embedding Analysis
        
        Statistical Drift
        
        Statistical Tests (KS, MMD)
        
        Prompt Drift
        
        Semantic Similarity (Embedding-based)
        
        Statistical Tests (Statistical)
        
        Concept Drift
        
        Clustering Analysis (Embedding-based)
      - Outlier Detection
        
        Input outliers
        
        Point Anomaly
        
        Out-of-Distribution Responses
        
        Data Anomalies
        
        Pattern Anomaly
        
        Contextual Mismatches
        
        Sequences Irregularities
        
        Output outliers
        
        Point Anomaly
        
        Excessive Response Length
        
        Hallucinations
        
        Inappropriate Responses
        
        Data Leakers
        
        Pattern Anomaly
        
        Contextual Mismatches
        
        Verbosity Patterns
        
        Sequence Irregularities
        
        Behavioral outliers
        
        Point Anomaly
        
        Internal Model Outliers
        
        Confident Fabricators
        
        Pattern Anomaly
        
        Contextual Mistmatches
        
        Multi-Agent Miscoordination
        
        Topic Wanderers
        
        Sequence Irregularities
        
        User Behavior Anomalies
  - - - Accuracy and Quality
        
        Can I trust critical outputs like classifications or translations?
        
        Task specific Performance
        
        Monitoring Focus
        
        Accuracy Validation
        
        Translation Tasks
        
        3 more items...
        
        Classification Tasks
        
        3 more items...
        
        Text Quality Metrics
        
        ROUGE Score
        
        3 more items...
        
        BLEU Score
        
        3 more items...
        
        Confidence Measurement
        
        Perplexity Analysis
        
        3 more items...
        
        Output Certainty
        
        3 more items...
        
        Generation Quality
      - Safety & Ethics
      - Efficiency