Arize AI
An ML observability platform that has expanded to support LLM applications. Arize provides monitoring, tracing, and evaluation for both traditional ML models and LLM-based systems. Phoenix, their open-source tool, focuses specifically on LLM observability and evaluation.
Implements
Concepts this tool claims to implement:
- Benchmark primary
LLM evaluation with built-in and custom evaluators. Evals for hallucination, toxicity, relevance, and custom criteria. Experiment tracking and comparison.
- Hallucination secondary
Hallucination detection and scoring. Compare generated responses against retrieved context. Track hallucination rates over time.
- Retriever secondary
RAG-specific observability. Trace retrieval quality, embedding drift, and document relevance. Debug retrieval failures.
Integration Surfaces
Details
- Vendor
- Arize AI
- License
- Elastic License 2.0 (Phoenix)
- Runs On
- local, cloud
- Used By
- human, system
Links
Notes
Arize comes from traditional ML observability and brings that expertise to LLMs. Phoenix is their open-source LLM-specific offering. Strong on embedding visualization and drift detection. Good for teams already using Arize for ML who are adding LLM capabilities.