Suggest a tool

Ragas

Repository: vibrantlabsai/ragasHomepage

Metrics as of , from the GitHub or GitLab API of each repository. Refreshed monthly.

What Ragas is

Ragas is a Python toolkit for evaluating and optimizing applications built on large language models. It scores application output with both LLM-based and traditional metrics. It generates test datasets for evaluation. It integrates with LLM frameworks such as LangChain and with observability tools. Custom metrics, such as a DiscreteMetric with allowed values and a prompt, score responses through an LLM client. The `ragas quickstart` command creates example projects from templates, including one for RAG evaluation. It collects anonymized usage data, which the RAGAS_DO_NOT_TRACK environment variable disables.

Written from the project's README, read .

Category
AI and LLM evaluation
License
Apache-2.0
Language
Python
Changelog
Releases on GitHub

Repository metrics

Repository metrics
Stars15,8241
Forks1,7181
Open issues and PRs6031
Contributors2461
Commits in the last 90 days01
Last commit2026-02-241
Last releasev0.4.3, 2026-01-131
Fetched

1 Fetched from the GitHub or GitLab API on . Hover a value for its own date.

Status

slowLast commit 91 to 365 days before the fetch date.

Computed from the last commit date and the archive flag on the fetch date. See the status rules.

Alternatives

Listed AI and LLM evaluation tools, same primary language first, then by GitHub stars. Each line gives one fact from the tool's documentation where it differs from Ragas's, with its source. See also Ragas alternatives.

  • OpenAI Evals: Model providers: Models on the OpenAI API; completion functions in evals/registry/completion_fns or any CompletionFn implementation. source: Docs: How to run evals

  • DeepEval: Model providers: OpenAI, Azure OpenAI, Ollama, Anthropic, Gemini, LiteLLM; custom models through DeepEvalBaseLLM. source: Docs: Introduction to LLM Metrics

  • Language Model Evaluation Harness: Model providers: Hugging Face transformers, vLLM, SGLang, GGUF via llama.cpp, NeMo, Megatron-LM; OpenAI, Anthropic, LiteLLM, local API servers. source: README

  • garak: Model providers: Hugging Face, Replicate, OpenAI, AWS Bedrock, LiteLLM, Cohere, Groq, NIM, ggml/GGUF, REST endpoints. source: README

  • Giskard: Model providers: Provider SDK extras such as openai and anthropic; default judge model openai/gpt-4o-mini. source: README

  • TruLens: Model providers: OpenAI, Azure OpenAI, LiteLLM, Google Gemini, AWS Bedrock, Snowflake Cortex, HuggingFace, LangChain models, OrcaRouter. source: README

  • HELM: Model providers: Models from various providers through one interface, such as OpenAI, Anthropic Claude, Google Gemini. source: README

  • Inspect: Model providers: OpenAI, Anthropic, Google, Grok, Mistral, DeepSeek; AWS Bedrock, Azure AI; Groq, Together AI; local models. source: Docs: Model Providers

Comparisons

How to install

pip install ragas
From README, read .

Questions

Is Ragas open source?

Yes. Ragas is released under Apache-2.0, an OSI-approved license, as reported by the GitHub API on 2026-09-22.

Is Ragas maintained?

On 2026-09-22, the last commit to the default branch was on 2026-02-24, so the listed status is slow. Rule: Last commit 91 to 365 days before the fetch date.

How many GitHub stars does Ragas have?

15,824 stars on 2026-09-22, from the GitHub API. The number is refreshed at each monthly update.

What language is Ragas written in?

The repository's primary language, as reported by the GitHub API, is Python.

How do I install Ragas?

The README gives this command: pip install ragas

Sources

  1. GitHub REST API: repository, read
  2. GitHub REST API: commits, read
  3. GitHub REST API: latest release, read
  4. GitHub REST API: contributors, read
  5. README, read