Platform Powers End-to-End Continuous Improvement of Agentic Applications
SAN FRANCISCO, March 18, 2025 -- Galileo, the AI Evaluation company, today announced an integration with NVIDIA NeMo™, enabling customers to continuously improve their custom generative AI models. Now, customers can evaluate models comprehensively across the development lifecycle, curating feedback into datasets that power additional fine-tuning. As a result, customers ship GenAI apps that are more reliable, trusted, and cost-effective.
Data Flywheel for AI
The majority of enterprises are developing GenAI applications – including agents and RAG-based chatbots – but it can be challenging to ship and scale these applications due to the non-deterministic outputs of Large Language Models (LLMs). There's even more complexity when AI teams wish to test new LLMs, which are constantly evolving in capability and price point. The solution is to build an AI data flywheel, enabling continuous testing and refinement, collecting data about user interactions for subsequent improvement. When AI teams use data to improve outcomes (whether by fine-tuning, prompt engineering, or in-context learning), they gain a competitive advantage.
Galileo and NVIDIA accelerate a data flywheel by collecting and curating better data about the interactions of an AI application. Galileo complements NVIDIA NeMo Evaluator with customizable evaluators that empower teams to iterate and better measure the success of their AI applications. For example, Galileo can evaluate whether an AI agent is calling the right tools and if RAG-based outputs are hallucinations or pulled from provided context. Galileo evaluators include explanations for the score as well as recommendations about how to improve it. Galileo also serves as a control plane for guardrails, extending NeMo Guardrails with low-latency, low-cost models that intercept risky outputs before they are presented to users.
"NVIDIA NeMo is a powerful toolchain for end-to-end generative AI development," said Yash Sheth, COO and Co-founder of Galileo. "The Galileo platform, combined with NVIDIA NeMo, can accelerate an AI data flywheel from data curation, fine-tuning, generative AI development into production, for a continuously improving system. We're giving developers the tools they need to find the best models, fine-tune quickly, and respond to any drift that arises in the real world."
"The Galileo platform, integrated with NVIDIA NeMo, can turbo-charge an AI data flywheel," said Santiago Pombo, Group Product Manager at NVIDIA. "Customers can leverage the best datasets and metrics to customize, evaluate and scale their LLMs with confidence, and use NeMo Guardrails within Galileo Protect to build safe, secure and robust solutions. Galileo's real-time observability also instills trust in production by continuously evaluating systems running on top of NVIDIA NIM, sending alerts if something goes wrong or interactions drift from the training data."
Key highlights:
- Data Error Potential analysis with Galileo Fine-Tune improves dataset quality for NeMo Customizer.
- Galileo Evaluate complements NeMo Evaluator benchmarks with custom metrics, test sets, debugging insights and optimization recommendations.
- NeMo Guardrails integrated with Galileo Protect provide safe, accurate guardrails with deep control for de-risking AI-powered applications.
Availability
Galileo is generally available now. The integrations with NVIDIA NeMo are available starting April 2025 for all customers.
To learn more, visit www.galileo.ai to sign up or get a demo.
About Galileo
San Francisco-based Galileo is the leading platform for AI evaluations. Powered by Evaluation Foundation Models (EFMs), Galileo's platform supports AI teams across the development lifecycle from prototyping to production. Fortune 500 companies and startups use Galileo to accelerate AI development, increase AI accuracy, and add real-time guardrails. Learn more at www.galileo.ai.
This News is brought to you by Qube Mark, your trusted source for the latest updates and insights in marketing technology. Stay tuned for more groundbreaking innovations in the world of technology.
