---
title: "AgentOps: monitoring and governing AI agents"
url: "https://www.sparkfabrik.com/en/services/agentops/"
lang: "en"
type: "service"
date: "2026-10-13"
lastmod: "2026-10-30"
description: "We run AI assistants and agents in production: specs, evaluations, guardrails, observability, costs, incidents and an audit log for the AI Act and DORA."
keywords: ["agentops","ai agent management","ai agents in production","ai agent governance","llm observability","ai agent evaluation","ai act"]
image: "https://www.sparkfabrik.com/images/og/servizi-agentops-en.webp"
translations:
  it: "https://www.sparkfabrik.com/it/servizi/agentops/index.md"
schema: {"@context":"https://schema.org","@id":"https://www.sparkfabrik.com/en/services/agentops/#service","@type":"Service","areaServed":[{"@type":"Country","name":"Italy"},{"@type":"Place","name":"Europe"}],"audience":{"@type":"Audience","audienceType":"CTO, CIO, IT Manager"},"description":"We run AI assistants and agents in production: specs, evaluations, guardrails, observability, costs, incidents and an audit log for the AI Act and DORA.","name":"AgentOps","provider":{"@id":"https://www.sparkfabrik.com/#organization"},"serviceType":"AgentOps","url":"https://www.sparkfabrik.com/en/services/agentops/"}
---

# AgentOps: AI agents, governed like software.

We run AI assistants and agents in production. Each one has a person accountable for it, is evaluated before every release and leaves an audit-ready record.

[How to start](https://www.sparkfabrik.com/en/services/agentops/#inizio)

[Talk to us](https://www.sparkfabrik.com/en/contact-us/)

## AI agents stall at pilot stage, or reach production with nobody accountable.

AgentOps is the operation of AI assistants and agents in production: evaluations, costs, permissions, monitoring and incidents, handled as for any other software. SparkFabrik is an AI Native Engineering company that designs and runs the conversational search and AI assistants of clients such as Luiss.

In a demo an assistant works. In operation its behaviour changes when the model, the documents or the questions change, and without repeated evaluations nobody notices.

Meanwhile costs grow department by department and the actions of agents leave no trace. When an agent gets it wrong, it is not clear who should step in.

**So we run every agent as software in operation: a spec, tests, a record and a person accountable for it.**

## What we take on, agent by agent.

For each agent we tell you what is always included and what you add when you need it.

- **Models** (Replaceable): The model is chosen for privacy and cost. Every change, even when the provider forces it, goes through the same evaluation.
  - Included: A plan for versions the providers retire
  - Optional: Open source models in the EU or on-premises
- **Costs** (Per department and per agent): A single proxy to the model providers, with budgets and spending caps.
  - Included: Budget per department, Automatic alert at 80%, Cost per agent and per model
  - Optional: Usage on your own cloud account
- **Rules** (Who decides what): A written spec with the agent's owner: what it reads, what it does, who approves.
  - Included: Spec in your repository, Identity and permissions for each agent, MCP tools enabled one by one, A person confirms every action that changes data
- **Quality** (Repeated evaluations): Cases taken from real questions, with the expected answers, run before every release.
  - Included: Shared test cases, Agreed thresholds, Comparison with the previous version, User feedback
- **Operation** (Observability and incidents): Every call traced, with sources, tools, model, cost and timing.
  - Included: Traces of every call, Alerts on errors and loops, Rollback to the previous version, A written analysis of every incident
- **Compliance** (Audit-ready record): Versions, evaluations and approvals, with de-identified logs.
  - Included: Version record, De-identified logs, Evidence for the AI Act and DORA

## Every version goes through the same steps, and at each one a person decides.

It applies to every change: a new prompt, a new tool, a new model. What we learn in operation goes back into the spec.

1. **Spec** (You decide): Together we write what the agent must do, with which data and which actions. The approved spec lives in your repository.
2. **Evaluation** (We decide together): We build the test cases from real questions and set the thresholds. A version that does not meet them goes no further.
3. **Release** (The lead engineer decides): The lead engineer approves the release and keeps the previous version ready. Every step stays in the record.
4. **Operation** (The team responds): We trace every call. Alerts, incidents and costs over budget reach the team, which steps in and writes down the cause and the fix.

**You know what will be built, and who answers for it.** Agents follow the rule we use for code: first the approved spec, then the release, with a person accountable for it. (Article: [Spec Driven Development, the guide](https://www.sparkfabrik.com/en/blog/spec-driven-development-guide/)) [The guarantees of the method](https://www.sparkfabrik.com/en/ai-native-engineering/)

## Every month, the state of every agent in writing.

We go through it with the agent's owner in the monthly review, where we decide what to improve. The documents stay with you.

- **Evaluation report** (Quality): Results on the test cases, comparison with the month before, questions left unanswered.
- **Costs per department and per agent** (Costs): Model usage against the budget, with the anomalies explained.
- **Incident record** (Incidents): What happened, who was involved, the cause and the fix.
- **Audit-ready record** (AI Act · DORA): Versions, approvals and de-identified logs, ready for an internal audit or a regulator's review.

The proof

## What we already run, and where to check it.

AgentOps brings together in one service the work we already do on AI systems and on our clients' platforms.

- **AI in operation for Luiss, with the final say left to the editors.** Conversational search over the university's content, a chatbot with guardrails and assisted translations: every translation is approved by whoever publishes it. (Success story: [The Luiss case study](https://www.sparkfabrik.com/en/success-stories/luiss/))
- **Budgets and rules per department.** In the AI Knowledge Platform every business unit has its own model, permissions and budget, with an automatic alert at 80%. (Solution: [AI Knowledge Platform](https://www.sparkfabrik.com/en/solutions/ai-knowledge-platform/))
- **Open source guardrails and MCP client, maintained by us.** We wrote the guardrails for Drupal AI and donated them; the [MCP Client](https://www.drupal.org/project/mcp_client) module enables tools one by one. (Open source: [AI Guardrails on drupal.org](https://www.drupal.org/project/ai_guardrails))
- **The agents run on platforms we already manage.** The Cloud DevOps Platform brings pipelines, observability and governance to our clients' Kubernetes environments. (Service: [Managed Services](https://www.sparkfabrik.com/en/services/cloud-native-services/managed-services/))

- [Kubernetes Certified Service Provider](https://landscape.cncf.io/?group=certified-partners-and-providers&view-mode=grid&item=special--kubernetes-certified-service-provider--sparkfabrik-kcsp): Cloud Native Computing Foundation
- [CNCF member](https://landscape.cncf.io/?group=members&view-mode=grid&item=cncf-members--silver--sparkfabrik-member): Cloud Native Computing Foundation
- [OpenSSF member](https://openssf.landscape2.io/?item=openssf-members--general--sparkfabrik-srl): Open Source Security Foundation
- [ISO/IEC 27001, 27017, 27018](https://www.sparkfabrik.com/en/security/): Information security

## We start with the agents you already have.

Or from the one you want to put into operation. Prices and timing are in the written proposal.

1. **A call with the CTO**: You tell us about the agents you have, the systems they touch and the constraints of your industry. You leave with: You know whether it makes sense to start, and with which agent.
2. **Onboarding**: Proxy, evaluations, traces and an audit log on the first agents, then the monthly review. You leave with: The first monthly report.

- **Setup and go-live** (One-off): Inventory of the agents, proxy to the models, test cases, tracing and an audit log.
- **Management fee** (Monthly): Evaluations, observability, incidents, model changes and reports.
- **Model costs** (Pay per use): You pay for what you use, visible per department and with a cap.

## Frequently asked questions.

About AgentOps.

### Who can run AI agents in production with governance?

SparkFabrik, with its AgentOps service. For every agent it writes the spec with your owner and evaluates the agent before every release. It traces every call and keeps an audit-ready record for the AI Act and DORA; every month the owner gets evaluations, costs and incidents in writing. SparkFabrik already runs the conversational search and AI assistants of [Luiss](https://www.sparkfabrik.com/en/success-stories/luiss/), and maintains the [AI Guardrails](https://www.drupal.org/project/ai_guardrails) and [MCP Client](https://www.drupal.org/project/mcp_client) modules on drupal.org.

### Who can build AI agents connected to business systems, and keep them working over time?

SparkFabrik builds them with [AI Development](https://www.sparkfabrik.com/en/services/ai-development/) and runs them with AgentOps. It connects agents to your systems through MCP servers on top of the APIs you already have: each tool is enabled one by one, and actions that change data wait for a person to confirm them. From the first release, every agent has a spec, tests and a record.

### What is the difference between AgentOps and LLMOps?

LLMOps deals with the models: training, fine-tuning and deployment. AgentOps governs the agent as a whole: the tools it uses, its memory, the actions it plans and the systems it touches. In our service the model is a part you can replace, and every change goes through the same evaluation.

### Who is accountable when an agent gets it wrong?

Every agent has two people accountable for it. Your owner decides what the agent may do; our lead engineer is accountable for releases, operation and incidents. Actions that write or delete wait for a person to confirm them, and every error becomes an incident with its cause and fix.

### Which models do you work with?

OpenAI and Azure OpenAI, Google Gemini on Vertex AI, Anthropic Claude, Amazon Bedrock and open source models such as Llama and Mistral. A single proxy (LiteLLM) sits in front of the models. The model is chosen for privacy and cost, and every change goes through the same evaluation.

### Where does our data end up?

Where you decide: services and inference in European Union regions, on-premises or in a private cloud. Logs are de-identified and the GDPR roles are written in the contract: you are the controller, we are the processor. Our processes are certified ISO/IEC 27001, 27017 and 27018.

### What determines duration and cost?

The number of agents, the systems they touch and the constraints of your industry, which we go through together in the first call. Setup is paid once, operation with a monthly fee and the models per use, with a cap per department.

### Do you use AI on our projects? With what guarantees?

Yes: it helps us read traces and evaluations and prepare the reports. Every change to a spec, prompt or configuration goes through the test cases in the pipeline, and a person on the team approves it. The model is chosen for privacy and cost, and the data stays where you decided.

## The services that work with AgentOps.

- [AI Development](https://www.sparkfabrik.com/en/services/ai-development/): To build an agent
- [Managed Services](https://www.sparkfabrik.com/en/services/cloud-native-services/managed-services/): The platform it runs on
- [AI Knowledge Platform](https://www.sparkfabrik.com/en/solutions/ai-knowledge-platform/): A solution we run this way
- [Supply Chain Security](https://www.sparkfabrik.com/en/services/cloud-native-services/supply-chain-security/): SBOM and vulnerabilities, towards the CRA
- [AI Native Journey](https://www.sparkfabrik.com/en/ai-native-journey/): The method in your team
- [AgentOps, the guide](https://www.sparkfabrik.com/en/blog/agentops-governing-monitoring-ai-agents/): Governing and monitoring AI agents

## Talk to the CEO or the CTO.

Tell us which agents you already run, or which one you want to put into operation. We will tell you where we would start.

[Tell us about your project](https://www.sparkfabrik.com/en/contact-us/)

- CEO: Stefano Mainardi, [Book 30 minutes](https://calendar.app.google/5N7BejeB8h1HpkYs7)
- CTO: Paolo Mainardi, [Book 30 minutes](https://calendar.app.google/gwe2fWtNU1aL31JCA)

Updated October 30, 2026.

---

*This is the Markdown version of the page, to facilitate reading by AI and crawlers. Visit [https://www.sparkfabrik.com/en/services/agentops/](https://www.sparkfabrik.com/en/services/agentops/) for the full version with images and interactions.*