Azure News - 2026-07-13
2026-07-13
最終更新: 2026-08-27 21:13:49 JST
Azure Architecture Blog
The AI Agent Lifecycle: A Simple Guide
- Link: https://techcommunity.microsoft.com/t5/azure-architecture-blog/the-ai-agent-lifecycle-a-simple-guide/ba-p/4535729
- Published: 2026-07-13 09:52:49
- Fetched: 2026-08-27 21:13:49
詳細を表示
The Bigger Picture
Building an AI agent is fundamentally different from building traditional software.
With a website or application, teams typically design, develop, test, and release. Once deployed, the focus shifts primarily to maintenance and feature enhancements. AI agents operate differently. They don't just execute predefined instructions they interpret information, reason, and make decisions.
In banking, those decisions can influence customer experiences, operational efficiency, compliance outcomes, and risk management. As a result, deploying an AI agent is not the finish line; it's the beginning of an ongoing process of learning, monitoring, and improvement.
The lifecycle of an enterprise AI agent reflects this reality.
Stage 1 —> Design: Decide What It Can and Cannot Do
Before writing a single line of code, answer three questions:
- What is this agent allowed to do?
- What must it never do?
- Who is accountable when something goes wrong?
Simple example:
A loan agent is allowed to check credit scores, apply lending policy, and recommend a decision. It is never allowed to approve a loan above $50,000 without a human sign-off. The Head of Credit Risk is accountable.
Design produces one critical output: the risk classification. A low-risk FAQ bot and a high-risk loan decisioning agent need completely different levels of testing, guardrails, and oversight. Getting this wrong at design time is expensive to fix later.
Microsoft helps here with:
| Tool | How It Helps |
|---|---|
| Microsoft Foundry Model Catalog | Browse and compare models and select the right one for the risk level before any code is written |
| Azure AI Content Safety | Review the built-in risk categories to understand what the platform can enforce, informing the guardrail boundary decisions |
| Microsoft Responsible AI Impact Assessment |
Structured tooling to assess and document harms, likelihood, severity, and mitigations, producing a risk classification artefact. |
Stage 2 —> Build: Put the Safety Controls In, Not On
Build the agent but more importantly, build the safety controls at the same time. Not afterwards.
The core stack:
Simple example:
The loan agent is built with: a PII redaction step (strip account numbers before they reach the model), a credit bureau tool, a policy lookup tool, an output checker (does the response cite a real policy?), and a Human in the loop gate (flag any decision over $25k for human review).
Establish the golden dataset during this phase: a representative set of real-world loan scenarios with SME validated expected outcomes. This serves as the ground truth for evaluating accuracy, consistency, and regression performance throughout the agent lifecycle.
Microsoft helps here with:
| Tool | How It Helps |
|---|---|
| Microsoft Foundry Agent Service | The primary platform for building and hosting the agent, tool registration, memory, and orchestration in one place |
| Azure OpenAI Service | The LLM backbone with configurable built-in content filters on every inference call |
| Azure AI Content Safety | Input and output guardrails, content moderation and Prompt Shield for injection and jailbreak detection |
| Azure AI Language | PII detection and redaction across 100+ entity types before data reaches the model |
| Azure AI Search | The RAG pipeline retrieves verified policy documents to ground every agent response |
| Azure Functions (Premium) | Hosts custom guardrail logic (e.g. policy compliance checks) inside the bank's private network |
| Microsoft Foundry Tracing | Instruments every tool call and reasoning step, essential for evaluation and audit |
Stage 3 —> Test: Find the Failures Before Customers Do
Testing occurs in three waves:
- Automated testing evaluates the agent against the golden dataset, measuring accuracy, groundedness, and safety.
- Human review brings in domain experts to assess decision quality, reasoning, and compliance.
- Red teaming stress-tests the agent with adversarial prompts to uncover vulnerabilities and safety gaps.
The stage concludes with a quality gate, a formal sign-off that the agent meets the required standards. No sign-off, no deployment.
Simple example:
The loan agent achieves 98% accuracy against the golden dataset. A compliance officer reviews a sample of 50 decisions and confirms that the reasoning meets requirements. During red-team testing, a vulnerability is discovered: the agent can be manipulated through instructions embedded within a PDF. The issue is addressed and remediated before deployment.
Microsoft helps here with:
| Tool | How It Helps |
|---|---|
| Microsoft Foundry Evaluation SDK | Runs the full golden dataset evaluation in parallel structured scores per row, side-by-side comparison between runs |
| Built-in Safety Evaluators | Out-of-the-box scoring for violence, hate, self-harm, sexual content, and indirect prompt injection |
| Built-in Quality Evaluators | Groundedness, relevance, coherence, and fluency no configuration needed |
| Agent Evaluators | TaskAdherence and ToolCallAccuracy checks the agent followed the right process, not just gave the right answer |
| Microsoft Foundry Versioned Datasets | Locks the golden dataset by version the same benchmark is used for every regression test |
Stage 4 —> Deploy: Start Small, Expand Carefully
Do not flip a switch and send all traffic to the new agent. Start in shadow mode.
Shadow mode: Agent processes requests → responses NOT shown to customers Purpose: does it behave in production like it did in test? Pilot (5%): A small slice of real customers get agent responses Watch error rates for 2 weeks Full rollout: Expand only when quality metrics stay within thresholds
Everything must be live before the first customer interaction: monitoring dashboards, alerting, human review queues, and a fallback plan if the agent needs to be pulled.
Simple example:
The loan agent goes live in shadow mode for one week. No unexpected failures. Expands to 5% of applications. Error rate stays below 0.1% for two weeks. Full rollout approved.
Microsoft helps here with:
| Tool | How It Helps |
|---|---|
| Microsoft Foundry Agent Services, Azure Kubernetes Services, Azure Container Apps | Hosts and auto-scales the agent runtime canary deployments enable the staged rollout without a full infrastructure team |
| Azure API Management | The API gateway enforces rate limits, authentication, and routing before any request reaches the agent |
| Azure Application Insights | Latency, volume, and error rate dashboards live from the first interaction |
| Azure Private Endpoints + Managed Identity | All traffic stays inside the bank's network no public endpoints, no passwords in code |
| Microsoft Foundry Deployment Management | Version-pins the model deployment enables instant rollback if the new version degrades |
Stage 5 —> Operate: Watch Everything, Always
A deployed agent is not a finished product. It is a living system. Watch five things continuously:
| What to Watch | Why |
|---|---|
| What's coming in | Are users trying to manipulate the agent? |
| How fast it responds | Is it meeting SLA? |
| Quality of outputs | Is it still giving correct answers? |
| Guardrail trigger rates | Are more things being blocked or slipping through? |
| Business outcomes | Are loan decisions still aligned with policy? |
Sample a portion of live interactions and route them to human reviewers. When a reviewer corrects the agent, that correction is a training signal — collected, annotated, and fed back into the next iteration.
Simple example:
Three weeks after launch, monitoring shows the agent's policy compliance score has dropped from 98% to 94%. Human reviewers identify that a recent policy update was not reflected in the agent's RAG knowledge base. The team is alerted before any customers are affected.
Microsoft helps here with:
| Tool | How It Helps |
|---|---|
| Microsoft Foundry Online Evaluation |
Asynchronously samples live traffic and evaluate it, providing continuous quality monitoring without impacting response latency. |
| Azure AI Content Safety (runtime) | Prompt Shield enforces guardrails on every production interaction in real time |
| Azure Monitor + KQL |
Provides dashboards and alerts across all monitoring signals, including latency, quality, guardrail compliance, and business outcomes. |
| Microsoft Foundry Tracing |
Captures every production trace, including tool interactions and execution history, providing a complete audit trail for review and compliance |
| Traces to Dataset |
Automatically converts production traces into versioned evaluation datasets, feeding seamlessly into the next optimization cycle. |
Stage 6 —> Iterate: The Agent Is Never Finished
Every signal from production triggers a loop back into the lifecycle.
| Signal | What Happens |
|---|---|
| Quality score drops | Loop back to Build and update the RAG index |
| New attack pattern detected | Loop back to Build and patch the guardrail, re-test |
| Human overrides spiking | Loop back to Design and rethink the HITL threshold |
| New regulation published | Loop back to Design and full cycle restarts |
Simple example:
APRA publishes new guidance on AI in credit decisions. The loan agent must be updated to include a new mandatory disclosure in every decision output. The team loops back to Design, specifies the new requirement, updates the agent in Build, re-tests against an updated golden dataset, and redeploys within three weeks.
Microsoft helps here with:
| Tool | How It Helps |
|---|---|
| Microsoft Foundry Fine-tuning |
Fine-tunes the model using human-reviewed annotations, enabling the agent to improve with every feedback cycle. |
| Microsoft Foundry Dataset Versioning |
Promotes newly annotated traces into the next version of the golden dataset, ensuring regression testing remains current. |
| Microsoft Foundry Experiment Tracking |
Maps evaluation outcomes to specific prompt revisions, making it easy to identify the exact change that introduced a regression. |
| Azure Monitor Alerts |
Automatically triggers when quality thresholds are breached, initiating the optimization cycle without requiring manual intervention. |
Why the Loop Is the Most Important Part
Most teams focus on stages 1–4. The loop through stage 6 is what separates agents that stay safe from agents that drift into risk over time.
Regulators do not just ask "was it safe when you launched it?" They ask, "is it safe now and can you prove it has been improving?"
The iterate loop, supported by Microsoft Foundry's continuous evaluation and monitoring capabilities, is how you answer yes.
Summary
Design what the agent can and cannot do. Build the safety controls into the system at the same time as the agent itself. Microsoft Foundry Agent Service, Azure AI Content Safety, and Azure AI Search provide the core infrastructure. Test it in three waves before any customer sees it, using Microsoft Foundry's evaluation SDK and built-in evaluators. Deploy it carefully in stages, watching every metric through Azure Monitor and Application Insights. Once live, monitor it continuously using Microsoft Foundry's online evaluation. And when something changes a policy, a regulation, a performance drift loop back to the right stage and run the cycle again. The agent is never finished. The loop is the product.