Azure News - 2026-08-20

Many discussions about Microsoft Foundry Agent Service eventually arrive at the same question: should this workload be implemented as a Prompt Agent or a Hosted Agent? While the documentation explains both options well, the architectural decision usually comes down to something much simpler: where do you want the orchestration logic to live?

First, what actually makes something an agent? 

A basic AI assistant generates an answer. An agent can also decide what to do next, call tools, access data, maintain context and complete work across multiple steps. 

At the center of most agents are three building blocks: 

• Model: provides language understanding, generation, and reasoning. 

• Instructions: define the job, boundaries, role, and expected behaviour. 

• Tools: connect the agent to knowledge and actions such as search, APIs, databases, code execution, MCP servers or business systems. 

For enterprise use, that is only the starting point. You also need identity, authorization, network controls, content safety, session management, evaluation, tracing, versioning, rollback, and cost controls. Foundry Agent Service provides the surrounding platform capabilities, while letting you choose how much runtime logic your team owns. 

Where should orchestration logic live and who should own the runtime?

Understanding the Runtime Boundary

When evaluating Foundry Agent Service, many teams focus on models. In practice, models are rarely the architectural differentiator.

Most architecture reviews eventually come down to three questions:

• Who owns orchestration?
• Who owns state?
• Who owns operations?

Foundry Agent Service provides a managed platform for these concerns, but the amount of control retained by engineering teams depends on the selected agent type.

For most teams, the architectural decision usually comes down to one of two operating models

• Prompt agents: declarative agents defined by a model, instructions, and tools, with a managed runtime. 

• Hosted agents: code-based agents that you package and run in Foundry, while the service manages the endpoint, identity, scaling, sessions, and observability. 

Prompt Agents

With a Prompt Agent, engineering teams focus primarily on defining the model, instructions, tools, knowledge sources and identity configuration, while Foundry takes responsibility for the surrounding runtime.

Why teams start here 

Prompt agents are usually the fastest route from an idea to a working, governed agent. They are a good fit when the behaviour can be expressed clearly through instructions and supported tools. 

• You need to deliver quickly. 

• The agent follows a fairly straightforward reasoning and tool-use loop. 

• Foundry-supported tools cover the required integrations. 

• You do not need custom libraries, middleware, or orchestration code. 

• You want Foundry to own compute, scaling, and patching. 

• Reviewers need an agent definition that is easy to inspect. 

Good examples 

Enterprise knowledge assistant. Employees ask about policies, engineering standards, procedures, or product information. The agent retrieves approved content and cites its sources. 

Document review assistant. The agent checks a proposal or design against an approved rubric and returns structured findings, while a human keeps responsibility for the final decision. 

Employee self-service agent. The agent answers questions and performs a small number of tightly scoped actions, such as checking request status or creating a support case. 

A useful warning sign A Prompt agent is probably becoming the wrong fit when the prompt starts looking like application code. Large branching instructions, retry logic written in prose, state-machine behaviour, custom payload handling, framework middleware or real-time media are all signs that runtime logic belongs in code instead. 

Hosted Agents

Hosted Agents move the responsibility boundary. Instead of defining behaviour through configuration alone, engineers deploy an actual application into Foundry Agent Service. Hosted Agents are framework-agnostic. Whether your team builds with Agent Framework, LangGraph, Semantic Kernel, OpenAI Agents SDK, or a custom runtime, Foundry can host the application while managing the surrounding operational services.

When Hosted agents make sense 

• You need a particular agent framework or custom orchestration engine. 

• The flow includes branching, parallel work, fan-out and fan-in, or human approvals. 

• Business rules require a deterministic state machine around model reasoning. 

• You need custom packages, middleware, algorithms, retries, caching, or error handling. 

• The client sends custom payloads or webhooks. 

• The session needs persistent files or custom state. 

• The design includes multi-agent orchestration or real-time voice. 

Good examples

A bank onboarding workflow where uploaded documents must be validated, checked against multiple systems, and routed to a human when confidence drops below a threshold.

A fraud investigation agent that gathers transaction history, enriches data from multiple internal systems, applies bank-specific risk rules, requests additional evidence when required, and generates a recommended outcome for an investigator. The process involves long-running workflows, branching logic and audit requirements that are better suited to code-based orchestration.

A lending workflow that coordinates document collection, credit bureau checks, income verification, affordability assessments, policy exceptions, and approval routing. The process spans multiple systems and often requires deterministic decision paths that extend beyond prompt-driven orchestration.

A security operations agent that aggregates alerts from SIEM platforms, enriches incidents with threat intelligence, executes automated containment actions, opens tickets, requests approvals for high-impact remediation steps, and maintains a complete audit trail of decisions and actions.

The trade-off 

More control also means more ownership. Your team must secure and patch the code and dependencies, test the runtime, manage supply-chain risk, and think about compute sizing, cold starts, session lifecycle, and cost. Hosted agents reduce platform plumbing, but they do not remove application engineering. 

Choosing Prompt Agent/Hosted Agents 

A pattern I see quite often is teams arriving at the solution before they've fully articulated the requirement. The conversation usually starts with "We need a Hosted Agent," but after digging into the workload, the real requirements turn out to be things like persistent state, webhook processing, custom orchestration, background execution, framework-specific capabilities, or human approval workflows.

These are runtime concerns, not agent concerns and they're usually the factors that determine whether a Hosted Agent is necessary. Hosted Agents are valuable because they give engineering teams control over those aspects of execution while still offloading much of the operational infrastructure to Foundry.

This is also where teams most commonly choose the wrong agent type. A frequent assumption is that existing investments in frameworks such as LangGraph or Semantic Kernel automatically imply a Hosted Agent architecture. In practice, many of these workloads are relatively simple orchestration scenarios that can be implemented effectively as Prompt Agents, with lower operational overhead and less infrastructure to manage.

My advice is usually to start by identifying the runtime requirements rather than selecting an agent type. Once those requirements are clear, the right architecture often becomes obvious.

What I would choose today

Hosted agents in Foundry Agent Service - Microsoft Foundry | Microsoft Learn