netaiguidance

Practical guidance for choosing and composing the .NET AI stack.

This repo complements the .NET AI scenarios by focusing less on "what exists" and more on when developers should use each component, how the pieces fit together, and where the boundaries are.

See decision-flow.md for a Mermaid decision flowchart covering the current guidance.

Working definitions

• AI feature — a discrete AI capability such as chat, summarization, extraction, classification, or generation.
• Agent — a software component that uses an AI model to pursue a goal over multiple steps, choosing actions based on context and previous results.
• Tool — a callable capability the model or agent can use, such as an API, database query, search operation, or business action.
• MCP server — a process that exposes tools, resources, or prompts over the Model Context Protocol for other AI clients to use.
• MCP client — an application or agent that connects to MCP servers and consumes those exposed capabilities.
• Platform — the surrounding infrastructure for model hosting, orchestration, telemetry, safety, governance, and deployment.

Positioning draft

Use this as the starting point for iterating on “when should a dev use this?”

Component	Use it when...	Do not lead with it when...	Common companions
MEAI	You need to add AI behavior to an app: chat, summarization, structured outputs, tool calling, embeddings, or streaming.	The hardest problem is ingestion, retrieval, orchestration, or infrastructure.	MEDI, MEVD, Evaluations, MCP Client, MAF, Foundry
Evaluations	You need repeatable quality checks for prompts, models, agents, or AI features, including regressions and side-by-side comparisons.	You have not yet built the AI behavior you want to measure.	MEAI, AI Toolkit, Azure AI Foundry
MEDI	Your challenge is getting source content into shape for AI: reading, chunking, enriching, and preparing data for grounding or RAG.	You only need a direct model call and no serious data preparation pipeline.	MEAI, MEVD
MEVD	You need semantic search, retrieval, embeddings-backed lookup, or RAG over your own data.	You do not need vector retrieval or grounding.	MEAI, MEDI
MCP Server	You want to expose tools, resources, or prompts so external assistants, IDEs, or agents can discover and use them.	The capability is local to one app and ordinary in-process function calling is enough.	MCP Client, MEAI, MAF, Copilot SDK
MCP Client	Your app or agent needs to connect to existing MCP servers and consume external tools or resources.	You do not need interoperability beyond your own process boundary.	MEAI, MCP Server, MAF, Copilot SDK
MAF	The system must pursue goals across multiple steps with routing, planning, handoffs, or multiple collaborating agents.	A single LLM call or straightforward tool-calling loop is sufficient.	MEAI, MCP Client, MCP Server, Aspire
AI Toolkit	You want a better developer workflow for trying models, prompts, and evaluations during development.	You need the runtime abstraction or production architecture itself.	Foundry Local, MEAI, Evaluations
Copilot SDK	You want a pre-built agent harness with tools, context, and automatic tool calling out of the box.	You want a blank-slate app stack or full low-level control from MEAI/MAF.	MCP Client, MCP Server, MEAI
Azure AI Foundry	You need managed model hosting, safety, governance, enterprise controls, and a cloud deployment target.	You are only deciding how app code should call a model.	MEAI, Aspire
Foundry Local	You need local or local-first AI for privacy/compliance reasons, or you want some alignment with Azure AI Foundry without assuming a perfect 1:1 transition.	Any local runtime will do, or you expect local-to-cloud to be a drop-in identical move.	MEAI, AI Toolkit
Aspire	The solution spans multiple services and you need orchestration, service discovery, and observability.	The app is still a single service or proof of concept.	MAF, MEAI, Foundry

Suggested walkthrough order

If we review these one at a time, this is a sensible sequence:

1. MEAI — the default starting point
2. Evaluations — measuring quality and regressions
3. MEDI — preparing data for AI
4. MEVD — retrieval and vector search
5. MCP Server — exposing tools and capabilities
6. MCP Client — consuming external MCP capabilities
7. MAF — true agent orchestration
8. Azure AI Foundry — enterprise hosting and governance
9. Foundry Local — local-first and privacy-sensitive execution
10. Aspire — multi-service orchestration and telemetry
11. AI Toolkit — developer experimentation workflow
12. Copilot SDK — pre-built runtime for code-aware assistants

Tech-by-tech review cards

These are the short positioning statements we can refine one by one:

• MEAI — start here when you want to add AI behavior directly into a .NET app.
• Evaluations — add this when the team needs to measure quality, safety, and regressions instead of relying on intuition.
• MEDI — use this when the hard part is preparing real-world content for grounding or RAG.
• MEVD — use this when the app needs semantic retrieval and vector-backed access to its own data.
• MCP Server — use this when your system should publish tools, resources, or prompts for other AI clients.
• MCP Client — use this when your app or agent should consume tools and resources from external MCP servers.
• MAF — use this when the system must pursue a goal over multiple steps with orchestration, routing, or handoffs.
• Azure AI Foundry — use this when enterprise hosting, governance, and managed cloud controls are the deciding factors.
• Foundry Local — use this when local execution matters for privacy/compliance, with some alignment to Azure AI Foundry but not a perfect 1:1 path.
• Aspire — use this when the AI solution is really a distributed .NET system that needs orchestration and observability.
• AI Toolkit — use this when developers need a faster experimentation loop for models, prompts, and evaluations.
• Copilot SDK — use this when you want an agent harness with tools, context, and auto tool calling out of the box.

Component notes

MEAI

Definition: MEAI (Microsoft.Extensions.AI) is the app-facing foundation for adding model-powered behavior to a .NET application.

Use it when a developer wants to add AI directly into normal application code:

• chat or conversational UX
• streaming responses
• summarization, extraction, or classification
• structured outputs
• embeddings
• tool/function calling

Why it matters:

• it gives .NET developers a clean abstraction for model interaction
• it fits naturally into DI, configuration, and existing app architectures
• it is usually the first layer of an AI-enabled .NET application

Important boundary:

• MEAI is not automatically an agent framework
• a one-shot call, chat feature, or tool call loop can be built with MEAI without becoming "agentic"
• if the system needs goal-directed, multi-step orchestration, that is where MAF starts to matter

When not to lead with MEAI alone:

• if the hard problem is preparing enterprise content for AI, add MEDI
• if the hard problem is retrieval / RAG, add MEVD
• if tools must be discoverable across assistants or apps, add MCP Server
• if the app needs to consume external MCP capabilities, add MCP Client
• if the challenge is multi-step autonomous workflows, add MAF
• if the main concern is hosting, governance, and enterprise controls, add Azure AI Foundry

Rule of thumb: start with MEAI unless the real difficulty is ingestion, retrieval, orchestration, or platform operations.

Evaluations

Definition: Evaluations (Microsoft.Extensions.AI.Evaluations.*) is the quality and regression layer for AI features built with the .NET AI stack.

Use it when a team needs to answer questions like:

• "Is this prompt change actually better?"
• "Did switching models hurt quality or safety?"
• "Did the agent regress on key scenarios?"
• "Can we measure behavior before shipping?"

Why it matters:

• AI behavior changes easily as prompts, models, and tools evolve
• subjective "looks good to me" testing does not scale
• evaluations give teams a repeatable way to compare outputs and catch regressions

Important boundary:

• evaluations are not the runtime feature itself
• they support decision-making, quality gates, and iteration
• they are most valuable once a team already has a feature, workflow, or agent worth measuring

Rule of thumb: if the app already has AI behavior and you are changing prompts, models, or orchestration, add Evaluations before relying on intuition alone.

MEDI

Definition: MEDI (Microsoft.Extensions.DataIngestion) is the ingestion and preparation layer for AI-ready data in .NET.

Use it when the hard part is not the model call itself, but getting source material into a form the AI system can actually use:

• reading content from files, stores, or enterprise sources
• chunking documents for retrieval and grounding
• normalizing and enriching content with metadata
• preparing data to feed a vector index or downstream RAG pipeline

Why it matters:

• many AI apps fail before retrieval because the data is messy, oversized, or poorly structured
• ingestion quality strongly affects downstream answer quality
• it gives teams a composable pipeline for turning raw content into useful AI context

Important boundary:

• MEDI is not the retrieval layer
• it does not replace MEVD
• it comes before retrieval when the job is to prepare and shape the data that MEVD or another store will later query

When to reach for it:

• enterprise documents need to be processed before indexing
• you are building grounding or RAG over real-world content
• data preparation, chunking, or enrichment logic is starting to sprawl

When not to lead with it:

• if the app just needs chat, extraction, summarization, or tool calling over immediate input, start with MEAI
• if your content is already prepared and the need is semantic lookup, lead with MEVD

Rule of thumb: use MEDI when your biggest AI problem is data readiness, not model access.

MEVD

Definition: MEVD (Microsoft.Extensions.VectorData) is the vector data storage and retrieval layer for semantic search, similarity lookup, and grounding in .NET AI apps.

Use it when the app needs to retrieve the most relevant information from its own data, especially through:

• semantic search
• embeddings-backed retrieval
• grounding for RAG
• similarity search or hybrid retrieval patterns

Why it matters:

• it gives .NET applications a consistent way to work with vector stores
• it helps separate vector storage and retrieval concerns from model invocation concerns
• it is often the point where an app moves from "AI over prompt input" to "AI over enterprise knowledge"

Important boundary:

• MEVD is not the model layer
• MEVD is not the ingestion layer
• MEDI prepares the data, MEVD stores and retrieves the data, and MEAI uses that retrieved context with the model

When not to lead with it:

• if a plain AI feature is enough and no retrieval is involved, start with MEAI
• if the hard problem is preparing raw documents and content, start with MEDI

Rule of thumb: reach for MEVD when the hard problem is not "how do I call an LLM?" but "how do I find the right context from my data before I call it?"

MCP Server

Definition: an MCP Server exposes capabilities — tools, resources, or prompts — over the Model Context Protocol so other assistants, IDEs, and agents can discover and use them through a standard protocol.

Use it when:

• you want to make internal APIs or business actions available to multiple AI clients
• you want interoperability instead of custom one-off integrations
• the same capability should be reusable across tools, products, or agent systems

Why it matters:

• it turns app capabilities into reusable AI-facing endpoints
• it reduces duplicated tool integration work across assistants
• it creates a cleaner boundary between capability providers and capability consumers

Important boundary:

• an MCP Server is about publishing capabilities
• if the capability only needs to be used inside one app, ordinary in-process function calling may be simpler

Rule of thumb: build an MCP Server when the goal is to make capabilities reusable across AI clients, not just callable inside one application.

MCP Client

Definition: an MCP Client is the consumer side of the protocol: it connects to MCP servers and brings their exposed capabilities into an app, assistant, or agent runtime.

Use it when:

• your app needs to call tools that are already exposed elsewhere through MCP
• you want an agent or assistant to consume capabilities without custom per-tool plumbing
• you are composing with an external ecosystem of AI-facing tools

Why it matters:

• it lets apps and agents integrate with external capabilities in a standard way
• it keeps interoperability from becoming a pile of bespoke adapters

Important boundary:

• an MCP Client is about consuming capabilities, not publishing them
• if everything the app needs is local and in-process, ordinary function/tool calling may still be simpler

Rule of thumb: add an MCP Client when the value comes from connecting to external MCP-enabled capabilities, not when everything already lives inside your own process.

MAF

Definition: MAF (Microsoft Agent Framework) is the orchestration layer for systems that are truly agentic: they pursue a goal across multiple steps, make decisions along the way, use tools, and may coordinate multiple agents.

Use it when the system needs:

• planning or stepwise execution
• routing across tools or specialist agents
• handoffs between agents or humans
• stateful, multi-step workflows that adapt as results come back

Why it matters:

• this is where AI behavior moves beyond simple prompts and into orchestrated work
• it gives structure to systems that would otherwise become fragile tool-calling loops

Important boundary:

• not every AI feature needs MAF
• if a direct MEAI call or a simple tool-calling loop solves the problem, stay simpler
• MAF matters when orchestration complexity is the real challenge, not just model access

Rule of thumb: reach for MAF when the system must keep working toward a goal over multiple steps, not just generate one good answer.

Azure AI Foundry

Definition: Azure AI Foundry is the managed cloud platform layer for enterprise AI solutions, with two primary functions: model management and hosted agents.

Use it when priorities include:

• managed model hosting
• hosted agent capabilities
• persistent memory for hosted agents
• built-in tools
• code-free RAG through file upload / grounding services
• out-of-the-box evaluations, comparisons, and safety checks
• safety, filtering, and governance controls
• enterprise compliance and operational consistency
• centralized access to models and cloud deployment infrastructure

Why it matters:

• many teams need more than "call a model" — they need enterprise controls, auditability, and managed infrastructure
• hosted agents especially shine when teams want persistent memory, built-in tools, and RAG-like grounding without building the full ingestion pipeline themselves
• Foundry also provides useful out-of-the-box eval value, including built-in evaluators, safety-oriented checks, and comparison workflows for models and agents
• it is often the bridge between a local prototype and a production cloud deployment

Important boundary:

• Foundry is not the app-facing programming abstraction; MEAI still plays that role in .NET code
• Foundry becomes the right lead when the real question is where and under what controls the model runs
• for code-first, app-level evaluation workflows in .NET, Microsoft.Extensions.AI.Evaluations.* is still the more direct developer-facing layer

Rule of thumb: lead with Azure AI Foundry when the hardest problem is enterprise-grade hosting and governance, not just model invocation from app code.

Foundry Local

Definition: Foundry Local is a local development and local-first deployment option for teams that need to keep AI workloads close to the machine or environment, often for privacy, compliance, or network-bound reasons.

Use it when:

• local development needs better production parity
• the organization is local-first or all-local because of privacy, compliance, or data residency requirements
• you want the local experience to align somewhat with Azure AI Foundry
• teams need to experiment locally without sending sensitive data to the cloud

Why it matters:

• local dev environments shape how quickly teams can iterate
• many shops stay local-first because sensitive data cannot leave controlled environments
• alignment can reduce some friction when a cloud path exists, even if the experience is not identical

Important boundary:

• Foundry Local is about the development and deployment path, not the higher-level app architecture itself
• local -> cloud is not a clean 1:1 move; expect differences in features, hosting model, and operations
• its value is local execution with some alignment, not perfect equivalence

Rule of thumb: choose Foundry Local when local execution is required for privacy or compliance, or when you want a local-first path that can align with Azure AI Foundry without assuming exact portability.

Aspire

Definition: Aspire is the orchestration, service-wiring, and observability layer for distributed .NET applications, including AI systems that span multiple services.

Use it when the solution includes things like:

• separate API, agent, retrieval, and model-facing services
• service discovery and distributed configuration
• tracing, telemetry, and end-to-end visibility across the system

Why it matters:

• AI systems often stop being "just one app" once retrieval, tools, gateways, and worker services are involved
• Aspire helps teams keep those parts understandable and observable
• its visuals make it easier to trace AI flows across services and understand what happened end-to-end

Important boundary:

• Aspire is not specifically the AI runtime; it is the multi-service application layer around it
• it does not replace MEAI, MAF, or Azure AI Foundry

Rule of thumb: add Aspire when the AI application stops being just an app and starts behaving like a distributed platform.

AI Toolkit

Definition: AI Toolkit is a VS Code extension pack for AI development that speeds up experimentation with models, prompts, agents, and evaluations. It includes Microsoft Foundry integration directly in the editor experience.

Use it when teams want:

• a model catalog and playground inside VS Code
• a faster loop for testing prompts and models
• a friendlier workflow for experimenting during development
• agent builder / agent inspector workflows for creating, debugging, and visualizing agents
• support for iterative evaluation alongside implementation
• bulk runs, tracing, or fine-tuning as part of the development loop
• skills that make Copilot more effective in AI-oriented development contexts

Why it matters:

• developer productivity matters when prompts, models, and evaluation criteria change frequently
• it helps teams discover what works before hardening the architecture
• it can improve Copilot's usefulness in AI workflows by bringing in AI-specific skills and context
• it brings together practical tooling such as model catalog, playground, agent builder, agent inspector, evaluation, and tracing in one place

Important boundary:

• AI Toolkit is not the core runtime architecture for the production app
• it complements MEAI, Evaluations, and Foundry Local

Rule of thumb: reach for AI Toolkit when the bottleneck is experimentation speed and developer workflow, not runtime composition.

Copilot SDK

Definition: Copilot SDK is a pre-built agent harness and runtime that brings tools, context, and automatic tool calling out of the box.

Use it when:

• you want more built-in runtime behavior than a blank-slate MEAI app would provide
• you want tools and context to be wired in quickly
• automatic tool calling and agent-harness behavior are more valuable than assembling everything manually
• you want a faster path to a working assistant or agent runtime

Why it matters:

• some assistant and agent scenarios benefit from a purpose-built harness instead of assembling every layer manually
• it provides useful built-in behavior out of the box so teams can focus faster on the experience they are building

Important boundary:

• it is more opinionated and pre-wired than MEAI
• if the goal is a fully custom app architecture, direct MEAI or MAF composition may be a better fit

Rule of thumb: use Copilot SDK when you want a ready-made agent harness with built-in tools, context, and auto tool calling, rather than composing those pieces yourself from scratch.