Skip to content

feat(agents): support declarative prompt-voice agents (managed model) - #9364

Open
v1212 wants to merge 22 commits into
Azure:mainfrom
v1212:users/wujia/prompt-voice-agent-managed-model
Open

feat(agents): support declarative prompt-voice agents (managed model)#9364
v1212 wants to merge 22 commits into
Azure:mainfrom
v1212:users/wujia/prompt-voice-agent-managed-model

Conversation

@v1212

@v1212 v1212 commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Summary

Closes #9336

Adds azd support for a new declarative prompt-voice agent kind that creates a managed speech-to-speech (voice) agent on Azure AI Foundry, end-to-end through azd initazure.yamlazd deploy.

Scope (intentionally narrow): prompt-voice + managed model only — i.e. scaffold (init) and deploy only. BYOM, hosted-voice, tools, avatar, and cascaded models are deliberately left as follow-up PRs. azd-native text invoke, list, and a Portal Playground link are out of scope for this PR (see Follow-ups).

What's included

  • yaml (agent_yaml/yaml.go, parse.go): new prompt-voice authoring kind + VoiceAgent struct/parsing/validation.
  • map (agent_yaml/map.go): translate authoring kind prompt-voice → data-plane service kind voice; default the audio pipeline (PCM16 @ 24 kHz, server_vad, whisper-1, DragonHD default voice); v1 = implicit model_type: managed.
  • agent_api (agent_api/models.go, operations.go): VoiceAgentDefinition wire structs + CreateVoiceAgent with the required preview header Foundry-Features: VoiceAgents=V1Preview.
  • project (project/agent_definition.go, service_target_agent.go): voice-aware definition read/write + an isolated deployVoiceAgent deploy path — the existing hosted/container path is byte-for-byte unchanged.
  • init (cmd/init.go, init_from_templates_helpers.go): --kind / --model / --voice flags, voice manifest synthesis, and a new interactive prompt option.
  • schema + tests: azure.ai.agent.json gains the prompt-voice kind + voice service properties; unit tests cover the map translation, voice-type selection, and manifest parse/validation.

Compatibility

All changes are additive — new case branches with untouched defaults and omitempty fields. Existing azd init / invoke / deploy / list flows and the hosted/workflow code paths are unchanged.

Testing

Automated

  • go build ./..., go vet ./..., and cspell clean.
  • New unit tests (map_voice_test.go, parse_voice_test.go, init_test.go, nextstep/state_test.go, voice_deploy_dispatch_test.go) plus the existing suite pass.

Live validation

  • Against a real Foundry project: an agent created via the real CreateVoiceAgent code appears in the project list and its stored definition is byte-identical to the service's own managed reference agent (kind:voice / model_type:managed / audio pipeline / voice config). Connect reaches session.created.

How to test end-to-end

Build the branch binary and install the branch build of the azure.ai.agents extension first:

cd cli/azd && go build -o azd-branch.exe ./     # then: AZD=./azd-branch.exe
# install the branch-built azure.ai.agents extension from local source (overrides the released build)

1) Interactive (recommended)

mkdir voice-test && cd voice-test
"$AZD" ai agent init      # choose "Create a prompt voice agent", then follow the standard init prompts
"$AZD" provision          # standard azd provision (subscription / location / resource group / Foundry project)
"$AZD" deploy             # POST /voice_agents -> stderr "Voice agent '<name>' created successfully!"
"$AZD" env get-values | grep AGENT_    # expect AGENT_<KEY>_NAME and AGENT_<KEY>_ENDPOINT
  • The interactive azd ai agent init follows the same prompt flow as other agent kinds (agent name, then the standard Foundry-project selection). --model (default gpt-realtime) and --voice are flags, not prompts — pass them on the same command to override, e.g. azd ai agent init --model gpt-realtime --voice alloy.
  • azd provision and azd deploy add no voice-specific interaction — they behave exactly like the existing hosted/code experience.
  • No on-disk infra/ is required. When infra/ is absent, the Foundry provider synthesizes the embedded ARM template in-memory and resolves the required parameters via the interactive prompts. --infra is optional (only needed to eject IaC to disk).
  • Resulting azure.yaml service: host: azure.ai.agent, kind: prompt-voice, modelType: managed, model: { id: gpt-realtime } (a voice: field appears only when --voice was passed).

2) Non-interactive (CI / scripted)

--no-prompt cannot prompt for the three required provision parameters, so eject IaC once and set them explicitly:

export AZURE_SUBSCRIPTION_ID=<sub> AZURE_LOCATION=eastus2 AZURE_TENANT_ID=<tenant>

"$AZD" init --minimal --environment voice-test --no-prompt
"$AZD" ai agent init --kind prompt-voice --agent-name my-voice-agent \
       --model gpt-realtime --voice alloy --no-prompt
"$AZD" ai agent init --infra --no-prompt                              # eject embedded Bicep to ./infra
"$AZD" env config set infra.parameters.location eastus2
"$AZD" env config set infra.parameters.resourceGroupName rg-voice-test
"$AZD" env config set infra.parameters.foundryProjectName fp-voice-test
"$AZD" provision --no-prompt
"$AZD" deploy --no-prompt
"$AZD" env get-values | grep AGENT_

3) Runtime check (client)

azd-native text invoke is out of scope for this PR (managed voice agents are Voice Live realtime over WebSocket; the /voice_agents data plane has no HTTP text-invoke endpoint). Validate runtime instead with a voice client connecting to the deployed agent — a session reaches session.created and supports voice/text turns. A read-only control-plane GET /voice_agents/<name> (with header Foundry-Features: VoiceAgents=V1Preview) is a quick sanity check: expect state=enabled, definition.model_type=managed.

Known gaps

  • End-to-end voice session runtime still surfaces errors that reproduce equally on the service's own reference managed agent (e.g. session.audio.output.voice string coercion at the Voice Live layer, and a demo-client session.update framing issue) — being investigated; not specific to this change.

Follow-ups

  • BYOM (self_deployed) model type
  • hosted-voice, tools, avatar, cascaded models
  • azd-native text invoke, list, and Portal Playground link for voice agents
  • azd ai agent delete for voice services — the current delete path targets /agents/<name> and returns CodeAgentNotFound (404) for a voice agent, which lives under /voice_agents/<name>. Voice-aware teardown (and an idempotent redeploy/update path over the create-only /voice_agents route) is a follow-up.
  • Remove AZURE_VOICE_OVERRIDDEN_HOST / x-ms-overridden-host once the public Foundry APIM voice route is generally rolled out; will be filed as a tracking issue.

Add azd support for a new 'prompt-voice' agent kind that creates a
managed speech-to-speech (voice) agent on Azure AI Foundry.

- yaml: new prompt-voice kind + VoiceAgent authoring struct/parsing
- map: translate authoring kind prompt-voice -> data-plane kind voice,
  defaulting the audio pipeline (PCM16@24k, server_vad, whisper-1,
  DragonHD default voice) and v1 implicit managed model_type
- agent_api: VoiceAgentDefinition wire structs + CreateVoiceAgent with
  Foundry-Features: VoiceAgents=V1Preview preview header
- project: voice-aware agent_definition read/write + isolated
  deployVoiceAgent deploy path (container path unchanged)
- init: --kind/--voice flags, voice manifest synthesis, prompt option

Scope: prompt-voice + managed model only. BYOM, hosted-voice, tools,
avatar, and cascaded models are follow-ups. Draft: needs further
end-to-end session testing and optimization.
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).
21 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

@github-actions github-actions Bot added the ext-agents azure.ai.agents extension label Jul 30, 2026
- map_voice_test.go: cover CreateVoiceAgentAPIRequest defaults/overrides,
  managed enforcement, BYOM rejection, missing-model error, and the
  isOpenAIVoice/buildVoiceConfig voice-type selection
- parse_voice_test.go: cover prompt-voice manifest parsing and
  ValidateAgentDefinition (ok / missing model.id / self_deployed rejected)
- azure.ai.agent.json: add prompt-voice to the kind enum and document the
  voice service properties (modelType/model/instructions/voice/store)
- cspell.yaml: allow BYOM
@v1212
v1212 requested a balanced review from Copilot July 30, 2026 09:16
The azure.ai.projects synthesis copy must stay byte-identical to the
azure.ai.agents copy (TestAgentsSynthesisCopyMatches). Mirror the
prompt-voice comment update made in the agents synthesizer.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds managed declarative prompt-voice agents across initialization, configuration, API mapping, and deployment.

Changes:

  • Adds voice-agent YAML/schema models and validation.
  • Maps voice manifests to Foundry’s preview API contract.
  • Adds voice-specific initialization and deployment paths with tests.

Reviewed changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
schemas/azure.ai.agent.json Adds voice-agent schema fields.
internal/synthesis/synthesizer.go Marks voice agents as non-container.
internal/project/service_target_agent.go Adds voice deployment handling.
internal/project/agent_definition.go Adds inline voice configuration conversion.
internal/pkg/agents/agent_yaml/yaml.go Defines voice authoring models.
internal/pkg/agents/agent_yaml/parse.go Parses and validates voice manifests.
internal/pkg/agents/agent_yaml/parse_voice_test.go Tests voice parsing and validation.
internal/pkg/agents/agent_yaml/map.go Maps voice manifests to API requests.
internal/pkg/agents/agent_yaml/map_voice_test.go Tests voice request mapping.
internal/pkg/agents/agent_api/operations.go Adds the preview create operation.
internal/pkg/agents/agent_api/models.go Defines voice API wire models.
internal/cmd/init.go Adds voice initialization flags and flow.
internal/cmd/init_from_templates_helpers.go Adds the interactive voice option.
cspell.yaml Adds voice-related terminology.

Comment thread cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go Outdated
Comment thread cli/azd/extensions/azure.ai.agents/internal/cmd/init.go
Comment thread cli/azd/extensions/azure.ai.agents/internal/cmd/init.go Outdated
@github-actions github-actions Bot added the ext-projects azure.ai.projects extension label Jul 30, 2026
Resolve four correctness gaps in the declarative prompt-voice flow surfaced
in review, keeping every existing hosted/container/workflow path unchanged:

- init: validate --kind (and its --image incompatibility) before either the
  image or prompt-voice synthesis fast path, so `--kind prompt-voice --image`
  is rejected instead of silently creating a hosted image agent.
- init: skipACR now also covers prompt-voice (managed, no container), while a
  new isHostedAgent decision drives hosted-region filtering. selectFoundryProject
  gains a distinct filterHostedRegions parameter so a voice agent skips ACR
  without being constrained to hosted-agent regions.
- deploy: resolve an explicit AGENT_DEFINITION_PATH override before the
  voice/container dispatch (resolveVoiceAgentForDeploy), so an override wins for
  voice just as it does for the container path.
- deploy contract: make Endpoints() and next-step isDeployed voice-aware. Voice
  agents record only NAME + base ENDPOINT (no agent-version / per-protocol
  endpoints), so both consumers now treat the base endpoint as the deployment
  marker instead of reporting a created voice agent as undeployed.

Adds unit tests for the skipACR/isHostedAgent split, the override-precedence
dispatch, and the voice deployed-marker fallback.
@v1212
v1212 marked this pull request as ready for review August 3, 2026 09:13
Copilot AI review requested due to automatic review settings August 3, 2026 09:13
@v1212
v1212 requested review from huimiu and hund030 as code owners August 3, 2026 09:13
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 2 pipeline(s).
20 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Note

This error may be related to your runner configuration. You can now configure runners for Copilot code review separately from Copilot cloud agent by creating a copilot-code-review.yml file with your setup steps. Read the docs for details.

@jongio jongio left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ran through the voice path end to end against the current head. Build is clean and the new tests pass locally. Three things I'd like to sort out before this ships, plus two nits.

Medium

  1. The new deployed-agent detection keys off "VERSION empty and ENDPOINT set" rather than the service kind, in both Endpoints() and nextstep.isDeployed(). Inline comments on both.
  2. Voice create is an unconditional POST to /voice_agents with no version model. What's the intended behavior for a second azd deploy? Inline comment on operations.go.
  3. Delete has no voice path. AgentClient.DeleteAgent builds %s/agents/%s, and DeleteAction.cleanupEnvVars clears the three AGENT_<KEY>_* vars regardless. So azd ai agent delete against a prompt-voice service hits /agents/<name>, gets a 404, and classifyDeleteError turns that into CodeAgentNotFound. The user is told the agent doesn't exist while it's still live in the project, and there's no way to tear it down from azd. If delete is intentionally out of scope for this PR, could you add it to the follow-ups list in the description so it doesn't get lost?

Low

  1. Orphaned comment fragment in init.go. Inline comment.
  2. isOpenAIVoice classifies by name shape. Inline comment.

One question on AZURE_VOICE_OVERRIDDEN_HOST: the comment says it exists to bypass the public Foundry APIM while the voice route rolls out. Worth filing an issue to remove it once the route is live, otherwise it tends to stick around forever.

Also, the description still says Draft but the PR is open and review is requested. Probably just needs updating.

Comment thread cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go Outdated
Comment thread cli/azd/extensions/azure.ai.agents/internal/cmd/nextstep/state.go
Comment thread cli/azd/extensions/azure.ai.agents/internal/cmd/init.go Outdated
- Endpoints()/isDeployed(): gate voice base-endpoint fallback on the
  service's actual prompt-voice kind instead of the env-var shape, so a
  partially-failed hosted deploy still surfaces CodeMissingAgentEnvVars
- add nextstep isVoiceService helper (mirrors project kind gate; the two
  stay in separate packages to avoid a project->nextstep import cycle)
- CreateVoiceAgent: document create-only redeploy semantics
- isOpenAIVoice: classify via explicit OpenAI voice set + Azure Neural
  locale-prefix pattern instead of a bare '-' check
- init.go: drop orphaned comment fragment
- tests: cover hosted lingering-endpoint gate and voice name classification
Copilot AI review requested due to automatic review settings August 3, 2026 12:43
@v1212

v1212 commented Aug 3, 2026

Copy link
Copy Markdown
Contributor Author

@jongio thanks for the review — pushed 438466c addressing the inline threads (replied on each; left them unresolved for you to confirm). For the remaining points from the review body:

  • azd ai agent delete on a voice service (404 CodeAgentNotFound because it targets /agents/<name> not /voice_agents/<name>): out of scope for this init+deploy PR — added a Follow-ups entry for voice-aware teardown (plus the idempotent redeploy/update path over the create-only /voice_agents route).
  • Nit: > Draft text — removed from the description (PR is out of draft).
  • Nit: AZURE_VOICE_OVERRIDDEN_HOST / x-ms-overridden-host cleanup — added a Follow-ups entry; will file a tracking issue to remove it once the public Foundry APIM voice route is rolled out.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 22 out of 22 changed files in this pull request and generated 1 comment.

Suppressed comments (8)

cli/azd/extensions/azure.ai.agents/internal/cmd/init_foundry_project_setup.go:148

  • [azd-code-reviewer] The interactive existing-project branch has the same coupling: prompt-voice sets skipACR=true, which now turns on hosted-region filtering and hides otherwise valid voice projects. Use the separate hosted-agent decision here as well.
				skipACR, // filterHostedRegions: this path is code/container only (non-voice)

cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go:440

  • [azd-code-reviewer] This voice check ignores AGENT_DEFINITION_PATH, although deployment now lets that override determine voice/container dispatch. A voice override on a hosted service deploys successfully but endpoint discovery still treats it as hosted and fails on the missing VERSION; next-step state has the same mismatch. Persist or resolve a common effective-kind marker in all three consumers.
	if _, isVoice, err := VoiceAgentFromResolvedService(serviceConfig, p.projectPath); err != nil {

cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:1646

  • [azd-code-reviewer] This resolved name is not pinned to flags.agentName. runInitFromManifest later reaches downloadAgentYaml, which calls resolveInitAgentName again, so the interactive voice flow asks for the agent name twice. Pin the result before entering the manifest flow and add a regression test.
					resolvedName, err := resolveInitAgentName(ctx, azdClient, flags, "voice-agent")

cli/azd/extensions/azure.ai.agents/internal/cmd/init_foundry_project_setup.go:74

  • [azd-code-reviewer] This still equates skipping ACR with requiring hosted-agent regions. Prompt-voice reaches configureFoundryProject with skipACR=true, so a voice init using --project-id incorrectly filters out projects in regions that do not support hosted agents. Pass a separate filterHostedRegions decision through this helper.

This issue also appears on line 148 of the same file.

			skipACR, // filterHostedRegions: this path is code/container only (non-voice)

cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:1288

  • [azd-code-reviewer] The prompt-voice fast path silently ignores --model-deployment, even though the shared flag help says it takes precedence over --model. Because this PR supports managed models only, reject this combination instead of creating the default gpt-realtime agent.
				if flags.image != "" {

cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:2216

  • [azd-code-reviewer] Recording isVoiceAgent does not preserve the documented two-question scaffold flow. A synthesized voice manifest has no model resources, so this function continues into configureFoundryProject, whose interactive branch prompts for project choice, subscription, and location during init rather than deferring them to azd provision as the PR description states.
	if kind, err := agentManifestKind(agentManifest); err == nil {
		a.isVoiceAgent = kind == agent_yaml.AgentKindPromptVoice

cli/azd/extensions/azure.ai.agents/internal/pkg/agents/agent_api/operations.go:188

  • [azd-code-reviewer] The linked REST contract does not have the create-only behavior documented here: POST /voice_agents creates an agent or a new version of an existing one, and POST /voice_agents/{agent_name} provides update semantics. Treating every second deploy as an unavoidable failure makes azd deploy non-repeatable despite API support; implement the existing-agent path rather than deferring it.
// Redeploy semantics: the voice data-plane exposes create-only POST /voice_agents
// with no version/upsert model (unlike hosted agents, which mint a new
// agent-version per deploy). A second `azd deploy` of the same voice service
// therefore re-POSTs with the same name and the service rejects it with a
// non-success status, which this method surfaces as a deploy error rather than
// silently overwriting the existing agent. Idempotent redeploy/update is tracked
// as a follow-up (see the PR "Follow-ups" section); until the service adds an
// update route, redeploy requires deleting the existing voice agent first.

cli/azd/extensions/azure.ai.agents/internal/pkg/agents/agent_api/operations.go:212

  • [azd-code-reviewer] CreateVoiceAgent has no operation-level test, leaving the required preview header, optional overridden-host header, route, and accepted response statuses unverified. Add a fake-transport test alongside operations_test.go that asserts these request details and response parsing.
	// Voice agents are a preview feature; the service rejects the request with
	// 403 preview_feature_required unless this opt-in header is present.
	req.Raw().Header.Set("Foundry-Features", voiceAgentsPreviewFeature)

	if overriddenHost != "" {
		req.Raw().Header.Set("x-ms-overridden-host", overriddenHost)

Comment thread cli/azd/extensions/azure.ai.agents/internal/pkg/agents/agent_yaml/map.go Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 27 out of 27 changed files in this pull request and generated no new comments.

Suppressed comments (5)

cli/azd/extensions/azure.ai.agents/internal/project/agent_definition.go:848

  • [azd-code-reviewer] The shared kind lookup can identify a voice agent from the service directory's agent.yaml, but this fallback only parses inline/config properties. For the on-disk shape covered by TestKind_ManifestFallback and the new endpoint test, IsPromptVoice returns true here, then VoiceAgentFromResolvedService returns found=false, and deploy falls through to the container path instead of deploying the voice definition. Parse the same on-disk manifest source when the service entry has no definition.
		)

cli/azd/extensions/azure.ai.agents/internal/project/agent_definition.go:954

  • [azd-code-reviewer] Inline voice definitions bypass validateAgentServiceDefinition, unlike hosted definitions and file-based voice definitions. A manually authored service with an empty/invalid name therefore reaches createAgentAPIRequest, which silently deploys it as unspecified-agent-name instead of returning the manifest validation error. Validate the reconstructed voice agent before returning it.
func resolveVoiceAgentForDeploy(

cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:1327

  • [azd-code-reviewer] Supplying --kind prompt-voice together with --manifest skips this branch, so --kind becomes a no-op: a hosted manifest still initializes a hosted agent, and the new --model/--voice values are not applied. This is especially surprising because the validation message explicitly offers --manifest as an alternative to --agent-name. Either reject this flag combination or verify that the loaded manifest is prompt-voice and apply the requested overrides.
			if flags.kind != "" && flags.manifestPointer == "" {

cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:3285

  • [azd-code-reviewer] The generic post-init resolver treats this managed voice service as locally runnable. When --project-id or a reused project already supplies the Foundry endpoint, ResolveAfterInit takes its ready-state branch and prints azd ai agent run plus invoke --local; prompt-voice has no local source/runtime, and invoke is explicitly out of scope. Make the next-step state/resolver voice-aware or use voice-specific guidance limited to provision/deploy.
	fmt.Printf(

cli/azd/extensions/azure.ai.agents/schemas/azure.ai.agent.json:65

  • [azd-code-reviewer] Requiring the id property still allows model: { id: "" }, while both manifest validation and deploy reject an empty model ID. Add minLength: 1 so editor/schema validation matches the runtime requirement stated below.
      "description": "Voice agent (kind: prompt-voice) system prompt for the assistant."

Copilot AI review requested due to automatic review settings August 4, 2026 05:26

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 27 out of 27 changed files in this pull request and generated 1 comment.

Suppressed comments (3)

cli/azd/extensions/azure.ai.agents/schemas/azure.ai.agent.json:58

  • [azd-code-reviewer] The schema still accepts model: { id: "" }, while both manifest validation and deployment reject an empty model.id. Add a non-empty constraint so editor validation matches runtime validation as the new conditional promises.
        "id": { "type": "string", "description": "Model name (e.g. 'gpt-realtime')." }

cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go:1788

  • [azd-code-reviewer] This adds a consumed extension environment variable without adding it to the canonical environment-variable reference. Document AZURE_VOICE_OVERRIDDEN_HOST under the azure.ai.agents debug/internal variables, including its expected host format and unset behavior, so this temporary routing escape hatch is supportable.
// voiceOverriddenHostEnvKey optionally routes the /voice_agents call directly to
// a regional data-plane host (bypassing the public Foundry APIM, whose voice
// route may not yet be rolled out). When unset, default endpoint routing is used.
//
//nolint:gosec // env var key name, not a credential
const voiceOverriddenHostEnvKey = "AZURE_VOICE_OVERRIDDEN_HOST"

cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:145

  • [azd-code-reviewer] The canonical environment-variable reference still says AZD_AGENT_SKIP_ACR is set only for code-deploy scenarios, but this branch now sets it for prompt-voice agents too. Update that entry so documented provisioning behavior matches this new path.
// This happens when:
// - Code deploy mode is selected (ZIP upload, no container build)
// - Pre-built image is provided via --image flag (user manages their own registry)
// - The manifest is a prompt-voice agent (managed, no container image)
func (a *InitAction) skipACR() bool {
	return a.isCodeDeploy || a.flags.image != "" || a.isVoiceAgent

Comment thread cli/azd/extensions/azure.ai.agents/internal/cmd/init.go
Previously --kind prompt-voice was silently ignored when --manifest was
also supplied, so a hosted manifest would create a hosted service despite
the user explicitly selecting the voice kind. Reject the combination
early, matching the existing --kind/--image validation.
Copilot AI review requested due to automatic review settings August 4, 2026 05:44

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 27 out of 27 changed files in this pull request and generated no new comments.

Suppressed comments (3)

cli/azd/extensions/azure.ai.agents/internal/pkg/agents/agent_yaml/map.go:530

  • [azd-code-reviewer] A whitespace-only model.id passes both manifest validation and this check, then is sent as the managed model name. Normalize the ID with strings.TrimSpace, reject the normalized empty value, and use the normalized value in the request; keep the schema and ValidateAgentDefinition checks aligned so hand-authored manifests fail before the service call.
	if voiceAgent.Model == nil || voiceAgent.Model.Id == "" {
		return nil, fmt.Errorf("model.id is required for a prompt-voice agent")

cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go:1788

  • [azd-code-reviewer] This introduces a supported environment override, but cli/azd/docs/environment-variables.md does not list it. That file is the repository's source of truth for environment variables; document whether this must be an azd environment value or shell variable, the expected host format, and that it is a temporary/internal routing override.
const voiceOverriddenHostEnvKey = "AZURE_VOICE_OVERRIDDEN_HOST"

cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go:1802

  • [azd-code-reviewer] The tests cover dispatch and the HTTP client separately, but never exercise this new deploy orchestration. Add a provider-level test that verifies the successful call writes both AGENT_<KEY>_NAME and AGENT_<KEY>_ENDPOINT and returns the endpoint artifact, plus failure cases for a missing project endpoint and environment persistence errors.
func (p *AgentServiceTargetProvider) deployVoiceAgent(
	ctx context.Context,
	serviceConfig *azdext.ServiceConfig,
	va agent_yaml.VoiceAgent,
	azdEnv map[string]string,
	progress azdext.ProgressReporter,
) (*azdext.ServiceDeployResult, error) {

@jongio jongio left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-approving against bea4d9c. Verified the three commits since 7f5667f.

4db597e pins flags.agentName after the interactive resolve, matching what resolveAgentNameFromManifestPointer already does, so runInitFromManifest short-circuits instead of prompting a second time. The mutation can't leak, since the only thing running after that switch is ejectInfraAfterInit, which never reads the name.

bea4d9c rejects --kind prompt-voice combined with --manifest before either synthesis fast path, mirroring the existing --image guard. That closes the silent-adopt hole Copilot flagged.

The merge of main didn't drift anything. Net diff against the merge base is still 27 files, +2123/-29, byte-identical to the pre-merge PR, so conflict resolution didn't alter this branch's own contribution. All three agentkind.IsPromptVoice call sites (deploy dispatch, Endpoints, next-step reader) still share one lookup with the same AGENT_DEFINITION_PATH precedence. Main's new setServiceEnvironment call in addToProject correctly doesn't apply to addVoiceAgentToProject, since EnvironmentVariables only exists on ContainerAgent and a voice agent has no way to declare env.

Build and vet are clean and the agents extension tests pass. The two internal/synthesis bicep-stale failures reproduce identically on main at 493e6a7, so they aren't from this branch.

One leftover error hint, inline.

Comment thread cli/azd/extensions/azure.ai.agents/internal/cmd/init.go Outdated
When 'azd ai agent init' for a prompt-voice (managed) agent runs inside an
existing azd project, add it as a new azure.ai.agent service to the current
azure.yaml (src/<name> layout), matching hosted and other agents, instead of
scaffolding a separate nested <name>/ project. Applies to both the interactive
voice menu path and the '--kind prompt-voice' fast path. A brand-new (empty)
init still creates the <name>/ project folder.
Copilot AI review requested due to automatic review settings August 4, 2026 07:33
Since --kind prompt-voice combined with --manifest is now rejected, drop the
'(or provide --manifest ...)' remediation that would walk the user into a dead
end. Addresses PR review feedback.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 27 out of 27 changed files in this pull request and generated no new comments.

Suppressed comments (5)

cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:1286

  • [azd-code-reviewer] The prompt-voice compatibility check covers --image and --manifest, but container/code flags still pass through. --kind prompt-voice --deploy-mode container is silently ignored; with --deploy-mode code, validateCodeDeployFlags may require --runtime/--entry-point, after which all three values are still ignored because a VoiceAgent bypasses promptDeployMode. Reject deployment-mode-specific flags when --kind prompt-voice is selected so accepted CLI input is never discarded.
				if flags.image != "" {
					return exterrors.Validation(
						exterrors.CodeInvalidParameter,
						"--kind prompt-voice cannot be combined with --image",
						"a voice agent is managed and has no container image; drop --image",

cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go:1788

  • [azd-code-reviewer] This new deployment override is absent from cli/azd/docs/environment-variables.md, which is the repository's source of truth for extension environment variables. Document its expected host format, default routing behavior, and temporary/internal support status so users do not have to infer how to configure it from source.
const voiceOverriddenHostEnvKey = "AZURE_VOICE_OVERRIDDEN_HOST"

cli/azd/extensions/azure.ai.agents/internal/project/agent_definition.go:969

  • [azd-code-reviewer] This kind probe now intercepts an invalid AGENT_DEFINITION_PATH before the existing container loader can classify it. For example, malformed YAML in a hosted override returns a raw YAML error here instead of the existing CodeInvalidAgentManifest error and recovery suggestion. Parse the explicit override through voiceAgentFromDefinitionFile first, then use agentkind only when resolving the service entry.
	isVoice, err := agentkind.IsPromptVoice(svc, projectRoot, agentDefinitionPath)
	if err != nil {
		return agent_yaml.VoiceAgent{}, false, err

cli/azd/extensions/azure.ai.agents/schemas/azure.ai.agent.json:58

  • [azd-code-reviewer] Requiring the id property still permits model: { id: "" }, while both manifest validation and deployment reject an empty model ID. Add a minimum length so editor/schema validation matches the runtime requirement stated below.
        "id": { "type": "string", "description": "Model name (e.g. 'gpt-realtime')." }

cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:145

  • [azd-code-reviewer] Extending skipACR to voice agents makes the current AZD_AGENT_SKIP_ACR entry in cli/azd/docs/environment-variables.md:161 inaccurate because it says the variable is set automatically only for code-deploy scenarios. Update that entry to include managed prompt-voice initialization.
	return a.isCodeDeploy || a.flags.image != "" || a.isVoiceAgent

Copilot AI review requested due to automatic review settings August 4, 2026 07:45

@jongio jongio left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-approving against 03f8a6c. Verified both commits since bea4d9c.

7b30f47 lines up the prompt-voice target directory across the interactive and non-interactive paths. Inside an existing project both now hold targetDir at "." and append the service to the current azure.yaml, while a fresh init still creates the <name>/ folder and the cd hint. I walked all four combinations (interactive and flag-driven, crossed with existing and empty project) and they agree. Reordering the manifestInCwd branch keeps the old flags.src = "." behavior, and because --kind and --manifest are rejected together at line 1288, the synthesized voice manifest always lands in a temp dir, so manifestInCwd can't shadow the new voice branch.

03f8a6c drops the --manifest remediation from the prompt-voice hint, which matches that validation. Leaving the --image hint at line 1314 alone also tracks, since --image has no equivalent incompatibility check and still accepts --manifest.

Build, vet, and gofmt are clean here, and the cmd package tests pass.

Non-blocking, for a follow-up: the targetDir and folderDisplay decision now lives in three places: the non-interactive manifest branch, the template branch, and the interactive voice branch. That duplication is the same drift 7b30f47 is fixing, and the template branch still nests into <name>/ inside an existing project. The rest of this package factors decisions like this into a helper and unit tests it, the way synthesizeVoiceManifestFile and resolveInitAgentName are covered. Worth pulling this one out too so the next kind can't drift?

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 27 out of 27 changed files in this pull request and generated no new comments.

Suppressed comments (4)

cli/azd/extensions/azure.ai.agents/schemas/azure.ai.agent.json:58

  • [azd-code-reviewer] required: ["id"] still accepts id: "", while CreateVoiceAgentAPIRequest rejects an empty ID. Editors therefore report this configuration as valid only for deploy to fail. Add a minimum length to keep schema and runtime validation aligned.
        "id": { "type": "string", "description": "Model name (e.g. 'gpt-realtime')." }

cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go:1788

  • [azd-code-reviewer] This adds a new extension environment setting, but it is absent from the azure.ai.agents table in cli/azd/docs/environment-variables.md. Users cannot discover that this expects a regional host in the active azd environment or that leaving it unset uses the default APIM route. Document it alongside the other extension-specific variables.
const voiceOverriddenHostEnvKey = "AZURE_VOICE_OVERRIDDEN_HOST"

cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:145

  • [azd-code-reviewer] skipACR() now persists AZD_AGENT_SKIP_ACR=true for prompt-voice, but cli/azd/docs/environment-variables.md:161 still describes automatic use only for code deploy. Update that entry so the documented behavior matches generated voice projects.
	return a.isCodeDeploy || a.flags.image != "" || a.isVoiceAgent

cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:1651

  • [azd-code-reviewer] The added tests cover the synthesis helper and state predicates, but neither the interactive voice branch nor the --kind prompt-voice command path is driven through service creation. The conflict validation, target-directory handling, and inline AddService shape can regress while these tests still pass. Add an action-level test that exercises each entry path and asserts the generated voice service properties.
				case initModeVoice:

@trangevi trangevi left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most of the changes in the cmd files don't have corresponding tests, please see what can be added there

Comment thread cli/azd/extensions/azure.ai.agents/internal/cmd/init.go Outdated
Comment thread cli/azd/extensions/azure.ai.agents/internal/cmd/init.go
Comment thread cli/azd/extensions/azure.ai.agents/internal/cmd/init.go
Comment thread cli/azd/extensions/azure.ai.agents/internal/cmd/init.go
Comment thread cli/azd/extensions/azure.ai.agents/internal/cmd/init_adopt.go Outdated
Comment thread cli/azd/extensions/azure.ai.agents/internal/cmd/init_from_code.go Outdated
Copilot AI review requested due to automatic review settings August 6, 2026 03:39
@github-actions github-actions Bot added the area/extensions Extensions (general) label Aug 6, 2026
…ice-agent-managed-model

# Conflicts:
#	cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 31 out of 31 changed files in this pull request and generated no new comments.

Suppressed comments (8)

cli/azd/extensions/azure.ai.agents/internal/cmd/init_adopt.go:910

  • [azd-code-reviewer] The adopt flow can resolve to a container deployment (usesContainer == true), but this unconditional value newly applies hosted-agent region filtering to that path. Previously filtering occurred only when skipACR was true, so valid container projects outside hosted-agent regions can disappear from selection. Keep the split parameter, but preserve the old condition for this non-voice flow.
			"run this command in an empty directory (or pass a new target directory) to "+
				"adopt the sample, or add an individual agent to this project with "+
				"'azd ai agent init -m <agent.manifest.yaml>'",
		)

cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go:1788

  • [azd-code-reviewer] This introduces a new environment-variable input without adding it to cli/azd/docs/environment-variables.md, the repository’s source of truth for variables read by azd and its extensions. Document its host-only format, default routing behavior, and temporary/internal status so operators can use and remove the workaround safely.
		configDir := ""

cli/azd/extensions/azure.ai.agents/internal/project/agent_definition.go:972

  • [azd-code-reviewer] Inserting the voice helpers removed the opening sentence of this exported function’s Go doc, leaving a fragment that no longer names or explains AgentDefinitionToServiceProperties. Restore the function-prefixed sentence to keep generated documentation and the repository’s public-function convention intact.
// it round-trips through azure.yaml.

cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:145

  • [azd-code-reviewer] skipACR now also returns true for prompt-voice agents, but cli/azd/docs/environment-variables.md still says AZD_AGENT_SKIP_ACR is set only for code-deploy scenarios. Update the variable’s documented semantics so users do not infer that a voice initialization unexpectedly skipped registry provisioning.
	return a.isCodeDeploy || a.flags.image != "" || a.isVoiceAgent

cli/azd/extensions/azure.ai.agents/internal/cmd/init_from_code.go:338

  • [azd-code-reviewer] This specific-project branch now filters every local-code deployment to hosted-agent regions, including deployMode == "container". The equivalent interactive branch below filters only code deploys, and the previous skipACR coupling did the same. A valid container deployment can therefore reject its specified Foundry project as ineligible. Preserve the existing condition here.
		filterHostedRegions := true

cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:2243

  • [azd-code-reviewer] The earlier incompatibility check only runs when --kind is set. A user can still pass --manifest voice.yaml --image ... (or voice with code-deploy-only flags): this is where the manifest is first recognized as voice, but addVoiceAgentToProject later returns before consuming those options, so initialization succeeds while silently ignoring them; --image also makes isHostedAgent() apply the wrong region filter. Reject container/code-only options once the parsed manifest is known to be prompt-voice.
// lives inside the (untyped) Template payload rather than on AgentManifest, so we
// round-trip the template into an AgentDefinition to read it. Mirrors the
// extraction addToProject performs before dispatching on kind.

cli/azd/extensions/azure.ai.agents/internal/cmd/doctor/checks_agent_status.go:733

  • [azd-code-reviewer] This classifier omits the AGENT_DEFINITION_PATH override even though deploy, Endpoints, and next-step classification all give that override highest precedence. A voice agent deployed from an override over a hosted or kind-less service is therefore retained in this hosted-only probe and incorrectly checked for AGENT_<KEY>_VERSION. Pass the process override here as the other consumers do, and add an override regression test.
		isVoice, err := agentkind.IsPromptVoice(svc, resp.Project.GetPath(), "")

cli/azd/extensions/azure.ai.agents/schemas/azure.ai.agent.json:58

  • [azd-code-reviewer] The schema currently accepts model.id: "", while both manifest validation and deploy reject an empty ID. Add a minimum length so editor/schema validation actually matches the runtime requirement stated by the new conditional comment.
        "id": { "type": "string", "description": "Model name (e.g. 'gpt-realtime')." }

Copilot AI review requested due to automatic review settings August 6, 2026 03:46

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 31 out of 31 changed files in this pull request and generated no new comments.

Suppressed comments (5)

cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:1275

  • [azd-code-reviewer] The prompt-voice fast path still accepts hosted-only flags that it never consumes. For example, --model-deployment, --protocol, or --deploy-mode container pass validation, but the synthesized voice manifest uses only --model and --voice, so automation can succeed while silently ignoring requested settings. Reject incompatible flags in this validation block (especially --model-deployment, since its help says it takes precedence) and cover those combinations.
			if flags.kind != "" {

cli/azd/extensions/azure.ai.agents/internal/cmd/doctor/checks_agent_status.go:733

  • [azd-code-reviewer] This classifier omits the AGENT_DEFINITION_PATH override even though deploy, Endpoints, and next-step all pass it to agentkind. If an inline hosted service is deployed through a prompt-voice override, doctor classifies it as hosted and runs the NAME/VERSION probe against a deployment that intentionally has no VERSION, producing a false failure. Pass the active override here too and add an override-precedence regression test.
		isVoice, err := agentkind.IsPromptVoice(svc, resp.Project.GetPath(), "")

cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go:1875

  • [azd-code-reviewer] This new extension-specific environment key is absent from cli/azd/docs/environment-variables.md, which is the repository's source of truth for variables consumed by azd and its extensions. Document where the override must be set (active azd environment versus process environment), its default behavior, and that it is a temporary/internal preview workaround.
const voiceOverriddenHostEnvKey = "AZURE_VOICE_OVERRIDDEN_HOST"

cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:145

  • [azd-code-reviewer] This changes AZD_AGENT_SKIP_ACR to be set for prompt-voice agents, but cli/azd/docs/environment-variables.md:161 still says the extension sets it only for code-deploy scenarios. Update that entry so users do not infer that a true value necessarily means code deploy.
func (a *InitAction) skipACR() bool {
	return a.isCodeDeploy || a.flags.image != "" || a.isVoiceAgent

cli/azd/extensions/azure.ai.agents/internal/project/agent_definition.go:1013

  • [azd-code-reviewer] The generated inline azure.yaml shape is the primary scaffold path, but the new voice writer/reader pair has no round-trip test. Existing voice dispatch tests cover only an on-disk override, so regressions in modelType, model, instructions, voice, or store serialization would not be caught before deploy. Add a test that writes with VoiceAgentDefinitionToServiceProperties, reads with VoiceAgentFromResolvedService, and asserts every field.
func VoiceAgentDefinitionToServiceProperties(
	va agent_yaml.VoiceAgent,
	extra *ServiceTargetAgentConfig,
) (*structpb.Struct, error) {
	inline := voiceAgentDefinitionToInline(va)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/extensions Extensions (general) ext-agents azure.ai.agents extension ext-projects azure.ai.projects extension

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Voice-based prompt agents

5 participants