feat(agents): support declarative prompt-voice agents (managed model) - #9364
feat(agents): support declarative prompt-voice agents (managed model)#9364v1212 wants to merge 22 commits into
Conversation
Add azd support for a new 'prompt-voice' agent kind that creates a managed speech-to-speech (voice) agent on Azure AI Foundry. - yaml: new prompt-voice kind + VoiceAgent authoring struct/parsing - map: translate authoring kind prompt-voice -> data-plane kind voice, defaulting the audio pipeline (PCM16@24k, server_vad, whisper-1, DragonHD default voice) and v1 implicit managed model_type - agent_api: VoiceAgentDefinition wire structs + CreateVoiceAgent with Foundry-Features: VoiceAgents=V1Preview preview header - project: voice-aware agent_definition read/write + isolated deployVoiceAgent deploy path (container path unchanged) - init: --kind/--voice flags, voice manifest synthesis, prompt option Scope: prompt-voice + managed model only. BYOM, hosted-voice, tools, avatar, and cascaded models are follow-ups. Draft: needs further end-to-end session testing and optimization.
|
Azure Pipelines: Successfully started running 1 pipeline(s). 21 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
- map_voice_test.go: cover CreateVoiceAgentAPIRequest defaults/overrides, managed enforcement, BYOM rejection, missing-model error, and the isOpenAIVoice/buildVoiceConfig voice-type selection - parse_voice_test.go: cover prompt-voice manifest parsing and ValidateAgentDefinition (ok / missing model.id / self_deployed rejected) - azure.ai.agent.json: add prompt-voice to the kind enum and document the voice service properties (modelType/model/instructions/voice/store) - cspell.yaml: allow BYOM
The azure.ai.projects synthesis copy must stay byte-identical to the azure.ai.agents copy (TestAgentsSynthesisCopyMatches). Mirror the prompt-voice comment update made in the agents synthesizer.
There was a problem hiding this comment.
Pull request overview
Adds managed declarative prompt-voice agents across initialization, configuration, API mapping, and deployment.
Changes:
- Adds voice-agent YAML/schema models and validation.
- Maps voice manifests to Foundry’s preview API contract.
- Adds voice-specific initialization and deployment paths with tests.
Reviewed changes
Copilot reviewed 15 out of 15 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
schemas/azure.ai.agent.json |
Adds voice-agent schema fields. |
internal/synthesis/synthesizer.go |
Marks voice agents as non-container. |
internal/project/service_target_agent.go |
Adds voice deployment handling. |
internal/project/agent_definition.go |
Adds inline voice configuration conversion. |
internal/pkg/agents/agent_yaml/yaml.go |
Defines voice authoring models. |
internal/pkg/agents/agent_yaml/parse.go |
Parses and validates voice manifests. |
internal/pkg/agents/agent_yaml/parse_voice_test.go |
Tests voice parsing and validation. |
internal/pkg/agents/agent_yaml/map.go |
Maps voice manifests to API requests. |
internal/pkg/agents/agent_yaml/map_voice_test.go |
Tests voice request mapping. |
internal/pkg/agents/agent_api/operations.go |
Adds the preview create operation. |
internal/pkg/agents/agent_api/models.go |
Defines voice API wire models. |
internal/cmd/init.go |
Adds voice initialization flags and flow. |
internal/cmd/init_from_templates_helpers.go |
Adds the interactive voice option. |
cspell.yaml |
Adds voice-related terminology. |
Resolve four correctness gaps in the declarative prompt-voice flow surfaced in review, keeping every existing hosted/container/workflow path unchanged: - init: validate --kind (and its --image incompatibility) before either the image or prompt-voice synthesis fast path, so `--kind prompt-voice --image` is rejected instead of silently creating a hosted image agent. - init: skipACR now also covers prompt-voice (managed, no container), while a new isHostedAgent decision drives hosted-region filtering. selectFoundryProject gains a distinct filterHostedRegions parameter so a voice agent skips ACR without being constrained to hosted-agent regions. - deploy: resolve an explicit AGENT_DEFINITION_PATH override before the voice/container dispatch (resolveVoiceAgentForDeploy), so an override wins for voice just as it does for the container path. - deploy contract: make Endpoints() and next-step isDeployed voice-aware. Voice agents record only NAME + base ENDPOINT (no agent-version / per-protocol endpoints), so both consumers now treat the base endpoint as the deployment marker instead of reporting a created voice agent as undeployed. Adds unit tests for the skipACR/isHostedAgent split, the override-precedence dispatch, and the voice deployed-marker fallback.
|
Azure Pipelines: Successfully started running 2 pipeline(s). 20 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
There was a problem hiding this comment.
Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.
Note
This error may be related to your runner configuration. You can now configure runners for Copilot code review separately from Copilot cloud agent by creating a copilot-code-review.yml file with your setup steps. Read the docs for details.
jongio
left a comment
There was a problem hiding this comment.
Ran through the voice path end to end against the current head. Build is clean and the new tests pass locally. Three things I'd like to sort out before this ships, plus two nits.
Medium
- The new deployed-agent detection keys off "VERSION empty and ENDPOINT set" rather than the service kind, in both
Endpoints()andnextstep.isDeployed(). Inline comments on both. - Voice create is an unconditional POST to
/voice_agentswith no version model. What's the intended behavior for a secondazd deploy? Inline comment onoperations.go. - Delete has no voice path.
AgentClient.DeleteAgentbuilds%s/agents/%s, andDeleteAction.cleanupEnvVarsclears the threeAGENT_<KEY>_*vars regardless. Soazd ai agent deleteagainst a prompt-voice service hits/agents/<name>, gets a 404, andclassifyDeleteErrorturns that intoCodeAgentNotFound. The user is told the agent doesn't exist while it's still live in the project, and there's no way to tear it down from azd. If delete is intentionally out of scope for this PR, could you add it to the follow-ups list in the description so it doesn't get lost?
Low
- Orphaned comment fragment in
init.go. Inline comment. isOpenAIVoiceclassifies by name shape. Inline comment.
One question on AZURE_VOICE_OVERRIDDEN_HOST: the comment says it exists to bypass the public Foundry APIM while the voice route rolls out. Worth filing an issue to remove it once the route is live, otherwise it tends to stick around forever.
Also, the description still says Draft but the PR is open and review is requested. Probably just needs updating.
- Endpoints()/isDeployed(): gate voice base-endpoint fallback on the service's actual prompt-voice kind instead of the env-var shape, so a partially-failed hosted deploy still surfaces CodeMissingAgentEnvVars - add nextstep isVoiceService helper (mirrors project kind gate; the two stay in separate packages to avoid a project->nextstep import cycle) - CreateVoiceAgent: document create-only redeploy semantics - isOpenAIVoice: classify via explicit OpenAI voice set + Azure Neural locale-prefix pattern instead of a bare '-' check - init.go: drop orphaned comment fragment - tests: cover hosted lingering-endpoint gate and voice name classification
|
@jongio thanks for the review — pushed 438466c addressing the inline threads (replied on each; left them unresolved for you to confirm). For the remaining points from the review body:
|
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 22 out of 22 changed files in this pull request and generated 1 comment.
Suppressed comments (8)
cli/azd/extensions/azure.ai.agents/internal/cmd/init_foundry_project_setup.go:148
- [azd-code-reviewer] The interactive existing-project branch has the same coupling: prompt-voice sets
skipACR=true, which now turns on hosted-region filtering and hides otherwise valid voice projects. Use the separate hosted-agent decision here as well.
skipACR, // filterHostedRegions: this path is code/container only (non-voice)
cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go:440
- [azd-code-reviewer] This voice check ignores
AGENT_DEFINITION_PATH, although deployment now lets that override determine voice/container dispatch. A voice override on a hosted service deploys successfully but endpoint discovery still treats it as hosted and fails on the missing VERSION; next-step state has the same mismatch. Persist or resolve a common effective-kind marker in all three consumers.
if _, isVoice, err := VoiceAgentFromResolvedService(serviceConfig, p.projectPath); err != nil {
cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:1646
- [azd-code-reviewer] This resolved name is not pinned to
flags.agentName.runInitFromManifestlater reachesdownloadAgentYaml, which callsresolveInitAgentNameagain, so the interactive voice flow asks for the agent name twice. Pin the result before entering the manifest flow and add a regression test.
resolvedName, err := resolveInitAgentName(ctx, azdClient, flags, "voice-agent")
cli/azd/extensions/azure.ai.agents/internal/cmd/init_foundry_project_setup.go:74
- [azd-code-reviewer] This still equates skipping ACR with requiring hosted-agent regions. Prompt-voice reaches
configureFoundryProjectwithskipACR=true, so a voice init using--project-idincorrectly filters out projects in regions that do not support hosted agents. Pass a separatefilterHostedRegionsdecision through this helper.
This issue also appears on line 148 of the same file.
skipACR, // filterHostedRegions: this path is code/container only (non-voice)
cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:1288
- [azd-code-reviewer] The prompt-voice fast path silently ignores
--model-deployment, even though the shared flag help says it takes precedence over--model. Because this PR supports managed models only, reject this combination instead of creating the defaultgpt-realtimeagent.
if flags.image != "" {
cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:2216
- [azd-code-reviewer] Recording
isVoiceAgentdoes not preserve the documented two-question scaffold flow. A synthesized voice manifest has no model resources, so this function continues intoconfigureFoundryProject, whose interactive branch prompts for project choice, subscription, and location during init rather than deferring them toazd provisionas the PR description states.
if kind, err := agentManifestKind(agentManifest); err == nil {
a.isVoiceAgent = kind == agent_yaml.AgentKindPromptVoice
cli/azd/extensions/azure.ai.agents/internal/pkg/agents/agent_api/operations.go:188
- [azd-code-reviewer] The linked REST contract does not have the create-only behavior documented here:
POST /voice_agentscreates an agent or a new version of an existing one, andPOST /voice_agents/{agent_name}provides update semantics. Treating every second deploy as an unavoidable failure makesazd deploynon-repeatable despite API support; implement the existing-agent path rather than deferring it.
// Redeploy semantics: the voice data-plane exposes create-only POST /voice_agents
// with no version/upsert model (unlike hosted agents, which mint a new
// agent-version per deploy). A second `azd deploy` of the same voice service
// therefore re-POSTs with the same name and the service rejects it with a
// non-success status, which this method surfaces as a deploy error rather than
// silently overwriting the existing agent. Idempotent redeploy/update is tracked
// as a follow-up (see the PR "Follow-ups" section); until the service adds an
// update route, redeploy requires deleting the existing voice agent first.
cli/azd/extensions/azure.ai.agents/internal/pkg/agents/agent_api/operations.go:212
- [azd-code-reviewer]
CreateVoiceAgenthas no operation-level test, leaving the required preview header, optional overridden-host header, route, and accepted response statuses unverified. Add a fake-transport test alongsideoperations_test.gothat asserts these request details and response parsing.
// Voice agents are a preview feature; the service rejects the request with
// 403 preview_feature_required unless this opt-in header is present.
req.Raw().Header.Set("Foundry-Features", voiceAgentsPreviewFeature)
if overriddenHost != "" {
req.Raw().Header.Set("x-ms-overridden-host", overriddenHost)
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 27 out of 27 changed files in this pull request and generated no new comments.
Suppressed comments (5)
cli/azd/extensions/azure.ai.agents/internal/project/agent_definition.go:848
- [azd-code-reviewer] The shared kind lookup can identify a voice agent from the service directory's
agent.yaml, but this fallback only parses inline/config properties. For the on-disk shape covered byTestKind_ManifestFallbackand the new endpoint test,IsPromptVoicereturns true here, thenVoiceAgentFromResolvedServicereturnsfound=false, and deploy falls through to the container path instead of deploying the voice definition. Parse the same on-disk manifest source when the service entry has no definition.
)
cli/azd/extensions/azure.ai.agents/internal/project/agent_definition.go:954
- [azd-code-reviewer] Inline voice definitions bypass
validateAgentServiceDefinition, unlike hosted definitions and file-based voice definitions. A manually authored service with an empty/invalidnametherefore reachescreateAgentAPIRequest, which silently deploys it asunspecified-agent-nameinstead of returning the manifest validation error. Validate the reconstructed voice agent before returning it.
func resolveVoiceAgentForDeploy(
cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:1327
- [azd-code-reviewer] Supplying
--kind prompt-voicetogether with--manifestskips this branch, so--kindbecomes a no-op: a hosted manifest still initializes a hosted agent, and the new--model/--voicevalues are not applied. This is especially surprising because the validation message explicitly offers--manifestas an alternative to--agent-name. Either reject this flag combination or verify that the loaded manifest is prompt-voice and apply the requested overrides.
if flags.kind != "" && flags.manifestPointer == "" {
cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:3285
- [azd-code-reviewer] The generic post-init resolver treats this managed voice service as locally runnable. When
--project-idor a reused project already supplies the Foundry endpoint,ResolveAfterInittakes its ready-state branch and printsazd ai agent runplusinvoke --local; prompt-voice has no local source/runtime, and invoke is explicitly out of scope. Make the next-step state/resolver voice-aware or use voice-specific guidance limited to provision/deploy.
fmt.Printf(
cli/azd/extensions/azure.ai.agents/schemas/azure.ai.agent.json:65
- [azd-code-reviewer] Requiring the
idproperty still allowsmodel: { id: "" }, while both manifest validation and deploy reject an empty model ID. AddminLength: 1so editor/schema validation matches the runtime requirement stated below.
"description": "Voice agent (kind: prompt-voice) system prompt for the assistant."
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 27 out of 27 changed files in this pull request and generated 1 comment.
Suppressed comments (3)
cli/azd/extensions/azure.ai.agents/schemas/azure.ai.agent.json:58
- [azd-code-reviewer] The schema still accepts
model: { id: "" }, while both manifest validation and deployment reject an emptymodel.id. Add a non-empty constraint so editor validation matches runtime validation as the new conditional promises.
"id": { "type": "string", "description": "Model name (e.g. 'gpt-realtime')." }
cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go:1788
- [azd-code-reviewer] This adds a consumed extension environment variable without adding it to the canonical environment-variable reference. Document
AZURE_VOICE_OVERRIDDEN_HOSTunder theazure.ai.agentsdebug/internal variables, including its expected host format and unset behavior, so this temporary routing escape hatch is supportable.
// voiceOverriddenHostEnvKey optionally routes the /voice_agents call directly to
// a regional data-plane host (bypassing the public Foundry APIM, whose voice
// route may not yet be rolled out). When unset, default endpoint routing is used.
//
//nolint:gosec // env var key name, not a credential
const voiceOverriddenHostEnvKey = "AZURE_VOICE_OVERRIDDEN_HOST"
cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:145
- [azd-code-reviewer] The canonical environment-variable reference still says
AZD_AGENT_SKIP_ACRis set only for code-deploy scenarios, but this branch now sets it for prompt-voice agents too. Update that entry so documented provisioning behavior matches this new path.
// This happens when:
// - Code deploy mode is selected (ZIP upload, no container build)
// - Pre-built image is provided via --image flag (user manages their own registry)
// - The manifest is a prompt-voice agent (managed, no container image)
func (a *InitAction) skipACR() bool {
return a.isCodeDeploy || a.flags.image != "" || a.isVoiceAgent
Previously --kind prompt-voice was silently ignored when --manifest was also supplied, so a hosted manifest would create a hosted service despite the user explicitly selecting the voice kind. Reject the combination early, matching the existing --kind/--image validation.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 27 out of 27 changed files in this pull request and generated no new comments.
Suppressed comments (3)
cli/azd/extensions/azure.ai.agents/internal/pkg/agents/agent_yaml/map.go:530
- [azd-code-reviewer] A whitespace-only
model.idpasses both manifest validation and this check, then is sent as the managed model name. Normalize the ID withstrings.TrimSpace, reject the normalized empty value, and use the normalized value in the request; keep the schema andValidateAgentDefinitionchecks aligned so hand-authored manifests fail before the service call.
if voiceAgent.Model == nil || voiceAgent.Model.Id == "" {
return nil, fmt.Errorf("model.id is required for a prompt-voice agent")
cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go:1788
- [azd-code-reviewer] This introduces a supported environment override, but
cli/azd/docs/environment-variables.mddoes not list it. That file is the repository's source of truth for environment variables; document whether this must be an azd environment value or shell variable, the expected host format, and that it is a temporary/internal routing override.
const voiceOverriddenHostEnvKey = "AZURE_VOICE_OVERRIDDEN_HOST"
cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go:1802
- [azd-code-reviewer] The tests cover dispatch and the HTTP client separately, but never exercise this new deploy orchestration. Add a provider-level test that verifies the successful call writes both
AGENT_<KEY>_NAMEandAGENT_<KEY>_ENDPOINTand returns the endpoint artifact, plus failure cases for a missing project endpoint and environment persistence errors.
func (p *AgentServiceTargetProvider) deployVoiceAgent(
ctx context.Context,
serviceConfig *azdext.ServiceConfig,
va agent_yaml.VoiceAgent,
azdEnv map[string]string,
progress azdext.ProgressReporter,
) (*azdext.ServiceDeployResult, error) {
jongio
left a comment
There was a problem hiding this comment.
Re-approving against bea4d9c. Verified the three commits since 7f5667f.
4db597e pins flags.agentName after the interactive resolve, matching what resolveAgentNameFromManifestPointer already does, so runInitFromManifest short-circuits instead of prompting a second time. The mutation can't leak, since the only thing running after that switch is ejectInfraAfterInit, which never reads the name.
bea4d9c rejects --kind prompt-voice combined with --manifest before either synthesis fast path, mirroring the existing --image guard. That closes the silent-adopt hole Copilot flagged.
The merge of main didn't drift anything. Net diff against the merge base is still 27 files, +2123/-29, byte-identical to the pre-merge PR, so conflict resolution didn't alter this branch's own contribution. All three agentkind.IsPromptVoice call sites (deploy dispatch, Endpoints, next-step reader) still share one lookup with the same AGENT_DEFINITION_PATH precedence. Main's new setServiceEnvironment call in addToProject correctly doesn't apply to addVoiceAgentToProject, since EnvironmentVariables only exists on ContainerAgent and a voice agent has no way to declare env.
Build and vet are clean and the agents extension tests pass. The two internal/synthesis bicep-stale failures reproduce identically on main at 493e6a7, so they aren't from this branch.
One leftover error hint, inline.
When 'azd ai agent init' for a prompt-voice (managed) agent runs inside an existing azd project, add it as a new azure.ai.agent service to the current azure.yaml (src/<name> layout), matching hosted and other agents, instead of scaffolding a separate nested <name>/ project. Applies to both the interactive voice menu path and the '--kind prompt-voice' fast path. A brand-new (empty) init still creates the <name>/ project folder.
Since --kind prompt-voice combined with --manifest is now rejected, drop the '(or provide --manifest ...)' remediation that would walk the user into a dead end. Addresses PR review feedback.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 27 out of 27 changed files in this pull request and generated no new comments.
Suppressed comments (5)
cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:1286
- [azd-code-reviewer] The prompt-voice compatibility check covers
--imageand--manifest, but container/code flags still pass through.--kind prompt-voice --deploy-mode containeris silently ignored; with--deploy-mode code,validateCodeDeployFlagsmay require--runtime/--entry-point, after which all three values are still ignored because aVoiceAgentbypassespromptDeployMode. Reject deployment-mode-specific flags when--kind prompt-voiceis selected so accepted CLI input is never discarded.
if flags.image != "" {
return exterrors.Validation(
exterrors.CodeInvalidParameter,
"--kind prompt-voice cannot be combined with --image",
"a voice agent is managed and has no container image; drop --image",
cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go:1788
- [azd-code-reviewer] This new deployment override is absent from
cli/azd/docs/environment-variables.md, which is the repository's source of truth for extension environment variables. Document its expected host format, default routing behavior, and temporary/internal support status so users do not have to infer how to configure it from source.
const voiceOverriddenHostEnvKey = "AZURE_VOICE_OVERRIDDEN_HOST"
cli/azd/extensions/azure.ai.agents/internal/project/agent_definition.go:969
- [azd-code-reviewer] This kind probe now intercepts an invalid
AGENT_DEFINITION_PATHbefore the existing container loader can classify it. For example, malformed YAML in a hosted override returns a raw YAML error here instead of the existingCodeInvalidAgentManifesterror and recovery suggestion. Parse the explicit override throughvoiceAgentFromDefinitionFilefirst, then useagentkindonly when resolving the service entry.
isVoice, err := agentkind.IsPromptVoice(svc, projectRoot, agentDefinitionPath)
if err != nil {
return agent_yaml.VoiceAgent{}, false, err
cli/azd/extensions/azure.ai.agents/schemas/azure.ai.agent.json:58
- [azd-code-reviewer] Requiring the
idproperty still permitsmodel: { id: "" }, while both manifest validation and deployment reject an empty model ID. Add a minimum length so editor/schema validation matches the runtime requirement stated below.
"id": { "type": "string", "description": "Model name (e.g. 'gpt-realtime')." }
cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:145
- [azd-code-reviewer] Extending
skipACRto voice agents makes the currentAZD_AGENT_SKIP_ACRentry incli/azd/docs/environment-variables.md:161inaccurate because it says the variable is set automatically only for code-deploy scenarios. Update that entry to include managed prompt-voice initialization.
return a.isCodeDeploy || a.flags.image != "" || a.isVoiceAgent
jongio
left a comment
There was a problem hiding this comment.
Re-approving against 03f8a6c. Verified both commits since bea4d9c.
7b30f47 lines up the prompt-voice target directory across the interactive and non-interactive paths. Inside an existing project both now hold targetDir at "." and append the service to the current azure.yaml, while a fresh init still creates the <name>/ folder and the cd hint. I walked all four combinations (interactive and flag-driven, crossed with existing and empty project) and they agree. Reordering the manifestInCwd branch keeps the old flags.src = "." behavior, and because --kind and --manifest are rejected together at line 1288, the synthesized voice manifest always lands in a temp dir, so manifestInCwd can't shadow the new voice branch.
03f8a6c drops the --manifest remediation from the prompt-voice hint, which matches that validation. Leaving the --image hint at line 1314 alone also tracks, since --image has no equivalent incompatibility check and still accepts --manifest.
Build, vet, and gofmt are clean here, and the cmd package tests pass.
Non-blocking, for a follow-up: the targetDir and folderDisplay decision now lives in three places: the non-interactive manifest branch, the template branch, and the interactive voice branch. That duplication is the same drift 7b30f47 is fixing, and the template branch still nests into <name>/ inside an existing project. The rest of this package factors decisions like this into a helper and unit tests it, the way synthesizeVoiceManifestFile and resolveInitAgentName are covered. Worth pulling this one out too so the next kind can't drift?
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 27 out of 27 changed files in this pull request and generated no new comments.
Suppressed comments (4)
cli/azd/extensions/azure.ai.agents/schemas/azure.ai.agent.json:58
- [azd-code-reviewer]
required: ["id"]still acceptsid: "", whileCreateVoiceAgentAPIRequestrejects an empty ID. Editors therefore report this configuration as valid only for deploy to fail. Add a minimum length to keep schema and runtime validation aligned.
"id": { "type": "string", "description": "Model name (e.g. 'gpt-realtime')." }
cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go:1788
- [azd-code-reviewer] This adds a new extension environment setting, but it is absent from the
azure.ai.agentstable incli/azd/docs/environment-variables.md. Users cannot discover that this expects a regional host in the active azd environment or that leaving it unset uses the default APIM route. Document it alongside the other extension-specific variables.
const voiceOverriddenHostEnvKey = "AZURE_VOICE_OVERRIDDEN_HOST"
cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:145
- [azd-code-reviewer]
skipACR()now persistsAZD_AGENT_SKIP_ACR=truefor prompt-voice, butcli/azd/docs/environment-variables.md:161still describes automatic use only for code deploy. Update that entry so the documented behavior matches generated voice projects.
return a.isCodeDeploy || a.flags.image != "" || a.isVoiceAgent
cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:1651
- [azd-code-reviewer] The added tests cover the synthesis helper and state predicates, but neither the interactive voice branch nor the
--kind prompt-voicecommand path is driven through service creation. The conflict validation, target-directory handling, and inlineAddServiceshape can regress while these tests still pass. Add an action-level test that exercises each entry path and asserts the generated voice service properties.
case initModeVoice:
trangevi
left a comment
There was a problem hiding this comment.
Most of the changes in the cmd files don't have corresponding tests, please see what can be added there
…ice-agent-managed-model # Conflicts: # cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 31 out of 31 changed files in this pull request and generated no new comments.
Suppressed comments (8)
cli/azd/extensions/azure.ai.agents/internal/cmd/init_adopt.go:910
- [azd-code-reviewer] The adopt flow can resolve to a container deployment (
usesContainer == true), but this unconditional value newly applies hosted-agent region filtering to that path. Previously filtering occurred only whenskipACRwas true, so valid container projects outside hosted-agent regions can disappear from selection. Keep the split parameter, but preserve the old condition for this non-voice flow.
"run this command in an empty directory (or pass a new target directory) to "+
"adopt the sample, or add an individual agent to this project with "+
"'azd ai agent init -m <agent.manifest.yaml>'",
)
cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go:1788
- [azd-code-reviewer] This introduces a new environment-variable input without adding it to
cli/azd/docs/environment-variables.md, the repository’s source of truth for variables read by azd and its extensions. Document its host-only format, default routing behavior, and temporary/internal status so operators can use and remove the workaround safely.
configDir := ""
cli/azd/extensions/azure.ai.agents/internal/project/agent_definition.go:972
- [azd-code-reviewer] Inserting the voice helpers removed the opening sentence of this exported function’s Go doc, leaving a fragment that no longer names or explains
AgentDefinitionToServiceProperties. Restore the function-prefixed sentence to keep generated documentation and the repository’s public-function convention intact.
// it round-trips through azure.yaml.
cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:145
- [azd-code-reviewer]
skipACRnow also returns true for prompt-voice agents, butcli/azd/docs/environment-variables.mdstill saysAZD_AGENT_SKIP_ACRis set only for code-deploy scenarios. Update the variable’s documented semantics so users do not infer that a voice initialization unexpectedly skipped registry provisioning.
return a.isCodeDeploy || a.flags.image != "" || a.isVoiceAgent
cli/azd/extensions/azure.ai.agents/internal/cmd/init_from_code.go:338
- [azd-code-reviewer] This specific-project branch now filters every local-code deployment to hosted-agent regions, including
deployMode == "container". The equivalent interactive branch below filters only code deploys, and the previousskipACRcoupling did the same. A valid container deployment can therefore reject its specified Foundry project as ineligible. Preserve the existing condition here.
filterHostedRegions := true
cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:2243
- [azd-code-reviewer] The earlier incompatibility check only runs when
--kindis set. A user can still pass--manifest voice.yaml --image ...(or voice with code-deploy-only flags): this is where the manifest is first recognized as voice, butaddVoiceAgentToProjectlater returns before consuming those options, so initialization succeeds while silently ignoring them;--imagealso makesisHostedAgent()apply the wrong region filter. Reject container/code-only options once the parsed manifest is known to be prompt-voice.
// lives inside the (untyped) Template payload rather than on AgentManifest, so we
// round-trip the template into an AgentDefinition to read it. Mirrors the
// extraction addToProject performs before dispatching on kind.
cli/azd/extensions/azure.ai.agents/internal/cmd/doctor/checks_agent_status.go:733
- [azd-code-reviewer] This classifier omits the
AGENT_DEFINITION_PATHoverride even though deploy,Endpoints, and next-step classification all give that override highest precedence. A voice agent deployed from an override over a hosted or kind-less service is therefore retained in this hosted-only probe and incorrectly checked forAGENT_<KEY>_VERSION. Pass the process override here as the other consumers do, and add an override regression test.
isVoice, err := agentkind.IsPromptVoice(svc, resp.Project.GetPath(), "")
cli/azd/extensions/azure.ai.agents/schemas/azure.ai.agent.json:58
- [azd-code-reviewer] The schema currently accepts
model.id: "", while both manifest validation and deploy reject an empty ID. Add a minimum length so editor/schema validation actually matches the runtime requirement stated by the new conditional comment.
"id": { "type": "string", "description": "Model name (e.g. 'gpt-realtime')." }
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 31 out of 31 changed files in this pull request and generated no new comments.
Suppressed comments (5)
cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:1275
- [azd-code-reviewer] The prompt-voice fast path still accepts hosted-only flags that it never consumes. For example,
--model-deployment,--protocol, or--deploy-mode containerpass validation, but the synthesized voice manifest uses only--modeland--voice, so automation can succeed while silently ignoring requested settings. Reject incompatible flags in this validation block (especially--model-deployment, since its help says it takes precedence) and cover those combinations.
if flags.kind != "" {
cli/azd/extensions/azure.ai.agents/internal/cmd/doctor/checks_agent_status.go:733
- [azd-code-reviewer] This classifier omits the
AGENT_DEFINITION_PATHoverride even though deploy,Endpoints, and next-step all pass it toagentkind. If an inline hosted service is deployed through a prompt-voice override, doctor classifies it as hosted and runs the NAME/VERSION probe against a deployment that intentionally has no VERSION, producing a false failure. Pass the active override here too and add an override-precedence regression test.
isVoice, err := agentkind.IsPromptVoice(svc, resp.Project.GetPath(), "")
cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go:1875
- [azd-code-reviewer] This new extension-specific environment key is absent from
cli/azd/docs/environment-variables.md, which is the repository's source of truth for variables consumed by azd and its extensions. Document where the override must be set (active azd environment versus process environment), its default behavior, and that it is a temporary/internal preview workaround.
const voiceOverriddenHostEnvKey = "AZURE_VOICE_OVERRIDDEN_HOST"
cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:145
- [azd-code-reviewer] This changes
AZD_AGENT_SKIP_ACRto be set for prompt-voice agents, butcli/azd/docs/environment-variables.md:161still says the extension sets it only for code-deploy scenarios. Update that entry so users do not infer that a true value necessarily means code deploy.
func (a *InitAction) skipACR() bool {
return a.isCodeDeploy || a.flags.image != "" || a.isVoiceAgent
cli/azd/extensions/azure.ai.agents/internal/project/agent_definition.go:1013
- [azd-code-reviewer] The generated inline
azure.yamlshape is the primary scaffold path, but the new voice writer/reader pair has no round-trip test. Existing voice dispatch tests cover only an on-disk override, so regressions inmodelType,model,instructions,voice, orstoreserialization would not be caught before deploy. Add a test that writes withVoiceAgentDefinitionToServiceProperties, reads withVoiceAgentFromResolvedService, and asserts every field.
func VoiceAgentDefinitionToServiceProperties(
va agent_yaml.VoiceAgent,
extra *ServiceTargetAgentConfig,
) (*structpb.Struct, error) {
inline := voiceAgentDefinitionToInline(va)
Summary
Closes #9336
Adds azd support for a new declarative
prompt-voiceagent kind that creates a managed speech-to-speech (voice) agent on Azure AI Foundry, end-to-end throughazd init→azure.yaml→azd deploy.Scope (intentionally narrow):
prompt-voice+ managed model only — i.e. scaffold (init) and deploy only. BYOM, hosted-voice, tools, avatar, and cascaded models are deliberately left as follow-up PRs. azd-native textinvoke,list, and a Portal Playground link are out of scope for this PR (see Follow-ups).What's included
agent_yaml/yaml.go,parse.go): newprompt-voiceauthoring kind +VoiceAgentstruct/parsing/validation.agent_yaml/map.go): translate authoring kindprompt-voice→ data-plane service kindvoice; default the audio pipeline (PCM16 @ 24 kHz,server_vad,whisper-1, DragonHD default voice); v1 = implicitmodel_type: managed.agent_api/models.go,operations.go):VoiceAgentDefinitionwire structs +CreateVoiceAgentwith the required preview headerFoundry-Features: VoiceAgents=V1Preview.project/agent_definition.go,service_target_agent.go): voice-aware definition read/write + an isolateddeployVoiceAgentdeploy path — the existing hosted/container path is byte-for-byte unchanged.cmd/init.go,init_from_templates_helpers.go):--kind/--model/--voiceflags, voice manifest synthesis, and a new interactive prompt option.azure.ai.agent.jsongains theprompt-voicekind + voice service properties; unit tests cover the map translation, voice-type selection, and manifest parse/validation.Compatibility
All changes are additive — new
casebranches with untoucheddefaults andomitemptyfields. Existingazd init/ invoke / deploy / list flows and the hosted/workflow code paths are unchanged.Testing
Automated
go build ./...,go vet ./..., andcspellclean.map_voice_test.go,parse_voice_test.go,init_test.go,nextstep/state_test.go,voice_deploy_dispatch_test.go) plus the existing suite pass.Live validation
CreateVoiceAgentcode appears in the project list and its storeddefinitionis byte-identical to the service's own managed reference agent (kind:voice/model_type:managed/ audio pipeline / voice config). Connect reachessession.created.How to test end-to-end
Build the branch binary and install the branch build of the
azure.ai.agentsextension first:1) Interactive (recommended)
azd ai agent initfollows the same prompt flow as other agent kinds (agent name, then the standard Foundry-project selection).--model(defaultgpt-realtime) and--voiceare flags, not prompts — pass them on the same command to override, e.g.azd ai agent init --model gpt-realtime --voice alloy.azd provisionandazd deployadd no voice-specific interaction — they behave exactly like the existing hosted/code experience.infra/is required. Wheninfra/is absent, the Foundry provider synthesizes the embedded ARM template in-memory and resolves the required parameters via the interactive prompts.--infrais optional (only needed to eject IaC to disk).azure.yamlservice:host: azure.ai.agent,kind: prompt-voice,modelType: managed,model: { id: gpt-realtime }(avoice:field appears only when--voicewas passed).2) Non-interactive (CI / scripted)
--no-promptcannot prompt for the three required provision parameters, so eject IaC once and set them explicitly:3) Runtime check (client)
azd-native text
invokeis out of scope for this PR (managed voice agents are Voice Live realtime over WebSocket; the/voice_agentsdata plane has no HTTP text-invoke endpoint). Validate runtime instead with a voice client connecting to the deployed agent — a session reachessession.createdand supports voice/text turns. A read-only control-planeGET /voice_agents/<name>(with headerFoundry-Features: VoiceAgents=V1Preview) is a quick sanity check: expectstate=enabled,definition.model_type=managed.Known gaps
session.audio.output.voicestring coercion at the Voice Live layer, and a demo-clientsession.updateframing issue) — being investigated; not specific to this change.Follow-ups
self_deployed) model typeinvoke,list, and Portal Playground link for voice agentsazd ai agent deletefor voice services — the current delete path targets/agents/<name>and returnsCodeAgentNotFound(404) for a voice agent, which lives under/voice_agents/<name>. Voice-aware teardown (and an idempotent redeploy/update path over the create-only/voice_agentsroute) is a follow-up.AZURE_VOICE_OVERRIDDEN_HOST/x-ms-overridden-hostonce the public Foundry APIM voice route is generally rolled out; will be filed as a tracking issue.