Skip to content

Support model discovery and in-session switching for BYOM providers #4376

Description

@Omzig

Describe the feature or problem you'd like to solve

Copilot CLI's BYOM configuration currently requires a single COPILOT_MODEL value. When using Google Vertex AI through its OpenAI-compatible endpoint and gcloud authentication, users must restart Copilot CLI to change models.

This is a gap in the model-selection experience: a Copilot subscription exposes multiple GitHub-hosted models through the model picker, but BYOM is effectively one provider/model per process. A subscription-based model experience should support many available models behind one provider configuration (many-to-one), rather than requiring a separate CLI process for each model (one-to-one).

Think BYOMS (Bring your own model subscription) instead of a single BYOM.

Proposed solution

Support model discovery and model selection for BYOM providers, including Vertex AI:

  • Discover models exposed by the configured provider, or allow a provider-specific model catalog/configuration.
  • Show available BYOM models in the same model picker used for GitHub-hosted models.
  • Allow switching models during a Copilot CLI session without restarting the CLI.
  • Preserve provider-level authentication and endpoint settings while changing only the selected model.
  • Support short-lived Google OAuth access tokens from gcloud/ADC without requiring users to manually restart the CLI when a token refresh is needed.

For Vertex AI, this should work with the OpenAI-compatible endpoint and Google Cloud authentication, while respecting the models enabled and authorized for the user's project and region.

Example prompts or workflows

copilot
# Model picker shows GitHub models plus authorized Vertex AI models.
# Selecting a different model changes the active BYOM model in the current session.

Additional context

Current workaround:

$env:COPILOT_PROVIDER_BASE_URL = "https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/endpoints/openapi"
$env:COPILOT_PROVIDER_API_KEY = gcloud auth print-access-token
$env:COPILOT_MODEL = "google/gemini-2.5-flash"
copilot

Changing COPILOT_MODEL requires restarting Copilot CLI. Related issue: #3399 (custom headers for BYOK).

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions