Skip to content

[Container Apps] Fix #10103: az containerapp up: Fail closed when app lookup fails - #10519

Draft
Aditya Pujara (a0x1ab) wants to merge 1 commit into
Azure:mainfrom
a0x1ab:agent-assist/azure-azure-cli-extensions-issue-10103-18e50bd97a6d
Draft

Aditya Pujara (a0x1ab) wants to merge 1 commit into
Azure:mainfrom
a0x1ab:agent-assist/azure-azure-cli-extensions-issue-10103-18e50bd97a6d

Conversation

@a0x1ab

@a0x1ab Aditya Pujara (a0x1ab) commented Oct 25, 2026 •

Copy link
Copy Markdown
Member

🤖 PR Validation — ⚠️ Review suggested

Breaking Changes Suppressions
️✔️ None ⚠️ 1 added

This PR adds warning or check suppressions. Fix the underlying warning where possible and justify each remaining suppression in a code comment or the PR description.

1 added suppression
File Line Kind Code
src/containerapp/azext_containerapp/_up_utils.py 479 pylint def create(self, no_registry=False, apply_image_defaults=True): # pylint: disable=arguments-differ

Description

Fixes #10103.

Fail closed when app lookup fails.

Root cause: In src/containerapp/azext_containerapp/_up_utils.py, ContainerApp inherited Resource.get, which swallowed every lookup exception and reported absence. In src/containerapp/azext_containerapp/custom.py, containerapp_up_logic independently swallowed every show failure and selected create. ContainerApp.create also used a suppressing core existence helper solely for logging. This allowed a transient ARM lookup failure to select full-spec creation against an existing app. Image-specific ingress defaults were applied before knowing whether the app existed. The restored candidate already fixes these paths; this repair additionally explains its arguments-differ suppression …

Fix: Require explicit HTTP 404 for app absence during preflight and final create/update selection; propagate uncertain errors unchanged and reject empty successful responses. Preserve successful-show updates and defer quickstart/foundry ingress defaults until confirmed creation. Remove the logging-only suppressing existence probe and retain cloud-build placeholder behavior. Add focused unit and live scenario coverage, patch release 1.3.1b1, and explain the existing app-specific create signature suppression.

Origin: Supplied history attributes containerapp_up_logic's create/update branch to 2f16dd4 (PR #4837, 2022-05-23) and the Resource/ContainerApp wrappers to cf4bdeb (PR #4726, 2022-04-29). Their intent was to create an absent app or update a present app. The fix preserves that intent but fails closed when state is unknown. No Git metadata exists here …

Why now: A transient ARM/runner connectivity failure exposed a latent catch-all branch. The supplied runner logs suggest an infrastructure incident but do not establish its exact cause. No dependency bump has been proven to cause this failure.

Blast radius: Only ContainerApp's lookup override changes Resource.get behavior; ContainerAppEnvironment and other Resource subclasses are unchanged. Related commands/flows are az containerapp up with images, source, repository deployments, and cloud-build placeholders. Uncertain app lookups now stop rather than provisioning; explicit 404 creation and successful-show updates remain. Fixing only the final decision would leave …

How verified (trusted validation output):

  • Trusted validation passed (1 command).
  • New or changed tests fail on the base revision and pass with the change: src/containerapp/azext_containerapp/tests/test_containerapp_up_unit.py::ContainerAppUpCommandUnitTest::test_app_appearing_after_preflight_updates_without_defaults, src/containerapp/azext_containerapp/tests/test_containerapp_up_unit.py::ContainerAppUpCommandUnitTest::test_final_lookup_failure_never_deploys_after_successful_preflight, src/containerapp/azext_containerapp/tests/test_containerapp_up_unit.py::ContainerAppUpCommandUnitTest::test_preflight_failure_stops_before_provisioning.
Before After
Reproduced: bash: python - <<'REPRO' import os import sys from unittest.mock import patch from requests.exceptions import … Not reproduced: PR #10519 @ 2b7db3c, extension 1.3.1b1: Not reproduced (exit 0)

Reproduction run: https://github.com/Azure/issue-sentinel/actions/runs/40193867782

Risks and limitations (from the job report):

  • Required self-check now passes with authentic upstream baseline comparison, exit 0 and zero findings. Existing …
  • The self-check receipt reports azdev-linter not run because of an azdev linter error, and base-only …
  • New live ScenarioTests require recordings from the live-test lane; none were created or changed here. …

This checklist is used to make sure that common guidelines for a pull request are followed.

Related command

az containerapp up

General Guidelines

  • Have you run azdev style <YOUR_EXT> locally? (pip install azdev required)
  • Have you run python scripts/ci/test_index.py -q locally? (pip install azdev required)
  • My extension version conforms to the Extension version schema

For new extensions:

About Extension Publish

There is a pipeline to automatically build, upload and publish extension wheels.
Once your pull request is merged into main branch, a new pull request will be created to update src/index.json automatically.
You only need to update the version information in file setup.py and historical information in file HISTORY.rst in your PR but do not modify src/index.json.

Independent review: an independent reviewer checked this change against 11 acceptance criteria derived from the issue (11 required); every required criterion is backed by evidence in the diff.

…hen app lookup fails

Root cause: In src/containerapp/azext_containerapp/_up_utils.py,
ContainerApp inherited Resource.get, which swallowed every lookup
exception and reported absence. In
src/containerapp/azext_containerapp/custom.py, containerapp_up_logic
independently swallowed every show failure and selected create.
ContainerApp.create also used a suppressing core existence helper solely
for logging. This allowed a transient ARM lookup failure to select
full-spec creation against an existing app. Image-specific ingress
defaults were applied before knowing whether the app existed. The
restored candidate already fixes these paths; this repair additionally
explains its arguments-differ suppression …

Change: Require explicit HTTP 404 for app absence during preflight and
final create/update selection; propagate uncertain errors unchanged and
reject empty successful responses. Preserve successful-show updates and
defer quickstart/foundry ingress defaults until confirmed creation.
Remove the logging-only suppressing existence probe and retain
cloud-build placeholder behavior. Add focused unit and live scenario
coverage, patch release 1.3.1b1, and explain the existing app-specific
create signature suppression.

Origin: Supplied history attributes containerapp_up_logic's
create/update branch to 2f16dd4 (PR Azure#4837, 2022-05-23) and the
Resource/ContainerApp wrappers to cf4bdeb (PR Azure#4726, 2022-04-29).
Their intent was to create an absent app or update a present app. The
fix preserves that intent but fails closed when state is unknown. No Git
metadata exists here …

Why now: A transient ARM/runner connectivity failure exposed a latent
catch-all branch. The supplied runner logs suggest an infrastructure
incident but do not establish its exact cause. No dependency bump has
been proven to cause this failure.

Blast radius: Only ContainerApp's lookup override changes Resource.get
behavior; ContainerAppEnvironment and other Resource subclasses are
unchanged. Related commands/flows are az containerapp up with images,
source, repository deployments, and cloud-build placeholders. Uncertain
app lookups now stop rather than provisioning; explicit 404 creation and
successful-show updates remain. Fixing only the final decision would
leave …

Alternatives considered:
- Changing Resource.get globally was rejected because it would alter
  unrelated resource wrappers.
- Fixing only containerapp_up_logic was rejected because earlier
  preflight could still provision after an uncertain lookup.

Verified:
- Trusted validation passed (1 command).
- New or changed tests fail on the base revision and pass with the
  change:
  `src/containerapp/azext_containerapp/tests/test_containerapp_up_unit.py::ContainerAppUpCommandUnitTest::test_app_appearing_after_preflight_updates_without_defaults`,
  `src/containerapp/azext_containerapp/tests/test_containerapp_up_unit.py::ContainerAppUpCommandUnitTest::test_final_lookup_failure_never_deploys_after_successful_preflight`,
  `src/containerapp/azext_containerapp/tests/test_containerapp_up_unit.py::ContainerAppUpCommandUnitTest::test_preflight_failure_stops_before_provisioning`.
- Unit tests: 17 executed, 0 skipped.
- flake8 and pylint passed.
- Pre-publication gates passed: release-artifact, generated-ownership,
  command-help, suppression, regression-test, independent-review.
- Scenario recordings must be refreshed by a live test run.
- Repository CI and live tests run on this PR.

Fixes Azure#10103
@x-engineering-agent x-engineering-agent Bot added X Engineering Agent In-Progress The agent is still preparing and validating this draft X Engineering Agent Generated Agent opened this PR do-not-merge labels Oct 25, 2026
@x-engineering-agent
x-engineering-agent Bot requested a review from a team October 25, 2026 15:42
@a0x1ab
Aditya Pujara (a0x1ab) force-pushed the agent-assist/azure-azure-cli-extensions-issue-10103-18e50bd97a6d branch from 01f9710 to 2b7db3c Compare October 25, 2026 16:01
@x-engineering-agent x-engineering-agent Bot added Request X Engineering Agent Ask the agent to triage this issue or review this PR and removed X Engineering Agent In-Progress The agent is still preparing and validating this draft labels Oct 25, 2026
@x-engineering-agent

x-engineering-agent Bot commented Oct 25, 2026 •

Copy link
Copy Markdown

X Engineering Agent Pull request review: Automated repair stopped

Automated repair stopped

Aditya Pujara (@a0x1ab)

Automated repair stopped after 3 attempts
Review

This PR is still failing after three automated attempts. The fail-closed lookup handling addresses ambiguous lookup failures and restricts implicit ingress defaults to creation, but regression validation remains incomplete.

Upstream CI

Azure.azure-cli-extensions, build 365639 failed, including these checks:

  • Azure.azure-cli-extensions (Integration Tests, Build Tests Python314)
  • Azure.azure-cli-extensions (Integration Tests, Build Tests Python312)
  • Azure.azure-cli-extensions (Integration Tests, Build Tests Python310)
  • Azure.azure-cli-extensions (Integration Tests, Build Tests Python313)
  • Azure.azure-cli-extensions (Integration Tests, Build Tests Python311)

Relevance: PR-related for the newly added test; other failures remain uncertain. Each job quotes FAILED src/containerapp/azext_containerapp/tests/latest/test_containerapp_up.py::ContainerAppUpImageTest::test_containerapp_up_image_preserves_existing_configuration. Each reports 22 failed tests, including existing registry/job scenarios. The supplied traceback does not establish the underlying cause. Inspect the added test's detailed failure before remediation; replay that test and rerun all five integration jobs. Do not change unrelated job code without supporting diagnostics.

Test validation

  • Live test: Failed with conclusion failure. Workflow run
  • Regression coverage: Not applicable to the changed files.

Agent live-test relevance: uncertain. The supplied evidence is only "the run concluded failure"; it does not identify the failing step or establish whether scenario tests executed. Inspect the existing workflow run, resolve its demonstrated cause, and rerun the focused ContainerAppUpImageTest scenarios. Infrastructure failures require rerun/escalation, not speculative source changes.

Confirmed recording gap

P2 - Test semantic-strength: src/containerapp/azext_containerapp/tests/latest/test_containerapp_up.py:49,108,169 adds three playback-enabled scenarios without matching cassettes at this head. Record and commit these files under src/containerapp/azext_containerapp/tests/latest/recordings/:

  • test_containerapp_up_image_preserves_existing_configuration.yaml
  • test_containerapp_up_image_preserves_custom_domain_binding.yaml
  • test_containerapp_up_image_creates_missing_app_with_defaults.yaml

Despite the structured coverage applicability result above, issue-scenario regression coverage is not verified. The scenarios contain meaningful assertions, but absent recordings leave their request/response sequences unverified in playback. Replay all three identically named ContainerAppUpImageTest methods, preserving assertions for environment variables, secret values, managed identity, ingress/port 3000, custom-domain certificate binding, and confirmed-404 creation with external ingress/port 80. The unchanged recordings/test_containerapp_up_image_e2e.yaml covers 404 creation, not preservation of an existing configuration. Rerun src/containerapp/azext_containerapp/tests/test_containerapp_up_unit.py; its mocked lookup/error/dispatch checks alone do not prove ARM configuration preservation. Missing cassettes are a confirmed gap, not an established explanation for every failure.

No additional production defect or unresolved Copilot reviewer feedback was confirmed.

Question for maintainers

Will the Container Apps owning squad take over repairing the missing scenario recordings and diagnosing the outstanding CI/live-test failures?

Risk assessment

48/100 · Medium

The Medium rating is driven by public CLI behavior, dependency or supply chain, failure-handling behavior.

  • Change scope: 6 changed files, 490 changed lines (+462 / -28), including 3 production files.
  • Affected components: containerapp
  • Risk drivers: public CLI behavior (+18); dependency or supply chain (+18); failure-handling behavior (+12)
  • Regression evidence: Changed regression tests are included, reducing risk.
  • Confidence: High because changed-line patches were available for every production file.
  • Required review: Owning-squad review is required for containerapp before merge.

Posted by x-engineering-agent (Reviewer)

Live test — Failed

FAIL (exit 1)

Selectors: azext_containerapp.test_containerapp_up_image_preserves_existing_configuration azext_containerapp.test_containerapp_up_image_preserves_custom_domain_binding azext_containerapp.test_containerapp_up_image_creates_missing_app_with_defaults (extension)
PR head ref: agent-assist/azure-azure-cli-extensions-issue-10103-18e50bd97a6d
PR head sha: 2b7db3c178d8eb3fbe567a017b09adfc7b40716b
PR base ref: main
New test files in PR: false

Changed test files run
src/containerapp/azext_containerapp/tests/latest/test_containerapp_up.py

Workflow run: https://github.com/Azure/issue-sentinel/actions/runs/40870107709

Live-test recordings: 2 regenerated — archived in workflow artifact live-test-pr-10519 (recordings/).

The recording was not committed because the live test failed. Fix the test failure before publishing its regenerated cassette.

Recording files
src/containerapp/azext_containerapp/tests/latest/recordings/test_containerapp_up_image_preserves_custom_domain_binding.yaml
src/containerapp/azext_containerapp/tests/latest/recordings/test_containerapp_up_image_preserves_existing_configuration.yaml
Last 80 lines of azdev output
    @ResourceGroupPreparer(location="eastus2")
    def test_containerapp_up_image_preserves_custom_domain_binding(self, resource_group):
        env_name = self.create_random_name(prefix='env', length=24)
        app_name = self.create_random_name(prefix='containerapp', length=24)
        original_image = "mcr.microsoft.com/k8se/quickstart"
        updated_image = "mcr.microsoft.com/k8se/quickstart:latest"
>       self.cmd(f'containerapp env create -g {resource_group} -n {env_name} --logs-destination none')

azure-cli-extensions/src/containerapp/azext_containerapp/tests/latest/test_containerapp_up.py:113: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
azure-cli/src/azure-cli-testsdk/azure/cli/testsdk/base.py:177: in cmd
    return execute(self.cli_ctx, command, expect_failure=expect_failure).assert_with_checks(checks)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
azure-cli/src/azure-cli-testsdk/azure/cli/testsdk/base.py:252: in __init__
    self._in_process_execute(cli_ctx, command, expect_failure=expect_failure)
azure-cli/src/azure-cli-testsdk/azure/cli/testsdk/base.py:315: in _in_process_execute
    raise ex.exception
.venv/lib/python3.12/site-packages/knack/cli.py:233: in invoke
    cmd_result = self.invocation.execute(args)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
azure-cli/src/azure-cli-core/azure/cli/core/commands/__init__.py:677: in execute
    raise ex
azure-cli/src/azure-cli-core/azure/cli/core/commands/__init__.py:820: in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
azure-cli/src/azure-cli-core/azure/cli/core/commands/__init__.py:812: in _run_job
    return cmd_copy.exception_handler(ex)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
azure-cli-extensions/src/containerapp/azext_containerapp/_client_factory.py:28: in _polish_bad_errors
    raise ex
azure-cli/src/azure-cli-core/azure/cli/core/commands/__init__.py:789: in _run_job
    result = cmd_copy(params)
             ^^^^^^^^^^^^^^^^
azure-cli/src/azure-cli-core/azure/cli/core/commands/__init__.py:335: in __call__
    return self.handler(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
azure-cli/src/azure-cli-core/azure/cli/core/commands/command_operation.py:120: in handler
    return op(**command_args)
           ^^^^^^^^^^^^^^^^^^
azure-cli-extensions/src/containerapp/azext_containerapp/custom.py:771: in create_managed_environment
    return create_managed_environment_logic(
azure-cli-extensions/src/containerapp/azext_containerapp/custom.py:857: in create_managed_environment_logic
    r = containerapp_env_create_decorator.create()
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
azure-cli/src/azure-cli/azure/cli/command_modules/containerapp/containerapp_env_decorator.py:178: in create
    handle_raw_exception(e)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

e = HTTPError('Conflict({"error":{"code":"MaxNumberOfRegionalEnvironmentsInSubExceeded","message":"The subscription \'2e4d...ainer App Environments in West US 2. For more information, visit: https://go.microsoft.com/fwlink/?linkid=2208872"}})')

    def handle_raw_exception(e):
        import json
    
        stringErr = str(e)
    
        if "WorkloadProfileNameRequired" in stringErr:
            raise CLIInternalError("Workload profile name is required. Please provide --workload-profile-name.")
    
        if "Error starting job" in stringErr:
            raise CLIInternalError("There was an error starting the job execution. Please check input parameters and try again.")
    
        if "Requested job execution" in stringErr and "not found" in stringErr:
            raise CLIInternalError("Requested job execution not found. Please check input parameters and try again.")
    
        if "{" in stringErr and "}" in stringErr:
            jsonError = stringErr[stringErr.index("{"):stringErr.rindex("}") + 1]
            jsonError = json.loads(jsonError)
    
            if 'error' in jsonError:
                jsonError = jsonError['error']
    
                if 'code' in jsonError and 'message' in jsonError:
                    code = jsonError['code']
                    message = jsonError['message']
>                   raise CLIInternalError('({}) {}'.format(code, message))
E                   azure.cli.core.azclierror.CLIInternalError: (MaxNumberOfRegionalEnvironmentsInSubExceeded) The subscription '2e4d10de-8014-45bd-b341-de3ba295cade' cannot have more than 1 Container App Environments in West US 2. For more information, visit: https://go.microsoft.com/fwlink/?linkid=2208872

azure-cli/src/azure-cli/azure/cli/command_modules/containerapp/_client_factory.py:56: CLIInternalError
!!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!
=================== 1 failed, 1 passed in 170.03s (0:02:50) ====================

Workflow run

X Engineering Agent · Tester · React with +1 or -1 to train the agent


Ask the Agent

Tick a box to queue an action. Requires write access to this repository.

  • Re-review this PR
  • Run Agent live tests on this PR

@x-engineering-agent

Copy link
Copy Markdown

Live reproduction on the PR head is inconclusive

The issue was reproduced live before this change, but the re-run on head 2b7db3c178d8 could not confirm the fix: the PR head could not run the reproduction to the end.

This PR stays a draft until a maintainer verifies the fix (or a new push re-runs the reproduction).

Run: https://github.com/Azure/issue-sentinel/actions/runs/40210011398

@x-engineering-agent x-engineering-agent Bot added the X Engineering Agent Reviewed Agent reviewed this PR label Oct 25, 2026
@yonzhan

Copy link
Copy Markdown
Collaborator

Container Apps

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

act-observability-squad Auto-Assign Auto assign by bot ContainerApp do-not-merge Request X Engineering Agent Ask the agent to triage this issue or review this PR X Engineering Agent Generated Agent opened this PR X Engineering Agent Reviewed Agent reviewed this PR

Projects

None yet

4 participants