Release a stuck apply lock

Goal

For platform teams. In this task you free the apply lock of an application and an environment, so that the next apply can start. The release does not stop work that a provisioner is already doing.

Before you start

You have the role operator. A person with only the role owner:<team> gets 403 and the text Operator role required.
You have the CLI, and you are signed in. Sign in and get access shows how.
You know which apply is stuck. The usual sign is a 409 with the title Apply already in progress that a feature team reports.

Steps

Know the lock rule

The control plane lets one apply run at a time for each application and each environment. The apply holds the lock while its run has the status running or accepted.

A second apply for the same application and environment is refused at once with 409. The text names the run that holds the lock.
A plan never takes the lock, and a lock never blocks a plan.
A different environment of the same application, and a different application, have their own lock.

This is what a feature team sees:

error: the control plane could not apply joomiz.yaml (HTTP 409)
  Apply already in progress
  another apply is in progress for this application and environment: run b631ac57-8f93-4f0f-8735-5546572dee44, started at 2026-10-28T19:16:23.439Z, status accepted
  hint: another apply of this application and environment is running. Follow it with `joomiz runs watch b631ac57-8f93-4f0f-8735-5546572dee44`, and apply again when it has ended.
  If that apply is stuck, an operator can release the lock with `joomiz runs abandon b631ac57-8f93-4f0f-8735-5546572dee44`. That does not stop work in progress.

The same refusal over HTTP is a problem detail with the type tag:joomiz.com,2026:problems/apply-in-progress. The field holder names the run:

{"detail":"another apply is in progress for this application and environment: run b631ac57-8f93-4f0f-8735-5546572dee44, started at 2026-10-28T19:16:23.439Z, status accepted","status":409,"title":"Apply already in progress","type":"tag:joomiz.com,2026:problems/apply-in-progress","holder":{"runId":"b631ac57-8f93-4f0f-8735-5546572dee44","startedAt":"2026-10-28T19:16:23.439Z","status":"accepted"}}
HTTP 409

Find the run that holds the lock

The refusal names the run. You can also list the runs that are not final. A run that waits for a provisioner has the status accepted, and a run that is working has the status running:

joomiz runs --status accepted

ID                                    KIND   APPLICATION                 ENVIRONMENT  STATUS    STARTED
b631ac57-8f93-4f0f-8735-5546572dee44  apply  orders-database (4569ce39)  test         accepted  2026-10-28T19:16:23.439Z

Read the run:

joomiz runs show b631ac57-8f93-4f0f-8735-5546572dee44

run b631ac57-8f93-4f0f-8735-5546572dee44
  kind:         apply
  application:  orders-database (4569ce39-2dcb-4431-ae13-083b06393362)
  environment:  test
  caller:       owner:finance-team
  status:       accepted
  started:      2026-10-28T19:16:23.439Z
  finished:     not yet
  revision:     e4fd59f0-96cf-40a6-88ae-48f304576ad5
  plan:         be7e4eed-c905-4d2c-88f4-54a9217943b3
  apply:        5d498bf7-005f-4923-b2d7-438d23be903d
groups:
  managed-resource/postgres  accepted  9ms  created 1, updated 0, removed 0  operation 6fc7890b-7061-4b52-9b4d-05ae28768fa2

The line caller says which role started the run. This documentation says acted as. The table under groups: has one line for each type of resource of the run, with the state of its work.

Decide if you release the lock

Release the lock only when the run does not end by itself. First read what the control plane does without you:

A run with the status running that writes no event for 15 minutes is marked abandoned by the control plane. The setting joomiz.runs.stale-after changes the 15 minutes, and the control plane looks every minute. Settings reference lists both settings.
A run with the status accepted is not touched by that check. It ends when the provisioner gives a final answer, or when the longest wait that you registered for the provisioner has passed. The longest wait is at most 30 minutes.

If you do not want to wait, release the lock.

Release the lock

Run the command with the ID of the run:

joomiz runs abandon b631ac57-8f93-4f0f-8735-5546572dee44

run b631ac57-8f93-4f0f-8735-5546572dee44 is now abandoned: the apply lock of its application and environment is released.
This does not stop work that is still in progress. A provisioner may still be changing resources; check them before the next apply.

The release does not stop work that is already running. The provisioner may still be changing resources. The control plane only stops treating the run as the holder of the lock. Check the resources before the next apply.

The same release over HTTP is PATCH /runs/{id} with the body {"status":"abandoned"}. The answer is the run with the status abandoned. Control plane API reference shows it.

A person with only the role owner:<team> cannot release a lock:

error: the control plane refused the request (HTTP 403)
  Operator role required
  This operation requires the operator role. Releasing an apply lock is a platform operation, not an application-owner one.

Read what the release did

The run now has the status abandoned, and the line ended says who ended it:

run b631ac57-8f93-4f0f-8735-5546572dee44
  kind:         apply
  application:  orders-database (4569ce39-2dcb-4431-ae13-083b06393362)
  environment:  test
  caller:       owner:finance-team
  status:       abandoned
  started:      2026-10-28T19:16:23.439Z
  finished:     2026-10-28T19:16:26Z
  ended:        abandoned by an operator
  duration:     2.5s
  revision:     e4fd59f0-96cf-40a6-88ae-48f304576ad5
  plan:         be7e4eed-c905-4d2c-88f4-54a9217943b3
  apply:        5d498bf7-005f-4923-b2d7-438d23be903d
groups:
  managed-resource/postgres  accepted  9ms  created 1, updated 0, removed 0  operation 6fc7890b-7061-4b52-9b4d-05ae28768fa2
This run was abandoned. Some changes may have been made. Run "joomiz apply" again to find out; applying twice is safe. If another apply changed this environment since, apply refuses the plan as stale; plan again first.

The log of the run keeps the event run_abandoned with the word operator. A run that the control plane abandoned says abandoned by the system. This run wrote no event for longer than joomiz.runs.stale-after, because its control plane stopped in the middle of the call:

run 401e08b9-db9e-40e4-b42f-8844df431b31
  kind:         apply
  application:  sweep-demo (0008a264-25c7-4502-94bf-e237fe92e034)
  environment:  test
  caller:       owner:finance-team
  status:       abandoned
  started:      2026-10-28T21:15:05.822Z
  finished:     2026-10-28T21:21:24.120Z
  ended:        abandoned by the system
  duration:     6m 18s
  revision:     42f2ffc0-9840-4d7c-bf47-99a3ab9e3697
  plan:         49d2e925-84af-485b-9ca8-ffcf62b5cb75
groups:
  managed-resource/postgres  unknown  (the control plane stopped before this group finished)
This run was abandoned. Some changes may have been made. Run "joomiz apply" again to find out; applying twice is safe. If another apply changed this environment since, apply refuses the plan as stale; plan again first.

The type that was running shows the state unknown. The control plane does not know what the provisioner did. The run is abandoned in the same way, and the advice at the end is the same. The audit trail records each release by an operator with the action run.abandoned and the ID of the operator. Read and manage the audit trail shows how to read it.

The provisioner keeps working. When the provisioner gives its answer, the control plane still writes the event group_resolved to the run, and the status of the run stays abandoned:

watching run b631ac57-8f93-4f0f-8735-5546572dee44
#1  2026-10-28T19:16:23.439Z  run_started  status running
#2  2026-10-28T19:16:23.442Z  group_queued  managed-resource/postgres
#3  2026-10-28T19:16:23.443Z  group_running  managed-resource/postgres
#4  2026-10-28T19:16:23.452Z  group_finished  managed-resource/postgres  accepted  9ms  created 1, updated 0, removed 0  operation 6fc7890b-7061-4b52-9b4d-05ae28768fa2
#5  2026-10-28T19:16:23.454Z  run_finished  status accepted
#6  2026-10-28T19:16:26Z  run_abandoned  status abandoned  abandoned by an operator
#7  2026-10-28T19:17:37.875Z  group_resolved  managed-resource/postgres  succeeded  1m 14s  operation 6fc7890b-7061-4b52-9b4d-05ae28768fa2
run b631ac57-8f93-4f0f-8735-5546572dee44 finished with status abandoned

A second release is refused with 409 and the text Run cannot be abandoned:

error: the control plane refused the request (HTTP 409)
  Run cannot be abandoned
  the run is abandoned; only a running or accepted run can be abandoned

Check

Ask the feature team to run joomiz apply --environment <name> again. The control plane accepts it, and it starts a new run:

error: the control plane could not fully apply joomiz.yaml to environment test (via https://joomiz.example.com/apply?environment=test)
  managed-resource/postgres: pending — not applied yet: the provisioner accepted these changes and is still making them, as operation 5ca4af2b-e650-47ae-aca8-72a206b05689
    orders-db (updated): failed — not applied yet: the provisioner accepted these changes and is still making them, as operation 5ca4af2b-e650-47ae-aca8-72a206b05689
apply failed (plan 78394874-5d29-41a4-abc1-4d1c43ce8bbf, apply 69dbfd96-6c9f-4742-927e-8cf4b634bba0)

List the runs of the environment. The released run has the status abandoned, and the new run has its own ID:

ID                                    KIND   APPLICATION                 ENVIRONMENT  STATUS     STARTED
ad9dee87-765a-439f-bfea-bbb0ef17ad57  apply  orders-database (4569ce39)  test         accepted   2026-10-28T19:28:13.880Z
63d8876f-f99a-41fc-9947-513aa0f8b039  apply  orders-database (4569ce39)  test         abandoned  2026-10-28T19:27:09.971Z
23d1567d-9f0b-4872-9755-489697834134  plan   orders-database (4569ce39)  test         succeeded  2026-10-28T19:25:56.614Z

Look at the resources that the provisioner handles. Tell the feature team if something is half done. Applying the same plan again is safe, because every provisioner must give the same end state when it repeats a change. If another apply changed the environment in the meantime, the control plane refuses the old plan as stale, and the team plans again.

Troubleshooting

Symptom	Cause	Fix
`403` and `Operator role required`	Your sign-in has only the role `owner:<team>`.	Sign in with the role `operator`.
`404` and `Run not found`	The ID is wrong, or the run does not exist.	Copy the ID from the refusal or from `joomiz runs`.
`409` and `Run cannot be abandoned`	The run already has a final status.	Read it with `joomiz runs show`. The lock is already free.
The next apply is refused again at once	Another apply holds the lock now.	Read the run that the text names, and decide again.
The next apply is refused as stale	Another apply changed the environment after the plan.	Ask the team to run `joomiz plan` and then `joomiz apply`.
`503` and `audit-unavailable`	The control plane could not write the audit event, so it did not release the run.	Try again later.

Find out what went wrong in a run in Investigate a failed run.