Skip to content
PAUIOps
Open menu

Operate · 11 min

Compose monitoring & recovery

Opt projects into service-level Compose monitoring, understand the recovery ladder, and respond when automatic repair stops.

Check release availability first

Enable one project deliberately

  1. Open the project in Shell.
  2. Find Auto-start / restart below the Published ports service list.
  3. Confirm the project uses the intended Compose file and service set.
  4. Enable the switch as an administrator with Docker access.
  5. Watch the initial checking state resolve to running, stopped, degraded, repairing, retry queued, unavailable, or Manual fix needed.

Understand what Manager checks

SignalWhat PAUIOps does
Missing, created, stopped, exited, or dead serviceMarks the service affected and prepares docker compose up -d for that service
Paused servicePrepares docker compose unpause
Unhealthy, restart-looping, or unresponsive servicePrepares a targeted service restart
Compose network driftTreats a running container detached from its expected network as affected
No Compose healthcheckKeeps state monitoring and can probe published TCP ports; the UI identifies the missing healthcheck
Scale zero, completed one-time job, or pauiops.watch: ignoreLeaves the intentionally non-running service out of automatic recovery

PAUIOps reads the resolved service model from docker compose config and runtime rows from docker compose ps --all. A failed inspection—such as an unavailable Docker daemon, invalid Compose file, or missing project directory—is reported as unavailable and never consumes the recovery budget or issues a repair command.

Resolve declared host-port collisions first

Terminal and Files compare the resolved Compose bindings across projects the current user can access. A collision lists both projects, services, Compose files, protocol, host binding, and published port so the operator can change the intended stack before a start or recovery attempt fails.

Follow the bounded recovery ladder

LevelAllowed response
TargetedUnpause, start, or restart only the affected services
Service upRun up -d for every failing service
Force recreateRun up -d --force-recreate for failing services
Whole projectRun docker compose up -d for the project
Circuit breakerStop issuing commands and show Manual fix needed

Nothing restarts until the configured consecutive-failure threshold is reached. Attempts use cooldown, exponential backoff, a sliding recovery window, dependency-first ordering, and startup grace before the next judgment.

Recover from Manual fix needed

  1. Inspect the affected services, last recovery result, project Compose file, container logs, and Manager journal.
  2. Repair the underlying configuration, image, dependency, network, healthcheck, port, or application problem.
  3. Verify the stack manually.
  4. Toggle Auto-start / restart off and back on. Re-enabling clears the circuit breaker, failure counters, attempt window, recovery level, and remembered one-time-job classification.
Trace a watcher incident
sudo journalctl -u pauiops.service -f | grep "compose watcher"
docker compose ps --all
docker compose logs --since 30m <service>
docker inspect <container> --format '{{json .State}}'

Keep healthchecks in the project

PAUIOps does not edit or inject healthchecks into docker-compose.yml. Add an application-level healthcheck to every important long-running service when a process can remain running while the application is hung. The healthcheck command must exist inside that service image.

Example HTTP healthcheck
services:
  web:
    healthcheck:
      test: ["CMD-SHELL", "wget -qO- http://127.0.0.1/health >/dev/null || exit 1"]
      interval: 30s
      timeout: 5s
      retries: 3
      start_period: 20s