22_sharded_eks#3
FAILEDAttribution
22_sharded_eks_3.mdMDevidence:
The render paired psmdb-operator 1.15.0 with percona-server-mongodb 7.0.2-1. MongoDB 7.0 needs operator 1.16 or newer, and 1.15's init runs the legacy mongo shell, which the 7.0 image no longer ships: 'mongo: command not found' 91 times, no replica set ever initiated, the bootstrap job waited 60 minutes. A platform evidence gap made attempt 1 blind — the operator's log was collected only after a restart, and the operator had not restarted — so the first repair re-applied the same pair. The human operator stopped the ladder after that attempt rather than spend about four hours of EKS on a known cause, and banked the row as a failure. Earlier runs of this slot were cut by three off-ledger incidents (an operator IP rotation under a /32 allow-list, a provider credit exhaustion, a single 502 on session create), each re-run.
- date2026-10-26T14:09Z
- deploy statusNone (Nones)
- gate––––healthz=— connection=— logs=— metrics=—
- teardown cleanFalse
resolution:
Fixed (5a7592ef): an operator older than its image's major is refused at plan with the minimum version named, a runtime verdict names the missing shell, and the operator log is read whenever the operator runs, not only after restarts. Scenario 22 passed on its 31 Oct re-run.
Recorded reason (scrubbed)
operator-stopped 19:50 IST: render paired psmdb-operator 1.15.0 with percona-server-mongodb:7.0.2-1 (7.0 needs >= 1.16.0); operator init failed 'mongo: command not found' x91, no replica set initiated, bootstrap waited 60 min. Platform evidence gap: operator log not collected (operator had 0 restarts) so ladder attempt 1 was blind. Fixed 5a7592ef (plan rule + verdict + operator log always read); blind ladder stopped to save ~4 h EKS; 22 re-runs in the re-run pass on the fixed code.