Verify by read-back

The single rule that shaped mcidrac’s write paths:

A step is not “ok” because the BMC accepted the request. It is “ok” because the state read back correct.

The gap between accepted and applied

BMCs are full of ways for a 200 to lie. Dell validates BIOS values when the config job runs during POST — not when you PATCH them — so a typo’d enum “succeeds” now and fails minutes later. A PATCH to several iDRAC attributes where one is read-only returns 200 with the complaint tucked into @Message.ExtendedInfo (at the top level of the body, a different place than errors use). InsertMedia can accept a URL the iDRAC then fails to stream. An eject can be acknowledged and not happen.

For a human clicking a UI, these gaps are annoying. For an LLM operator that chains the next destructive action on the word “applied,” they’re dangerous — which is why every mcidrac mutation closes the gap with a read-back:

• idrac_attributes_set re-reads the attributes and diffs — the result is applied only if every value matches, otherwise partial with wanted-vs-actual
• idrac_vmedia_mount confirms Inserted is true before saying “inserted (verified)”; unmount confirms the device is empty
• idrac_boot_to_iso re-reads the OEM boot attributes and refuses to issue the reset until they verify

Never lose the ledger

Multi-step workflows have a second failure mode: step 3 fails after steps 1–2 changed the world, and the error message describes only step 3. idrac_boot_to_iso ejects media, mounts new media, sets boot attributes, and reboots — if the boot-attribute PATCH fails, the caller must know the old media is already gone and a new ISO is already mounted.

So failures carry the ledger: every completed step, in the error itself, with a warning that the BMC is in a partial state and what to inspect before retrying. The same principle keeps idrac_bios_set honest when the optional reboot fails — the config job it created is still queued and will fire at the next boot, so the tool says exactly that instead of letting the reboot error swallow the job ID.

Where this came from

The first implementation returned “ok” when HTTP calls succeeded, like most tooling does. A rigorous failure-mode review (in the Margaret Hamilton tradition: verify state, then report it) found the gap in nearly every mutating tool, and the fix became the house rule. The behavioral simulator in the test harness exists largely to keep it enforced — its fake iDRAC rejects read-only attributes and refuses double-mounts precisely so the tests fail if a tool ever stops checking.