Robot Acceptance Retest Evidence: Building the Re-Verification Record

Acceptance retest evidence must demonstrate three things to be complete. First, it must demonstrate that the corrective action was applied: the software version, configuration state, or hardware identity at the time of retesting must confirm that the change intended to address the original failure is present in the tested configuration. This is the anti-regression check that ensures the retest is measuring the performance of the corrected system, not a system that still has the original problem.

Second, it must demonstrate that the retest was conducted under conditions that reproduce the original failure scenario: the same operational scenario, the same environmental conditions, and the same success criteria as the original acceptance test for the criterion being retested. This is what makes the retest result directly relevant to the original acceptance issue. Third, it must demonstrate the outcome: a pass result against the specified acceptance criterion, captured in a primary evidence format that a reviewer can inspect independently of the deployment team's assertion that the test passed.

The most defensible retest evidence is produced when the retest conditions match the conditions of the original acceptance test for the same criterion. When conditions must differ from the original test, the differences should be documented with an explanation of why they differ and an assessment of whether the different conditions are more or less demanding than the original. In some cases, exactly reproducing the original conditions is not possible: a failure that occurred during a specific operational event that cannot be staged exactly may require an approximation of the original conditions.

In these cases, the retest plan should document what approximation was used, why it is a valid approximation of the original failure conditions, and any limitations on the comparison between the retest result and the original acceptance criterion. A retest conducted under conditions that are less demanding than the original failure scenario produces weaker evidence than one conducted under conditions that match or exceed the original scenario.

Retest evidence should be organized explicitly for comparison with the original acceptance test evidence for the same criterion. A side-by-side comparison record that shows the original test conditions, the original test result, the corrective action applied, the retest conditions, and the retest result is the most useful format for acceptance reviews, because it allows the customer to assess both the original problem and the resolution in a single view.

For quantitative criteria - positioning accuracy, cycle time, throughput - the retest evidence should include the same metrics as the original test, measured in the same way, so that the comparison is numerical rather than qualitative. For qualitative criteria - correct error handling, appropriate response to specific scenarios - the retest evidence should include the same observations categories as the original test record, with the updated results noted for each.

Where the retest evidence is in a different format from the original, the reason for the format difference should be documented and the mapping between the formats should be explained.

Some acceptance issues are addressed through workarounds - operational or configuration adjustments that prevent the original failure from occurring without addressing its root cause. When a workaround is used as the corrective action, the retest evidence documents the performance of the system with the workaround in place. This is an appropriate test of whether the workaround achieves the acceptance criterion in practice, but the retest evidence record must explicitly note that the corrective action is a workaround rather than a root cause fix.

The workaround's implications must also be documented: if the workaround changes the robot's operational envelope - for example, by restricting it from a zone where the original failure was observed - the acceptance record must reflect this changed operational envelope. A customer accepting a system where a workaround is in place needs to know that the system's operation is contingent on the workaround and that the workaround represents an accepted limitation rather than a verified resolution.

An acceptance issue is closed when the retest evidence is reviewed by the deployment lead and confirmed to meet the closure standard: passing evidence for the retested criterion, captured under appropriate conditions, with the corrective action confirmed as applied. The closure review should also confirm that the retest evidence is organized and accessible in the acceptance package, not stored in a location that is accessible during the deployment but may not be accessible to the customer or to future reviewers after project handover.

Closed acceptance issues should be linked in the acceptance record to the original failure observation, the corrective action, the retest plan, and the retest evidence, creating a complete chain from problem to resolution that any reviewer can follow. This chain is what transforms the acceptance record from a list of open and closed issues into a defensible document that answers the question of how every problem identified during deployment was verified as resolved.

• Confirm the corrective action is applied by verifying the configuration state before beginning the retest
• Define the retest conditions to match the original failure scenario conditions as closely as possible
• Document any differences between the retest conditions and the original test conditions with an explanation
• Capture retest evidence in a format that enables direct comparison with the original acceptance test evidence
• Note explicitly in the retest record whether the corrective action is a root cause fix or a workaround, and document the workaround's implications if applicable
• Conduct a closure review before marking the acceptance issue as closed
• Link the closed acceptance issue record to the original observation, corrective action, retest plan, and retest evidence