Retesting After Robot Software Updates: Scope and Evidence Guide

The scope of retesting after a software update depends on the components that were updated and the nature of the changes. Application layer updates that modify task execution logic require retesting of the acceptance criteria that govern task execution behavior: throughput, error rate, task completion accuracy, and any specific workflow scenarios that were part of the original acceptance test set. Navigation stack updates require retesting of navigation accuracy, path planning behavior, obstacle avoidance, and any scenarios that involve complex navigation challenges specific to the deployment environment.

Platform or OS updates require retesting of any criteria that depend on system call behavior, networking behavior, or resource allocation, which may include timing-sensitive criteria and integration criteria. Perception stack updates require retesting of all criteria that depend on the accuracy of object detection, environment mapping, or scene understanding. The scope assessment should be documented with the rationale for each included and excluded criterion, enabling a reviewer to understand why specific criteria were or were not included in the retest plan.

Many software updates for robot systems update not only the primary application or platform software but also underlying dependencies: libraries, data files, perception models, and configuration schemas. A dependency version change can affect acceptance criteria that depend on the dependency's behavior even if the primary software's own code changes are minor. A navigation software update that also updates the planning library may change path planning behavior in ways that are not described in the primary software's change notes.

A perception software update that includes a new version of the object detection model may change detection behavior in ways specific to the facility's environment and object types. The retest scope assessment should extend to dependency changes that are included in the software update package, not only to the changes in the primary software code. For software updates delivered as monolithic packages without detailed component-level change notes, the assessment may need to consult the manufacturer for information about which dependencies changed and the nature of the changes.

Software updates to a robot that include changes to integration interfaces - the APIs, data formats, or communication protocols through which the robot interacts with facility systems - require retesting of the integration behaviors that were verified at acceptance. An integration interface change that breaks backward compatibility with the facility system's integration configuration will manifest as integration failures: tasks not received, status messages not accepted, or error responses from the facility system that the robot does not handle correctly.

Even interface changes that are described as backward compatible may produce behavioral differences in specific scenarios that were not tested in the manufacturer's compatibility testing. Integration retesting after a software update should cover the full integration test set from the original acceptance test plan, since integration behaviors are sensitive to the complete set of changes in the update, not only the changes explicitly described as integration-related.

For deployments with multiple integration points, the retest plan should prioritize the most frequently used and most operationally critical integrations.

Retesting software updates in a production deployment environment raises operational disruption concerns: the robot is taken out of service for retesting, which reduces throughput during the testing period. For deployments where operational continuity is important, testing updates in a staged environment before deploying to production is the preferred approach. A staged environment for software update retesting should replicate the production deployment environment as closely as possible: same facility layout, same integration systems at their production versions, and same operational workflow scenarios.

Testing in a staged environment and documenting the test results before deploying to production allows the deploying team to confirm the update is compatible with the acceptance criteria before any operational disruption occurs. For deployments without staged environments, retesting in the production environment should be scheduled during low-activity periods and the retesting scope should be managed to minimize the time the robot is out of production use.

Robot software updates in many deployments are frequent: the robot manufacturer may issue minor releases on a monthly or quarterly cadence. Managing a retest process for every minor update is operationally intensive, and a practical risk-based approach is needed. Not every update requires the same level of retesting: a security patch that does not change functional behavior may require only a basic functional confirmation; a minor tuning update in the navigation layer may require retesting only of the navigation criteria most sensitive to the changed parameters; a major release that revises multiple components requires a comprehensive review of the full acceptance test set.

A risk-based retesting policy, documented and agreed with the customer, defines the retesting requirements for different categories of software updates. This policy allows the deploying team to apply routine updates without full acceptance retesting while requiring comprehensive retesting for significant changes. The policy itself, together with each update's classification under the policy, is part of the deployment record.

• Review software update change notes and identify acceptance criteria that may be affected by the changes
• Extend the scope assessment to dependency version changes included in the update package
• Include integration criteria in the retest plan for any update that modifies integration interfaces
• Test updates in a staged environment before deploying to production where possible
• Capture retest evidence in formats consistent with the original acceptance test evidence
• Investigate any criterion that fails or produces a degraded result in the updated software
• Define and document a risk-based retesting policy that scales the retest scope to the significance of each update