Test Analysis – Field Testing Series – Part 5

This post is part of a series on testing robots with a focus on field testing. It will start with why we test robots, work its way through planning for a field test, to performing the test, and finally what to do after the test. We will conclude with a description of many common tests that we subject our robots to and how to perform them.
I hope you enjoy this series and please leave comments below.
Links:
Why Test Robots? – Field Testing Series – Part 1
Designing for Success – Field Testing Series – Part 2
Test Preparation – Field Testing Series – Part 3
Test Execution – Field Testing Series – Part 4
Test Analysis – Field Testing Series – Part 5
Common Robot Tests – Field Testing Series – Part 6
—————————————————–

After a test it is important to compile the results within a short amount of time. If this was an extended test campaign people often want to take a few days off but it is important to take care of the test recap as soon as possible. The test recap does not always include the data that was collected from the robot, in fact it can often take a significant amount of time after the test to get all of the data presented. Some of the items that should be done immediately after the test include revising the manifest that was created when planning for the test so you can account for things you thought you would need but did not (take these items off from the manifest very carefully) and more importantly add items to the manifest the you forgot.

The other big item that should happen right after a test is the creation of a test report. Many times this will be based on the test plan. The test report should have information about relevant conditions (rain, sunny, muddy, sandy, etc..), when the test occurred, where the test occurred, what was done, and what happened. The test report should be created immediately after the test so that information is still fresh in your mind and so that it can be passed on to the rest of the team for analysis.

A lessons learned report is also a valuable outcome from a field test. For short field tests this might be included in the test report however for longer duration field tests it is useful to get the entire team together to document what worked and what did not work.This helps organize what should be maintained and what needs to be improved for the next field test. There will be some things that did not work well and as Brian Wilcox said you must “learn good practice from bad experiences”.

Before going on another field test make sure to fix the underlying problem with the robot. Often times a fuse will blow and be replaced but people will fail to determine and fix the root cause. This can lead to larger problems in the future that could have been handled before the problem grew.

In the field robotics world we are often complacent with results that are no statistically significant. We need to make a concerted effort to analyze test results and determine what is significant and what is not. Software packages that allows data to be plotted with error bars is often a nice way of presenting data in a meaningful way. In addition to building statistical confidence in a robot from testing, regression testing is important to demonstrate that while we improved a system we did not harm other components.

The test is done. The results are in. But lets step back and look at common robot tests in the next post.

As always comments are welcome (and encouraged)

Image Source: http://frc.ri.cmu.edu/depthx/images/drake1.jpg

Liked it? Take a second to support David Kohanbash on Patreon!