Test Execution – Field Testing Series – Part 4

This post is part of a series on testing robots with a focus on field testing. It will start with why we test robots, work its way through planning for a field test, to performing the test, and finally what to do after the test. We will conclude with a description of many common tests that we subject our robots to and how to perform them.
I hope you enjoy this series and please leave comments below.
Links:
Why Test Robots? – Field Testing Series – Part 1
Designing for Success – Field Testing Series – Part 2
Test Preparation – Field Testing Series – Part 3
Test Execution – Field Testing Series – Part 4
Test Analysis – Field Testing Series – Part 5
Common Robot Tests – Field Testing Series – Part 6
—————————————————–

It is finally time to begin testing. After a lot of preparation the robot is in the field and ready for operations. Many people have an expectation that testing will be very quick and will require minimal time since everything will go as planned. Just about all (if not all) testing veterans can testify that things usually take longer than planned and something will probably go wrong. This is especially true during initial testing of the robot, as the test team gets more experience and the robot gets more testing less things go wrong and tests can be conducted more seamlessly. There are many elements involved in a test and getting them to fall into place seamlessly can require a lot of practice and repetition. Having a team that performed similar experiments multiple times can reduce the test execution time dramatically. For many tests there is a lot of preparation that needs to happen in order for the test to occur. For each test you want to maintain a repeatability in conditions. For tests in sand this might mean having a standard ground preparation or finding areas of sand with similar properties to be used for each of the tests.

A former colleague of mine and army veteran said “most plans don’t survive first contact with the enemy.” It is important to remember this when executing your test plan. Many times plans will need to change in the field in order to insure success. Many field tests require participation from multiple groups, this can add a lot of complexity to the test and demand even more flexibility.

During testing it is important to pay attention to the test plan to make sure tests and schedules are still on track. By actively using a test plan you can reduce the chance of making a mistake of forgetting a step. Creating checklists for common tests is a good idea. One item that is often overlooked is checking the log files during the field tests. There have been many sad stories of people spending a lot of time conducting a field test only to return home and realize that all of the logs are corrupt or the desired data does not appear in the logfile.

Another wise thing that I heard from a colleague is “if you don’t have images it didn’t happen.” While this cannot be taken literally there is a lot of truth to this idea. When performing a test it is important to get imagery and video of the test setup and the test. This will help you remember what happened, help you explain it to others, and demonstrate the robot to others at a later point in time. Many people had a hard time believing the notion that cars can driven by themselves without any human operator on board. However after showing them video from testing not, only did they believe that it was possible, but they were excited about it.

While public outreach and education can get in the way of testing and be disruptive, this can also be a very valuable and rewarding experience. While conducting a field test of a lunar rover at a field site there were many local people who stopped by to see what we were working on. One local farmer looked at the wheels that one of our groups was using and advised us that a different wheel would work much better on the robot, and he was also able to provide us with those wheels for testing. This farmer was correct, the wheels he recommended worked better and used less power when driving. This local farmer taught all of us the importance of specific knowledge that we might not have but that others with little background in robotics can have and can contribute.

One area of testing that is often overlooked in the human component. Insuring that personnel are not hungry and not tired is very important. When people are hungry or tired they tend to make mistakes and not work optimally . During one field test that we were not prepared for everybody was lacking sleep. As a result people were working less than optimal and we placed ourselves in a potently dangerous situation. Some of these potentially dangerous situations included working on elevated surfaces in the dark and while very tired, and driving one and a half hours on
flat “boring” highways while tired.

As much as anyone can prepare for a test there is always the possibility of something unexpected to go wrong. During these times it is important to stay calm and not overreact. Taking a step back and examining the problem can often lead to a solution without further damaging the robot. Creating checklists in advance of diagnostic and debugging steps is good for situations like this. In one case we had a robot get stuck in an awkward configuration the night before a demonstration. Instead of approaching this methodologically we started to disassemble and overreact. If we had operated with calmer heads and we fully analyzed what happened we would have realized the simple fix instead of the risky disassembly operation that we partook in. And in all cases think “How will it look in an accident report” (also from our friend the astronaut) and don’t do things that are dumb and that you will regret later.

The test is done. You are ready to go home. But now we need to analyse the test in the next post.

As always comments are welcome (and encouraged)

Image Source: http://www.frc.ri.cmu.edu/projects/lri/scarab/images/MK_testing/IMG_6516.html

Liked it? Take a second to support David Kohanbash on Patreon!