Designing for Success – Field Testing Series – Part 2

This post is part of a series on testing robots with a focus on field testing. It will start with why we test robots, work its way through planning for a field test, to performing the test, and finally what to do after the test. We will conclude with a description of many common tests that we subject our robots to and how to perform them.
I hope you enjoy this series and please leave comments below.
Links:
Why Test Robots? – Field Testing Series – Part 1
Designing for Success – Field Testing Series – Part 2
Test Preparation – Field Testing Series – Part 3
Test Execution – Field Testing Series – Part 4
Test Analysis – Field Testing Series – Part 5
Common Robot Tests – Field Testing Series – Part 6
—————————————————–

This post takes a small tangent to talk about designing for success with lots of testing bits thrown in.

Every design should start with a systems requirement document. This document lays the framework from which everything else is built upon. The systems requirement document should have all of the mandatory requirements clearly defined with metrics that describe what success of the requirement is. It should also have desired and self-imposed requirements. For the desired and self-imposed requirements, metrics should also be clearly defined but can sometimes be more qualitative than the mandatory requirements. The system requirements document should also clearly define what requirements are mandatory and which are desirable and self-imposed so that the engineers know where to prioritize work/testing and what is absolutely needed.

Even before the robot gets built there are many design decisions that affect how the robot will perform, how testable the robot will be, and the ease of servicing and diagnosing problems. The ability to easily diagnose and fix problems in the field can be the difference between a failed test and a highly successful one. When starting a design there are many aspects that should be considered. The two primary categories for testable systems are robot factors and human factors.

Robot factor questions are about things inherent to the robot. Some questions are as simple as how will we transport the robot. Will it fit in a truck? How about out the front door of the lab? What is the operational mode of the robot while field testing? Other questions are related to potential failures. How would we replace a wheel or sensor in the field? Are the fuses accessible? How do we calibrate the sensors in the field? It is also necessary to perform a “reality check” on the design. If a robot is large and traveling at 60mph can the user gain access to the emergency stop button (Is there an Emergency stop button)? While these considerations might seem obvious it is important to review them and actively consider these questions.

Sandstorm the self driving car. The operator had to squeeze in to the sides of the center computing rack.

Design reviews play an important part is the design of a robot. Having a design review with people with applicable experience is invaluable. Make sure to listen and consider their advice and input. In addition to members of the design team there must be external reviewers so that fresh eyes are looking at the design. Reviewers must not be afraid to be critical and questions things they do not understand or have a doubt about.

Once the design stage is complete fabrication can begin. In the fabrication stage it is important to assemble the robot and keep notes and pictures. There are many times when questions develop about how something is assembled and it is useful to have these resources. As the robot is being built it is important to keep all documentation, block diagrams, CAD models, and schematics reflective of the as built system. It is a good idea to create a binder that stays with the robot and travels to field events that has the as-built (especially electrical) diagrams of the robot.

Mechanical systems often suffer from tolerance issues and thermal issues. When designing parts remember to add appropriate tolerances between parts (the added thickness from anodizing or painting a part MUST be considered and is often not negligible). It is also important to remember both the conditions the robot is designed for and any conditions the rover might experience while in testing. If a robot components need to be assembled with a sledge-hammer there is a good change it will bind when subjected to changing conditions. Gear boxes and lubrication’s are another area where paying attention to thermal conditions is very important. There are many cases where traditional grease lubricants must be traded for oil and visa versa. Sealing components and systems from dust, rain, etc… is also important.

Electrical systems must also be designed for testing and field operations. Having complete and accurate schematics in a binder that travels with the robot has proven to be essential for debugging electrical issues and failures in the field (I know I said this before, it is here again for emphasis). Almost more important than having the schematics on hand is carefully labeling every wire. Every wire and cable should be labeled with what it is and where it goes (you can buy heat shrink labels that can be printed on with a label maker and you can put it on your wires). If there is any ambiguity in a wires location, a connectors location, or a connectors orientations it must be labeled and made obvious. If someone can mess it up, someone will! I had a robot where the screw terminal battery charging wires came loose and somebody decided to fix. They swapped power and ground causing a bunch of damage to robot components (not a good day, it probably should have had some protection). Getting to the Fuses should be easily accessible and protect vital components. If there are any components with a high voltage/current it should be blocked in a way that people can not touch it and labeled as such. Some key electrical components are emergency stops (e-stops) that cut off all power to the robot, motion stops (m-stops) that provide a safe way to stop all motion (often useful if power is required to insure a safe stop), shunt regulators to protect robot systems from motors being back driven (make sure the shunt regulator has a path to the motors even if a fuse blows or the e-stop is engaged), and proprioceptive sensors (power, voltage, current, temperature, etc..).

Software also plays a critical role in the design of a robot. Software must be robust to handle failures, well documented, and should be carefully tested. Implementing watchdogs is a good way to catch if a process or components (including sensors) die and allows the system to react accordingly in a controlled manner. It is important that software is robust to all input states. For example if a process takes user input, a person should be able to pound on the keyboard randomly without causing an uncaught error. Software version control is also very important. Knowing what version of code is running on the robot and what version was running when an error occurred is important. It is very easy to make small changes on the robot and forget about them without having a formal version control process. Every process should have a way of displaying the usage instructions and a quick summary of what it does without having to look at source code (think having a “–help” option for the process). Team software code reviews are often not effective as the reviewers do not fully understand and cannot follow the large amounts of code being presented. It is often more valuable to have just one team member (or multiple individually) try to create a flow chart of how someone else’s code works and let them provide the feedback and review. And remember as Astronaut Dr. Jay Apt said “simple verifiable code is better than complex unverifiable code”. KEEP YOUR CODE SIMPLE and do not try to impress the world with complex code. Simple code is easier to debug, easier to fix, easier for others to understand, and easier for YOU to understand (trust me).

All members of the rover team (especially those that work with hardware) should have a lockout tag that can be put on the robot to let others know it is unsafe to operate. Only the person who put the tag there should be allowed to remove it unless they specifically pass that responsibility on to another (for example if a mechanical team member tags the robot due to an electrical fault, an electrical team member needs to review and certify that the robot is ready to be operated).

A note on schedule. Schedules are often made haphazardly and are often not followed. When developing a schedule it is important to be realistic and leave a little bit of flex time for the unexpected. It is also important to remember that if a system needs to be completed by the first of the month that does not mean the mechanical hardware gets finished then. It takes time for the electrical, software, and system to be tested once the mechanical portions are complete. Things that seem easy and fast can often take the most time. As a general rule everything takes twice as long to complete as you think it will.

This is a good point to develop an error tracking method. This can be as simple as a spreadsheet or as complex as a full commercial software tools package. Keeping track of errors is the best way to make sure that all errors are tracked down and fixed and can be used to track long-term system reliability and find weak areas in the system.

Once the robot is built it is time to begin full system testing. Developing a good testing methodology is key for a successful test program. Test plans should be based on the systems requirement that was created in the beginning of the design phase (you did create remember to create this?). Test plans should be developed for meeting these objectives.

By nature testing is a cyclic process of design, testing, evaluating the results and then going back to the design phase to fix identified problems and iterate for a better system. It is also important to have regression testing to make sure that while fixing one item another item was not broken or suffer a loss in performance. When a stable version of software is produced especially before an important robotic demonstration the code should be frozen and thoroughly tested so that last-minute changes won’t have unintended consequences during the test or demonstration. Finally once the test is completed it is critical that the results get turned over to the design team so that they can be analyzed and improved upon.

Stay tuned for the next post on pre-planning for a successful field test

As always comments are welcome (and encouraged)

Image Source: http://www.nasa.gov/centers/ames/research/technology-onepagers/fieldRobotics.html

Liked it? Take a second to support David Kohanbash on Patreon!