I recently had the fortune to attend a talk titled Multi-Modal Data for Perception in Smoke-Filled Underground Mines by Joe Bartels. It was an interesting talk about pushing sensor technology into difficult environments that can have smoke, dust or fog . The following are notes that I took during the talk, as always I add a little of my own content to the post. I want to thank Joe for giving the talk and sharing the images that are used below.
Sonar & Radar
Sonar and Radar works better penetrating the smoke, but lacks the resolution of LIDAR. This can be a problem for creating detailed maps.
Sonar also suffers from a lot of multipath
This work was focused in mines where external lighting is required for visual cameras, as opposed to LIDAR/Radar/Sonar which do not need illumination to operate.
Typical fog lights are angled downwards at the road to minimize light scattering and minimize having the light reflecting back into the camera. Also the lights should be placed low so that they are far away from the sensor camera.
The downward light helps illuminate a path on the ground, but is not high enough for seeing the environment and building a map (such as for SLAM or data purposes).
Scattered light is not on the epi-polar lines which interferes with the principles of epi-polar geometry and stereo reconstruction.
You can add a line filter to the camera which allows only a single line of light at a time into the camera sensor. This prevents the scattered light from entering the camera sensor. The downside to this is the reduced data returned from the camera.
- Thermal cameras often work well and see through smoke. They are based on emissivity (energy emitted from an object) as opposed to the light reflecting from the object (such as with a typical camera).
- Emissivity of 0 would be a mirror, while an emissivity of 1 would be a true blackbody.
- A problem is that the differential response is small in non man-made materials.
- 2 Types of sensors: short wave & long wave. The short wave sensors are more expensive, have better resolution, but less visibility through smoke.
- Lacks the sharp features for many mapping applications
- You need to be careful since lights and people can appear to be similar temperatures
- Wet surfaces can be confusing to interpret
Calibrating Thermal Cameras
People often use a standard calibration target however this is far from ideal
The next step that people often try is to artificially heat/cool a target, however it is hard to make the target a constant temperature and the temperature changes are short-lived. When you do this the black will heat up quickly. When viewed on a thermal camera the checkerboard colors get inverted.
A better way to calibrate thermal cameras is to make a target of dissimilar materials (multi-modal) that have different thermal properties.
Different types of smoke/fire produce different perception challenges.
Smoke Candles – White smoke that hangs around. It goes to the ceiling and then slowly drops down.
Wood Fire – Lighter color smoke that hangs near the top of the room. Carbon particles are suspended in the air.
Diesel Fire – Dark smoke that sticks to the ceiling and is very persistent. Carbon particles are suspended in the air.
Dust – Harder than smoke since the particles are bigger, however the particle suspension time in the air is shorter. Radar tends to be a good choice in dusty environments.
Fog – Fog is based on small water droplets suspended in the air. Fog tends to stay lower in the space and not hug the ceiling. Particularly cold fog stays near the ground, as it heats up it rises before dissipating. This can be simulated by putting dry ice into hot water.
From personal (observed) experience, I need to warn you that if you choose to generate smoke in an indoor environment you need to be careful about alarming others and potentially setting off smoke alarms. Smoke has a way of seeping through cracks and being observed in places that you did not expect. Also if you think a smoke candle is a dud, do not throw it in the dumpster right away, wait a little in case it starts smoking and somebody calls the fire department.
Also as with everything else we do, be careful, and do not do dumb things.
I hope you found the information above interesting, I know I did.
Have you worked in smoke? Leave it in the comments below.