LIDAR (Laser Scanner) Review Comparisons
by David Kohanbash on May 15, 2014
This is part of a series of posts talking about some of the common LIDAR’s used in robotics. At the end of these sensor profile articles there will be a final post that compares the sensors based on data collected by each of the sensors.
Posts:
Evaluating LIDAR Guide
LIDAR Fundamentals
SICK TiM551 – LIDAR Overview
SICK LMS111 – LIDAR Overview
SICK LMS511 – LIDAR Overview
Hokuyo UTM-30LX – LIDAR Overview
LIDAR Review Comparisons
I hope this series (and particularly the final comparison post) proves useful.
Enjoy!
Recommendations
Here are my high level recommendations. Obviously other things like power, mass, size, and ruggedness need to play into your choice of LIDAR’s.
TiM551-2050001 ($2000) – I would highly recommend this sensor for applications that need obstacle avoidance/detection at distances of less than 8 meters. However due to the 1 degree resolution and ±60mm range accuracy this sensor is not ideal for mapping applications.
LMS111-10100 ($5700) – I would recommend this sensor for applications that need obstacle avoidance/detection or mapping at distances of less than 18 meters.
UTM-30LX ($5590) – I would recommend this sensor for applications that need obstacle avoidance/detection at a range from 18 meters to 30 meters. This sensor is not the best for mapping; while it has a resolution of 0.25 degrees the range tends to be a little noisy and have an accuracy of ±50mm.
LMS511-10100 ($7000) – I would recommend this sensor for applications that need obstacle avoidance/detection at a range from 30 meters to 40 meters (80 if really needed). I would also recommend this sensor for the most detailed mapping work of this group. You might need to do some basic filtering of the data.
Object Detection
This test was designed to show the range response in different sensors when exposed to the same objects. You can clearly see how it is easier to pick out items with some of the sensors over the other ones. I should also point out that these are zoomed in a bit and in practice for obstacle avoidance probably would not matter. For mapping this can have a large impact.
With the SICK TiM551 it was pretty difficult to “see” which points correlated with which item. Of note the glass was “invisible” to the sensor, and it had no problem detecting the mirror.
The SICK LMS111 was pretty clear and I had no problem detecting which points were for which objects. You could make a decent map from this.
The Hokuyo UTM-30LX had a nice result with its 0.25 degree resolution.
The SICK LMS-511 was also very easy for me to determine what I was seeing in the point cloud. This sensor is good for mapping. If you are doing short-range obstacle avoidance then this sensor is probably overkill.
Environmental Tests
Pointed at the Sun Test
All of the lasers performed well in a sunlit outdoor environment. When pointed directly at the sun that portion of the data was reported as invalid and the rest of the sensor data (for example from the sides) seemed to be good.
Medium Rain Test
During this test there was a heavy drizzle (or medium rain) as can sort of be seen in the satellite image above.
In this test I had the sensor under my roof and the rain started to fall 1m in front of the sensor. There was a silver car (2009 Honda Accord) parked 5.8m in front of the sensor that acted as my target for this.
All of the sensors performed well. There were two observed phenomena. The first is random points when the laser beam hits a water drop. The second is the surfaces that I detected was not as clean and appeared to have more noise in the rain. Here is my summary of these effects:
Sensor | Random Points | Noisy Surfaces |
---|---|---|
SICK TiM551 | Few random points. This could be because it only has 1 degree resolution compared to the higher resolution in the other sensors. | The surface jumped the least with this sensor. Only around 1cm. |
SICK LMS111 | Very few random points. Almost as good as the TiM551. | There was about 8cm of movement in the points on the surface of the car. |
Hokuyo UTM-30LX | A lot of spurious points from the rain. | About 2cm of movement in the points on the car bumper. |
SICK LMS511 | This is kind of tricky. The LMS511 was the noisiest of the sensors I evaluated which I think might be due to how I used the multiple returns. With that said I did not notice any more random points then normal when operating in the rain. | The points on the car jumped around by about 4cm. |
Puddle Testing
For this test I filled a ceramic flower-pot with about 2cm of water to the top of the flower-pot.
I am not sure if it is better to see the top of the water or the surface below the water. I think for a mobile robot I would rather see the bottom of the water so that it does not become an obstacle. In some early LIDAR’s I used the sensors would report incorrectly when exposed to puddles and would make our autonomy software think that the puddle was an obstacle.
All three of the SICK sensors did a good job detecting the bottom of the container and saw right through the water. The Hokuyo had this weird 20-30cm variation where the 2cm of water was.
Laser Accuracy
Spot Size Comparison
This was a bit of a difficult test for me since I do not have a good camera that does not have an IR filter (and I did not want to break one of my cameras). So to do this test I used a Sony Handycam DCR-TRV38 in nightshot mode. This was good as it could see the laser points, however I could not adjust image settings so the image quality is bad in some cases (sorry about that).
When I did this test I setup the laser about 100 inches (254cm) from a white poster. On the white poster I taped a US Quarter (0.955 inch / 2.4cm) to help with size comparisons.
I would like to thank Hokuyo/Kirinsonfor lending me the UTM-30LX for this review and SICK for the other LIDARs.
More information about the SICK product line can be found here.
More information about the Hokuyo UTM-30LX can be found here.
Disclaimer: I am only borrowing these sensors and will return it when I am done with this evaluation. I have received no money or goods from the company in exchange for this review.
Comments
[…] Posts: Evaluating LIDAR Guide LIDAR Fundamentals SICK TiM551 – LIDAR Overview SICK LMS111 – LIDAR Overview SICK LMS511 – LIDAR Overview Hokuyo UTM-30LX – LIDAR Overview LIDAR Review Comparisons […]
[…] Posts: Evaluating LIDAR Guide LIDAR Fundamentals SICK TiM551 – LIDAR Overview SICK LMS111 – LIDAR Overview SICK LMS511 – LIDAR Overview Hokuyo UTM-30LX – LIDAR Overview LIDAR Review Comparisons […]
[…] Posts: Evaluating LIDAR Guide LIDAR Fundamentals SICK TiM551 – LIDAR Overview SICK LMS111 – LIDAR Overview SICK LMS511 – LIDAR Overview Hokuyo UTM-30LX – LIDAR Overview LIDAR Review Comparisons […]
Great stuff, I learnt a lot. I’m looking to use LIDAR in a marine environment, any additional suggestions you might make?
Hi
I am assuming you are above water. I think you will really want to test your sensor with how it handles water/sun. Particularly I would be concerned with the sun reflecting off from the water.
You might need to add some filtering in software to handle those cases.
[…] LIDAR often fails in smoke, however newer multi echo return LIDAR’s do […]
[…] Posts: Evaluating LIDAR Guide LIDAR Fundamentals SICK TiM551 – LIDAR Overview SICK LMS111 – LIDAR Overview SICK LMS511 – LIDAR Overview Hokuyo UTM-30LX – LIDAR Overview LIDAR Review Comparisons […]