The Truth System Challenge for Outdoor Robots

Developing a platform to test the accuracy of GNSS solutions for small outdoor robots is notoriously difficult. Learn how Swift has designed the first-of-its-kind robotic test infrastructure and get a sneak peek into test results.

All GNSS solutions have a certain level of accuracy they offer, typically ranging from a couple centimeters (e.g., components used in surveying equipment) up to about three meters (e.g., components used in cell phones). To determine the accuracy a given solution delivers, it must be tested extensively in various conditions and compared to a precisely known location – the ground truth. The ground truth is measured using a very high-end reference system.

For products with lower accuracy requirements, it is sufficient to use a reference system that is simply more accurate than the intended use case requires. For instance, if the system under design only requires precision of one meter, a reference system that provides decimeter accuracy is adequate to test the solution. But when it comes to robotics, accuracy is often measured in centimeters, so a ground truth reference system (or “truth system”) is critical.

If you’re new to the concept of GNSS and reference systems, you may be wondering why go through all this testing – why not just use the reference system in the product if you already know it can deliver the accuracy needed? As a general rule, the more accurate a system is, the more expensive it is. Since reference systems must be more accurate than the intended use case, it would drive the cost of the product up unnecessarily. For instance, a typical reference system used for automotive testing costs approximately $70-80 thousand – nearly twice the cost of the average car. Using a reference system only for testing enables robotics OEMs to achieve exactly the accuracy they need without overpaying for precision they don’t need.

Accuracy Requirements for Outdoor Robots

The most common types of outdoor robots that require precise GNSS positioning are robotic lawn mowers, small delivery robots, and agricultural robots. Agricultural robots have been around for decades and many of them tend to be quite different from the small robots used for mowing lawns or delivering packages. These small robots are newer to the market and have a unique set of challenges due to their size and the types of locations where they operate, so that is where we will focus.

Robotic lawn mowers and delivery robots don’t have exactly the same accuracy requirements because the use cases are a bit different. Lawn mowers are equipped with blades that can damage flowers and lawn equipment or harm pets and people, so they typically require accuracy of 2-4 centimeters, though in certain environments they may be able to operate safely and effectively with slightly lower precision. Delivery robots tend to operate on sidewalks and will need to navigate crosswalks and intersections. These environments typically require decimeter-level precision to operate safely.

To develop a reference system reliable enough to test the performance of GNSS solutions for these types of small outdoor robots down to as low as 2 centimeters, a highly accurate truth system will be required.

Designing a Truth System for Outdoor Robots

Consider again that pricey reference system used in automotive. To test the accuracy of the GNSS system designed for the vehicle, the reference system can be easily attached to the vehicle and drive tests can be conducted. But many of the robotics products that leverage GNSS positioning are significantly smaller than a car – they may be robotic lawn mowers or small package delivery robots. These types of robots often can’t bear the weight of additional equipment. One solution would be to build a larger version of the robot for test purposes that could support the truth system, but this introduces new problems by changing the elevation of the receivers from the ground, altering line-of-sight to the sky and creating a test environment that does not match the intended use case.

There are other problems with attempting to install the truth system on board the robot as well. These robots typically operate in suburban areas with tree cover, fences, and houses blocking GNSS signals. Even the best truth system will fail to generate a precise position if the GNSS signal is blocked. In automotive testing, there are many other sensors on board the vehicle that can be leveraged to determine the ground truth when the vehicle passes through areas with a blocked signal. But in slow-moving robots like lawn mowers and delivery robots, inertial sensors and wheel ticks don’t provide granular enough information to determine the ground truth.

Simply put: a truth system for small outdoor robots cannot be attached to the robot itself.

A Breakthrough Idea: Using a Survey Total Station as a Truth System

Here at Swift, we had a novel idea. What kind of equipment measures the location of objects in an area with incredible precision, while remaining in a fixed location itself? A total station used by surveyors!

The total station can be placed under open sky with clear line-of-sight to the entire test area. Even if the robot is operating in an area with blocked GNSS signals, the total station can determine its precise relative location, which is then converted to absolute coordinates. By this means, the on-board GNSS system can be effectively tested against the ground truth.

It would be incredible if solving the robotic truth system challenge was as simple as placing a total station in the test area, but of course the story doesn’t end there. When you’re leveraging a highly sophisticated system specifically designed for a particular purpose to do something else entirely, it’s no surprise that many adjustments will be required.

Any discussion of how you modify a total station for use as a truth system for robots gets very technical very quickly. At a high level, many of the required modifications stem from the fact that a total station is designed to measure stationary objects, so it doesn’t need to account for things like time, heading, or attitude. Swift has devised methodologies to account for these shortcomings, and has successfully developed the first-of-its-kind truth system for outdoor robots.

Testing Skylark GNSS Corrections With Swift’s Truth System

We are using this innovative test setup to determine the performance of our Skylark^® Precise Positioning System in robotic lawn mowers.

Most robotic lawn mowers on the market today leverage a local RTK base station to achieve precision. Skylark is a cloud-based service that corrects errors in the GNSS signal, dramatically improving the accuracy of on-board positioning systems in robotic lawn mowers without the need to install a local base station.

Swift’s test infrastructure includes the truth system described above and a robotic test platform that mimics the operation of a robotic lawn mower.

We have designed test cases to measure the performance of Skylark alongside common positioning engines used in robotic lawn mowers and run these tests in multiple environments where mowers typically operate. These results are compared to a single baseline to determine whether Skylark achieves comparable accuracy to the local base stations currently in use in robotic lawn mower deployments.

The following table shows the performance results achieved by Skylark in an area with moderate building obstruction. Skylark’s accuracy was nearly identical to the single baseline solution for both of the positioning engines tested.

If you have questions about how to get started with cloud-based corrections in your small outdoor robot, or if you’d like to learn more about how we built our test infrastructure, get in touch with our team.