How it started...
The foundations of the 3D mapping platform LiBorg were laid about four years ago, during the imec.icon project GIPA. In the frame of this project, industrial partner Sweco (the former engineering firm Grontmij) was looking for a system that could efficiently monitor large-scale building sites or constructions, to verify, for example, if a construction is proceeding according to plan. Michiel Vlaminck: “Today, static laser scanners are used for 3D reconstruction. They need to be set up manually at various spots within the environment. This requires considerable expertise, and is often a very expensive and time-consuming process. Sweco was therefore looking for a more efficient and cost-effective system, able to reconstruct 3D models on the fly.”
LiBorg 2.0
In the course of that project, LiBorg was born – a 3D mapping platform that was continuously refined after the GIPA project was ended. In its current implementation – LiBorg 2.0 – the mapping platform consists of a mobile robot, sensors including a lidar scanner, and reconstruction software allowing to generate in real time a 3D model of nearly any environment. Hiep Luong: “Our off-the-shelf lidar scanner is equipped with 16 laser beams that are continuously spinning at 10Hz. By measuring the time that each laser beam needs to move back and forth from scanner to object, the distance to the object can be calculated. By putting these distances in a coordinate system, a three-dimensional point cloud of the environment can be created. The system is also equipped with an RGB camera. A synchronization module allows data from the color camera to be synchronized with the lidar, providing a means to add realistic colors to the point cloud.”
LiBorg 2.0: a robot with lidar scanner, reconstruction software and RGB camera.
A unique localization system
The lidar scanner is also used to localize the robot, without using other techniques such as gps. And that makes the mapping platform unique. Michiel Vlaminck: “To generate a 3D model of an environment, the location of the robot within the environment must be known at any time. In a first step, this is realized by relating the newly incoming sensor data to the sensor data that have been captured before. This technique is called SLAM (simultaneous localization and mapping): while the map of the environment is being built, the robot puts itself into the map. In subsequent steps, the localization is being refined. For example, in a second step, data are related to the complete model that was created so far. In a final step, the robot moves in a closed loop – ending at its (well-known) departure point. This allows correcting the localization errors that have accumulated during the entire trajectory.”
The benefits at a glance...
This localization technique makes the platform independent of external localization systems such as gps or wheel odometry (allowing to calculate distances based on the number of wheel rotations). Each of these techniques has its own limitations. For applications that require a high accuracy, satellite navigation can currently only be used outdoor, and can as such not be deployed for mapping tunnels, mines or buildings. Hiep Luong: “In addition, we are currently investigating the integration of our LiBorg system in a drone. Here, of course, we cannot make use of, for example, wheel odometry. A platform that is independent of an external localization system has fewer constraints and can be used for all applications.”
Integrating the LiBorg system on a drone (picture) broadens the range of applications.
But the fully automatic operation of the system and its ability to generate 3D models on the fly – anywhere, anytime – are without doubt the biggest advantages of the system compared to existing mapping solutions. Michiel Vlaminck: “This technology allows to quickly and efficiently map nearly any environment, without human intervention. During movement of the robot, a 3D model of the environment is created – automatically and in real time. Currently, we steer the robot by means of a controller, but we are also investigating whether it can move fully automatically. And, in contrast to other existing systems, LiBorg can reach and map even the smallest corners of the environment.”
3D map of streets, reconstructed with LiBorg 2.0
Trade-off between speed and accuracy
LiBorg allows 3D models to be generated with high accuracy, but for each application, a trade-off must be made between accuracy and speed. Hiep Luong: “There are different ways to express the system’s accuracy. One way is by comparing the points from the calculated point cloud with the real environment. For an environment of 4000m2, we typically reach an accuracy of 1cm for two arbitrary points of the cloud. This is comparable to or even better than state-of-the-art. Another way is to assess the accuracy of the estimated position of the robot in the map. For a trajectory of 12 meter, the technique is accurate up to 5mm.”
The 3D model is being created on the fly, meaning that the time needed to create a model is determined by the time that the robot needs (or has) to move around. Hiep Luong: “LiBorg 2.0 can accurately generate 3D models on the fly. But if an application requires a more accurate result, we can also do more complex calculations ‘off board’. This of course requires more time.”
Data intensive
The on-the-fly processing of data that are being generated by the lidar scanner is executed by an ARM processor that runs on a Jetson TX2 board – a platform for embedded devices that is integrated within LiBorg. Michiel Vlaminck: “The lidar scanner generates about 300,000 points per second – a lot of data to process in real time. Therefore, a large part of my research activities focused on the fast processing of data, for example by making the processing algorithm more memory efficient. Therefore, we first translated the point cloud into a more efficient data structure. This way, we are able to filter the 3D point cloud and remove redundant points. If you don’t do that, and data keep on coming in at a rate of 300,000 points per second, the memory gets fully occupied, slowing down the operation of the system.”
From train station to building information model for architects
Thanks to these properties, LiBorg is suitable to map all types of environments. Michiel Vlaminck: “In the frame of the GIPA project, we were asked to map a chemical site. But besides that, our system can also be used for inspecting public buildings such as train stations or shopping centers, and for mapping more critical infrastructure such as tunnels, bridges, sewers or mines. LiBorg can also be deployed to monitor escape routes or to help planning events.”
The mobile mapping platform is especially suited for monitoring construction sites on a regular basis, and as such detect changes automatically. This brings us to another application: creating BIM models (building information models). Hiep Luong: “We are currently investigating if we can translate the scan data into a BIM model. A BIM model is a simplified CAD model allowing architects to check if the construction of a building proceeds according to plan, and to digitize the building’s life cycle. Our robot can repeatedly monitor a site, even at night. It can also be deployed after the construction works have ended, to regularly check changes made to the building. That’s why we are currently investigating if we can integrate change detection in the algorithm that we developed for LiBorg. That will be the focus of further research and development.”
Want to know more?
- Read more about the imec.icon project GIPA
- LiBorg 2.0 was among the demos that were shown at the 2018 Imec Technology Forum in Antwerp, Belgium. Watch the movie (Kanaal-Z, in Dutch only) to see the lidar based robot (and other imec technologies) at work.
- In this work, an off-the-shelf radar was integrated in the LiBorg system. Imec is also working on the next-generation lidar systems. More info on the website.
