Feature focus: how to tidy point clouds with a sky filter
At Pix4D, we spend a lot of time discussing the applications of point clouds and how they work. This article is focused on the tools we use to actually deliver clean and accurate point clouds or 3D models, and specifically talking about the sky segmentation tool which is used to tidy up point clouds when the sky is present in images.
Reliable point clouds are critical to a professional’s workflow when working with digitized versions of reality. As each point represents the precise location of a point with exact cartesian coordinates, they are a valuable resource for planning and surveying. Whether it’s for a 3D map, digital twin, or something different altogether, a clean point cloud can keep work both accurate and realistic.
What is sky segmentation in point clouds?
Sky segmentation solves the problem of improving the quality of point clouds for oblique projects. When people capture images with a drone that is looking directly down, the sky is not usually present in the point cloud. In oblique imagery such as for the sides of buildings, or projects from PIX4Dcatch, the sky is much more present because the camera captures elements of the sky around objects.
The sky is a problem in some point clouds because the sky appears uniform, and the horizon creates artifacts and noise in the point cloud. This disruption creates noise at the border of an object, which can be confusing and affect how useful the overall point cloud is.
The sky segmentation tool masks out the sky. Pix4D’s modeling and algorithm teams have worked to flag each pixel that is in the sky, which can then be edited out of the overall point cloud. This was previously possible to do manually by selecting individual points, but it took much longer and was quite complicated. Now, the sky segmentation tool can mark elements of the sky automatically. The process happens thanks to using images from densification to make the point cloud. Sky segmentation does so using machine learning and a segmentation model which recognizes what is the sky, and what is not.
Which Pix4D products have the sky segmentation tool?
Sky segmentation is currently available for PIX4Dmatic and PIX4Dengine, two of Pix4D's specialized data processing platforms. PIX4Dcatch users that process their projects on PIX4Dcloud will automatically have sky segmentation built in to their point clouds on projects from Android with the default processing piepline and from iOS with LiDAR data. PIX4Dmatic is the first of its kind to use both photogrammetry and LiDAR point clouds imported from PIX4Dcatch.
In PIX4Dengine the sky segmentation happens automatically during processing whereas in PIX4Dmatic users can choose whether or not to apply the tool (called sky mask) based on their project and desired outputs.
What's the secret behind cleaner point clouds?
The question you may have is whether it actually makes a difference. Working with machine learning means that the algorithm is constantly improving and providing better results. The photogrammetry algorithm now knows how to ignore the pixels in the images that are marked as the sky. You can see the impact it makes in these examples below. You can clearly see the difference it has on results in the accuracy of the point clouds.
Interested in having a go? Make sure your software is updated to the latest version to use the feature in your own work. If you haven’t got a valid Pix4D license yet, this may be your sign to start a free trial with Pix4D. If you've got more questions, you can always contact our sales team!