Machine learning meets photogrammetry

Pix4D's photogrammetry software suite puts machine-learning tools for photogrammetry applications in your hands.
First publishedSep 17, 2017
UpdatedJan 19, 2021

Machine-learning in Pix4Dmapper photogrammetry software allows you to classify 3D point clouds into categories like buildings, roads or vegetation.

And this is just the beginning of Pix4D’s latest journey.

We believe in the ability of photogrammetry and machine learning techniques to revolutionize today’s workflows and to enable many new ones. In the end, it will allow the conversion of raw image input to 3D reality models with attributed semantic information.

That means, instead of having operators inspecting and measuring 3D reality models manually, they will directly receive automatically-generated answers to questions like:

  • How many trees are within the project area and at what locations?
  • What is their height and species?
  • What is the total road surface area in your area of interest?
  • What is the amount and distribution of roofs that are suitable for solar cell coverage?
  • How many cars are at your parking lot and at what locations?

Answering these very specific questions will make feasible workflows that allow photogrammetric processing to be directly connected to GIS databases to update their vectorized information based on any new drone data collected.

There is still work to be done. Machine learning techniques are as good as the training data that is used to build the classification models. We opted to give our users the tools to control and refine the classification. As a baby learns how to see and interpret its environment gradually as it grows, our machine learning techniques evolve with the training data and the results will model more object categories and become more reliable.

As of today, professionals will use the new machine learning based point classification mainly to automatically generate digital terrain models (DTMs). In the near future, point classification will also form the basis to extract buildings and model them as a semantic composition of geometry elements as roof, facade, windows, doors and balconies for example.

"Our growing R&D teams in Lausanne, Berlin and Denver are dedicated to this challenge."

– Christoph Strecha, CEO and Founder Pix4D

What is machine learning?

Machine learning is an application of artificial intelligence (AI) that provides systems the ability to find patterns in massive amounts of data, automatically learn them and improve from experience without the need of extra programming.

In a few words, machine learning techniques teaches computers to do what comes naturally to humans and animals: learn from experience.

Machine learning algorithms based on geometry

Our first step in this direction is our novel machine-learning based point cloud classification.

We have trained algorithms based on geometry and pixel values to understand object classes.

We are able to collect user inputs to train new algorithms which can adapt to many topics, for instance in aggregates separating stockpiles from the bare terrain and measure volumes automatically with unprecedented accuracy, or digitizing automatically new road and urban areas.

Discover how to generate the point cloud classification using machine learning techniques.

an image showing classifications overlaid on a point cloud. The colours indicate there are buildings, vegetation and human-made objects.
The colours in the point cloud indicate there are buildings, vegetation and human-made objects.

Point-cloud classification guide

large stand-alone infrastructure
vertical structure or large stand-alone infrastructure
High vegetation
High vegetation:
trees or bushes which are taller than around half a meter
Human-made objects
Human-made objects:
artifical objects above ground
surfaces covered with asphalt
soil, extremly short grass or scattered lawn

Machine learning and photogrammetry in action

Extract bare-earth terrain by excluding above-ground objects

A lot of hydrological or geological analysis need to be done with bare terrain models. In Pix4Dmapper photogrammetry software, you can use the point classification function to separate all the above-ground objects and improve the classification using the point editing tools.

Two side-by-side images showing above-ground objects.
The above-ground objects can be easily spotted using the color classification.

Ignore vegetations on top of a stockpile for more accurate volume measurement

To get an accurate volume measurement, it is crucial to remove vegetation or human-made objects from the point cloud. With the point classification, it would be more time-saving to achieve more reliable volume calculations.

Three side-by-side images showing a stockpile, and how removing vegetation and human-made objects can make measuring more accurate.
Measuring can be more accurate when the vegetation and human-made objects are removed.

Prevent electricity outage by vegetation growth control

Vegetation is one of the leading causes to power line outage. It is extremely important for the energy company to keep track on the vegetation growth to be able to trim it before it causes damages. With the point classification, the extracted infrastructures can be grouped and manually digitized for further analysis.

A point cloud with powerlines highlighted in blue, and vegetation clearly marked in green.
Users can easily spot the vegetation and the powerlines just by their color.

Machine learning for automating telecom asset inspection

One of our latest software products for drone inspections, Pix4Dinspect, is a prime example of Pix4D's machine learning algorithms. Pix4Dinspect uses artificial intelligence and machine learning to automate the analysis of point clouds generated from data gathered by scans using the flight planning application Pix4Dscan. Machine learning automates the inspection of 3D digital twins of telecommunication towers, removing the need to manually detect and measure equipment components.

The resulting digital tower isn’t just a replica of reality, but an intelligent digital twin. The precision of the automatic measurements can be more than 1 degree, more accurate than most manual inspections. As it is, the use of Pix4Dinspect and Pix4Dscan means there is no need to send a person up the dangerous climb of a cell tower for a visual inspection. It means that not only does machine learning improve accuracy, but it helps keep staff safe as well as saving money and time.

cell tower inspected using Pix4Dinspect
The inspection time of telecommunication towers is dramatically reduced, whilst the accuracy is greatly increased.

The potential for machine learning in continuing to improve workflows in photogrammetry is only just being realized - and we'll keep sharing our discoveries as we innovate!

Improve your efficiency today
Classify million of points quickly using machine-learning technology

Related articles