ODM 3.0, the leading open-source drone mapping engine, has been released!
This release has focused on improving output quality, removing some legacy parameters and fixing long-standing issues with the software. You can check the change log to see what has been removed. If your scripts use legacy parameters, they will be ignored and you will get a warning. This follows our philosophy of “not breaking stuff”.
It can be difficult to convey in words what I mean by “improving output quality”. So I’m not going to, and instead, I will direct you to our UAVArena page.
If you want to get nostalgic, compare the ODM 3.0.0 engine outputs with those of ODM 0.9.9. How far we’ve gone!
If you want to get excited, compare the results to those of alternative proprietary software, or against a previous 2.x release.
Digital Surface Model. Left: ODM 2.8.0. Right: ODM 3.0.0
Orthophoto relief displacement improvements. Note the trailer on the bottom left and the truck. Left: ODM 2.8.0. Right: ODM 3.0.0
ODM 3.0.0 is available today as a standalone application or can be used with the WebODM GUI, which as of today also ships with the latest version of ODM.
Find-GCP is a program that detects ArUco markers in digital photos and is able to create OpenDroneMap ground control points files automatically. The project has been utilized by members of our community for a long time and has provided means to automate ground control point detection workflows using commodity hardware. Kudos to the GeoForAll Lab at the Department of Geodesy and Surveying of the Budapest University of Technology and Economics for their efforts in making this software and in particular to its lead maintainer, Zoltan Siki.
We’re pleased to announce that as of today Find-GCP has joined the OpenDroneMap family as a member project! We will do our best to support its adoption and further its development / integration with the wider ecosystem.
We strive to make ODM as fast as possible. GPUs are very good at performing parallel calculations on blocks of data. While they can’t be employed at every stage of the pipeline, they can help in several areas.
GPU-based features extraction has been in ODM for a while (if you ventured to use the opendronemap/odm:gpu docker image) and has given some modest speed improvements when run on consumer hardware compared to its CPU implementation. But perhaps the most useful area for leveraging the GPU in a photogrammetry pipeline is point cloud densification.
WebODM Native, which already contains the latest ODM binaries (Windows)
Early benchmarks point to a ~10x speed improvement in point cloud densification times! Considering that point cloud densification is (was?) the longest step in the processing pipeline, we’re talking about a significant improvement.
Brighton Beach dense point cloud processing time
You will need an NVIDIA card to make use of the new GPU-based point cloud densification code.
Check it out and let us know in the community forum if you notice improvements!
OpenDroneMap is now compatible with the Apple M1 (no, not with this arm). The community came together to fund this feat. Thanks to everyone that contributed!
What was challenging about this?
Apple M1 speaks a different language than Intel chips (the mainstream chips that most people’s computers have). This language is a dialect of ARM. Apple has added a piece of software to guarantee some compatibility with applications not specifically built for its new chip, but the compatibility layer fails with software such as ODM that has many optimizations and low-level functions.
The steps to a port are always more or less:
Try to build the software on the new architecture
Get showered with errors
Fix a few errors
Repeat from 1 until success
Re-test on every other architecture (in case you’ve broken something in the process)
Automate the build process
Some really strange bugs came up during this process (like this one or this one). We upgraded the base layer of the software to be based on the cutting edge Ubuntu 21.04, since certain packages we depend on aren’t available for ARM. This broke our deployment to Snap. So we had to fix that with some conditional build logic. In the process our Windows build broke. We finally got everything ready for building and deploying, but the build times were now exceeding 6+ hours (up from 2+), which is a limit on GitHub. So we setup our own GitHub builder node, which is faster and enjoys higher time limits.
A challenge was getting PostgreSQL + PostGIS running. To guarantee compatibility with previous releases and a smooth upgrade experience for long-time users, WebODM’s docker images need PostgreSQL 9.5. As we plan for a future database migration / update which will require us to include multiple Postgres installations in the same database image, we opted to build from source both packages (PostgreSQL 9.5 is no longer included in newer Ubuntu releases).
After several days (and lots of coffee), everything came together!
So there you have it! From the command line:
git clone https://github.com/OpenDroneMap/WebODM
cd WebODM/
./webodm.sh start
Will get you up and running on the Apple M1 (and on other ARMv7 devices such as newer Raspberry PIs too)!
What’s next? Maybe a sans-docker port to MacOS? We’ll see. Stay tuned.
I’m proud to to announce that as of last week, our esteemed community member, long-time contributor and all-around champion of open source Saijin_Naib has become the first non-developer paid contributor to be sponsored by UAV4GEO.
He’s no stranger to OpenDroneMap. He has been involved with the project for a long time and has demonstrated incredible enthusiasm across these years while helping people, rallying for fundraising rounds and advocating for a world with more free and open source software. UAV4GEO is delighted to be able to support his efforts to continue doing what he has been doing all along: helping people with OpenDroneMap. He will also spend time to expand and improve the documentation across all projects and provide hands-on learning experiences to the community. We look forward to see what he will bring next.
I think every open source project one has to choose between two priorities: code or people. If you want to invest into building features, you invest in software developers. If you want to build a community, you invest in the community. With finite resources, you cannot prioritize both. I think a strong community, with it’s diverse set of experiences and ideas, over the long-run, will lead to better code. And for communities to thrive, they need people like Saijin_Naib that will jump in to lend a hand when things get confusing or when somebody needs some guidance.
So, congratulations Saijin_Naib on your new position of IT Specialist and Community Manager! See you on the forum.
One of the cool things about open source software is that it has a tendency to eventually run everywhere. If you have the source code available and certain constraints allow it, with sufficient time a port can be made to different platforms. This is not always the case with proprietary software, where the vendor decides which platforms to support.
In the beginning, ODM could only run on Ubuntu Linux. Later, we opted to leverage docker, which is an amazing (yet confusing) technology we use to bring many of the OpenDroneMap tools to other platforms such as MacOS and Windows. The docker abstraction allows us to focus on building the software without worrying about the platforms. If you can install docker, you can run OpenDroneMap software. Simple. But not without issues. Especially on Windows, docker has several quirks.
We initially delayed efforts to make a native port to Windows for two main reasons:
To focus developer time on quality of outputs, speed and features.
Previous attempts at making a Windows native port (2 of them across 3 years) failed to yield a working prototype (and lots of lessons learned).
Third time is the charm. I’m pleased to announce that starting today, you can run ODM, NodeODM and WebODM entirely natively on Windows. Even without docker. So where/how can you install it?
WebODM (user interface): see the downloads page. We currently ask that people make a contribution and purchase the pre-built native Windows installer for WebODM, to support the project. All changes required to run WebODM natively, including the recent Desktop Mode support, are already on GitHub (but you will need to do some work).
This was no easy feat. Let me tell you a bit about the challenges we encountered:
ODM has a several primary dependencies (OpenCV, OpenSfM, mvs-texturing, PoissonRecon, PDAL, untwine, entwine, GRASS, OpenMVS, dem2mesh, etc.). These dependencies in turn have in some cases lots of other dependencies.
Most of these dependencies were/are written with non-Windows systems in mind.
Path handling requires special care on Windows. For example, you cannot create symbolic links on Windows, unless you have administrative permissions (that’s a no go). Paths have backward slashes instead of forward slashes. Single quotes are no good for wrapping paths with spaces in Windows shell commands and functions such as pipes.quote in Python will use single quotes. CTRL+C will prompt the user for a confirmation to terminate a process (instead of just terminating the process), which will hang forever in headless processes like NodeODM.
Building Python packages from source code (which ODM requires) leaves several hard-coded paths in our packages, making deployment to other machines tricky.
Python virtual environments weren’t meant to be used for deployment (they also have hard-coded paths). But ODM needs a dedicated Python environment to run.
On the plus side, we had already proven last year that the user interface could work natively on Windows:
So all that remained was to get the ODM and NodeODM components to work too.
We set off to build each major dependency (and contributed several patches to other upstream FOSS projects in the process). We used a tool from Microsoft, VCPKG, to help us build several secondary dependencies for the projects that needed them. This was probably the hardest and most time consuming task. Libraries missing headers, architecture set instructions problems, pre-processor define issues, runtime dependencies not being found, path resolution issues, CMake variable propagation troubles. The list goes on. But divide and conquer, lots of coffee and one by one, we managed to build them all (the entire saga can be read here, thanks to all the community members that followed the process and gave encouragement along the way).
We refactored/cleaned up the directory structure and eliminated the build directory in the process, allowing us to keep all of our dependencies in a single SuperBuild/install directory. This made it easier to package the software in subsequent steps.
We fought the idiosyncrasies of Windows Batch scripting and the Windows shell with some clever tricks to enable a user to simply run: run d:\my\dataset from a command prompt (the ODM Console) and allow to cancel it via a CTRL+C.
We hot-patch several hard-coded paths as part of the ODM startup process to make sure our Python modules work as expected.
We rewrote the cutline computation algorithm in Python instead of using GRASS, which allowed us to shed some space requirements (and improve performance).
We wrote an installer script using InnoSetup (another awesome FOSS) and we got pretty clever to automatically sign the final installer with a code signing certificate (otherwise Windows and other AV software would complain).
Lots of testing, which yielded its own set of problems, like undefined behavior in MVE/libjpeg causing a file descriptor leak due to C/C++ exception throwing, which were resolved after some debugging.
With ODM working, we turned the page to NodeODM. Luckily, NodeODM is a relatively simple piece of software and is written in portable NodeJS. Save a few minor and not so minor issues, we managed to use nexe to generate a portable executable for Windows. Here we found out that Windows exit codes do not propagate well unless you explicitly call exit on every Batch script (but that breaks console-based workflows), so we wrote some clever workarounds.
With NodeODM and ODM both working, the path to full Windows native WebODM was finally possible. Using the same technique that we explored last year, we generated a single executable setup that installs everything that you need. We finally improved the Desktop Mode capability of WebODM to give a more native feel to the application, which remains a web-based application running under an Electron wrapper.
We finally built an automatic updater system to allow people to check for the latest version of the software.
What does this mean for docker? Nothing. Docker is and will continue to remain a core part of our toolset and deployment mechanism. This is simply another way to run WebODM that we hope you will enjoy.
This is a guest post authored by Zoltán Siki, the creator of the awesome Find-GCP project.
Have you tried to create the gcp_list.txt file for a project with hundreds of images? It is a time-consuming and boring task to find each Ground Control Point (GCP) in 6-10 images and the chance of misidentification are high.
Some pieces of commercial software have an automatic GCP recognition system based on image processing algorithms, but no similar solution can be found in the open-source UAV image processing world.
I met ArUco codes some years ago and I used these black and white rectangular markers for indoor navigation and automatic movement detection. When I started to use ODM I realized the same technology can be used to find image coordinates of GCPs. There is a contrib package in OpenCV open-source library to detect marker positions and poses in images. The detection can be automatized using different ArUco markers for each GCP.
Different ArUco markers
At the Geo4All lab of the Department of Geodesy and Surveying at the Budapest University of Technology and Economics a small project was started to implement a solution in Python. The code and some instructions are available on GitHub Its usage is straightforward:
Install Python3, OpenCV and OpenCV contrib on your machine
Download the source code from GitHub (zip or clone)
Generate ArUco markers (aruco_make.py or dict_gen_3x3.py) and print them in a suitable size
Put the markers on the field before making UAV images and measure coordinates with the necessary accuracy
Make the flight
Collect the GCP’s 3D coordinates in a TXT file
Run gcp_find.py
Add the generated gcp_list.txt to your ODM project and start ODM processing
Original image and detected markers
Actually there are command line tools plus a simple visual check in the project. A GUI would be a useful addition to the project to edit and review GCP matching results possibly integrated in WebODM. More details are available on the GitHub page of the project and in an article in Baltic Journal of Modern Computing.
Contributions to this project are welcome, usage experience, case studies, bug reports and fixes, enhancements…
Whenever ghosting occurs, the problem is related to band alignment. In multispectral datasets, sensors such as the Sentera 6x, take multiple pictures at once using different cameras, each capable of capturing a different part of the light spectrum. This enables us to have images for various bands such as red, green, blue and near infrared. The images however are not perfectly aligned, due to the relative position of the cameras and slightly different lens distortion.
Red, Green, Blue, NIR and Rededge band mis-alignment
ODM now includes state-of-the-art automatic band alignment using a hybrid approach of feature-based and Enhanced Correlation Coefficient (ECC) maximization alignment, which is capable of estimating the alignment parameters from a sample computed on a subset of the input images and applied to the entire reconstruction.
After alignment, things start to look much better!
Same bands, properly aligned
After aligning the bands, we simply make sure that our orthophoto / texturing generation process uses the sections of one band consistently with all the others, and the ghosts are officially… busted!
Before ODM 2.3.0
After ODM 2.3.0
If you’ve used OpenDroneMap in the past to process multispectral datasets and you’ve been less-than-enthusiast about the results… it’s time to give it another shot!
Chapter 2. Speed
We’ve also gotten a lot faster at processing these datasets too, courtesy of the awesome team at Mapillary which introduced more efficient multi-threading and a change in how we process multispectral datasets, which now require ~1/3 of the time compared to before. That’s a lot faster!
ODM is now also capable of generating Entwine Point Cloud (EPT) point clouds more quickly and with less memory usage, thanks to the awesome team at Hobu and their new Untwine software.
Chapter 3. AGPL License Switch
Starting from ODM 2.3.0, NodeODM 2.1.0 and WebODM 1.6.0, all three projects join the family of free software projects (which already included ClusterODM) that use the AGPL license.
This move is designed to continue guaranteeing the software freedom of our users in a world of ever-increasing cloud services. Despite some questionable claims of the otherwise, by certain large organizations, AGPL is an excellent license for an open source project such as OpenDroneMap (as this humorous blog post by Drew DeVault explains). This choice will continue to help OpenDroneMap be as open and inclusive as it possibly can.
Today we’d like to share an impressive open source UAV hardware and software project, possible thanks to the effort of Ryan Brazeal and supporters.
When we think of open source UAV hardware projects we usually think of small, classroom friendly, tiny drones. Well, this one is not a toy, but will give a run for the money to the most sophisticated and professional rigs.
OpenMMS “presents a single source of information that lists every nut, bolt, OEM sensor, and manufacturing step required to build a modern, tightly-integrated MMS [Mobile Mapping System] sensor. The project also includes a robust suite of novel open-source software applications that rigorously process the collected data and generate industry-standard geospatial deliverables”.
Now for the kicker: “The complete OpenMMS solution provides many of the same features and achieves the same (or better) accuracy specifications as comparable commercial systems. The hardware combines a lidar sensor, global navigation satellite system sensors, an inertial sensor, cameras, and computers, to create a tightly-integrated digital mapping tool. The software is easy to use, fast, supports multiple OS, and, when possible, leverages multi-core (CPU) and graphics card (GPU) processing.”.
While OpenMMS is an independent project, here at OpenDroneMap we’re fully committed to support other open source projects in UAV mapping. We have opened a community channel on our forum for OpenMMS that will provide a venue for users to talk and share ideas about the project.
We look forward to see what the OpenMMS team will bring up next.
Today we’re excited to announce a new major release of ODM! What have we been working on? Well a lot of things. The two most important ones are the ones we hope you won’t notice, because they don’t affect functionality, but have been part of necessary “infrastructure” updates to make sure that ODM continues to work in the future. Namely:
We’ve upgraded the codebase from Python 2 to Python 3. Python 2 has been deprecated and will not receive updates past 2020. With Python 3 support, ODM can continue moving forward with the rest of the Python ecosystem.
We’ve upgraded the base OS target from Ubuntu Linux 16.04 to 18.04. 18.04 will continue to receive extended security maintenance updates until 2028 and compatibility with 18.04 has been frequently requested from our community.
If you’re using docker, these changes are (should be) transparent. If you’re running ODM natively, you will need to upgrade your base system to 18.04 before updating ODM (or continue using a 1.x release of ODM until you decide to upgrade).
Aside from these important under-the-hood upgrades, we couldn’t make an important release such as a 2.0 release without adding something new and shiny!
Image Geolocation Files allow you to specify/override the GPS location of your images without having to fiddle with tools such as exiftool. This is different than using Ground Control Point files. With these you simply specify the location of the camera centers instead of the location of points on the ground.
Image Masks allow you to mask out areas that you don’t want to include in your reconstruction. This is useful in many scenarios. For example, during cell tower inspections, photos typically include parts of the sky, which end up creating strange artifacts and negatively affect the reconstruction. By using masks, one can delineate areas to exclude from a reconstruction, thus obtaining a clean and more accurate result.
A new option, –feature-quality automatically adjusts the image sizes for feature extraction based on predetermined ratios instead of relying on the user input or making assumptions about the image size.
Static tiles that were previously computed in NodeODM for use in viewers such as Leaflet or OpenLayers can now be generated via the –tiles option directly in ODM.
Aside from the speed improvements of having updated Python, PDAL, numpy and many other libraries, we’ve specifically improved memory usage in split-merge to handle even larger DEMs, have improved DEMs compression and improved speed/network stability in ClusterODM/NodeODM.
We’ve fixed numerous bugs and increased overall stability. See the related PRs for geeky details (#1156 and #124). We also cleaned up the ODM repository from old, large binary objects that were inflating the git repository. A git clone previously took 1.1GB of downloads, now it takes only 12MB. If you forked ODM, read this announcement as it affects you.
Give ODM 2.0 a try! If you find any issues, please report them on GitHub.
