Oryx Simulations develops and manufactures real-time physics simulators for training of harbor crane operator in several of the world's major harbors. Currently, the modelling process is labor-intensive and a faster solution that can produce accurate, textured models of harbor scenes is desired. The accuracy requirements vary across the scene, and in some areas accuracy can be traded for speed. Due to the heavy equipment involved, reliable error estimates are important throughout the scene.

This report surveys the scientific literature of 3D reconstruction algorithms from aerial and terrestrial imagery and laser scanner data. Furthermore, available software solutions are evaluated. The conclusion is that the most useful data source is terrestrial images, optionally complemented by terrestrial laser scanning. Although robust, automatic algorithms exist for several low-level subproblems, no automatic high-level 3D modelling algorithm exists that satisfy all the requirements. Instead, the most successful high-level methods are semi-automatic, and their respective success depend on how well user input is incorporated into an efficient workflow.

Furthermore, the conclusion is that existing software cannot handle the full suite of varying requirements within the harbor reconstruction problem. Instead we suggest that a 3D reconstruction toolbox is implemented in a high-level language, Matlab. The toolbox should contain state-of-the-art low-level algorithms that can be used as ``building blocks'' in automatic or semi-automatic higher-level algorithms. All critical algorithms must produce reliable error estimates.

The toolbox approach in Matlab will be able to simultaneously support basic research of core algorithms, evaluation of problem-specific high-level algorithms, and production of industry-grade solutions that can be ported to other programming languages and environments.