From 2007-2010, I worked as a researcher at the Video and Image Processing Laboratory under Professor Avideh Zakhor at the University of California, Berkeley. This page provides an overview of the research that I did during this time.
Automated, 3D Modeling of Building Interiors
Summer 2009 – Summer 2010
Overview: The main objective of this proposal is to design, analyze, and develop architecture and algorithms, as well as associated statistical and mathematical framework for a human operated, portable, 3D indoor/outdoor modeling system, capable of generating photo-realistic rendering of the internal structure of multi-story buildings as well as external structure of a collection of buildings in a campus. Work on this project has been accepted for presentation at 3D Data Processing, Visualization and Transmission 2010 (3DPVT 2010), and submitted for publication to Computer Vision and Image Understanding: Special Issue on New Advances in 3D Imaging and Modeling.
Story from Engadget by Tim Stevens, August 11, 2010.
Video from ABC7 in the Bay Area, reported by Richard Hart, August 8, 2010
M. Carlberg, G. Chen, J. Kua, S. Shum, and A. Zakhor. Indoor Localization and Modeling Algorithms for a Human-Operated Backpack System, submitted to Computer Vision and Image Understanding: Special Issue on New Advances in 3D Imaging and Modeling, February 2010. [PDF]
G. Chen, J. Kua, S. Shum, N. Naikal, M. Carlberg, and A. Zakhor. Indoor Localization Algorithms for a Human-Operated Backpack System, to be presented at 3D Data Processing, Visualization, and Transmission 2010, Paris, France, May 2010. [PDF]
Urban Landscape Classification using Aerial LiDAR Point Clouds
Summer 2008 – Summer 2009
Overview: We identify different classes of urban landscape in large 3D point clouds. Each point is labeled as one of the following: water, ground, roof, tree, or other. Our classifiers process data fully in the 3D domain, similar to techniques used in ground-based robotic navigation. This work was presented at the IEEE International Conference on Image Processing (ICIP 2009).
Abstract: The classification of urban landscape in aerial LiDAR point clouds can potentially improve the quality of large scale 3D urban models, as well as increase the breadth of objects that can be detected and recognized in urban environments. In this paper, we introduce a multi-category classification system for aerial LiDAR point clouds. We propose the use of a cascade of binary classifiers for labeling each LiDAR return of an input point cloud as one of five categories: water, ground, roof, tree, and other. Each binary classifier identifies LiDAR returns corresponding to a particular class, and removes them from the processing pipeline. Categories of LiDAR returns that exhibit the most discriminating features, such as water and ground, are identified first. More complex categories, such as trees, are identified later in the pipeline after contextual information, such as the location of ground and roofs, has been obtained, and a significant number of LiDAR returns have already been removed from the pipeline. We demonstrate results on a North American dataset, consisting of 125 million LiDAR returns over 3 km2, and a European dataset, consisting of 200 million LiDAR returns over 7 km2. We show that our ground, roof, and tree classifiers, when trained on one dataset, perform accurately on the other dataset.
M. Carlberg, P. Gao, G. Chen, and A. Zakhor. Classifying Urban Landscape in Aerial LiDAR Using 3D Shape Analysis, IEEE International Conference on Image Processing 2009, Cairo, Egypt, February 2009. [PDF]
Supplemental material for paper, including images of classified point clouds for the entirety of the D1 dataset. [ZIP]
Fast Surface Reconstruction and Segmentation with Ground-Based and Airborne LiDAR
Fall 2007 – Summer 2008
Overview: We create 3D surface models of outdoor environments from 3D point clouds obtained using ground-based and aerial LiDAR. As an extension of surface reconstruction, we also introduce an algorithm that segments 3D scenes in a meaningful way. Our algorithms are fast and scalable, processing city-sized point clouds in 1-2 hours. This work was presented at 3D Processing, Visualization, and Transmission 2008 (3DPVT 2008)
Abstract: Recent advances in range measurement devices have opened up new opportunities and challenges for fast 3D modeling of large scale outdoor environments. Applications of such technologies include virtual walk and fly through, urban planning, disaster management, object recognition, training, and simulations. In this paper, we present general methods for surface reconstruction and segmentation of 3D colored point clouds, which are composed of partially ordered ground-based range data registered with airborne data. Our algorithms can be applied to a large class of LIDAR data acquisition systems, where ground-based data is obtained as a series of scan lines. We develop an efficient and scalable algorithm that simultaneously reconstructs surfaces and segments ground-based range data. We also propose a new algorithm for merging ground-based and airborne meshes which exploits the locality of the ground-based mesh. We demonstrate the effectiveness of our results on data sets obtained by two different acquisition systems. We report results on a ground-based point cloud containing 94 million points obtained during a 20 km drive.
M. Carlberg. Fast Surface Reconstruction and Segmentation with Terrestrial LiDAR Range Data. Master’s Thesis, Department of Electrical Engineering and Computer Sciences, University of California at Berkeley, May 2009. [PDF]
M. Carlberg, J. Andrews, P. Gao and A. Zakhor. Fast surface reconstruction and segmentation with ground-based and airborne LiDAR range data. 3D Processing, Visualization, and Transmission (3DPVT), Atlanta, GA, June 2008. [PDF ]
M. Carlberg, J. Andrews, P. Gao and A. Zakhor. Fast surface reconstruction and segmentation with ground-based and airborne LiDAR range data. Technical Report EECS-2009-5, EECS Department, University of California at Berkeley, January 2009. [PDF]
Demo video presented at 3DPVT 2008, showing portion of mesh of point cloud 2 from S1 dataset, before and after merging with airborne LiDAR .
Poster presented at 3DPVT 2008.