Next Article in Journal
Mapping the Leaf Economic Spectrum across West African Tropical Forests Using UAV-Acquired Hyperspectral Imagery
Next Article in Special Issue
Semantic Interpretation of Mobile Laser Scanner Point Clouds in Indoor Scenes Using Trajectories
Previous Article in Journal
Weed Mapping with UAS Imagery and a Bag of Visual Words Based Image Classifier
Previous Article in Special Issue
Panoramic Image and Three-Axis Laser Scanner Integrated Approach for Indoor 3D Mapping
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

Mobile Laser Scanned Point-Clouds for Road Object Detection and Extraction: A Review

1
Department of Geography and Environmental Management, University of Waterloo, 200 University Avenue, Waterloo, ON N2L 3G1, Canada
2
Fujian Key Laboratory of Sensing and Computing, School of Informatics, Xiamen University, 422 Siming Road South, Xiamen 361005, China
3
Department of Systems Design Engineering, University of Waterloo, 200 University Avenue West, Waterloo, ON N2L 3G1, Canada
4
Department of Geomatics Engineering, University of Calgary, 2500 University Dr. NW, Calgary, AB T2N 1N4, Canada
5
Department of Civil Engineering, Ryerson University, 350 Victoria Street, Toronto, ON M5B 2K3, Canada
*
Author to whom correspondence should be addressed.
The first two authors contributed equally to this paper.
Remote Sens. 2018, 10(10), 1531; https://0-doi-org.brum.beds.ac.uk/10.3390/rs10101531
Submission received: 5 July 2018 / Revised: 20 September 2018 / Accepted: 22 September 2018 / Published: 24 September 2018
(This article belongs to the Special Issue Mobile Laser Scanning)

Abstract

:
The mobile laser scanning (MLS) technique has attracted considerable attention for providing high-density, high-accuracy, unstructured, three-dimensional (3D) geo-referenced point-cloud coverage of the road environment. Recently, there has been an increasing number of applications of MLS in the detection and extraction of urban objects. This paper presents a systematic review of existing MLS related literature. This paper consists of three parts. Part 1 presents a brief overview of the state-of-the-art commercial MLS systems. Part 2 provides a detailed analysis of on-road and off-road information inventory methods, including the detection and extraction of on-road objects (e.g., road surface, road markings, driving lines, and road crack) and off-road objects (e.g., pole-like objects and power lines). Part 3 presents a refined integrated analysis of challenges and future trends. Our review shows that MLS technology is well proven in urban object detection and extraction, since the improvement of hardware and software accelerate the efficiency and accuracy of data collection and processing. When compared to other review papers focusing on MLS applications, we review the state-of-the-art road object detection and extraction methods using MLS data and discuss their performance and applicability. The main contribution of this review demonstrates that the MLS systems are suitable for supporting road asset inventory, ITS-related applications, high-definition maps, and other highly accurate localization services.

Graphical Abstract

1. Introduction

The advancement in mobile laser scanning (MLS) technology, integrated with laser scanners, location and navigation sensors (e.g., Global Navigation Satellite Systems (GNSS), and Inertial Measurement Unit (IMU)), and imagery data acquisition sensors (e.g., panoramic and digital cameras) on moving platforms has enhanced the performance of MLS in static urban objects detection, extraction, and modeling [1]. These tasks using various geospatial point cloud data with different geometric attributes, complex structure, and variable intensities, have become one of the popular topics in disciplines, such as photogrammetry, remote sensing, and computer vision [1]. From Figure 1a, imagery-based methods occupy the largest portion in the application for urban objects for the past ten years. MLS and airborne laser scanning (ALS) technologies are only dedicated to one fifth of the proportion. However, the MLS technology developed quickly in the past ten years. As can be seen from Figure 1b, publications related to MLS increased rapidly since 2008. Most state-of-art surveys for static urban objects are related to the data from imagery [2,3,4] and few are associated with point cloud data [4]. In Ref. [5], the review focuses mainly on existing urban reconstruction algorithms using different types of point clouds (e.g., MLS, ALS, and terrestrial laser scanning (TLS)).
This review provides a detailed and rigorous clarification for researchers who are interested in static urban object extraction, classification, and modeling using MLS technologies. In this paper, we review the state-of-the-art traditional and learning-based road object detection and extraction methods and discuss their performance and applicability. Such methods are classified into different categories according to targeted urban object types. A typical urban road environment consists of static road objects including road pavements, road-side trees, traffic signs, light poles, and road markings. We review object detection and extraction algorithms for these urban objects based on MLS data. This work includes three parts. First, the latest technologies of MLS are analyzed by listing the attributes and functions of these devices. Then, we review the latest extraction algorithm for on-road static urban objects, which includes road surfaces, road markings, driving lines, road cracks, and road manholes. Meanwhile, emerging off-road object detection and extraction methods are analyzed in detail. Similar methods are associated together and are grouped into the same categories. Finally, challenges and future research are discussed and a conclusion is presented.

2. Mobile Laser Scanning: The State-of-the-Art

Since the GNSS techniques have been publicly accessible services in the past several decades, MLS technologies have indicated the huge commercial potentials as advanced surveying and mapping techniques for effective and rapid geospatial data collection [6]. In comparison with TLS systems, ALS systems, and digital satellite imaging technologies, MLS systems have the flexible mobility and proven ability to collect highly dense point cloud data with cost-saving and time-efficiency measurements.
In general terms, an MLS platform is a vehicle-mounted mobile mapping system that is integrated with multiple onboard sensors, including light detection and ranging (LiDAR) sensors, GNSS units (e.g., GNSS antenna, an IMU, and a distance measurement indicator (DMI)), advanced digital cameras, and a centralized computing system for data synchronization and management [7]. Additionally, the GNSS, the IMU, and the DMI constitute a Position and Orientation System (POS) for mobile mapping applications. By using profiling scanning techniques and detecting laser pulse intensity reflected from the target surfaces, MLS systems can directly support the localization, navigation, object detection, accurate perception, object tracking, and three-dimensional (3D) urban digital modeling missions. Depending on the speed of light and flying time of laser pulses, the precise measurement range is calculated. Accordingly, highly accurate and geo-referenced 3D coordinates for MLS point clouds are ascertained depending on angular measurement, range measurement, position and orientation information [8]. In addition, MLS data acquisition rate is determined according to the scanning sensor-deflecting pattern and laser pulse repetition rate. Currently, the most high-end commercial MLS systems (e.g., RIEGL VMX-2HA MLS system) provide 500 scan lines and two million measurements per second, which can collect high-density 3D point clouds for an extended range of applications, including city modeling, tunnel surveying, intelligent transportation systems (ITS), architecture and façade measurement, and civil engineering.
In order to fully describe the feasibility of MLS systems for urban object detection and extraction, this part details (1) the components of MLS systems, (2) direct geo-referencing, and (3) error analysis.

2.1. Components of MLS System

MLS systems are applied to point clouds data acquisition of urban road networks and any objects on both roadsides using multiple sensors onboard moving vehicles. Figure 2 presents crucial subsystems of a Trimble MX-9 MLS system and a RIEGL VMX-2HA MLS system, respectively. Additionally, the GNSS, the IMU, and the DMI constitute a POS for mobile mapping applications.
(1) Laser scanner
These laser scanners continuously emit laser pulses with a near-infrared wavelength to the surfaces of specified targets and digitize the backscattered signals. Then, based on light travel time, the scanning distances between the scanners and scanning targets can be calculated to obtain coordinates and intensity information. Currently, two state-of-the-art techniques are applied to range measurements in the plurality of MLS systems: time of flight (TOF) and phase shift [9].
In a certain medium, laser pulses move at a finite and constant velocity [9]. Therefore, the scanning range between the laser sensors and the target can be determined by calculating time-of-flight (also known as time delays) between light pulse transmission and return when these pulses are backscattered from the target [10]. Most commercial MLS systems are equipped with TOF scanners, since such scanners provide a farther measurement range than phase-based scanners [11]. When compared to TOF scanners, phase-based scanners emit continuous amplitude-modulated waves using phase differences. Thus, the distance between the MLS system and the target is calculated depending on the phase shift from light beam transmission and reception.
(2) Digital camera
Advances in the development of digital cameras have driven the applicability of MLS systems [12]. In order to provide visual coverage, most commercial MLS systems are integrated with high-end digital cameras, so that point clouds are realistically rendered using optical color imagery data. Additionally, stable image geometry provides detailed information to enhance the reliability and applicability of point clouds data [13]. Therefore, most optical camera manufacturers focus on the research and development of customized digital cameras to meet any application specifics of MLS systems. For instance, RIEGL VMX-2HA MLS systems are equipped with nine digital cameras to capture rich geometry information (e.g., color and texture information) of targets. Since most laser scanners provide the reflectivity at the laser wavelength as texture information, while color image data is lack of coordinate reference, a combination of these two types of data can be applied to generate colorized point clouds, which is widely applied in elaborated visualization and 3D city model construction.
However, digital cameras play a secondary role that they are applied in visualization, while laser scanners that are integrated into MLS systems are the primary source of precisely georeferenced data [14]. Additionally, perspective distortions of images are caused by changes in imaging distances and angles when the vehicle-based MLS system is moving. Moreover, for manufacturers of MLS systems, it is challenging to ascertain a proper size format digital camera (e.g., medium format digital camera) according to both point density and the driving speed of such systems.
(3) Global navigation satellite system & Orientation system
As mentioned before, the GNSS, the IMU, and the DMI (or odometer) constitute a POS system, which can be utilized to ascertain localization information of the MLS system. The GNSS provides highly precise positioning information up to centimeter accuracy and offers two additional observations: velocity and time. However, the positioning accuracy would decline due to multipath propagation issues caused by high-rise obstacles (e.g., tall trees, buildings and tunnels). For instance, a GNSS receiver is surrounded by buildings or under tree canopies. To eliminate multipath effects and to overcome the limitation of GNSS signal loss, an IMU is therefore used to provide the immediate position, velocity, and attitude of an MLS system. Typically, an IMU can interpolate stable and continuous positioning information in case of GNSS dropouts [11]. Additionally, three gyroscopes and three accelerometers are installed in an IMU, which can be used to measure three-axis angular rotations and three-axis accelerations. Since the positioning and orienting precision would decrease with the increase of measurement time, the combination of GNSS/IMU can improve positioning accuracy [8]. Consequently, the IMU provides relative position and orientation information in the periods of weak satellite signals, while a GNSS presents detailed continuous positioning information to the IMU. Furthermore, a DMI that is installed on the vehicle’s wheel using data transmission cable is capable to offer the observation to constrain error drift, especially for urban areas with unstable GNSS signals.
(4) Central control system
A highly integrated control system, as a sophisticated software unit, is designed to control all sensors (e.g., laser scanners and digital cameras), synchronize data from such sensors, and store trajectory information obtained from a POS system.

2.2. Direct Geo-Referencing

The mechanism of direct geo-referencing is elucidated in Figure 3. According to the scanning angle α and the scanning range d of a certain point P, its position is thus determined in its coordinate system. In addition, such a position in the coordinate system of mapping frame can be transformed from the laser scanner system.
Table 1 lists parameters that are involved in the direct geo-referencing transformation, and the positioning information vector of target P is calculated using Equation (1) [14]:
[ X P Y P Z P ] = R M I M U ( ω , φ , κ ) · ( [ l X l Y l Z ] + [ L X L Y L Z ] + r P S ( α   d ) · R I M U S ( Δ ω , Δ φ , Δ κ ) ) + [ X G N S S Y G N S S Z G N S S ]
As shown in Table 1, [XP, YP, ZP]T presents the positioning information of target P in the certain mapping system; [XGNSS, YGNSS, ZGNSS]T shows the positioning information of the GNSS antenna in the same mapping system; ω, φ, κ are roll, pitch and yaw details of IMU in the mapping coordinate system; ∆ω, ∆φ, ∆κ are bore sight angles that bring the scanners into correspondence with the IMU; α and d denote the incident angle and shooting range of laser pulses; and, other parameters are identified via system calibration.

2.3. Error Analysis for MLS Systems

As shown in Equation (1), the relationship among observation parameters is defined to generate direct geo-referenced point clouds data. Thus, several typical errors of these parameters can result in accuracy reduction for data acquisition and correction.
(1) Laser scanning errors
Based on Equation (1), the data acquisition accuracy is impacted by ranging error and incident angle error. A ranging error is involved by the system errors and indeterminate intervals, which are utilized to measure the TOF and the width of output laser pulses. Moreover, an incident angle error is involved because of the angular resolution and the uncertainty of laser pulses divergence [15].
(2) IMU errors
An IMU sensor in an MLS system indicates roll, pitch, and yaw details that define the rotation matrix between the IMU and the mapping coordinate system. An IMU unit mainly includes two components: (1) three orthogonal accelerometers that are employed to measure changes in position along a specific axis; and, (2) three orthogonal gyroscopes that maintain an absolute angular reference according to the principles of conservation of angular momentum. Typically, common errors of the IMU contains accelerometer biases and gyroscope drifts, such as gravity misalignment, scale factor, and random walk (sensor noises). Generally, the IMU accuracy is specified in the manufacturers’ technical specification. For instance, an Applanix POS LV 520, which is configured within a RIEGL VMX-2HA MLS system, provides 0.005° in both roll and pitch angles and 0.015° in yaw angle within one standard deviation.
(3) Localization errors
The localization accuracy of GNSS systems is mainly impacted through many reasons, including multipath effects, atmospheric errors, receiver errors, and baseline length [16]. Thus, determining the absolute localization accuracy for a GNSS measurement is always challenging. By using differential GNSS and real-time kinematic (RTK) methods, the localization accuracy is often expected to be 1 cm + 1 ppm horizontally and 2 cm + 1 ppm vertically for a short kinematic baseline length with local GNSS-reference stations.
(4) Lever-arm offset errors
Highly precise geo-referenced MLS data can be obtained if the lever-arm offsets are known. Calibration and physical measurement methods are commonly applied to determine such offsets. The first approach is not extensively applied because it is challenging to implement when compared to the second method. However, physical measurement errors can evolve due to the assumption of the alignment of two sensor’s axes [8].
Based on the discussion about error sources, it demonstrates that the overall performance of an MLS system relies on the accuracy of resultant positions by laser scanners and a GNSS/IMU subsystem. Lever-arm offset errors are effectively eliminated by appropriate system calibration and measurement. Moreover, navigation solution can make an impact on the overall accuracy of MLS data collection. Multipath propagation effects and GNSS-signal loss that are caused by trees and constructions along the roadsides can decrease GNSS localization accuracy [16]. Accordingly, a postprocessing operation of trajectory data is performed to increase the MLS data accuracy.

2.4. Introduction of Several Commercial MLS Systems

MLS systems are attacking worldwide attention with time. To date, based on the design and development of MLS-related technologies (e.g., laser scanning, positioning devices, and digital imaging), many MLS systems have hit the market. The worldwide manufacturers in the surveying and digital mapping industries (e.g., Leica, RIEGL, and Velodyne) have been well-established as suppliers of laser scanners or MLS systems. Table 2 summaries the several commercial-available MLS systems and Figure 4 illustrates their configurations.
As illustrated in Table 2, Faro Focus3D laser scanner integrated within a Road-Scanner C MLS system provides the most accurate mobile measurements by using phase shift measuring technique, while the SICK LMS 291 laser scanners present the worst measurement accuracy with 35 mm absolute measurement accuracy. Furthermore, RIEGL series and Trimble series laser scanners can achieve measurement ranges over 420 m due to employing the TOF measuring technique. In particular, the scanning range of a RIEGL VQ-450 scanner approximates 800 m.
Additionally, RIEGL VMX-2HA, VQ-450, Trimble MX-9, and Lynx HS-600 scanners can operate with a full 360° field of view, while SICK LMS 291 and Faro Focus 350 scanners provide less than 360° coverage. Besides, the system portability is an important factor when suppliers design and develop their MLS systems, which enables customers to conduct the survey conveniently. An IP-S3 MLS system is compact and highly integrated within a small case for easy installing on the roofs of various motorized vehicles. Similarly, VMX-2HA, VMX-450, Trimble MX-9, and Lynx SG are relatively easily moved despite that they have a portable control unit mounted inside the trunk. However, the system portability of Road-Scanner C MLS system is hindered because of its large-sized control unit.
Moreover, point density is a significant attribute in many applications, including road information inspection, landslide detection, deformation monitoring, and forest health inspection using MLS point clouds. Additionally, point densities of different laser scanners are determined based on the scan rate and driving speed of the specified MLS platform [11]. For instance, a RIEGL VQ-450 scanner is able to acquire highly dense point clouds over 8000 points/m2 at a driving speed of 60 km/h. Moreover, point density will increase with the shorter measurement range.
As shown in Figure 4, all MLS systems can provide very high point cloud densities within short measuring distances (e.g., the distance to roadside buildings). Accordingly, the RIEGL VQ-450 and VUX-1HA, and Optech Lynx HS-600 Dual laser scanners can produce relatively large data volumes as compared to SICK LMS 291 scanners integrated within Topcon’s IP-S3 Compact+ system.
Meanwhile, driving speed has a great impact on point density. For this reason, users are able to determine the desired point density by adjusting the proper driving speed and incremental angle. For instance, for a Trimble MX-9 MLS system, the scan rate can reach 500 profiles per second from a moving vehicle at regular traffic speeds. When compared with other laser scanners that are listed in Table 2, the RIEGL VQ-2HA and Trimble MX-9 system can provide the best specifications of point density. Therefore, high-density point clouds enable a wide variety of applications of MLS systems for monitoring and detecting road conditions, especially in complex urban road environments.

2.5. Open Access MLS Datasets

To further expand the applications of MLS technologies, more and more national governments, research institutions, and universities are devoting to releasing and sharing their MLS datasets free to the public. The Robotics laboratory (CAOR) at MINES ParisTech, France released the Paris-rue-Madame dataset [22] that contains approximately 160 m long urban street MLS point clouds. This dataset, which was obtained by the LARA2-3D MLS system, has been classified and segmented, which makes the point-wise evaluation of MLS point-based detection, segmentation, and classification methods possible. More recently, MINES ParisTech also made Paris-Lille-3D dataset [23] publicly available. Such a dataset has been entirely labeled with 50 types of object classes to assist the related research communities on automated point cloud detection and classification methods. TUM City Campus MLS point cloud dataset [24] was collected by Fraunhofer IOSB using two Velodyne HDL-64 laser scanners. Several classes have been assigned and labeled as: unlabeled, high vegetation, low vegetation, natural terrain, artificial terrain, building, hardscape, artefact, and cars. This urban street dataset contains 1.7 billion points over 62 GB of storage, which is suitable for the development of methods for 3D object detection, SLAM-based navigation, and for transportation-related applications, including ITS and city modeling.
Moreover, the Perceptual Robotics Lab (PeRL) at the University of Michigan provides the Ford Campus Vision and LiDAR Data Set [25] obtained by a pickup truck-based autonomous vehicle integrated high-end RIEGL LiDAR sensors around the Ford Research campus, as well as downtown Dearborn, Michigan. This dataset approximates 100 GB with high-density point clouds to help research on real-time perceptual sensing, autonomous navigation, and high-definition maps for autonomous vehicles in prior unknown traffic environments. Furthermore, a 147.4 km long North Campus Long-Term Vision and LiDAR Dataset [26] is publicly available collected by the PeRL to facilitate research and development focusing on long-term autonomous driving function in dynamic urban street environments.
Based on a wagon-based MLS with high-resolution color and grayscale video cameras and Velodyne laser scanners, the KITTI Vision Benchmark Suite [27] project that was conducted by the Karlsruhe Institute of Technology and Toyota Technological Institute at Chicago released massive images, point cloud datasets, and benchmarks. Such datasets were collected in urban areas, rural areas and on highways around the city of Karlsruhe, which provided detailed road information (e.g., cars and pedestrians) to support computed vision related missions, such as 3D tracking and 3D object detection. Meanwhile, the Cityscapes Datasets [28] focusing on the semantic understanding of urban street scene were publicly accessible with a diversity of 50 cities and 30 classes. More recently, Large-scale Point Cloud Classification Benchmark dataset [29] that is a benchmark for classification with eight class labels was released to boost the applications in robotics, augmented reality and urban planning. Such datasets contain fine details of urban streets and provide labeled 3D point clouds with over four-billion points in total. These publicly accessible MLS datasets boost the development of city modeling, transportation engineering, autonomous driving, and high-definition navigation maps.

3. On-Road Information Extraction

The traffic information and inventory on the road, including road surfaces, road markings, and manmade holes, are significant and necessary elements in the geometric design of roadways to provide guidance, warning, and bans for all road users (e.g., drivers, cyclists, and pedestrians). According to [8], MLS data can be utilized for accurately detecting and extracting various types of on-road object inventory, which are mainly classified as five categories: road surfaces, road markings, driving lines, road cracks, and road manholes. This section contains a systematic and comprehensive literature review of backgrounds and studies that are related to urban on-road object detection and extraction using MLS point clouds. Figure 5 indicates the applications and related experiment results of on-road information extraction using MLS point clouds.

3.1. Road Surface Extraction

Road surfaces, as an important urban road design structure, play an important role in ITS development, urban planning, 3D digital city modeling, and high-definition roadmap generation [30]. The relatively large data volumes obtained by MLS systems in an urban environment consist of different kinds of objects, including roads, buildings, pole-like objects (e.g., trees, traffic signs, and poles), cars, and pedestrians. In general, efficient detection and accurate extraction of road surfaces are the prerequisites for the extraction and classification of other on-road inventories (e.g., road manholes) [31]. It aims to reduce the data volume and boost the subsequent process straightforward and reliably [7]. Furthermore, filtering out off-ground point clouds can minimize computational load and save memory space. A variety of methods and algorithms have been developed for road surface detection and extraction using MLS data. According to different data formats, such methods are mainly categorized into three groups: 3D point-based extraction, two-dimensional (2D) feature image-based extraction, and other data source based extraction [8]. Figure 6 presents several road surface extraction algorithms and a few representative experiment results.

3.1.1. 3D Point-Based Extraction

Road surface extraction is conducted using either MLS point clouds directly or 2D feature imagery derived from 3D point clouds. The majority of algorithms directly extracted pavement, while the others first extracted road edges or road boundaries [32,33,34]. By applying the derivatives of the Gaussian function to MLS point clouds, Kumar et al. [32] effectively extracted road edges. According to the parametric active contour and snake model, they implemented a revised automated method for road edge extraction from MLS data [33]. Additionally, Zai et al. [34] proposed a supervoxel generation algorithm that automatically extracted road boundaries and pavement surfaces while using MLS point clouds. Wang et al. [35] developed a computationally efficient approach integrating Hough forest framework with supervoxel to remove pavement from raw 3D mobile point clouds for further object detection. By calculating angular distances to the ground normals, Hervieu and Soheilian [36] effectively extracted road edges and road surfaces from MLS point clouds. Hata et al. [37] extracted ground surfaces by employing various filters, including differential filter and regression filter, on 3D MLS points. Guan et al. [38] proposed a curb-based road extraction method to separate pavement surfaces from roadsides with the assumption that road curbstones can represent the boundaries of pavement. Xu et al. [39] proposed an automated road curb extraction method using MLS point clouds. First, candidates’ points of pavement curbs were extracted by an energy function. Then, such candidate points were refined based on the proposed least cost path model. Riveiro et al. [40] implemented the road segmentation using a curvature analysis that is directly derived from MLS point clouds. Additionally, a voxel-based upward growing method was performed for direct ground point segmentation from entire MLS point clouds [7].
In order to enhance computational efficiency, trajectory data were commonly used [31,38,41]. Wu et al. [42] proposed a step-wise method for off-ground point removal. Firstly, raw point clouds were vertically partitioned along the vehicle’s trajectory. Then, the Random Sample Consensus (RANSAC) method was employed for ground point extraction by determining the average height of ground points.
As mentioned, MLS systems collect data by multiple scan lines. The spatial configuration of the scan line relies on the user-defined parameters of a specific MLS system, such as driving speed, sensor trajectory, and sensor orientation. Consequently, the high-density pavement points decrease the computation complexity in pavement segmentation by processing scan lines [38,43,44,45]. According to scan lines, Cabo et al. [43] proposed a method depending on line cloud grouping from MLS point clouds. The input point clouds were first organized into lines covering pavements. Subsequently, such lines were clustered based on a group of quasi-planar restrictions. Finally, road edges were extracted from the end points of lines. The scan line related methods have been demonstrated to be efficient, but they still have challenges in processing unordered MLS data that lack the characteristics of the scan lines’ sequence [30].
Furthermore, some studies focus on directly extracting road surfaces from MLS data, according to the data characteristics and road properties, including point intensity, road width, elevation, and smoothness [32,46,47]. According to the MLS system mechanism, point density is negatively correlated to the shooting range from the onboard laser scanners. Kumar et al. [32] filtered out off-ground points from raw MLS point clouds by taking point densities into consideration. Guo et al. [48] removed non-road surface points by combining road width and the elevation data. Other valuable data characteristics and road properties include local point patterns, laser pulse information, and height-related data (e.g., slope, height mean value, and height variance). In many existing methods, certain characteristics of MLS point clouds and road properties were combined to segment pavement points. Such methods extracted road surfaces using either local scales or global scales, but they were computationally intensive and time-consuming.
Some studies concentrated on a unique road object, while the others performed the classification which the whole 3D point-based scene was classified into different types of objects, including traffic lights, roadside trees, building facades, and road pavement [41,46,49,50]. By considering shape, size, orientation, and topotaxy, Pu et al. [41] were capable of automatically extracting road surfaces, ground objects, and off-ground objects within a 3D point-based scene. Fan et al. [46] classified the point clouds into six groups, including roads, cars, trees, traffic signs, poles, and buildings, based on point density, height histogram, and fine geometric details. Moreover, Díaz-Vilariño et al. [50] proposed a method using roughness descriptors to automatically segment and classify asphalt pavements while using MLS point clouds. Although these studies indicated promising solutions for road surface extraction from MLS point clouds, other data sources, including optical RGB imagery, are of vital importance in the classification of more types of roads and defect detection on the pavements.

3.1.2. 2D Feature Image-Based Extraction

Converting 3D MLS point clouds into 2D geo-referenced feature (GRF) images enables decreasing computational complexity at the stage of road surface extraction. By using the existing computer vision and image processing methods, road boundaries and pavements can be efficiently detected and extracted [51]. Pavement segmentation of range scan lines was implemented in 2D, Cabo et al. [43] conducted an experiment to differentiate road inventories (e.g., trees, high-rise buildings, and road surfaces) based on the height deviation. In order to minimize the computational effort, Kumar et al. [52] proposed a 2D approach by projecting the 3D point clouds onto the XOY plane and assigning to a lattice of adjacent squares covering the whole 3D points scene. Subsequently, the generated 2D elevation image was processed to detect the road curbs by using active contour models. Riveiro et al. [40] firstly projected the entire MLS point clouds onto a 2D space. The principal component analysis (PCA) was then employed to detect peaks (e.g., altitude and deflection angle) denoting the transversal limits of the road. With the assumptions that road surfaces are large planes with a certain distance to trajectory data of MLS systems and the normal vectors of road surfaces are approximately parallel to the Z-axis, Yang et al. [53] segmented road surfaces by generating GRF images to filter out off-ground objects from the raw MLS point clouds.
Although studies on road surface extraction using image processing algorithms from MLS point clouds have been pursued for years, fully automated road surface extraction is still a huge challenge. Additionally, it is still difficult to deal with the steep terrain environments from 2D GRF images.

3.1.3. Other Data-Based Extraction

To eliminate the limitations of MLS point clouds (e.g., data occlusion and unorganized data distribution), other data sources, including high-resolution satellite imagery, unmanned aerial vehicle (UAV) imagery, ALS data, and TLS data, were integrated with MLS point clouds. Choi et al. [54] created detailed 3D geometric models of road surfaces based on the data fusion of MLS point clouds, video data, and scanning profiles. Jaakkola et al. [49] employed image processing algorithms to the intensity and height images for road curbstone segmentation, and the road surfaces were modeled as triangulated irregular networks (TIN) using 3D point clouds. With the assistance of such data, the extraction accuracy can be boosted by providing additional pavement texture information. Boyko and Funkhouser [55] proposed a method to combine multiple aerial scans and TLS highly dense point clouds of urban road environments together for completely extracting road surfaces. With the assistance of ALS data, Zhou and Vosselman [56] implemented a sigmoid function to the 3D points near the detected curbstone, then adapted for processing of MLS data. Accordingly, Cavegn and Haala [57] proposed a method to efficiently extract pavement surfaces that are based on the TLS points, images, and scaled numeric maps.
Combining other data sources with MLS point clouds enables both accuracy and correctness enhancement of road surface extraction. However, different input data might be collected by different sensors at multiple times, under different weather conditions, with different densities and resolutions, and under various illuminations, it makes the data registration, data calibration, and data fusion challenging.
Table 3 is a summary of three categories of road surface extraction methods. Although the existing image processing algorithms can be efficiently performed on 2D GRF imagery derived from 3D MLS point clouds [44], it is challenging to deal with steep terrain environments. Thus, the accuracy of pavement extraction results can be improved based on the additional detailed road information supported by other data sources. When compared to 2D GRF image-based road surface extraction, 3D point-based methods are capable of extracting road surfaces directly from MLS data, and segmenting road surfaces in the local-global scales [35,38,58]. However, these methods are mostly computationally intensive and time-consuming.

3.2. Road Marking Extraction and Classification

Road markings, including lane lines, center lines, zebra crossings, and arrows, provide necessary road information (e.g., warning and guidance) for all road users. Identifying and extracting road markings accurately is significant for advanced driver assistance systems (ADAS) and autonomous driving systems to plan reliable navigation routes and prevent collisions in complex urban road networks. When considering urban road condition and road topography, the availability and clarity of road markings are critical elements in traffic management systems and accidents where road networks themselves are the cause. For instance, the fatal accident rate is high due to the damage of clearly painted road markings, especially in highly complex urban road and highway environments [59].
Road markings are highly retro-reflective materials painted on asphalt concrete pavements. Thus, the relatively high-intensity value is considered to be a unique characteristic for road marking extraction from using point clouds [40]. Accordingly, a variety of methods are mainly categorized into two classes depending on semantic knowledge (e.g., shape) and MLS intensity properties: 2D GRF image-driven extraction, and 3D point-driven extraction. Figure 7 shows several experiment results by using different road marking extraction methods.

3.2.1. 2D GRF Image-Driven Extraction

The majority of studies focused on the road marking extraction using 2D GRF images interpreted from 3D points. According to semantic information (e.g., size, orientation, and shape) of road markings, the existing image processing approaches were widely used, including Hough Transform, multi-thresholding segmentation, morphology, and Multi-Scale Tensor Voting (MSTV) [13,69,70,71]. Toth [13] conducted a simplified intensity threshold segmentation for the extraction with regard to intensity distribution in different searching windows. Accordingly, Li et al. [70] implemented a global intensity filter to roughly segment road markings based on the generated density-based images. In order to efficiently extract lane-shaped road markings (e.g., broken lane lines and continuous edge lines), Yang et al. [71] performed a Hough Transform method in four connected regions of the georeferenced intensity images. However, the applications of Hough transformation for road marking extraction is limited by defining the quantity of road marking types to be segmented, especially in the process of dealing with multiple types of road markings (e.g., arrows). In contrast, multi-thresholding segmentation methods were developed through exploiting the relationship between the intensity values and scanning ranges, which were commonly applied to overcome intensity inconsistency due to different scanning patterns and non-uniformity of point cloud distribution [69]. Accordingly, Guan et al. [38] and Ma et al. [47] dynamically segmented road markings using multiple thresholds regarding different scanning distances that are based on point densities, followed by a morphological operation with a linear structuring feature. Kumar et al. [12] performed a scanning range dependent thresholding algorithm to identify road markings from the georeferenced intensity and range imagery. Furthermore, the MSTV approach can suppress high-level noises and preserve complete road markings. In the study that was conducted by [14], road marking segmentation was improved by using weighted neighboring difference histogram based dynamic thresholding and MSTV algorithms from the noisy GRF images.

3.2.2. 3D Point-Driven Extraction

Meanwhile, most studies directly extracted road markings from MLS point clouds rather than the generated GRF images. Chen et al. [72] and Kim et al. [73] performed a profile-based intensity analysis algorithm to quickly segment pavement markings from 3D MLS points. Firstly, raw MLS point clouds were partitioned into point cloud data slices based on the trajectory of vehicles. Then, pavements were detected and extracted by considering the geometric properties of road edges, boundaries, and barriers. Finally, line-shaped pavement marks were efficiently extracted by determining the peak value of intensity in each scan line. Additionally, Yu et al. [74] directly derived road markings using MLS point clouds and classified them into large-sized (e.g. stop line and centerline), small-sized (e.g., arrow), and rectangular-shaped (e.g., zebra crossing) road markings. According to trajectory data and road curb lines, large-sized road markings were firstly extracted by using multiple threshold segmentation and spatial point density filtering methods, while the Otsu’s thresholding approach [75] was adopted to select multiple optimal thresholds. Then, small-sized road markings were extracted and classified according to Deep Boltzmann Machines (DBMs)-based neural networks. Finally, rectangular-shaped markings were efficiently classified by performing a PCA-based method. Soilán et al. [76] computed several discriminative features for individual road marking based on the geometric parameters and pixel distribution. Subsequently, each road marking was defined as a Geometry Based Feature (GBF) vector that was the input of a supervised neural network for road marking classification.
Table 4 summarizes road marking extraction and classification methods in terms of 2D image-driven and 3D MLS point-driven extraction. Converting MLS point clouds into 2D GRF images is effective to overcome intensity inconsistency and density variance issues due to different scanning patterns. However, complex types of road markings (e.g., words) make the extraction process via 2D feature image processing algorithms challenging. By comparison, MLS point-driven extraction methods concentrating on directly segment road markings from raw MLS point clouds can achieve both completeness and correctness improvement. In addition, detailed geospatial information road markings can be preserved for further applications. However, automated road marking extraction from the large-volume MLS point clouds especially with strong concavo-convex features and unevenly distributed point clouds is still a very difficult task [75]. Accordingly, many studies were conducted for automated road marking classification by using deep learning based neural networks (e.g., PointNet [77] and PointCNN [78]), but such methods need massive labeled road markings for training purposes and they have limitations in processing extra-large MLS data volume for urban road networks.

3.3. Driving Line Generation

Driving lines, defined as the driving routes for motorized vehicles or autonomous vehicles, play a critical role in the generation of high-definition roadmaps and fully autonomous driving systems [42]. When considering the turning speed limitation, driving lane departure, and vehicle-handling capability issues, one of the most challenging tasks to achieve fully autonomous driving is to enable autonomous vehicles to navigate and maneuver themselves at complex urban environments without direct human intervention [79]. Therefore, generating highly accurate driving lines within centimeter-level localization and navigation accuracy can boost the development of high-definition roadmaps and autonomous vehicles.
Recently, many worldwide automotive manufacturers (e.g., BMW, Ford, and Mercedes-Benz) and digital map suppliers (e.g., HERE, Civil Maps, Ushr, and nuTonomy) are generating driving lines from MLS data. Ma et al. [47] proposed a three-step process to generate horizontally curved driving lines. Firstly, with the assistance of the trajectory of the MLS system, the curb-based road surface extraction algorithms were adopted to segment road surfaces from raw MLS point clouds. Secondly, road markings were directly extracted by using the multi-threshold segmentation method, followed by a statistical outlier removal (SOR) filtering algorithm for noise removal. Finally, the conditional Euclidean clustering algorithm was performed, followed by a nonlinear least-squares curve fitting algorithm. Consequently, the driving lines were efficiently generated with 15 cm-level localization accuracy.
Additionally, Li et al. [80] firstly implemented a voxel-based upward growing algorithm to extract the ground points from the entire MLS point clouds. Then, road markings were categorized into two types (i.e., lane markings and textual road markings). Such road markings were separately extracted based on the distance-to-road-edge thresholding algorithm and related road design standards. Lastly, 3D high-definition roadmaps were generated with detailed road edge, road marking, and the estimated lane centerline information. Furthermore, Li et al. [81] proposed a step-wise method for road transition line generation while using MLS data. The region growing algorithm was first proposed to enhance the curb-based road surface extraction performance. Subsequently, the multi-thresholding segmentation and geometric feature filtering algorithms were conducted for lane marking extraction. Finally, a node structure generation algorithm was developed to generate lane geometries and lane centerlines, followed by a cubic Catmull-Rom spline approach for transition line generation. Figure 8 indicates the generated driving lines from MLS data for high-definition map generation.

3.4. Road Crack Detection

Road cracks, as a common type of distress in asphalt concrete pavement, are caused by road surface fractures due to overloaded heavy-duty trucks, thermal deformation, moisture corrosion, and road slippage or contraction. Rapid and accurate perception of road cracks not only enables the local transportation departments to perform monitoring and repair for traffic efficiency, but it also provides necessary information to the ITS for the probabilities of potential road surface distresses and traffic risks [82].
With the development of laser imaging techniques, MLS data are commonly utilized in road crack detection applications. In addition to providing an efficient promising solution for road crack detection using high-resolution digital imagery, MLS point clouds are valuable data sources for evaluating road surface distresses. Morphological mathematics based algorithms were implemented for crack extraction from pavement GRF images [83]. Tsai et al. [84] summarized the previous six commonly used pavement crack extraction and classification methods: Canny edge detection, regression or relaxation thresholding, crack seed verification, multi-scale wavelets, iterative clipping, and dynamic optimized method. However, most methods are computationally intensive and time-consuming. The intensity information of road cracks normally indicates lower values in comparison with their neighbors, Yu et al. [85] employed the Otsu thresholding algorithm to extract crack candidates. Next, a spatial density filtering algorithm was performed for outlier removal. Finally, crack points were clustered into crack-lines, and crack skeletons were extracted by performing an L1-medial skeleton extraction algorithm. However, due to either grayscale similarity or discontinuity, the correctness and robustness of the thresholding-based segmentation algorithms mainly rely on road surface materials and surrounding environments, which can cause unreliable crack extraction results.
Meanwhile, many studies focused on extracting road cracks using advanced data mining, artificial intelligent (AI)-based, and neural network methods [86,87]. However, the training process needs a lot of labeled data, and the selection of parameters mostly depends upon data quality and crack variations. Accordingly, Guan et al. [14] developed a method framework, called ITVCrack, to automatically extract pavement cracks by using the iterative tensor voting (ITV) method. The curvilinear cracks were efficiently delineated based on the ITV-based extraction algorithm, and a four-pass-per-iteration morphological thinning algorithm was afterward performed to remove noise. Additionally, Chen et al. [82] proposed a method for detecting asphalt pavement cracks by generating Digital Terrain Model (DTM) from MLS point clouds. Then, local height changes that may relate to cracks were detected based on a high-pass filter. Finally, a two-step matched filter (i.e., Gaussian-shaped kernel) was used to extract crack features.

3.5. Road Manhole Detection

Road manholes, as a major road infrastructure on the urban environment, provide entry to conduits that are utilized to perform drainage, steam, liquefied gas, communication cables, optical fiber, and other underground utility networks. Typically, road manholes are covered using concrete-made or metal-made materials to prevent road users from falling into the wells. Such manhole covers can be detected while using MLS systems to achieve higher time-efficiency and better cost-savings than human field surveying, in a safe and effective way. Based on the various data format, the majority of existing methods for road manhole cover detection are classified into two types: 2D imagery-based methods and 3D point-based methods.
Due to the complex background of pavement images, Cheng et al. [88] proposed an optimized Hough transformation approach for the manhole cover detection. Firstly, all the contours were determined by contour tracking from binary edge imagery. False detections were then filtered out using contour filters. Finally, circular-shaped manhole covers were extracted using the revised Hough transform algorithm. Additionally, in the study conducted by [89], the single view and multiple view processing methods were performed for manhole cover detection from road surface images. Subsequently, Niigaki et al. [90] developed a novel method to identify blurred manhole covers from low-quality road surface images. By focusing on the uniformity and separability of the intensity distribution, the Bhattacharyya coefficient was fine-tuned and three operators (i.e., uniformity operator, oriented separability operator, and circular object operator) were defined to detect manhole covers.
Meanwhile, MLS point clouds have also been widely used for road manhole cover detection. Guan et al. [91] proposed an MSTV algorithm to detect road manhole covers from MLS point clouds. First, ground points were extracted and rasterized into 2D GRF images. Next, the manhole cover candidates were determined based on distance-dependent intensity thresholding. Then, an MSTV algorithm was implemented for noise suppression and manhole cover pixel preservation. Finally, manhole covers were extracted through distance-based clustering and morphological operations. Yu et al. [92] presented a novel method for automated road manhole cover detection using MLS data. First, in order to improve the computational efficiency, off-ground points were filtered out through a curb-based road surface segmentation algorithm and rasterized into GRF intensity images via inverse distance weighted (IDW) interpolation approach. Then, a supervised deep learning neural network was trained to build a multi-layer feature generation network to determine high-level features of local image patches. Next, the relationships between the high-level patch features and the probabilities of the presence of manhole covers were learned by training a random forest model. Finally, based on the multi-layer feature generation network and random forest model, road manhole covers were efficiently extracted from GRF intensity images.

4. Off-Road Information Extraction

In mobile LiDAR systems, laser scanners continually collect 3D points along the roads, which produces complicated and multi-dimensional, large volume data that requires specific processing. Thus, efficient storage and management of such data are necessary for complicated archive, rapid data dissemination, and processing for multi-level application, including interrogation, filtering, and product derivatization [93]. In the view of scanned MLS data, the ground points always occupy the largest part; however, our focused objects are set on the ground. Thus, data preprocessing that segment the raw data into different groups roughly, and then select the related information locally to extract off-road objects is particularly challenging.
Most researchers usually segment the raw data into three groups: ground, on-ground and off-ground points [42,61,94,95,96]. Table 3 lists six representative ground removal methods. Among these methods, voxelization and segmentation along the trajectory are commonly used. Voxelization is applied to simplify the huge amount and heterogeneous distribution of data. Thus, the computing cost is reduced by mapping the whole data in a three-dimensional grid without suffering a significant information loss by choosing correct scale, because the reduced version is stored and directly linked to the original data [97]. Trajectory data indicates precisely real-time positioning information of the vehicle, which can provide localization and orientation information of MLS systems [98]. Such trajectory data is commonly used as a filter to remove points that are far away from the road in the raw point clouds.
The remaining off-ground points will be reduced to a large extent when compared with the raw data when the ground and the on-ground points are removed. Within the off-road parts, those objects with various intensity values are still unorganized. A rough clustering is commonly performed to group the points into different clusters. Euclidean clustering and DBSCAN are two main density-based spatial clustering methods.
  • Euclidean clustering. This method is based on Euclidean space to calculate the distances between discrete points and their neighbors. Then the distances are compared with a threshold to group nearest points together. This method is simple and efficient. However, some disadvantages include no initial seeding system and no over- and under-segmentation control. In [2,97], Euclidean clustering was applied to segment the off-terrain points into different clusters. Due to the lack of under-segmentation control, Yu et al. [7,66,96] proposed an extended voxel-based normalized cut segmentation method to split the outputs of Euclidean distance clustering, which are overlapped or occlusive objects.
  • DBSCAN. DBSCAN is the most commonly used clustering methods without a priori knowledge of the number of clusters. Besides, the remaining data can be identified as noise or outliers by the algorithm. Riveiro et al. [99] and Soilán et al. [64] used the DBSCAN algorithm to obtain and cluster a down-sampled set of points. Before that, a Gaussian mixture model (GMM) was presented to determine the distribution of those points that matched sign panels. For some points which were not related to traffic signs, PCA was used to analyze the curvature to filter the noise. Yan et al. [100] used the module that was provided by the Scikit-learn to perform DBSCAN. The points with height-normalized above ground features were clustered and then identified and classified as potential light poles and towers.
After acquiring the rough off-road objects clusters, more detailed extraction solutions are analyzed in the following sections: pole-like object extraction, power line extraction, and multiple object extraction. Attributes of off-road urban objects are listed in Figure 9. These features play important roles in static urban object extraction in the following algorithms.

4.1. Pole-Like Objects Extraction

4.1.1. Traffic Sign Detection

Accurate recognition and localization of traffic signs are significant in intelligent traffic-related applications (e.g., autonomous driving [101] and driver-assisted systems), which help machines to respond in a timely and accurate manner in different situations. The high accuracy of points (about 1 cm) in the meter-level area along with imagery provides promising solutions for traffic sign detection (TSD) and recognition (TSR) while using mobile laser scanning techniques. Accurate geometry and localization information is provided by point clouds; whereas, detailed texture and semantic information is contained in digital images [96]. Existing algorithms apply geometric and spatial features of traffic signs [65,102,103] or learn these features automatically [95,104,105,106] to achieve TSD. TSR is commonly achieved by integrating imagery data and 3D point clouds together. These methods can be grouped into TSD using 3D point clouds and TSR using imagery data [1,2,94,95,104], as follows:
1. TSD using point clouds
Traffic sign point clouds have obvious geometric and spatial features. Table 5 lists the attributes of them, which are employed in the most of existing traffic sign detection methods using point clouds [65,97,99,105,108]. The metallic material of traffic signs makes their surfaces have a strong reflectance intensity, for laser pulses hitting the retroreflective surfaces and returning to the receivers without much signal loss. Besides, traffic signs are commonly installed at a certain height along the roadsides. The sizes of them are limited to a certain range and their shapes are regular, such as rectangle, circle, and triangle.
Wen et al. [65] set the minimum number of points in clusters (setting as 50) to remove small parts. Besides, point intensities below 60,000 are also filtered as non-sign objects. Yu et al. [66] also used intensity filter to extract traffic sign clusters. But, before that, an improved voxel-based normalized cut segmentation method was applied to separate occlusive and overlapped objects when Euclidean distance clustering failed. Ai et al. [94] used scanning distances and incidence angles to normalize each clusters’ retro-intensity value. The comparison between the median value from the population of the normalized retro-intensity and the selected threshold was conducted to assess the traffic sign retro-reflectivity condition. In [63], traffic sign attributes, such as planar surfaces and an enclosed range of heights, were used in detection solutions. A height filter was proposed to eliminate clusters with heights that were smaller than 25 cm. Planar filters remove objects with reflective features that are not planar or small. Huang et al. [61] used the two same attributes to remove irrelevant points, then they applied a shape-based filtering to exclude the pole-like objects. Guan et al. [108] detected traffic signs using the intensity that extracted objects with low intensity and geometrical structures and removed non-planar parts.
2. TSD Using Point Clouds and Imagery
After finishing TSD tasks, TSR is commonly achieved by integrating point clouds and imagery data together. The fusion of point clouds and imagery are composed of two steps: (1) transform the 3D point clouds coordinate system from the world coordinate system to the camera coordinate system and (2) map the 3D points from the camera coordinate system to the 2D image plane [65]. After that, the TSR task is completed based on the projected area. One example of TSD and TSR is shown in Figure 10. Learning-based methods, such as machine learning methods and deep learning algorithms, are commonly used techniques in the TSR process.
  • Machine learning methods. Machine learning based TSR methods are most commonly used in the imagery-based TSR tasks in the past years. Wen et al. [65] assigned traffic sign types by a support vector machine (SVM) classifier trained by integral features composed of Histogram of Gradients (HOG) and color descriptor. Tan et al. [109] developed a latent structural SVM-based weakly supervised metric learning (WSMLR) method for the TSR and learned a distance metric between the captured images and the corresponding sign templates. For each sign, recognition was done via soft voting by the recognition results of its corresponding multi-view images. However, machine learning methods need manually designed features that are subjective and highly rely on the prior knowledge of researchers.
  • Deep learning methods. Recently, deep learning methods are applied in MLS regions for their powerful computation capacity and superior performance without human design features. By learning multilevel feature representations, deep learning models are effective in TSR. Yu et al. [96] applied the Gaussian–Bernoulli deep Boltzmann machine (GB-DBM) model for TSR tasks. The detected traffic signs point clouds were projected into 2D images, which produced a bounding box on the image. The TSR was achieved among the bounding box. To train a TSR classifier, they arranged the normalized image into a feature vector and then input it into to the hierarchical classifier. Then a GB-DBM model was constructed to classify traffic signs. This model contains three hidden layers and one logistic regression layer, which contains quantity activation units. These units were linked to the predefined class. Guan et al. [108] also conducted TSR using GB-DBM model. Their method was proved for rapidly handling large-volume MLS point clouds toward TSD and TSR. Arcos-García et al. [63] applied a DNN model to finish TSR from the segmented RGB images. This DNN model combined several convolutional, spatial transformers, non-linearity, contrast normalization, and max-pooling layers, which were proved useful in the TSR stage. These deep learning methods are proven to generate good experimental results. However, the point cloud traffic sign detection results have impacts on the recognition results. If the traffic sign is missing in the first stage, the following TSR results will be inaccurate.

4.1.2. Light Pole Detection

Street light poles, used to adjust illumination condition to help the pedestrians and drivers at night, are cost-effectively monitored, maintained, and managed for the transportation agencies [110,111]. MLS techniques, which provide high-density and high-accuracy point clouds, are applied effectively in the light pole detection [15]. Existing light poles detection algorithms can be grouped into the following types: knowledge-driven and data-driven methods.
(1) Knowledge-driven
Knowledge-driven solutions have been widely used in the pole-like object extraction. Figure 11 lists different types of light poles, such as one-side light, two-side light, and beacon light, etc. Knowledge of light attributes in Figure 11 can be used to recognize and model pole-like objects. e.g., Li et al. [112] set two constraints to extract light poles: (1) poles are commonly installed near the road curbs; and, (2) poles stand vertically to the ground within a certain height range. Knowledge-driven methods can be classified into two types: matching-based extraction and rule-based extraction.
  • Matching-based extraction. Matching-based extraction is one of the representative knowledge-driven methods, which is achieved by matching the extracted objects with pre-selected modules. Yu et al. [66] extracted light poles using a pairwise 3D shape context, where a shape descriptor was designed to model the geometric structure of a 3D point-cloud object. One-by-one matching, local dissimilarity, and global dissimilarity were calculated to extract light poles. In [7], light poles were extracted by matching the segmented batch with the pre-selected point cloud object prototype. The prototype was chosen based on prior knowledge. The feature points for this voxel was chosen from the points nearest the centroid of the voxel. Then, the matching cost for feature points between the prototype and individual grouped object was conducted via 3D object matching framework. Finally, the extraction results derived from the filtered matching costs from all segmented objects. Cabo et al. [97] spatially discretized the initial point clouds and chose the initial 20–30% representative point clouds as the inputs. The reduction of the inputs data kept model accessible without losing any information by linking each point to its voxel. Then, they selected and clustered small 2D segments that are compatible with a component of a target pole. Finally, the representative 3D voxel of the detected pole-like objects were determined and the points from the initial point clouds linking to individual pole-like objects were derived. Wang et al. [113] also introduced a similar matching method to recognize lamp poles and street signs. First, the raw data within an octree was grouped into multi-levels via 3D SigVox descriptor. Then, PCA was applied to calculate the eigenvalues of points in each voxel, which were projected onto the suitable triangle of a sphere approximating icosahedron. This step was iterated within multiple scales. Finally, street lights were identified via measuring the similarity of 3D SigVox descriptors between candidate point clusters and training objects. However, these three algorithms highly rely on the parameters and matching module selection. Thus, it is challenging to achieve automatic extraction. One-by-one matching has high computation cost when compared with other methods. These methods are not suitable for the large-volume data processing.
  • Rule-based extraction. Rule-based extraction is the other typical knowledge-driven method. Geometric and spatial features of light poles are setting as rules in the process of extraction. Teo et al. [114] applied several geometric features to remove the non-pole-like road objects. This algorithm includes feature extraction for each object and non-pole-like objects removal using geometric features (e.g., area of cross-section, position, size, orientation, and shape). Besides, pole parameters (e.g., location, radius, and height) were computed to extract feature, as well as hierarchical filtering was used to filter the non-pole- like objects. A height threshold (3 m) was used to classify the data into two parts, and do segmentation in horizontal and vertical directions. However, not all the pole-like objects are higher than 3 m, the parameter choosing is subjective. Rodríguez-Cuenca et al. [115] utilized the specified characteristics of the poles in the pole-like object extraction. Since poles stand vertically to the ground, this attribute is employed by the Reed and Xiaoli (RX) anomaly detection method for a linear pole structure with pre-organized point clouds. Finally, the vertical elements were detected as man-made poles or trees using the clustering method. However, if pole-like objects are overlapped by other objects, the disconnected upper parts are discarded as noise. Li et al. [116] used a slice-based method to extract pole-like road objects. A set of common rules were constrained to split and merge the groups. E.g., the connected components that contain vertical poles were analyzed for integrating or separating; the detached components as well as their nearby components were checked for further merging. Rule-based methods are rapid and efficient. But, the quantity of rules is hard to determine. The data would be over-segmented with numerous rules or under-segmented with insufficient rules. Especially, some true objects with poor data distribution are usually removed by strict rules.
(2) Data-driven
Plenty data-driven methods have been proposed for light pole extraction using MLS techniques in the past decade. A set of features can be designed and applied to train or learn a pole extraction model that is based on massive labeled training data. Thus, these methods can also be regarded as learning-based methods. Guan et al. [117] chose a set of 50 training datasets, which covered a road patch of about 50 m separately, to construct a contextual visual vocabulary for depicting the features of light poles. Based on the generated contextual visual vocabulary, a bag-of-contextual-visual-words was constructed to detect pole-like objects from the filtered off-ground points. The extraction stage is similar to Yu et al. [62] research. Wu et al. [42] proposed a localization method based on integration of 2D images and point clouds. There are three main steps, i.e., raw localization map generation, “ball falling”, and position detection. Then, the output of the three steps processing was viewed as prior information for segmentation. Once the first segmentation was finished, several features were computed to describe the pole: the height, the average height, the standard deviation of height, the estimated volume, the number of pole points, and the number of the super-voxels whose area of the convex hull. Besides, once the second segmentation was achieved, a set of features were computed to describe the global objects: four kinds of height-related features, the pixel intensity related to the location map, the area of the convex hull for whole mapped points, and the approximated volume, etc. These local and global features were fed into SVM and random forests classifiers. Yan et al. [67] used the Ensemble of Shape Functions (ESF) as well as geometric characters to describe the characteristics of light poles. The ESF includes three shape functions (point distance, area, and angle) and a ratio function. Geometric features of the objects, including linearity, planarity, scattering of the covariance matrix, ratio of the minor axis, and the major axis of Oriented Bounding Box, etc. are considered. Then, these characteristics were input into Random forest to train a pole-like objects classifier. Yan et al. [100] proposed an unsupervised segmenting algorithm to group the above ground point clouds. Several decision rules were defined to extract potential light poles. However, learning-based methods cannot interpret data on their own, which means that human designed feature extraction is needed before training.

4.1.3. Roadside Trees Detection

With the improvement of urban environments, a lot of landscape trees are planted along the urban streets. Timely measurement and management of trees are required to ensure their healthy growth without interference with traffic conditions. MLS point clouds have intensive sampling density and rich information, which can provide a promising solution for single tree extraction and tree species classification. Existing methods can be classified into two parts: rule-based and deep learning methods. Figure 12 illustrates some of the tree classification methods.
Rule-based methods commonly separate trees into two parts: leaf-off trunks and tree crowns. Xu et al. [118] presented a bottom-up hierarchical segmenting algorithm to merge non-photosynthetic clusters. Merging was conducted based on the dissimilarity between two groups. Euclidean distance was used to calculate the proximity matrix; whereas, the principal direction was applied to compute the direction. The key contribution is the cluster combination optimization with minimum energy function and extraction of non-photosynthetic components using an automatic hierarchical clustering. Li et al. [119] extracted individual trees from MLS data based on the general constitution of trees. The trunk and crown are two components for each tree, which can be detected using the dual growing method. A coarse classification was conducted first to remove human-made objects. Automatic seeds selection was the second step to avoid manual initial parameters setting. Then, the trees were separated using a dual growing process, which circumscribed a trunk in a scalable growing radius and segmented a crown in limited growing regions. Rule-based methods need pre-defined thresholds to process the data in several steps, which is time-consuming.
Recently, deep learning methods have been used in the roadside tree extraction. Guan et al. [60] presented a method with two steps: tree preprocessing and tree classification. The tree preprocessing is similar to the work in [62]. Then, the geometric structures of trees were modeled while using a waveform representation. After that, the generation of high-level feature abstractions of the trees’ using waveform representations was conducted via deep learning architectures. Finally, these features were fed into SVM to train a tree classifier. Zhou et al. [120] also proposed a deep learning method to extract trees and their species. At first, individual trees were extracted by analyzing the density of the point clouds. Then, the voxel-based rasterization was applied to generate low-level feature representation. Finally, the tree species were classified by a deep learning model. However, deep learning architecture was applied to generate low-level or high-level features, which were fed into a machine learning classifier later. This operation cannot fully utilize the strength of deep learning methods, but just use them as a feature generator. Automated extraction focusing on deep learning should be considered in future research.

4.2. Power Lines Extraction

Power lines play an important role in connections among various nationwide grids and distributions of regional electricity. Accurate and timely detection and monitoring are essential in electric equipment management and electric power engineering related decision-makings. Recently, MLS data has been widely applied in power line extraction. Power lines acquired by MLS systems are featured with the following attributes: (1) power lines are placed above the ground; (2) power lines have a certain clearance distance; and, (3) power lines have highly linear distribution [121]. Existing methods are commonly designed based on these features.
Zhu et al. [122] proposed an approach focusing on the locations and heights of the pylons to model the 3D power lines. Gradients were used to group similar points that were mapped into a binary image. The processing of the binary image was constrained by the image region properties (e.g., region’s shape, length, and area), which proves effective in removing non-powerline objects. Guan et al. [123] extracted the power lines by mapping the point clouds into the 2D plane. They utilized incidence angles to estimate road ranges and applied elevation-difference and slope criteria to separates off-road points from ground points. Then, height, spatial density, and an integration of size and shape filters were used to extract power line point clouds from the clustered off-road objects. Identical power lines were extracted using Hough transform and Euclidean distance clustering. Finally, the x–y plane modeling of the 3D power line was featured with a horizontal line; whereas, the x–z plane was featured with a vertical catenary curve derived from a hyperbolic cosine function. Cheng et al. [121] used a voxel-based hierarchical method to calculate geometric features of the individual voxel to extract power lines. A bottom-up method was applied to filter the power lines. The experimental results of these methods demonstrate the applicability of MLS data for power line extraction.

4.3. Multiple Road Objects Extraction

Instead of focusing on the single urban object extraction, some researchers are dedicated to extracting multiple objects in urban areas. The development of implicit shape model and random forests renders Hough forests is capable of constructing an effective model, which can map the components features to the center of objects. Wang et al. [106] utilized the Implicit Shape Model to label object types and applied the framework of Hough Forest for detecting off-road objects: trees, light poles, cars, and traffic signs. Structural and reflective characteristics were employed to describe a 3D local, which was then mapped to predict the approximate position of the object centroid. Then, the peak points in the 3D Hough voting space was used to classify objects. A circular voting strategy was used to maintain the invariance of objects. Based on [106], Wang et al. [35] integrated super-voxel with Hough forest framework for detecting objects (e.g., cars, light poles, traffic signs) from 3D laser scanning point clouds. But, there exist two differences. First, over-segmenting was conducted to group the point clouds into spatially consistent super-voxels. Individual supervoxels, as well as its first order neighborhoods, were segmented into small components. Second, a random forest classifier was used to predict the possible location of the object center. Subsequently, a set of multiscale Hough Forest was applied by Yu et al. [59] to the encode high-order feature of 3D local patches to estimate car point cloud voxel centroids. Then the visibility estimation model was conducted to predict the completeness of cars by integrating Hough voting.
Yu et al. [62] proposed a contextual visual vocabulary approach integrated with spatial contextual information of feature regions to encode local abstract features of point cloud objects (e.g., light poles, traffic signposts, and cars). Then, related objects were extracted by measuring the similarity of the bag of contextual-visual words between the query object and the segmented semantic objects. Moreover, Luo et al. [124] used the similarities to detect road scenes, such as trees, lamp posts, traffic signs, cars, pedestrians, and hoardings from 3D point cloud data. At first, 3D components derived from point clouds were applied in the construction of a 3D patch-based match graph structure (3D-PMG), which effectively changes the labeling condition of category labels from labeled to unlabeled point cloud road scenes. Secondly, 3D components contextual information was analyzed with the integration of 3D-PMG and Markov random fields, which play a role in rectifying the transferring errors result from local patch similarities in multiple classes. Furthermore, Lehtomäki et al. [6] used several geometry-based features, i.e., local descriptor histograms (LDHs), spin images, general shape, and point distribution features, in the classification of the following roadside objects: trees, lamp posts, traffic signs, cars, pedestrians, and hoardings.

5. Challenges and Trend

This literature review reveals that MLS systems have proven their potential in static urban objects extraction and modeling, especially for road-related equipment inventory and condition detection. With the consistent advance of MLS technology and the declined costs of the system components, technique and algorithm challenges and future trends are discussed in this part.
(1) System design and cost
MLS techniques have been fast-developing especially with the advancement of laser scanners, high-resolution cameras, and accurate positioning systems in the past years. However, existing laser scanners are relatively expensive, which limits the wide applications of MLS. In future, cost-effective laser scanners which provide intensive point clouds with multiple attributes (e.g., colors and normal directions) are in high demand. For the digital camera, high-resolution and high-quality output images are needed for the extraction of semantic objects. The improvement of locating and positioning techniques for GNSS and IMU can provide an accessibility for highly accurate point clouds and image position information. The upgrade of hardware with powerful computation and low cost affects the efficiency and accuracy of dealing with large volume point clouds and imagery data.
(2) Software developments
Few existing point cloud processing programs are capable to process large volumes of 3D points in one or two steps. Complicated data processing, difficult fusion of multivariate data, and inconsistent data output format hinder the cross-application of point cloud data in different disciplines. Thus, large-volume data processing, seamless integration of multi-source data, and consistent data output software are urgently needed. Besides, there is no public platform for fusing data obtained from MLS systems. MLS systems produce 3D point clouds, imagery, position, and localization data. These data can be integrated together to generate advanced products. Efficient data fusion platform construction should be considered in future to improve the data utilization.
(3) Algorithm trend
Currently, the learning-based point cloud processing algorithms are the mainstream trend. Machine learning methods [77,125,126,127] train efficient classifiers by designing an effective feature descriptor. It highly relies on the prior knowledge of the human operators, which is a challenging task in the complex urban environments. Then, designing a proper feature descriptor for various sizes of point clouds is the future focus for machine learning methods. Recently published effective and available point cloud processing algorithms mainly focus on deep learning architectures and demonstrate their excellent performance, such as PointNet [71], PointNet++ [128], PointCNN [78], VoxelNet [129], and MSNet [130]. These methods mainly process small scene point clouds. It is challenging for them to deal with large scenes. Thus, the direct construction of efficient neural networks on the 3D points with various sizes and volumes is an interesting topic for future studies.
(4) Data storage and updating
With the advancement of data acquisition and data resolution, data, including point clouds and images collected by MLS systems, is in quite large-volume [131]. Accordingly, cost-effective and efficient data storage and updating have impacts on the working continuity and efficiency of MLS systems. Moreover, post-processing data storage and updating are important for MLS-related applications with regard to modeling, mapping, and managing. Challenges in large-volume data storage and timely data updating are the primary concerns when the worldwide LiDAR manufacturers develop new types of LiDAR sensors and information and communication technology (ICT) companies design commercial data processing and management software. However, data quality can be influenced due to system errors of MLS systems, ambient occlusion, and data calibration. Such data obtained from multiple sensors at different times, in various weather conditions, under different illuminations, and with different sampling densities, make the data calibration and data fusion challenging [132]. Furthermore, the evaluation of data updating and data correction results are cumbersome. Therefore, reliable, efficient, and cost-effective data management is one of the major challenges in the development and application of MLS techniques.

6. Conclusions

The urban road network plays an important role in the urban planning with regard to planning, design, construction, management, and maintenance. The inventory of road features contains both pavement geometrical properties (e.g., road width, slopes, and lanes) and road infrastructures (e.g., road markings, pavement cracks, road manholes, traffic signs, roadsides trees, and buildings). In order to enhance traffic safety and efficiency, managers, and decision-makers in cities and countries should pay much attention to periodically investigate urban spatial structures and road asset inventory. Such periodically surveyed data are used not only for ITS-related development to reconstruct and maintain the existing road networks, but also for city managers to formulate traffic policies and advance traffic management. As a vehicle-based mobile mapping system, an MLS system integrated with laser scanners, GNSS sensors, and digital cameras, collects high-density 3D point clouds from the surrounding objects in urban road environments and provides detailed georeferenced coordinate information of roads. To date, due to the high flexibility and accurate data acquisition over large-scaled surveyed areas, the industrial applicability of MLS systems has been proven to provide higher time-efficiency and better cost-saving in comparison to aerial imagery, terrestrial laser scanning, and human fieldwork. When considering such strengths of MLS systems, as demonstrated in previous studies, journal publications, and projects, an increasing number of governments and transportation departments have been applying mobile laser scanning techniques to urban road asset inventory.
In this paper, a literature review described and detailed the feasibility and applications of MLS systems to road asset inventory. The MLS technique, system component, direct geo-referencing, error analysis, and current applications of on-road and off-road inventory were accordingly discussed. The state-of-the-art road object detection and extraction algorithms, focusing on mobile laser scanning point clouds, have been reviewed. Accordingly, their performance and adaptability have been discussed. When compared to other review papers focusing on MLS applications, the main contribution of this review demonstrates that the MLS systems are suitable for supporting road asset inventory, ITS-related applications, high-definition maps, and other highly accurate localization services, such as road feature extraction and accurate perception.
MLS systems have indicated great potentials for rapid commercialization. With the further advancement of laser scanning technologies and the decreased costs of new MLS-related hardware, it can be predicted that MLS systems and big data management technologies will dominate the advanced remote sensing and photogrammetry fields, as well as accurately localization-based services, such as high-definition maps and autonomous driving. To summarize, this review indicates that MLS systems provide an effective and reliable solution for conducting road asset inventory in complex urban road environments.

Author Contributions

Conceptualization, L.M., Y.L., and J.L.; Methodology, L.M. and Y.L.; Formal Analysis, L.M.; Resources, L.M., Y.L., J.L., and M.C.; Data Curation, L.M., Y.L. and J.L.; Writing-Original Draft Preparation, L.M. and Y.L.; Writing-Review & Editing, L.M., J.L., and M.C.; Supervision, C.W., R.W., and J.L.; Project Administration, J.L. and C.W.”.

Funding

This research was funded in part by the National Natural Science Foundation of China under Grant number 41471379 and in part by the Fujian Collaborative Innovation Center for Big Data Applications in Governments.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Yang, B.; Dong, Z.; Zhao, G.; Dai, W. Hierarchical extraction of urban objects from mobile laser scanning data. ISPRS J. Photogramm. Remote Sens. 2015, 99, 45–57. [Google Scholar] [CrossRef]
  2. Leichtle, T.; Geiß, C.; Wurm, M.; Lakes, T.; Taubenböck, H. Unsupervised change detection in VHR remote sensing imagery—An object-based clustering approach in a dynamic urban environment. Int. J. Appl. Earth Obs. Geoinform. 2017, 54, 15–27. [Google Scholar] [CrossRef]
  3. Crommelinck, S.; Bennett, R.; Gerke, M.; Nex, F.; Yang, M.Y.; Vosselman, G. Review of automatic feature extraction from high-resolution optical sensor data for UAV-based cadastral mapping. Remote Sens. 2016, 8, 689. [Google Scholar] [CrossRef]
  4. Ma, L.; Li, M.; Ma, X.; Cheng, L.; Du, P.; Liu, Y. A review of supervised object-based land-cover image classification. ISPRS J. Photogramm. Remote Sens. 2017, 130, 277–293. [Google Scholar] [CrossRef]
  5. Wang, R.; Peethambaran, J.; Chen, D. LiDAR point clouds to 3-D urban models: a review. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2018, 11, 606–627. [Google Scholar] [CrossRef]
  6. Lehtomäki, M.; Jaakkola, A.; Hyyppä, J.; Lampinen, J.; Kaartinen, H.; Kukko, A.; Puttonen, E.; Hyyppä, H. Object classification and recognition from mobile laser scanning point clouds in a road environment. IEEE Trans. Geosci. Remote Sens. 2016, 54, 1226–1239. [Google Scholar] [CrossRef]
  7. Yu, Y.; Li, J.; Guan, H.; Wang, C. Automated extraction of urban road facilities using mobile laser scanning data. IEEE Trans. Intell. Transp. Syst. 2015, 16, 2167–2181. [Google Scholar] [CrossRef]
  8. Guan, H.; Li, J.; Cao, S.; Yu, Y. Use of mobile LiDAR in road information inventory: A review. Int. J. Image Data Fusion 2016, 7, 219–242. [Google Scholar] [CrossRef]
  9. Vosselman, G.; Maas, H.G. Airborne and Terrestrial Laser Scanning; CRC Press: Boca Raton, FL, USA, 2010. [Google Scholar]
  10. Lindner, M.; Schiller, I.; Kolb, A.; Koch, R. Time-of-flight sensor calibration for accurate range sensing. Comput. Vis. Image Underst. 2010, 114, 1318–1328. [Google Scholar] [CrossRef]
  11. Puente, I.; González-Jorge, H.; Martínez-Sánchez, J.; Arias, P. Review of mobile mapping and surveying technologies. Measurement 2013, 46, 2127–2145. [Google Scholar] [CrossRef]
  12. Kumar, P.; McElhinney, C.P.; Lewis, P.; McCarthy, T. Automated road markings extraction from mobile laser scanning data. Int. J. Appl. Earth Obs. Geoinf. 2014, 32, 125–137. [Google Scholar] [CrossRef] [Green Version]
  13. Toth, C.K. R&D of mobile LiDAR mapping and future trends. In Proceedings of the ASPRS 2009 Annual Conference, Baltimore, MD, USA, 9–13 March 2009; pp. 829–835. [Google Scholar]
  14. Guan, H.; Li, J.; Yu, Y.; Chapman, M.; Wang, H.; Wang, C.; Zhai, R. Iterative tensor voting for pavement crack extraction using mobile laser scanning data. IEEE Trans. Geosci. Remote Sens. 2015, 53, 1527–1537. [Google Scholar] [CrossRef]
  15. Olsen, M.J. NCHRP Report 748 Guidelines for the Use of Mobile Lidar in Transportation Applications; Transportation Research Board: Washington, DC, USA, 2013. [Google Scholar]
  16. McCormac, J.C.; Sarasua, W.; Davis, W. Surveying; Wiley Global Education: Hoboken, NJ, USA, 2012. [Google Scholar]
  17. Trimble. Trimble MX9 Mobile Mapping Solution. 2018. Available online: https://geospatial.trimble.com/products-and-solutions/mx9 (accessed on 11 May 2018).
  18. RIEGL. RIEGL VMX-450 Compact Mobile Laser Scanning System. 2017. Available online: http://www.riegl.com/uploads/tx_pxpriegldownloads/DataSheet_VMX-450_2015-03-19.pdf (accessed on 11 May 2018).
  19. RIEGL. RIEGL VMX-2HA. 2018. Available online: http://www.riegl.com/nc/products/mobile-scanning/produktdetail/product/scanner/56 (accessed on 11 May 2018).
  20. Teledyne Optech. Lynx SG Mobile Mapper Summary Specification Sheet. 2018. Available online: https://www.teledyneoptech.com/index.php/product/lynx-sg1 (accessed on 11 May 2018).
  21. Topcon. IP-S3 Compact+. 2018. Available online: https://www.topconpositioning.com/mass-data-and-volume-collection/mobile-mapping/ip-s3 (accessed on 11 May 2018).
  22. Vallet, B.; Brédif, M.; Serna, A.; Marcotegui, B.; Paparoditis, N. TerraMobilita/iQmulus urban point cloud analysis benchmark. Comput. Gr. 2015, 49, 126–133. [Google Scholar] [CrossRef] [Green Version]
  23. Roynard, X.; Deschaud, J.E.; Goulette, F. Paris-Lille-3D: A large and high-quality ground truth urban point cloud dataset for automatic segmentation and classification. CVPR 2017. under review. [Google Scholar] [CrossRef]
  24. Gehrung, J.; Hebel, M.; Arens, M.; Stilla, U. An Approach to Extract Moving Objects from Mls Data Using a Volumetric Background Representation. ISPRS. Ann. 2017, 4, 107–114. [Google Scholar] [CrossRef]
  25. Pandey, G.; McBride, J.R.; Eustice, R.M. Ford campus vision and lidar data set. Int. J. Robot. Res. 2011, 30, 1543–1552. [Google Scholar] [CrossRef]
  26. Carlevaris-Bianco, N.; Ushani, A.K.; Eustice, R.M. University of Michigan North Campus long-term vision and lidar dataset. Int. J. Robot. Res. 2016, 35, 1023–1035. [Google Scholar] [CrossRef]
  27. Menze, M.; Geiger, A. Object scene flow for autonomous vehicles. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 7–12 June 2015; pp. 3061–3070. [Google Scholar]
  28. Cordts, M.; Omran, M.; Ramos, S.; Rehfeld, T.; Enzweiler, M.; Benenson, R.; Schiele, B. The cityscapes dataset for semantic urban scene understanding. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 26–30 June 2016; pp. 3213–3223. [Google Scholar]
  29. Hackel, T.; Savinov, N.; Ladicky, L.; Wegner, J.D.; Schindler, K.; Pollefeys, M. Semantic3D. net: A new large-scale point cloud classification benchmark. ISPRS Ann. 2017, 4, 91–98. [Google Scholar]
  30. Wu, B.; Yu, B.; Huang, C.; Wu, Q.; Wu, J. Automated extraction of ground surface along urban roads from mobile laser scanning point clouds. Remote Sens. Lett. 2016, 7, 170–179. [Google Scholar] [CrossRef]
  31. Wang, H.; Luo, H.; Wen, C.; Cheng, J.; Li, P.; Chen, Y.; Wang, C.; Li, J. Road boundaries detection based on local normal saliency from mobile laser scanning data. IEEE Trans. Geosci. Remote Sens. Lett. 2015, 12, 2085–2089. [Google Scholar] [CrossRef]
  32. Kumar, P.; Lewis, P.; McElhinney, C.P.; Boguslawski, P.; McCarthy, T. Snake energy analysis and result validation for a mobile laser scanning data-based automated road edge extraction algorithm. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2017, 10, 763–773. [Google Scholar] [CrossRef]
  33. Kumar, P.; McElhinney, C.P.; Lewis, P.; McCarthy, T. An automated algorithm for extracting road edges from terrestrial mobile LiDAR data. ISPRS J. Photogramm. Remote Sens. 2013, 85, 44–55. [Google Scholar] [CrossRef] [Green Version]
  34. Zai, D.; Li, J.; Guo, Y.; Cheng, M.; Lin, Y.; Luo, H.; Wang, C. 3-D road boundary extraction from mobile laser scanning data via supervoxels and graph cuts. IEEE Trans. Intell. Transp. Syst. 2018, 19, 802–813. [Google Scholar] [CrossRef]
  35. Wang, H.; Wang, C.; Luo, H.; Li, P.; Chen, Y.; Li, J. 3-D point cloud object detection based on supervoxel neighborhood with Hough forest framework. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2015, 8, 1570–1581. [Google Scholar] [CrossRef]
  36. Hervieu, A.; Soheilian, B. Semi-automatic road pavement modeling using mobile laser scanning. ISPRS Ann. 2013, 2, 31–36. [Google Scholar] [CrossRef]
  37. Hata, A.Y.; Osorio, F.S.; Wolf, D.F. Robust curb detection and vehicle localization in urban environments. In Proceedings of the IEEE Intelligent Vehicles Symposium, Dearborn, MI, USA, 8–11 June 2014; pp. 1257–1262. [Google Scholar]
  38. Guan, H.; Li, J.; Yu, Y.; Wang, C.; Chapman, M.; Yang, B. Using mobile laser scanning data for automated extraction of road markings. ISPRS J. Photogramm. Remote Sens. 2014, 87, 93–107. [Google Scholar] [CrossRef]
  39. Xu, S.; Wang, R.; Zheng, H. Road curb extraction from mobile LiDAR point clouds. IEEE Geosci. Remote Sens. 2017, 2, 996–1009. [Google Scholar] [CrossRef]
  40. Riveiro, B.; González-Jorge, H.; Martínez-Sánchez, J.; Díaz-Vilariño, L.; Arias, P. Automatic detection of zebra crossings from mobile LiDAR data. Opt. Laser Technol. 2015, 70, 63–70. [Google Scholar] [CrossRef]
  41. Pu, S.; Rutzinger, M.; Vosselman, G.; Elberink, S.O. Recognizing basic structures from mobile laser scanning data for road inventory studies. ISPRS J. Photogramm. Remote Sens. 2011, 66, 28–39. [Google Scholar] [CrossRef]
  42. Wu, F.; Wen, C.; Guo, Y.; Wang, J.; Yu, Y.; Wang, C.; Li, J. Rapid localization and extraction of street light poles in mobile LiDAR point clouds: a supervoxel-based approach. IEEE Trans. Intell. Transp. Syst. 2017, 18, 292–305. [Google Scholar] [CrossRef]
  43. Cabo, C.; Kukko, A.; García-Cortés, S.; Kaartinen, H.; Hyyppä, J.; Ordoñez, C. An algorithm for automatic road asphalt edge delineation from mobile laser scanner data using the line clouds concept. Remote Sens. 2016, 8, 740. [Google Scholar] [CrossRef]
  44. Yang, B.; Fang, L.; Li, J. Semi-automated extraction and delineation of 3D roads of street scene from mobile laser scanning point clouds. ISPRS J. Photogramm. Remote Sens. 2013, 79, 80–93. [Google Scholar] [CrossRef]
  45. Zhou, Y.; Wang, D.; Xie, X.; Ren, Y.; Li, G.; Deng, Y.; Wang, Z. A fast and accurate segmentation method for ordered LiDAR point cloud of large-scale scenes. IEEE Trans. Geosci. Remote Sens. Lett. 2014, 11, 1981–1985. [Google Scholar] [CrossRef]
  46. Fan, H.; Yao, W.; Tang, L. Identifying man-made objects along urban road corridors from mobile LiDAR data. IEEE Trans. Geosci. Remote Sens. Lett. 2014, 11, 950–954. [Google Scholar] [CrossRef]
  47. Ma, L.; Li, J.; Li, Y.; Zhong, Z.; Chapman, M. Generation of horizontally curved driving lines in HD maps using mobile laser scanning point clouds. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens 2018. under review. [Google Scholar]
  48. Guo, J.; Tsai, M.J.; Han, J.Y. Automatic reconstruction of road surface features by using terrestrial mobile lidar. Autom. Constr. 2015, 58, 165–175. [Google Scholar] [CrossRef]
  49. Jaakkola, A.; Hyyppä, J.; Hyyppä, H.; Kukko, A. Retrieval algorithms for road surface modelling using laser-based mobile mapping. Sensors 2008, 8, 5238–5249. [Google Scholar] [CrossRef] [PubMed]
  50. Díaz-Vilariño, L.; González-Jorge, H.; Bueno, M.; Arias, P.; Puente, I. Automatic classification of urban pavements using mobile LiDAR data and roughness descriptors. Constr. Build. Mater. 2016, 102, 208–215. [Google Scholar] [CrossRef]
  51. Serna, A.; Marcotegui, B.; Hernández, J. Segmentation of façades from urban 3D point clouds using geometrical and morphological attribute-based operators. ISPRS Int. Geo-Inf. 2016, 5, 6. [Google Scholar] [CrossRef]
  52. Kumar, P.; Lewis, P.; McCarthy, T. The potential of active contour models in extracting road edges from mobile laser scanning data. Infrastructures 2017, 2, 9–25. [Google Scholar] [CrossRef]
  53. Yang, B.; Dong, Z.; Liu, Y.; Liang, F.; Wang, Y. Computing multiple aggregation levels and contextual features for road facilities recognition using mobile laser scanning data. ISPRS J. Photogramm. Remote Sens. 2017, 126, 180–194. [Google Scholar] [CrossRef]
  54. Choi, M.; Kim, M.; Kim, G.; Kim, S.; Park, S.C.; Lee, S. 3D scanning technique for obtaining road surface and its applications. Int. J. Precis. Eng. Manuf. 2017, 18, 367–373. [Google Scholar] [CrossRef]
  55. Boyko, A.; Funkhouser, T. Extracting roads from dense point clouds in large scale urban environment. ISPRS J. Photogramm. Remote Sens. 2011, 66, 2–12. [Google Scholar] [CrossRef]
  56. Zhou, L.; Vosselman, G. Mapping curbstones in airborne and mobile laser scanning data. Int. J. Appl. Earth Obs. Geoinf. 2012, 18, 293–304. [Google Scholar] [CrossRef]
  57. Cavegn, S.; Haala, N. Image-based mobile mapping for 3D urban data capture. Photogramm. Eng. Remote Sens. 2016, 82, 925–933. [Google Scholar] [CrossRef]
  58. Cheng, M.; Zhang, H.; Wang, C.; Li, J. Extraction and classification of road markings using mobile laser scanning point clouds. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2017, 10, 1182–1196. [Google Scholar] [CrossRef]
  59. Yu, Y.; Li, J.; Guan, H.; Wang, C. Automated detection of three-dimensional cars in mobile laser scanning point clouds using DBM-Hough-Forests. IEEE Trans. Geosci. Remote Sens. 2016, 54, 4130–4142. [Google Scholar] [CrossRef]
  60. Guan, H.; Yu, Y.; Ji, Z.; Li, J.; Zhang, Q. Deep learning-based tree classification using mobile LiDAR data. Remote Sens. Lett. 2015, 6, 864–873. [Google Scholar] [CrossRef]
  61. Huang, P.; Cheng, M.; Chen, Y.; Luo, H.; Wang, C.; Li, J. Traffic sign occlusion detection using mobile laser scanning point clouds. IEEE Trans. Intell. Transp. Syst. 2017, 18, 2364–2376. [Google Scholar] [CrossRef]
  62. Yu, Y.; Li, J.; Guan, H.; Wang, C.; Wen, C. Bag of contextual-visual words for road scene object detection from mobile laser scanning data. IEEE Trans. Intell. Transp. Syst. 2016, 17, 3391–3406. [Google Scholar] [CrossRef]
  63. Arcos-García, Á.; Soilán, M.; Álvarez-García, J.A.; Riveiro, B. Exploiting synergies of mobile mapping sensors and deep learning for traffic sign recognition systems. Expert Syst. Appl. 2017, 89, 286–295. [Google Scholar]
  64. Soilán, M.; Riveiro, B.; Martínez-Sánchez, J.; Arias, P. Traffic sign detection in MLS acquired point clouds for geometric and image-based semantic inventory. ISPRS J. Photogramm. Remote Sens. 2016, 114, 92–101. [Google Scholar] [CrossRef]
  65. Wen, C.; Li, J.; Luo, H.; Yu, Y.; Cai, Z.; Wang, H.; Wang, C. Spatial-related traffic sign inspection for inventory purposes using mobile laser scanning data. IEEE Trans. Intell. Transp. Syst. 2016, 17, 27–37. [Google Scholar] [CrossRef]
  66. Yu, Y.; Li, J.; Guan, H.; Wang, C.; Yu, J. Semiautomated extraction of street light poles from mobile LiDAR point-clouds. IEEE Trans. Geosci. Remote Sens. 2015, 53, 1374–1386. [Google Scholar] [CrossRef]
  67. Yan, L.; Li, Z.; Liu, H.; Tan, J.; Zhao, S.; Chen, C. Detection and classification of pole-like road objects from mobile LiDAR data in motorway environment. Opt. Laser Technol. 2017, 97, 272–283. [Google Scholar] [CrossRef]
  68. Smadja, L.; Ninot, J.; Gavrilovic, T. Road extraction and environment interpretation from LiDAR sensors. IAPRS 2010, 38, 281–286. [Google Scholar]
  69. Yan, L.; Liu, H.; Tan, J.; Li, Z.; Xie, H.; Chen, C. Scan line based road marking extraction from mobile LiDAR point clouds. Sensors 2016, 16, 903. [Google Scholar] [CrossRef] [PubMed]
  70. Li, Y.; Li, L.; Li, D.; Yang, F.; Liu, Y. A density-based clustering method for urban scene mobile laser scanning data segmentation. Remote Sens. 2017, 9, 331. [Google Scholar] [CrossRef]
  71. Yang, B.; Fang, L.; Li, Q.; Li, J. Automated extraction of road markings from mobile LiDAR point clouds. Photogramm. Eng. Remote Sens. 2012, 78, 331–338. [Google Scholar] [CrossRef]
  72. Chen, X.; Kohlmeyer, B.; Stroila, M.; Alwar, N.; Wang, R.; Bach, J. Next generation map making: geo-referenced ground-level LiDAR point clouds for automatic retro-reflective road feature extraction. In Proceedings of the 17th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, Seattle, WA, USA, 4–6 November 2009; pp. 488–491. [Google Scholar]
  73. Kim, H.; Liu, B.; Myung, H. Road-feature extraction using point cloud and 3D LiDAR sensor for vehicle localization. In Proceedings of the 14th International Conference on URAI, Maison Glad Jeju, Jeju, Korea, 28 June–1 July 2017; pp. 891–892. [Google Scholar]
  74. Yu, Y.; Li, J.; Guan, H.; Jia, F.; Wang, C. Learning hierarchical features for automated extraction of road markings from 3-D mobile LiDAR point clouds. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2015, 8, 709–726. [Google Scholar] [CrossRef]
  75. Otsu, N. A threshold selection method from gray-level histograms. IEEE Trans. Syst. Man Cybern. 1979, 9, 62–66. [Google Scholar] [CrossRef]
  76. Soilán, M.; Riveiro, B.; Martínez-Sánchez, J.; Arias, P. Segmentation and classification of road markings using MLS data. ISPRS J. Photogramm. Remote Sens. 2017, 123, 94–103. [Google Scholar] [CrossRef]
  77. Qi, C.R.; Su, H.; Mo, K.; Guibas, L.J. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 77–85. [Google Scholar]
  78. Li, Y.; Bu, R.; Sun, M.; Chen, B. PointCNN. arXiv, 2018; arXiv:1801.07791. [Google Scholar]
  79. Weiskircher, T.; Wang, Q.; Ayalew, B. Predictive guidance and control framework for (semi-)autonomous vehicles in public traffic. IEEE Trans. Control Syst. Technol. 2017, 5, 2034–2046. [Google Scholar] [CrossRef]
  80. Li, J.; Zhao, H.; Ma, L.; Jiang, H.; Chapman, M. Recognizing features in mobile laser scanning point clouds towards 3D high-definition road maps for autonomous vehicles. IEEE Trans. Intell. Transp. Syst. 2018. under review. [Google Scholar]
  81. Li, J.; Ye, C.; Jiang, H.; Zhao, H.; Ma, L.; Chapman, M. Semi-automated generation of road transition lines using mobile laser scanning data. IEEE Trans. Intell. Transp. Syst. 2018. under revision. [Google Scholar]
  82. Chen, X.; Li, J. A feasibility study on use of generic mobile laser scanning system for detecting asphalt pavement cracks. ISPRS Arch. 2016, 41. [Google Scholar] [CrossRef]
  83. Tanaka, N.; Uematsu, K. A crack detection method in road surface images using morphology. In Proceedings of the IAPR Workshop on Machine Vision Applications, Chiba, Japan, 17–19 November 1998; pp. 17–19. [Google Scholar]
  84. Tsai, Y.C.; Kaul, V.; Mersereau, R.M. Critical assessment of pavement distress segmentation methods. J. Transp. Eng. 2009, 136, 11–19. [Google Scholar] [CrossRef]
  85. Yu, Y.; Li, J.; Guan, H.; Wang, C. 3D crack skeleton extraction from mobile LiDAR point clouds. In Proceedings of the IEEE Geoscience and Remote Sensing Symposium, Quebec, QC, Canada, 13–18 July 2014; pp. 914–917. [Google Scholar]
  86. Saar, T.; Talvik, O. Automatic asphalt pavement crack detection and classification using neural networks. In Proceedings of the 12th Biennial Baltic Electronics Conference, Tallinn, Estonia, 4–6 October 2010; pp. 345–348. [Google Scholar]
  87. Gavilán, M.; Balcones, D.; Marcos, O.; Llorca, D.F.; Sotelo, M.A.; Parra, I.; Ocaña, M.; Aliseda, P.; Yarza, P.; Amírola, A. Adaptive road crack detection system by pavement classification. Sensors 2011, 11, 9628–9657. [Google Scholar] [CrossRef] [PubMed]
  88. Cheng, Y.; Xiong, Z.; Wang, Y.H. Improved classical Hough transform applied to the manhole cover’s detection and location. Opt. Tech. 2006, 32, 504–508. [Google Scholar]
  89. Timofte, R.; Van Gool, L. Multi-view manhole detection, recognition, and 3D localisation. In Proceedings of the 2011 IEEE International Conference on Computer Vision Workshops, 6–13 November 2011; pp. 188–195. [Google Scholar]
  90. Niigaki, H.; Shimamura, J.; Morimoto, M. Circular object detection based on separability and uniformity of feature distributions using Bhattacharyya coefficient. In Proceedings of the 21st International Conference on Pattern Recognition, Stockholm, Sweden, 11–15 November 2012; pp. 2009–2012. [Google Scholar]
  91. Guan, H.; Yu, Y.; Li, J.; Liu, P.; Zhao, H.; Wang, C. Automated extraction of manhole covers using mobile LiDAR data. Remote Sens. Lett. 2014, 5, 1042–1050. [Google Scholar] [CrossRef]
  92. Yu, Y.; Guan, H.; Ji, Z. Automated detection of urban road manhole covers using mobile laser scanning data. IEEE Trans. Intell. Transp. Syst. 2015, 16, 3258–3269. [Google Scholar] [CrossRef]
  93. Balali, V.; Jahangiri, A.; Machiani, S.G. Multi-class US traffic signs 3D recognition and localization via image-based point cloud model using color candidate extraction and texture-based recognition. Adv. Eng. Inf. 2017, 32, 263–274. [Google Scholar] [CrossRef]
  94. Ai, C.; Tsai, Y.C. Critical assessment of an enhanced traffic sign detection method using mobile lidar and ins technologies. J. Transp. Eng. 2015, 141, 04014096. [Google Scholar] [CrossRef]
  95. Serna, A.; Marcotegui, B. Detection, segmentation and classification of 3d urban objects using mathematical morphology and supervised learning. ISPRS J. Photogramm. Remote Sens. 2014, 93, 243–255. [Google Scholar] [CrossRef] [Green Version]
  96. Yu, Y.; Li, J.; Wen, C.; Guan, H.; Luo, H.; Wang, C. Bag-of-visual-phrases and hierarchical deep models for traffic sign detection and recognition in mobile laser scanning data. ISPRS J. Photogramm. Remote Sens. 2016, 113, 106–123. [Google Scholar] [CrossRef]
  97. Cabo, C.; Ordoñez, C.; García-Cortés, S.; Martínez, J. An algorithm for automatic detection of pole-like street furniture objects from mobile laser scanner point clouds. ISPRS J. Photogramm. Remote Sens. 2014, 87, 47–56. [Google Scholar] [CrossRef]
  98. Wang, H.; Cai, Z.; Luo, H.; Wang, C.; Li, P.; Yang, W.; Ren, S.; Li, J. Automatic road extraction from mobile laser scanning data. In Proceedings of the 2012 International Conference on Computer Vision in Remote Sensing, Xiamen, China, 16–18 December 2012; pp. 136–139. [Google Scholar]
  99. Riveiro, B.; Díaz-Vilariño, L.; Conde-Carnero, B.; Soilán, M.; Arias, P. Automatic segmentation and shape-based classification of retro-reflective traffic signs from mobile LiDAR data. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2016, 9, 295–303. [Google Scholar] [CrossRef]
  100. Yan, W.Y.; Morsy, S.; Shaker, A.; Tulloch, M. Automatic extraction of highway light poles and towers from mobile LiDAR data. Opt. Laser Technol. 2016, 77, 162–168. [Google Scholar] [CrossRef]
  101. Xing, X.F.; Mostafavi, M.A.; Chavoshi, S.H. A knowledge base for automatic feature recognition from point clouds in an urban scene. ISPRS Int. Geo-Inf. 2018, 7, 28. [Google Scholar] [CrossRef]
  102. Seo, Y.W.; Lee, J.; Zhang, W.; Wettergreen, D. Recognition of highway workzones for reliable autonomous driving. IEEE Trans. Intell. Transp. Syst. 2015, 16, 708–718. [Google Scholar] [CrossRef]
  103. Golovinskiy, A.; Kim, V.; Funkhouser, T. Shape-based recognition of 3D point clouds in urban environments. In Proceedings of the IEEE 12th International Conference on Computer Vision, Kyoto, Japan, 29 September–2 October; pp. 2154–2161.
  104. Gonzalez-Jorge, H.; Riveiro, B.; Armesto, J.; Arias, P. Evaluation of road signs using radiometric and geometric data from terrestrial lidar. Opt. Appl. 2013, 43, 421–433. [Google Scholar]
  105. Levinson, J.; Askeland, J.; Becker, J.; Dolson, J.; Held, D.; Kammel, S.; Kolter, J.; Langer, D.; Pink, O.; Pratt, V. Towards fully autonomous driving: systems and algorithms. In Proceedings of the IEEE Intelligent Vehicles Symposium, Baden-Baden, Germany, 5–9 June 2011; pp. 163–168. [Google Scholar]
  106. Wang, H.; Wang, C.; Luo, H.; Li, P.; Cheng, M.; Wen, C.; Li, J. Object detection in terrestrial laser scanning point clouds based on Hough forest. IEEE Trans. Geosci. Remote Sens. Lett. 2014, 11, 1807–1811. [Google Scholar] [CrossRef]
  107. Yang, B.; Dong, Z. A shape-based segmentation method for mobile laser scanning point clouds. ISPRS J. Photogramm. Remote Sens. 2013, 81, 19–30. [Google Scholar] [CrossRef]
  108. Guan, H.; Yan, W.; Yu, Y.; Zhong, L.; Li, D. Robust traffic-sign detection and classification using mobile LiDAR data with digital images. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2018, 11, 1715–1724. [Google Scholar] [CrossRef]
  109. Tan, M.; Wang, B.; Wu, Z.; Wang, J.; Pan, G. Weakly supervised metric learning for traffic sign recognition in a LIDAR-equipped vehicle. IEEE Trans. Intell. Transp. Syst. 2016, 17, 1415–1427. [Google Scholar] [CrossRef]
  110. Luo, H.; Wang, C.; Wang, H.; Chen, Z.; Zai, D.; Zhang, S.; Li, J. Exploiting location information to detect light pole in mobile LiDAR point clouds. In Proceedings of the IEEE International Geoscience and Remote Sensing Symposium, Beijing, China, 10–15 July 2016; pp. 453–456. [Google Scholar]
  111. Zheng, H.; Wang, R.; Xu, S. Recognizing street lighting poles from mobile LiDAR data. IEEE Geosci. Remote Sens. 2017, 55, 407–420. [Google Scholar] [CrossRef]
  112. Li, L.; Li, Y.; Li, D. A method based on an adaptive radius cylinder model for detecting pole-like objects in mobile laser scanning data. Remote Sens. Lett. 2016, 7, 249–258. [Google Scholar] [CrossRef]
  113. Wang, J.; Lindenbergh, R.; Menenti, M. SigVox–A 3D feature matching algorithm for automatic street object recognition in mobile laser scanning point clouds. ISPRS J. Photogramm. Remote Sens. 2017, 128, 111–129. [Google Scholar] [CrossRef]
  114. Teo, T.A.; Chiu, C.M. Pole-like road object detection from mobile lidar system using a coarse-to-fine approach. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2015, 8, 4805–4818. [Google Scholar] [CrossRef]
  115. Rodríguez-Cuenca, B.; García-Cortés, S.; Ordóñez, C.; Alonso, M.C. Automatic detection and classification of pole-like objects in urban point cloud data using an anomaly detection algorithm. Remote Sens. 2015, 7, 12680–12703. [Google Scholar] [CrossRef]
  116. Li, F.; Oude Elberink, S.; Vosselman, G. Pole-like road furniture detection and decomposition in mobile laser scanning data based on spatial relations. Remote Sens. 2018, 10, 531. [Google Scholar]
  117. Guan, H.; Yu, Y.; Li, J.; Liu, P. Pole-like road object detection in mobile LiDAR data via super-voxel and bag-of-contextual-visual-words representation. IEEE Trans. Geosci. Remote Sens. 2016, 13, 520–524. [Google Scholar] [CrossRef]
  118. Xu, S.; Xu, S.; Ye, N.; Zhu, F. Automatic extraction of street trees’ non-photosynthetic components from MLS data. Int. J. Appl. Earth Obs. Geoinf. 2018, 69, 64–77. [Google Scholar] [CrossRef]
  119. Li, L.; Li, D.; Zhu, H.; Li, Y. A dual growing method for the automatic extraction of individual trees from mobile laser scanning data. ISPRS J. Photogramm. Remote Sens. 2016, 120, 37–52. [Google Scholar] [CrossRef]
  120. Zou, X.; Cheng, M.; Wang, C.; Xia, Y.; Li, J. Tree classification in complex forest point clouds based on deep learning. IEEE Trans. Geosci. Remote Sens. Lett. 2017, 14, 2360–2364. [Google Scholar] [CrossRef]
  121. Cheng, L.; Tong, L.; Wang, Y.; Li, M. Extraction of urban power lines from vehicle-borne LiDAR data. Remote Sens. 2014, 6, 3302–3320. [Google Scholar] [CrossRef]
  122. Zhu, L.; Hyyppa, J. The use of airborne and mobile laser scanning for modeling railway environments in 3D. Remote Sens. 2014, 6, 3075–3100. [Google Scholar] [CrossRef]
  123. Guan, H.; Yu, Y.; Li, J.; Ji, Z.; Zhang, Q. Extraction of power-transmission lines from vehicle-borne lidar data. Int. J. Remote Sens. 2016, 37, 229–247. [Google Scholar] [CrossRef]
  124. Luo, H.; Wang, C.; Wen, C.; Cai, Z.; Chen, Z.; Wang, H.; Yu, Y.; Li, J. Patch-based semantic labeling of road scene using colorized mobile LiDAR point clouds. IEEE Trans. Intell. Transp. Syst. 2016, 17, 1286–1297. [Google Scholar] [CrossRef]
  125. Zhang, J.; Lin, X.; Ning, X. SVM-based classification of segmented airborne LiDAR point clouds in urban areas. Remote Sens. 2013, 5, 3749–3775. [Google Scholar] [CrossRef]
  126. Tootooni, M.S.; Dsouza, A.; Donovan, R.; Rao, P.K.; Kong, Z.J.; Borgesen, P. Classifying the dimensional variation in additive manufactured parts from laser-scanned three-dimensional point cloud data using machine learning approaches. J. Manuf. Sci. Eng. 2017, 139, 091005. [Google Scholar] [CrossRef]
  127. Ghamisi, P.; Höfle, B. LiDAR data classification using extinction profiles and a composite kernel support vector machine. IEEE Geosci. Remote Sens. Lett. 2017, 14, 659–663. [Google Scholar] [CrossRef]
  128. Qi, C.R.; Yi, L.; Su, H.; Guibas, L.J. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. In Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA, 4–9 December 2017; pp. 5105–5114. [Google Scholar]
  129. Zhou, Y.; Tuzel, O. VoxelNet: End-to-end learning for point cloud based 3D object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 22–25 July 2017. [Google Scholar]
  130. Wang, L.; Huang, Y.; Shan, J.; He, L. MSNet: Multi-scale convolutional network for point cloud classification. Remote Sens. 2018, 10, 612. [Google Scholar] [CrossRef]
  131. Balado, J.; Díaz-Vilariño, L.; Arias, P.; González-Jorge, H. Automatic classification of urban ground elements from mobile laser scanning data. Autom. Construct. 2018, 86, 226–239. [Google Scholar] [CrossRef]
  132. Li, J.; Yang, B.; Chen, C.; Huang, R.; Dong, Z.; Xiao, W. Automatic registration of panoramic image sequence and mobile laser scanning data using semantic features. ISPRS J. Photogramm. Remote Sens. 2018, 136, 41–57. [Google Scholar] [CrossRef]
Figure 1. (a) Ratio of publications related to ‘road object’, using airborne laser scanning (ALS) point cloud, imagery, and mobile laser scanning (MLS) point cloud, respectively, from 2008 to 2018. Source: MDPI; and, (b) Number of publications related to ‘mobile laser scanning’ from 2010 to 2018.
Figure 1. (a) Ratio of publications related to ‘road object’, using airborne laser scanning (ALS) point cloud, imagery, and mobile laser scanning (MLS) point cloud, respectively, from 2008 to 2018. Source: MDPI; and, (b) Number of publications related to ‘mobile laser scanning’ from 2010 to 2018.
Remotesensing 10 01531 g001
Figure 2. Components of MLS systems: (a) Trimble MX-9, and (b) RIEGL VMX-2HA.
Figure 2. Components of MLS systems: (a) Trimble MX-9, and (b) RIEGL VMX-2HA.
Remotesensing 10 01531 g002
Figure 3. Principle of geo-referencing [14].
Figure 3. Principle of geo-referencing [14].
Remotesensing 10 01531 g003
Figure 4. Current off-the-shelf commercial MLS systems: (a) Trimble MX-9; (b) RIEGL VMX-2HA; (c) Lynx Mobile Mapper™ SG; (d) StreetMapperIV; (e) Topcon IP-S3; and, (f) FARO® and Siteco Road-Scanner C. Source: [17,18,19,20,21].
Figure 4. Current off-the-shelf commercial MLS systems: (a) Trimble MX-9; (b) RIEGL VMX-2HA; (c) Lynx Mobile Mapper™ SG; (d) StreetMapperIV; (e) Topcon IP-S3; and, (f) FARO® and Siteco Road-Scanner C. Source: [17,18,19,20,21].
Remotesensing 10 01531 g004
Figure 5. Applications and experimental results of on-road information extraction using MLS point clouds.
Figure 5. Applications and experimental results of on-road information extraction using MLS point clouds.
Remotesensing 10 01531 g005
Figure 6. Methods and experimental results for road surface extraction from MLS point clouds.
Figure 6. Methods and experimental results for road surface extraction from MLS point clouds.
Remotesensing 10 01531 g006
Figure 7. Methods and experimental results for road marking extraction from MLS point clouds.
Figure 7. Methods and experimental results for road marking extraction from MLS point clouds.
Remotesensing 10 01531 g007
Figure 8. The generated driving lines using MLS point clouds: (a) the generated driving lines. Source: [80]; (b) the generated transition lines. Source: [81]; and, (c) the generated transition lines. Source: [81].
Figure 8. The generated driving lines using MLS point clouds: (a) the generated driving lines. Source: [80]; (b) the generated transition lines. Source: [81]; and, (c) the generated transition lines. Source: [81].
Remotesensing 10 01531 g008
Figure 9. Off-road objects attributes, including geometric attributes and spatial attributes .
Figure 9. Off-road objects attributes, including geometric attributes and spatial attributes .
Remotesensing 10 01531 g009
Figure 10. Traffic sign detection (TSD) and traffic sign recognition (TSR) using point clouds and imagery data [91]. (a) Raw data, (b) off-road data after ground removal, (c) off-road objects clustering results, (d) traffic signs, (e) projection, and (f) TSR results.
Figure 10. Traffic sign detection (TSD) and traffic sign recognition (TSR) using point clouds and imagery data [91]. (a) Raw data, (b) off-road data after ground removal, (c) off-road objects clustering results, (d) traffic signs, (e) projection, and (f) TSR results.
Remotesensing 10 01531 g010
Figure 11. Street light types [114].
Figure 11. Street light types [114].
Remotesensing 10 01531 g011
Figure 12. Methods and experimental results for tree extraction from MLS point clouds.
Figure 12. Methods and experimental results for tree extraction from MLS point clouds.
Remotesensing 10 01531 g012
Table 1. Parameters used in the direct geo-referencing transformation.
Table 1. Parameters used in the direct geo-referencing transformation.
ParametersRepresentationSource
( X P , Y P , Z P ) Coordinate of the laser point P in the mapping systemMapping frame
R M IMU ( ω , φ , κ ) Rotation information align mapping frame with IMUIMU
R IMU S ( Δ ω , Δ φ , Δ κ ) Transformation parameters from the laser scanner to IMU coordinate systemSystem calibration
r P S ( α   d ) Relative position information of point P in the laser sensor coordinate systemLaser scanners
l X , l Y , l Z The offsets between the IMU origin and the laser scanner originSystem calibration
L X , L Y , L Z The offsets between the GNSS origin and the IMU originSystem calibration
( X GNSS , Y GNSS , Z GNSS ) Position of GNSS sensor in mapping frameGNSS antenna
Table 2. Product specifications of several MLS systems.
Table 2. Product specifications of several MLS systems.
CorporationTrimbleRIEGLTeledy-Ne Optech3D Laser Mapping Ltd.& IGI mbHTOPCONSiteco
MLS SystemMX-9VMX-450VMX-2HALynx SGStreetMapperIVIP-S3 Compact+ROAD SCANNER-C
Laser scanner componentLaser scanner-RIEGL VQ-450 (2)RIEGL VUX-1HA (2)Optech Lynx HS-600 DualRIEGL VUX-1HA (2)SICK LMS 291(5)Faro Focus3D (2)
Laser wavelengthnear infrared1550 nmnear infrared905 nm905 nm
Measuring principleTOF measurement.
Echo signal digitalization.
TOF.TOF.
Echo signal digitalization.
TOF.Phase difference.
Maximum range420 m800 m420 m250 m420 m100 m330 m
Minimum range1.2 m1.5 m1.2 m-1.2 m0.7 m0.6 m
Measurement precision3 mm5 mm3 mm5 mm3 mm10 mm2 mm
Absolute accuracy5 mm8 mm5 mm5 cm5 mm35 mm10 mm
Scan frequency500 scans/sUp to 200 Hz500 scans/sUp to 600 scans/s500 scans/s75–100 Hz96 Hz
Angle measurement resolution0.001°0.001°0.001°0.001°0.001°0.667°H0.00076° V 0.009°
Scanner field of view360 ° 360 ° 360 ° 360 ° 360 ° 180 ° /90 ° H360 ° V 320 °
GNSS componentGNSS typesPOS LV 520POS LV-510POS LV-520POS LV 420POS LV-51040 channels, dual constellation, dual frequency GPS + GLONASS L1/L2Dual frequency GPS + GLONASS L1/L2
Roll & Pitch0.005 ° 0.005 ° 0.005 ° 0.015 ° 0.005 ° 0.4 ° 0.020 °
Heading0.015 ° 0.015 ° 0.015 ° 0.020 ° 0.015 ° 0.2 ° 0.015°
Imagery componentCamera typesPoint Gray Grasshopper® GRAS-50S5C (4)500 MP (6)Nikon® D810 (up to 9) or FLIR Ladybug® 5+Up to four 5-MP cameras and one Ladybug® cameraLadybug 3 & Ladybug 5 (6)-Ladybug 5 30-megapixel Spher. camera
Lens size2/3” CCD2/3” CCD2/3” CCD1/1.8” or 2/3” CCD2/3”1/3” CCD-
Field of view-80° × 65°104° × 81°57° × 47°--360°
Exposure-8-3215Max. 30
Reference [17][18][19][20]-[21]-
Table 3. Summary of several road surface extraction methods.
Table 3. Summary of several road surface extraction methods.
MethodInformationDescriptionPerformanceReference
Voxel-based upward growing method
  • Connectivity in vertical direction
  • Elevation
First, split the original point cloud into grids with certain width in the XOY plane. Second, use the octree data structure organizes the voxel in each grid. Third, the voxel grows upward to its nine neighbors, then continues to search upward to their corresponding neighbors and terminates when no more voxels can be searched. Finally, the voxel with the highest elevation is compared with a threshold to label it as ground or off-ground part.
  • Adaptive to large scenes with strong ground fluctuations.
  • May not function well for data with a lot of noise.
[7,59,60,61,62,63]
Ground Segmentati-on
  • Height difference
First, voxelize the input data into blocks. Second, cluster neighboring voxels whose vertical mean and variance differences are less than given thresholds and choose the largest partition as the ground. Third, select ground seeds using the trajectory. Finally, use K-Nearest Neighbor algorithm to determine the closest voxel for each point in the trajectory.
  • Making the region growing process faster.
  • Suitable for simple scenes
  • Implement in global scale
[63,64]
Trajectory-based filter
  • Trajectory data
  • 2D geo-referenced image
First, using the trajectory data and the distance to filter original data. Second, rasterize the filtered data onto the horizontal plane and then compute different features of point cloud in each voxel. The raster data structure is visualized as an intensity image. Finally, binary the image using different features. Run AND operation between both binary images to filter the ground points.
  • Fit for data with high intensity.
  • May cause over-segment due to rasterization.
[41,65]
Terrain filter
  • Point density
  • Euclidean distance
First, partition the input data into grids along horizontal XY-plane. Second, determine a representative point based on the given percentile. Then use it and its neighboring cell’s representative points to estimate a local plane. Third, define a rectangle box with a known distance to the estimated plane. Such points located outside the box are labeled as “off-terrain” points. The box is partitioned into four small groups. Repeat Step 1, 2, 3 until meet the termination condition. Finally, Apply the Euclidean clustering to group the off-terrain points into clusters.
  • Suitable for complex scene.
  • Time-consuming
  • Heavy computational burden.
[66]
Voxel-based ground removal method
  • Elevation
  • Point density
Firstly, the point clouds are vertically segmented into voxels with equal length and width on the XOY plane. Then, the ground points are removed by an elevation threshold in each voxel.
  • Fit for scenes with small fluctuations.
  • Simple and time-saving.
[67,68]
RANSAC-based filter
  • Trajectory data
  • Height deviation
First, divide the entire point clouds into partitions along the trajectory. Use plane fitting to estimate the average height of ground. Second, iteratively apply RANSAC algorithm to fit a plane. The iteration terminates when the elevation of one point is higher than the mean elevation or the number of points in the plane generated by RANSAC remains unchanged.
  • Quick and effectively.
  • Not suitable for complex scene.
  • Cannot work for scenes with multiple planes.
[42]
Table 4. Comparison of road marking extraction methods.
Table 4. Comparison of road marking extraction methods.
CategoriesMethodInformationAdvantagesLimitations
2D image-based extractionMultiple thresholding segmentation [13,40]
Intensity
Scanning distance
Semantic information
Efficient to overcome point permutation and intensity inconsistency issues of MLS point clouds.
Straightforward
Difficult to deal with complex road environments.
Hough Transform [71]
Intensity
Computation efficiency
Challenging to handle complex road markings types (e.g., arrows and words).
Multi-scale Tensor Voting [14]
Intensity values
Scanning range
Achieve extraction improvement
Suppress noises
Preserve road markings
Require rich prior knowledge to set stop iteration criteria.
Difficult to filter out small noisy fragments.
3D MLS-based extractionDeep learning neural network [74]
Small-sized road marking types
Efficient for small size road markings extraction.
Geospatial information of road markings is preserved.
Require lots of labeled samples for training.
Profile-based intensity analysis [73]
Intensity
Scanning range
Trajectory data
Do not require lane models.
Computation efficiency.
Difficult to handle complex road marking types.
Cannot work well in rural area without curbstones.
Table 5. This table shows the attributes of traffic signs.
Table 5. This table shows the attributes of traffic signs.
AttributesGeometric DimensionGeometric RelationsGeometric ShapeTopologyIntensity
Traffic signsHeight: 1.2–10 mPerpendicular to ground; parallel to pole-like objectscircle, rectangle, triangleplanar structureshigh

Share and Cite

MDPI and ACS Style

Ma, L.; Li, Y.; Li, J.; Wang, C.; Wang, R.; Chapman, M.A. Mobile Laser Scanned Point-Clouds for Road Object Detection and Extraction: A Review. Remote Sens. 2018, 10, 1531. https://0-doi-org.brum.beds.ac.uk/10.3390/rs10101531

AMA Style

Ma L, Li Y, Li J, Wang C, Wang R, Chapman MA. Mobile Laser Scanned Point-Clouds for Road Object Detection and Extraction: A Review. Remote Sensing. 2018; 10(10):1531. https://0-doi-org.brum.beds.ac.uk/10.3390/rs10101531

Chicago/Turabian Style

Ma, Lingfei, Ying Li, Jonathan Li, Cheng Wang, Ruisheng Wang, and Michael A. Chapman. 2018. "Mobile Laser Scanned Point-Clouds for Road Object Detection and Extraction: A Review" Remote Sensing 10, no. 10: 1531. https://0-doi-org.brum.beds.ac.uk/10.3390/rs10101531

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop