Research Review on Parking Space Detection Method

Abstract

The parking assist system is an essential application of the car’s active collision avoidance system in low-speed and complex urban environments, which has been a hot research topic in recent years. Parking space detection is an important step of the parking assistance system, and its research object is parking spaces with symmetrical structures in parking lots. By analyzing and investigating parking space information measured by the sensors, reliable detection of sufficient parking spaces can be realized. First, this article discusses the main problems in the process of detecting parking spaces, illustrating the research significance and current research status of parking space detection methods. In addition, it further introduces some parking space detection methods, including free-space-based methods, parking-space-marking-based methods, user-interface-based methods, and infrastructure-based methods, which are all under methods of parking space selection. Lastly, this article summarizes the parking space detection methods, which gives a clear direction for future research.

1. Introduction

With the rapid economic and social development, the number of motor vehicles in China has increased rapidly. In contrast, the construction of parking lots has been relatively slow, and the problem of parking difficulties has become increasingly prominent. Research on parking space detection methods can not only effectively increase the utilization rate of parking spaces but also alleviate the problem of limited parking space resources, and meet the requirements of the parking lot, including efficiency, safety, and management [1].

Currently, there are many detection methods for parking spaces. According to the categories of the selected sensors, they can be divided into visual and non-visual detection methods [2]. The vision-based detection methods mainly use monocular cameras, binocular cameras, or RGB-Depth (RGB-D) cameras. By using cameras for image acquisition, based on the captured images, computer vision algorithms are applied to detect parking spaces. Those approaches provide real-time visual assistance and rich image information, but it could be greatly affected by environmental factors. They are also usually computationally expensive, and it performs poorly in a dark environment. In contrast, non-visual detection methods mainly use ultrasonic sensors, short-range radars, or laser scanners. In this way, they send microwave signals to the surroundings and determine the distance from the sensor itself to the environment based on the “time of flight” (TOF). This kind of microwave sensor is independent of the lighting conditions in the close range. Data processing is straightforward and, therefore, fast, usually in a real-time speed. Therefore, in most scenarios, it can meet the requirements of parking measurement in terms of measuring distance and accuracy, but it also has some shortcomings, such as a large beam angle, poor direction, low resolution, short working distance, and other shortcomings.

Table 1. Advantages and disadvantages of various sensors.

Sensor Type	Advantages	Disadvantages
Ultrasonic radars and short-range radars	Low cost Long life Small size	Low accuracy Short-range Cannot be used in vertical parking mode
Laser scanners	High stability High precision	High cost Short life Easy to be affected by rain and snow
Vision sensors	Low cost Long life High accuracy Rich image information	Weak robustness

summarizes the advantages and disadvantages of different sensors.

After collating and summarizing the relevant literature, this paper subdivides the parking space detection methods, which are vision-based and non-visual-based parking space detection methods. Vision-based parking detection methods can be further divided into parking space markings and user interface-based methods. Moreover, non-visual-based parking space detection is divided into free-space-based and infrastructure-based methods. According to the reasoning process of effective parking spaces, each method contains several sub-methods, as shown in Figure 1.

Section 1 of this article introduces two mainstream parking space detection methods and the main problems. Section 2 introduces the relevant background and early research. Section 3, Section 4, Section 5 and Section 6 analyze and summarize parking space detection methods based on free space, parking space marking, user interface, and infrastructure. Section 7 summarizes the full text and looks forward to future research directions.

2. Related Background and Early Research

In recent years, with rapid economic development and increasing improvement of people’s living standards, the per capita ownership of automobiles has also increased year by year. According to the Traffic Management Bureau of the Ministry of Public Security, by the end of 2019, our country’s car ownership has reached 260 million. The number of car drivers has reached 397 million, indicating that China has entered the automobile era. Simultaneously, the increasing growth in the number of cars has also brought about static traffic and other problems. Static traffic refers to the parking of vehicles, including short-term parking due to passengers getting on and off or loading and unloading of goods, and long-term parking in parking lots. The problems of static traffic are mainly manifested in the following aspects: planning and layout, insufficient control of land use indicators, fewer parking spaces in parking lots, fewer parking spaces in public buildings, and a severe road occupation phenomenon. It can be seen from the above situation that the parking problem is particularly serious. Therefore, improving parking space detection technology has become the priority of alleviating parking difficulties. In recent years, with the continuous development of parking space detection technology, researchers have developed various parking space detection methods to help drivers park and enter the garage more quickly with real-time parking space information.

With the rapid growth of demand for parking assistance systems in recent years, researchers have proposed a variety of parking space detection methods, which can be divided into fully automatic and semi-automatic detection methods. The fully automatic parking space detection methods refer to there being no need for manual intervention. This detection system automatically selects the required parking space. However, there are great constraints on the types of parking spaces inside because all the detected parking space types are predefined. The system cannot detect that it is not within the defined range. In contrast, the semi-automatic parking space detection method means that, during the parking space detection process, human-computer interaction is required to complete the detection of available parking spaces. Compared with the fully automatic method, the semi-automatic method may produce more reliable results and consume less computing resources because it has additional information from the driver. For example, Toyota’s IPA (Intelligent Parking Assist System) is a typical semi-automatic parking system that displays potential parking spaces on the rear-view camera image through a human-machine interface display and enables the driver to use direction control buttons to change the location of the parking space.

Although semi-automatic parking space detection has greatly reduced the driver’s difficulty in parking, people still find it too cumbersome and complicated in daily use. Therefore, researchers have proposed a variety of fully automatic parking space detection methods. Kaempchen et al. [3] developed a stereo-vision-based method, which uses a feature-based stereo algorithm, a template matching algorithm on a depth map, and an iterative closest point algorithm to perform a three-dimensional fitting of a vehicle plane model. Xu et al. [4] proposed a parking space marking method based on Restricted coulomb energy (RCE) color segmentation, contour extraction by the least square method, and inverse perspective transformation theory. However, this system only relies on parking lot markings. It may be degraded due to poor visual conditions such as stains, shadows, and occlusions from adjacent vehicles on the markings.

With the continuous development of parking space detection technology, more researchers are exploring parking space detection methods based on semi-automatic and fully automatic methods, which promote the continuous progress and development of this field. This article will summarize and classify parking space detection methods and technologies. At the same time, it will analyze and explain the characteristics of current parking space detection technology, concluding the challenges faced by parking space detection technology in this field in order to better solve parking spaces for future researchers. All of this can provide a reference or help in detecting problems.

3. Free-Space-Based Methods

Free-space-based methods are to realize the detection of available parking spaces by identifying the surrounding environment of adjacent vehicles and analyzing the space structure around the vehicles. Among the commonly used sensors are ultrasonic sensors, laser sensors, stereo cameras, depth cameras, and other cameras. These sensors can perceive the environment around the vehicle and provide reliable reference data for parking space detection. According to the different discrimination methods of parking spaces, free-space-based methods can be further divided into direct-ranging-based methods, probability-maps-based methods, and 3D-reconstruction-based methods. This section will analyze and introduce the characteristics of these three methods, as shown in

Table 2. Free-space-based method.

Methods	Characteristics	Method Process
Direct-ranging-based methods	Using direct range sensor to measure the distance to surrounding obstacles
Probability-maps-based methods	Using probability maps to indicate the occupancy of the surrounding space
3D-reconstruction-based methods	Reconstructing the 3D environment from the data measured by the sensors

.

3.1. Direct-Ranging-Based Parking Space Detection Methods

Ranging sensors can emit microwave signals of a specific frequency and use an air medium to propagate. According to the reflection principle of microwave signals encountering obstacles, the distance between the sensor and the obstacle can be calculated by measuring the signal propagation time in the air. Therefore, installing a distance measuring sensor on the vehicle can sense the distance between the vehicle and the adjacent parked vehicle. The space structure around the vehicle can be estimated based on these continuously measured data. Available parking space can be obtained by comparing the size of free space and the vehicle. This method is referred to as a direct-ranging-based parking space detection method.

Based on the idea of sensor ranging, Pohl et al. [5] designed a semi-automated parking assistance system by seeking to reuse the built-in sensors of the vehicle and adding a few additional components to the vehicle. When the vehicle is running, the system continuously detects the distance to the surrounding vehicles through the ultrasonic sensor to detect whether there is available parking space, which can be seen in Figure 2. However, due to the characteristics of a sound wave reflection, only when the sound wave is orthogonal to the reflecting surface can the distance between the two be accurately measured. The reason lies in that the corners of the vehicle are not regular right angles. The ultrasonic sensor has a large error at the corner of the vehicle during the measurement process. In order to solve such problems, Jeong et al. [6] proposed a parallel parking assist system (PPAS) structure with software and hardware cooperation. The system uses a new sensor algorithm to reduce the vehicle angle error of the ultrasonic sensor and can successfully realize parking space detection and automatic steering control. Due to the cheap basic hardware and simple software design, the implementation of PPAS is possible.

Jiang et al. [7] proposed a multi-sensor-based parking space recognition method. Given the shortcomings of existing methods, this method uses lidar to detect the edge points of parking spaces. It also employs the camera to extract the contours of the parking space’s edge. Moreover, it combines the length of the library detected by the lidar with the contours of the parking space edges extracted from the image to determine whether the parking space meets the requirements. This method can further identify parking spaces in more complex scenes and improve parking space resources.

3.2. Probability-Maps-Based Parking Space Detection Methods

At present, there are many problems with the direct-ranging-based parking space detection method. Therefore, researchers use map representation methods and mathematical statistics to divide the environment into a series of small grids in which each is given a possible value to express the probability that the grid is occupied. The parking spaces around the vehicle are estimated by the degree of grid occupancy. This method is called the probability-maps-based parking space detection method.

Schmid et al. [8] proposed a parking space detection method based on a hierarchical three-dimensional occupancy grid, which uses a three-dimensional occupancy grid based on an octree hierarchical data structure to represent the surrounding environment. It can be seen in Figure 3. The detailed level of the dynamic control grid divides the intersection between the detected curb boundary and the vehicle boundary into smaller sections. In addition, it determines the size of the parking space by analyzing the maximum vertical distance between the vehicle boundary set and the intersection. It is widely acknowledged that detecting free parking spaces in a two-dimensional dense grid is easy to operate. This method solves the problem of detecting free parking spaces in a layered three-dimensional data structure. Furthermore, using a layered data structure dramatically reduces the cost of three-dimensional space representation.

Dube et al. [9] borrowed technology from the field of computer vision to extract parking spaces from the occupancy grid. According to Reference [10], a radar-based occupancy grid was established utilizing extracting parking candidate vehicles from this grid and describing and classifying these candidates to determine whether there are vehicles in the grid. This method solves the problem of detecting parallel and vertical parking at the same time. Loeffler et al. [11] used an occupancy grid based on the Dempster-Shafer theory [12]. This method accumulates the measured static cluster information in the grid, and two compensation factor (Gaussian factor and saturation factor) coefficients are introduced to improve the accuracy of parking space detection. This method may detect parking spaces on the other side of the road, even at higher vehicle speeds. Prophet et al. [13] designed an impressive two-dimensional vector target list, which uses a special contour filter to sort all targets in the environmental map, detect parallel parking spaces and vertical parking spaces, and park them. Perform a qualitative assessment. Compared with the commonly used grid map method, this method processes radar data in a specially designed two-dimensional vector target list. Therefore, quantization errors are avoided, and the computational burden is reduced.

Andre et al. [14] came up with a new parking garage vehicle location and tracking system with environment-based embedded lidar sensors. The method integrates data from multiple sensors, allowing vehicles to be tracked in a public parking lot coordinate system. Simultaneously, a combination of the grid-based method, the random sample consensus (RANSAC) algorithm, and the Kalman filter is used to realize real-time vehicle detection and tracking, and highly confident and accurate vehicle positioning can be achieved.

Scheunert et al. [15] developed a method for parking space detection, which uses a polarization mode dispersion (PMD) three-dimensional range-finder camera. The sensor allows reference to a large number of spatial point measurements, representing the cut of the observed scene in detail. The feature extraction is based on the feature consistency of the occupied grid and the detection channel. This method focuses on the feature extraction of PMD data and the feature fusion that defines parking spaces’ free space.

3.3. 3D-Reconstruction-Based Parking Space Detection Methods

3D-reconstruction-based parking space detection method realizes the detection of the parking space by reconstructing the three-dimensional space model around the vehicle. This method intuitively displays the space structure around the vehicle and is easy for the driver to understand.

Park et al. [16] proposed combining ultrasonic sensors and three-dimensional vision sensors to detect parking spaces. The shape of the parked vehicle is modeled by two vertical planes: a longitudinal plane and a transverse plane. The ultrasonic data calculate the former, and the three-dimensional vision data obtains the lateral plane. Moreover, the empty spaces between adjacent vehicles are the detected parking spaces. This method uses a vision sensor to make up for the inaccurate measurement of the ultrasonic at the corner of the vehicle.

In order to identify parking spaces, Zhou et al. [17] proposed a supervised learning method by detecting parked vehicles and explaining the parking environment. In this way, vehicle detection can be achieved by identifying vehicle bumpers from laser range scans. Among them, AdaBoost [18] is used to train a classifier based on the relevant geometric features of the data segment corresponding to the car bumper. In this method, the detected bumper is used as the symbol of the vehicle hypothesis. A topology map representing the structure of the parking space is constructed, and then the topology map is spatially analyzed to identify potential parking spaces. Lee et al. [19] preprocessed the three-dimensional point cloud data and then calculated the minimum size of the parking space based on the vehicle dynamics theory. Kaempchen et al. [3] also started from a set of three-dimensional point cloud data, using a vision-based detection method to determine the position of the three-dimensional point by estimating its position and then extracting potential vehicles from the depth map through template matching. The three-dimensional point cloud data is segmented. Finally, the plane of the vehicle model is fitted to it through the iterative closest point (IPC) algorithm to make the detected vehicle accurate and reliable.

Unger et al. [20] identified an efficient real-time motion stereo algorithm, allowing for a dense depth map of the vehicle’s observation lateral space on a mobile central processing unit (CPU). Due to the movement of the vehicle, images are acquired in consecutive time instances (that is, from different perspectives) to calculate the disparity map [21]. In order to eliminate abnormal points and improve the accuracy of depth information, the algorithm incorporates the time history of the disparity map. It extracts the ground and obstacles from these measurements, updating the environmental map accordingly.

Jung et al. [22] designed a target position method based on light belt projection, which provides an economical target location design method for indoor parking spaces with insufficient light without reducing the car’s interior. This method can add a low-cost light flat projector to the existing rear parking camera. The light streak feature is detected by the difference between the images of the light flat projector turned on and off. The light stripe projection theory is used to reconstruct the three-dimensional information of the parking lot. By normalizing the direction and constructing a depth map, this method can detect free space and select the closest point as the target position. Moreover, it can successfully determine the target position when the light conditions are poor and the reflective surface of adjacent parked vehicles is black.

Pelaez et al. [23] explored a method to assist the driver in parking by processing the data acquired by the three-dimensional time-of-flight (ToF) camera and reconstructing objects around the vehicle. This technology mainly focuses on the fusion of two parallel processing technologies. One is the visual processing of intensity images, and the other is the spatial processing of point clouds. Both are centered on detecting the license plate to estimate the position of the license plate and determine the free position in the parking lot. This method solves the problems of performance degradation under bright ambient light (mainly occurring in outdoor parking lots), resulting in shadows and brightness in the image, and limited detection of low-reflective objects, such as dark cars.

Meng et al. [24] designed a machine-vision-based parking space detection method. This method first customizes the parking space area, and sampling points are automatically generated in the customized area based on specific rules. In this way, it can greatly reduce the computing resources and storage resources of image processing. Second, parking space detection is carried out according to the change of the gray value of the relevant sampling points when the vehicle enters the parking space. Finally, the virtual parking space is customized by the height of the space, which solves the problem of vehicle occlusion in adjacent parking spaces. Various interferences such as strange objects on the road through shape matching and other algorithms are eliminated.

In the process of traditional 3D reconstruction, there are a large number of false 3D points, and the density of 3D points has a linear relationship with the number of detected features, which cannot be controlled. In References [25,26], it is mentioned that the detection accuracy of parking spaces can be promoted by improving the tracking efficiency of 3D points, the tracking density, and the quality of the 3D prints of the reconstructed obstacle model.

3.4. Summary

At present, the method of detecting parking spaces based on free space has become the mainstream method, and this method only needs to install cheap ranging sensors on the vehicle. At the same time, this method also has certain shortcomings. The detection performance of this method is completely dependent on the adjacent parked vehicles. When there are no parked vehicles around, this method will fail.

4. Parking-Space-Marking-Based Methods

Free-space-based parking space detection methods are the most widely used methods, but their performance depends on the pose of neighboring vehicles and the measurement accuracy of the sensors used. Therefore, without adjacent vehicles, free-space-based parking space detection methods cannot work normally and cannot meet the driver’s demand for parking space detection. Therefore, researchers proposed a parking space detection method based on parking space markings, using computer vision technology to determine the location of the parking space by identifying the parking space markings from the image. According to the different detection technologies, it can be divided into three categories: straight-line-detection-based methods, corner detection-based methods, and learning-based detection methods. This section will analyze and introduce these three detection methods, as shown in

Table 3. Parking-space-marking-based method.

Methods	Characteristics	Method Process
straight-line-based methods	Inferring the parking space by detecting straight line segments
corner detection-based methods	By detecting corner points, infer the parking space based on the corner point combination
learning-based detection methods	By extracting the features of the marked points, the parking space is determined according to the type of the marked points

.

4.1. Straight-Line-Based Parking Space Detection Methods

Parking space markings refer to parking spaces composed of guidelines and dividing lines attached to the ground in a specific parking lot. There are many types, such as rectangular parking spaces, parallelogram parking spaces, and diamond parking spaces. Since parking space markings are composed of straight-line segments, the detection of straight lines is essential in finding parking spaces.

Jung et al. [27] used a method based on the combination of pixel classification and feature matching to extract the spatial structure information of the parking lot. The parking lot marking is separated by plane constraints and transformed into a bird’s-eye view, and template matching is performed on the bird’s-eye view to determine the location of the parking lot. The obstacle depth map generated by the parallax of adjacent vehicles can be used as a criterion for template matching by limiting the search range and direction. This method utilizes the obstacle depth map and the bird’s-eye view of the parking lot markings, which effectively limits the search range and improves the operation speed and robustness to visual noise.

Since camera images are not robust for environmental conditions (such as night and backlight), Kamiyama et al. [28] came up with a method to identify parking spaces so that the parking spaces can be used during the day or night, regardless of whether the road is dry or wet. This method uses the statistical model of the received light intensity value of the laser rangefinder to classify the road surface. Then use Hough transform to estimate the target parking position. Jung et al. [27] proposed a parking space marking recognition algorithm based on a monocular vision for automatic selection of the automatic parking assistance system. A one-dimensional filter is designed in the Hough space, using prior knowledge about the feature of the marking line in the bird’s-eye view of the edge image and using the improved distance between the point and the line segment to distinguish the identified line segment. The algorithm can successfully identify the parking space, even if the adjacent vehicles block the parking space marking line seriously.

Hough transform is often used to detect the markings of parking spaces. However, when Hough transform detects parallel line pairs, its performance is not robust because of the influence of factors such as noise, clutter, illumination, and changes in weather conditions. Moreover, the Hough transform cannot detect multiple parallel line pairs at the same time. Wang et al. [29] used a polynomial fisheye distortion model to calibrate the camera. They employed an image stitching method based on the Levenberg-Marquardt algorithm to stitch four images of the fisheye camera into an omni-directional bird’s-eye view image. Second, the method based on the Radon transform is used to extract parking space features, which can detect multiple parking spaces at the same time.

When the detection distance of the ultrasonic sensor is less than 3 m, the result cannot be used to extract further information for identifying obstacles. Moreover, systems based on rear-view cameras cannot assist in situations, such as parallel parking that requires a wider field of view.

Hamada et al. [30] put forward a parking space detection algorithm based on panoramic images. This algorithm solves the tracking problem when the detected parking space falls off the surrounding field of view. Lee et al. [31] designed a tapered hat marking line extraction filter and an entropy-based line marking a clustering algorithm for accurate parking line detection. Even in the case of reduced visibility, the conical cap filter can extract line features powerfully. Then, the entropy-based line marker clustering algorithm quickly and effectively assigns the extracted line markers to each parking line segment. Compared with other linear detectors, including Hough transform, this method is faster and has stronger robustness in the case of distorted or blurred parking lines. In addition, when compared with other methods based on corner detection, the detected parking line provides a more accurate estimation of the target position, direction angle, and occlusion angle.

However, changes in light intensity and complex obstacle conditions will seriously affect the performance of the parking space detection system. In addition, the existing parking space detection methods only consider whether large objects occupy the parking space, and ignore the existence of small objects in the parking space. In order to overcome these shortcomings, Li et al. [32] came up with a vision-based automatic parking space detection method that uses a vehicle peripheral vision monitoring system (AVM) composed of four fisheye cameras to detect various parking space signs. This method utilizes a linear segment detector (LSD) based on edge information to detect parking space markings with a pair of parallel lines at a fixed distance in the AVM image. It then uses image segmentation algorithms and stereo vision algorithms to calculate small obstacles in the parking space marking height. This method is superior to the method based on the Hough transform in terms of continuity and completeness. In addition, this method can effectively reduce the frequency of false detections and missed detections. Wenhao et al. [33] came up with an improved LSD detection algorithm, which is divided into the detection thread and tracking thread. In the detection thread, an improved line extractor based on the line segment detector (LSD) [34] obtains the available parking space line edges at the beginning. Second, the parking space angle extractor obtains the structural information of the L-shaped component, which meets the specification requirements of most parking lots. Third, using the L-shaped results detected in the current frame and the L-shaped results obtained from the previous frame tracking, a search method is proposed to obtain candidate parking areas. In the tracking process, an algorithm based on vehicles and Kalman filtering is used to update the actual location of each parking lot and give a confidence score. Finally, with the help of ultrasound and reconfirmation schemes, most of the false alarms are removed to obtain the final detection results, including un-analyzable areas. Li et al. [35] put forward a method based on geometric features, which can obtain better results even when parking space markings are displayed in a fully automatic manner under various lights (dimmed and strong) and ground conditions (brick, curved, blurred, and marking). Based on the original line segment detector (LSD) algorithm, this method first uses line clustering to detect separation lines. Then, according to the geometric characteristics of the parking space, the separation lines are paired to form parking space candidates. Finally, the entrance is detected using straight-line and learning-based methods.

Li et al. [36] illustrated an automatic parking space detection and recognition system, which mainly solves the problem of incomplete parking space marking lines and shadows or obstacles in the existing technology, leading to the problem of inaccurate parking space detection and recognition. This method includes the camera detection module and the recognition module, and the camera is installed around the vehicle and stitched into a seamless panoramic image that can reflect the information around the vehicle. The module performs the detection of the parking space marking line and conducts the incomplete parking space detection through the marking line circumference. Then, the recognition module completes the parking space recognition under the shadow or obstacle in the parking space by calculating the difference value of the internal gray change of the parking space and the height of the obstacle.

Jae et al. [37] created a method that can be fully automated, identifying various types of parking space markings, and is robust for various lighting conditions. The method first extracts parallel line pairs from the around view monitor (AVM) image to detect the separation line of parking space markings. Then, the separation line detected in the current image is combined with the separation line detected in the previous image. According to the geometric constraints of the parking space, a parking space candidate is generated by combining the pairing of the dividing lines. These candidates were confirmed by identifying their entrance locations using line and corner features and using ultrasonic sensors to classify their occupancy. Finally, the free parking space identified in the current image is combined with the free parking space identified in the previous image. According to the vehicle odometer based on the in-vehicle motion sensor, the separation line found in the previous image and the current position of the parking space are tracked.

4.2. Corner-Based Parking Space Detection Methods

Methods based on traditional straight-line detection of parking spaces can only detect one or two types of parking spaces, but there are many types of parking spaces, such as diamonds or parallelograms. The parking space detection method based on corner features can detect a variety of parking space types and realize a variety of parking operations.

Suhr et al. [38] designed a method for fully automatic identification of parking space markings. This method models different types of parking space markings as a hierarchical tree structure for identification. As illustrated in Figure 4, this method mainly includes two processes: bottom-up and top-down. This method gradually generates candidate parking spaces through a bottom-up process. Afterward, according to the nature of the parking space marking type, climbing up the hierarchical tree structure from top to bottom can eliminate the false parking spaces, nodes, and corner points. Ultimately, the final parking space is determined. Although this method is fully automatic, its performance is better than the previous semi-automatic method under the condition of a small amount of calculation.

Suhr et al. [39] proposed an effective parking space detection and tracking system, which is divided into three stages. Furthermore, parking space marking detection employed the hierarchical tree structure method proposed by Reference [38] to identify various types of parking space markings. The occupancy classification stage of parking space uses ultrasonic sensor data to identify the vacancy of the detected parking space. The parking space occupancy rate calculates the probability by treating each parking space area as a single unit occupying the grid. As a result, when the vehicle enters the parking space, the parking space marking the tracking stage continuously estimates the location of the selected parking space. In the tracking process, the AVM image and the motion sensor-based ranging method are fused at the chamfer score level to achieve robustness to the inevitable occlusion caused by the vehicle. This method can identify the position and occupancy of various parking space markings in real-time and can track stably under actual conditions. The system can help the driver to quickly select an available parking space and support the parking control system by continually updating the designated target location.

4.3. Learning-Based Methods

Almost all the existing state-of-the-art methods in this field are based on low-level visual features, such as line segments and corner points, and detect some low-level visual algorithms. Due to noise, clutter, or lighting changes caused by non-repeatable environmental changes, these features are not significantly distinguishable, and, even worse, they are unstable. Therefore, how to detect parking spaces efficiently and accurately using visual methods in a complex and changeable environment is still a difficult problem. In order to solve this problem, researchers use deep learning technology to build a neural network model through a learning-based method, train the model from a large number of positive and negative samples, and realize the detection of parking spaces.

Xu et al. [4] demonstrated a parking space detection method based on a restricted coulomb energy (RCE) neural network color segmentation. According to the inconsistency between the parking space marking line and the background color, this method can automatically perform color segmentation through the training of the REC neural network and after adaptive training, so as to realize the detection of parking space. Li et al. [40] illustrated a parking space detection method based on data-driven learning. When a surround view image is given, a pre-trained detector is used to detect the marker points, and then an effective parking space is inferred from them. However, due to the influence of changing conditions such as ground, light, shadow, etc., it is challenging to detect parking spaces based on vision. Intending to deal with this problem, Lin et al. [41] proposed a new parking space detection method based on a Deep Convolutional Neural Network (DCNN), namely DEEPS, which takes surrounding images as input. In DEEPS, there are two key steps: identifying all the marked points on the input image and classifying the partial image patterns formed by the marked points, as shown in Figure 5. Due to the harsh conditions of outdoor parking lots and the high reflection of indoor parking lots, which reduces the reliability of parking space detection, Jang et al. [42] proposed a unified parking space detection method that can simultaneously detect parking spaces composed of multiple structures. This method can detect a variety of objects through a fully convolutional network of semantic segmentation. In addition, the vertical grid coding method can simultaneously detect vacancies identified by parking space markings and vacancies created by surrounding static objects without sensor fusion. Huang et al. [43] proposed a parking space detection method based on direction marker-point regression, namely Parking-Slot Detection Using Directional Marking-Point Regression (DMPR-PS). DMPR-PS does not use multiple off-the-shelf models but employs a new convolutional neural network (CNN)-based direction marker regression model. Given an image, the model can predict the position, shape, and direction of each marking point on the image. From the marking points, geometric rules can be used to infer the parking space on the image easily.

Hu et al. [44] expounded a deep-learning-based parking space detection method, which sums the loss function obtained based on deep learning and the cost function of the template obtained based on template matching to obtain the total cost function. The position of the point with the highest probability for any model is used as an initial value to search the total cost function to obtain the parking space position. This method combines template matching and deep learning to detect parking spaces in images. It is not only useful when the lighting conditions are good, and the splicing effect is good, but also when the light source is very complex. In this case, the road is reflective. Moreover, it can realize more robust and accurate parking space detection under poor accuracy conditions. Furthermore, the computational complexity of distance transformation and Canny edge detection is very low, and the amount of calculation is small. Therefore, it can run in real-time on the embedded platform, thus, having a direct application value.

4.4. Summary

Parking-space-markings-based methods identify free parking spaces by analyzing the markings on the road. Contrary to free-space-based methods, its performance does not depend on the presence and location of adjacent parked vehicles. However, these methods can only work normally in parking lots with parking space markings, and it is not robust to situations where the parking space markings are blocked or blurred.

5. User-Interface-Based Methods

Free-space-based and parking-space-markings-based detection methods require a tremendous amount of calculation. Therefore, a parking space method based on a user interface is proposed to solve that problem. It supports a simple and easy-to-use input method, such as a touch screen. The user can specify a seed point for the target location, and the detection process revolves around the seed point to search, which significantly reduces the scope of detection and reduces the amount of calculation. The steps of this method are divided into four steps, as seen in Figure 6.

Jung et al. [45] designed a seed point-based method, where the driver specifies the target position through a seed point in the rectangular parking space marking on the touchscreen-based human-machine interface (HMI) to mark the width of the line segment and the seed point. The location and direction of the parking space are identified through the intensity gradient of the direction, and the target parking space is identified using the “T”-shaped connection point. This method dramatically reduces the search range, but can only detect rectangular parking space types [46] based on Reference [27]. The user specifies two seed points on the man-machine interface, which significantly reduces the scope of the system search, thereby, reducing the amount of calculation and misrecognition. Furthermore, through the neural network to classify the connection mode of parking space markings, it can detect many types of parking spaces.

Hiramatsu et al. [47] developed a rear-view camera-based parking assistance system. Its functions include “parallel parking assistance” and voice guidance information superimposed on the rear-view camera image. The user can select the “parallel rail” or “normal (such as garage parking)” mode through the touch screen switch on display. Gi et al. [48] use cursor-like drag-and-drop operations to move and rotate the target position. The method in Reference [49] uses an interface based on arrow buttons, where the driver can locate the rectangle at the desired parking space by clicking several arrow buttons. As a result, this method has been applied to mass production of a Toyota Prius.

Summary

The user-interface-based method finds free parking spaces according to the driver’s manual input of specific information. Although this method requires the driver to have the disadvantage of repeated operations, it is useful as a backup tool for the failure of the automatic method and it is easy to implement.

6. Infrastructure-Based Methods

In large-scale parking lot management, the parking space detection method based on infrastructure is very suitable, and the target location is specified through the local-global positioning system, digital map, and communication with the parking management system [50,51]. This method relies on the infrastructure of the parking lot and needs to distribute various sensor modules and user data communication lines in the parking lot in advance, and communicate with the management system in the parking lot through the vehicle to realize the identification of the parking space. The specific process is seen in Figure 7.

Chen et al. [52] designed an intelligent guidance management system for parking space, including several parking space detectors located directly above the corresponding parking spaces, a parking space controller, several parking space controllers, and a parking space guidance controller, a data processing device, and a data transmission network. The parking guidance controller is connected with a number of parking guidance screens and a parking indicator box set at each parking space. The parking space detector detects whether there is parking in the corresponding parking space. First, it transmits corresponding signals to the parking space controller, and then the parking space controller obtains the best parking space and displays it on the parking space guide screen. Correspondingly, the parking space indicator box that controls the corresponding parking space flashes. After that, the user can be guided to quickly and accurately park the car into the allocated parking space. Yan et al. [53] proposed a safe and intelligent parking system based on a secure wireless network and sensor communication. The parking process is modeled as a random process. The vehicle broadcasts in the field and then uses the sensor infrastructure and decryption methods to protect the privacy of the driver and the security of information.

Li et al. [54] designed an intelligent reservation management system for parking spaces. This system includes an intelligent parking guidance unit, parking space reservation management unit, and control background. In addition, the intelligent parking guidance unit mentioned above includes multiple sets of level-guided screens. The first parking space detector is installed in the roadside parking space, and the geomagnetic vehicle detector is installed at the entrance and exit of the parking lot. The parking space reservation management unit includes the parking space reservation terminal and the parking space entrance, exit, and license plate recognition device on top of each parking space. Additionally, the second parking space detector and liftable ground lock for each parking space in the parking lot, data collection terminal, and guide screen for reserved parking space. The utility model can display the remaining parking space status of the parking lot in real-time. It can accurately indicate the parking space location, reserve the parking space in advance, prevent the reserved parking space from being occupied by others, and guide the car owner to go to the reserved parking space quickly.

Liu et al. [55] developed a global positioning system (GPS)-based automatic parking assistance method and system for unmanned vehicles, which is suitable for side parking, involving: collecting GPS location information of parking spaces in the target area, obtaining vehicle information of minimum turning radius information, and braking distance information. According to the GPS location information, it can mark the parking space, and the parking space is marked according to the parking space information to mark the entrance midpoint of the parking space. According to the parking space entrance midpoint, minimum turning radius information and braking distance information set the turning point and deceleration point. According to the preset turning angle and the turning point and deceleration point, the optimal driving trajectory before parking is generated. By using GPS to set auxiliary points, we guide the car’s position, angle and speed to the appropriate value when entering the parking space, generate the optimal trajectory before parking, reduce uncertainty, and improve parking accuracy.

Xie [56] designed a GPS-based smart parking navigation system, including a GPS navigation device for the car, a parking lot management server, and a parking detector configured in each parking space. Information is transmitted between the parking lot management server and parking spaces through the mobile communication system. The parking space detector transmits the detected parking space information to the parking lot management server. When looking for a parking space, the car driver needs to send the parking through the mobile communication system first. The requested information is sent to the parking lot management server. The parking request information contains the longitude and latitude information of the current location of the car, and the parking lot management server compares the parking request information with the parking space information and calculates the nearest parking space. After that, the result is transmitted to the car GPS navigation device through the mobile communication system. Finally, the car GPS navigation device navigates the car to the nearest parking space. This method can solve the problem of finding a parking space.

Cao et al. [57] provided a parking space guidance system for drones based on the Internet of Things, which includes a PC-side upper computer and a drone. Among them, the PC-side upper computer includes a parking guide model part, a real-time display part of Unmanned Air Vehicle (UAV) coordinates and wireless serial port remote control part. The design has good practicability and can be modified on existing equipment. The drone recognizes empty parking spaces through images. The upper computer automatically plans the optimal path and assigns the drone to guide the incoming vehicles to the corresponding parking spaces and return. The invention is suitable for large-scale and busy open-air parking because it can guide parking in a targeted manner, avoiding the trouble of finding parking space for vehicles and solving parking space grabbing. It links the warehousing vehicles together through drones and can rationally allocate parking resources to achieve energy saving and emission reduction.

Summary

Different from other methods, infrastructure-based methods use pre-built maps and underlying sensors to specify the target location through vehicle-to-infrastructure communication, using equipment outside the vehicle to help the driver easily find or reserve a parking space. Although these methods have the advantage of managing all parking spaces, they are limited to parking lots with pre-established infrastructure.

7. Summary and Outlook

In recent years, with the continuous improvement of drivers’ requirements for cars, the parking assistance system has been developed rapidly, and the parking space detection technology has also been improved. According to the different critical technologies of parking space detection, this paper elaborates on the related research of parking space detection. It analyzes and introduces parking space detection methods based on free space, parking space marking, user interface, and infrastructure. At present, the mainstream methods are based on free space and parking space markings because these two methods achieve the purpose of low-cost and fully automatic parking, but, at the same time, there are certain shortcomings. Among them, the detection efficiency of parking spaces depends entirely on the neighboring posture. When there are no neighboring vehicles, it is easy to cause a system detection failure. As we all know, the research object of the parking-space-marking-based method includes parking space markings. Therefore, when the parking space markings are worn or occluded seriously, the system will also fail to detect. In addition, this detection is not robust when it is in a dark environment. Figure 8 summarizes some methods and key technologies involved in this article.

With the continuous development of sensor technology and artificial intelligence deep learning technology, sensor technology is also developing in the direction of artificial intelligence. Intelligent sensors can integrate data collection, storage, and processing, and have functions such as independent selection and self-regulation, which will provide an excellent technical solution for parking space detection methods. This article believes that further research can be promoted from the following aspects in the future.

• Research on Multi-Sensor-Fusion-Based Parking Space Detection

The algorithms mentioned in this article are all based on the premise of a single sensor. Moreover, when it comes to the practicality of the algorithm, the detection system with only a single sensor has inherent limitations, such as the difficulty of tracking the camera when the camera moves quickly, and it is difficult to deal with dynamic obstacles. Therefore, fusing different sensor data to complement each other can make the system more robust and accurate. For example, the detection system of the inertial navigation combination is more suitable for complex scenes. The inertial sensor (IMU) can measure the acceleration and angular velocity of the sensor body and complements the camera sensor. After the fusion of the two, a more complete parking space detection system can be obtained. However, how to effectively combine the two is a problem worthy of in-depth discussion.

2.

Research on Artificial-Intelligence-Based Parking Space Detection

With the continuous development of artificial intelligence technology, the introduction of artificial intelligence technology into the parking space detection method can make the parking space detection process more intelligent. Xu et al. proposed a color-segmentation-based parking space detection method that relies on a REC neural network, which is the early research that used artificial intelligence technology to detect parking spaces. This method provides excellent guidance for parking space detection and obstacle detection. Whether artificial intelligence technology can be applied to more parking space detection methods is an important research topic.

3.

Research on Parking Space Detection Integrated with Task Requirements

The detection of parking spaces is not the ultimate goal. However, it is to complete diversified tasks by accurately detecting parking spaces, which puts forward higher requirements for detecting parking spaces. When detecting parking spaces, it is necessary to use the completion of the task as an indicator of detection. K. Hamada et al. [30] made a similar attempt. They regard the ground as a plane, detect parking spaces in the collected images, and estimate the two-dimensional rotation of the target in the current frame and the previous frame. It is panning to realize the tracking processing of the parking space in the panoramic image to complete the selection of the best parking space for the parking operation in the case of multiple parking spaces.