Potential risk assessment for safe driving of autonomous vehicles under occluded vision

This study aimed to explore how autonomous vehicles can predict potential risks and efficiently pass through the dangerous interaction areas in the face of occluded scenes or limited visual scope. First, a Dynamic Bayesian Network based model for real-time assessment of potential risks is proposed, which enables autonomous vehicles to observe the surrounding risk factors, and infer and quantify the potential risks at the visually occluded areas. The risk distance coefficient is established to integrate the perception interaction ability of traffic participants into the model. Second, the predicted potential risk is applied to vehicle motion planning. The vehicle movement is improved by adjusting the speed and heading angle control. Finally, a dynamic simulation platform is built to verify the proposed model in two specific scenarios of view occlusion. The model has been compared with the existing methods, the autonomous vehicles can accurately assess the potential danger of the occluded areas in real-time and can safely, comfortably, and effectively pass through the dangerous interaction areas.


Method
The potential risk γ c proposed in this paper refers to the probability or possibility of collision with TP, which came out of the occluded area all of a sudden, during AV passes through the interaction area. As expressed in γ c = [0, 1] , the potential risk varies with different traffic environments 36 . This paper takes the scene of visual field occlusion on one side of the AV as an example, as shown in Fig. 1, the occluded area may suddenly appear TPs (e.g. pedestrians passing through the clearance between vehicle and vehicle, bicycles or cars passing through the intersection), and the TP may enter the current driving area of the AV. Figure 1 shows common scenes of road and intersection with single occlusion or intermittent occlusion, such as pedestrian crossings, bus stops and intersections. The variables v t e and s t e represent the speed of the AV and distance between the AV and the interaction area at time t respectively. The variable t c represents the time required for the AV to pass through the interaction area which length is s i , t c = s i /v t e . At the current speed, it can be predicted that the AV will reach the Q point of the interaction area after t e = s t e /v t e seconds. Whether the AV continues to pass through www.nature.com/scientificreports/ the interaction area at the current speed depends on the predicted potential risk γ c value within the time t c . Let v p represent the average speed of the occluded TP in t times, d = v p · t indicates the distance traveled by traffic participants in t times. At any time t , the occluded area is divided into n t cells along the X direction, n t = (t e + t c )/�t . Let m t = t e /�t ; when the occluded TP move forward with v p , the TPs on the n c (n c = n t − m t ) cells will arrive at the risk interaction area in the t e ∼ (t e + t c ) period, resulting in potential risks to the AV. Using γ t c,i (i ∈ m t , m t + 1, · · · , n t ) to represent the risk probability generated by cell i at time t , the maximum value of γ t c,i is recorded as γ c : where O t represents the observed value of the AV around the interaction area at time t , which will be described in the later part. As shown in Fig. 1, the distance between the AV and the interaction area at t 1 is s t 1 e , and the visual range in the X direction of the interaction area is x t 1 . As the vehicle moves forward, the visual range gradually increases, and the visual range at t 2 and t 3 becomes x t 2 and x t 3 respectively. y p represents the minimum distance between the occlusion and the TPs, and d o represents the transverse distance between the centerline of the AV and the occlusion, so the visual range x t of the AV in the X direction of the interaction area at time t as: Equation (2) indicates that the visible range x t at time t is related not only to s t e but also to y p and d o . When the vehicle is driving, it is as far away from obstacles as possible, which is in line with the human driving experience. Potential risk assessment model. Simulate the principle of potential risk assessment by human drivers, the core idea of the proposed model is that the AV can inferring the potential risk probability of the occluded area by observing the surrounding environmental. To facilitate the description, pedestrians are the main TPs. The proposed risk assessment model based on DBN analysis is shown in Fig. 2, ρ l is used to represent pedestrians illegally crossing the road in the initial section (cell n t ) of the occluded area, its probability value P(ρ l ) can be predicted and inferred by AVs observing the state of environmental impact factors (yellow nodes). ρ l = 1 represents the event is occurrence, and P(ρ l = 1) is the occurrence probability, that is, the prior probability of risk assessment. Node "Zi" represents the event in which the road cell i is or not occupied by pedestrians at time t , and the node "Oi" represents the observation event of the AV to the road cell i . The AV infers the potential collision risk by observing whether pedestrians occupy the road, that is posteriori probability, P(Z = 1|O) . This paper uses GeNle software to help build Bayesian network. The reasoning mechanism of this software can perform fast and accurate causal reasoning.
It should be noted that the acquisition and measurement technology of variables is not the focus of this research, so it is not introduced in detail in this paper. We assume that the autonomous vehicle can accurately identify the surrounding traffic participants in real time by on-board sensors, and can also accurately measure and obtain various kinds of variables required by the model.
Prior probability P(ρ l = 1). In the occluded areas proposed above, the probability of pedestrians darting out P(ρ l = 1) is uncertain, and the probability varies with different factors, such as traffic environment and time. Determination of risk influencing factors is a crucial issue, which can be carried out by reviewing literature, consulting experts and analyzing historical data. In actual risk assessment, however, it is impossible to establish an assessment model that includes all indicators due to restrictions from the availability of data and the impact of indicators on assessment results. Akin et al. observed pedestrian crossing behavior near Michigan State Univer- Figure 2. The structure of risk assessment model. www.nature.com/scientificreports/ sity through previous video data and concluded that more than 90% of people would choose crosswalks in road sections without signal control 37 . In the study of pedestrian crossing behavior without signal lights in nine urban sections of Changsha in China 38 , 1275 groups of sample data show that 42.35% of pedestrians have "crossed the street illegally" on the main road. The proportion of pedestrians "crossing the street illegally" is 63.8% lower when there is a divider than when there is no divider, and the number of one-way lanes is 2, 3,4, the proportion is 36.7%, 50.3%, and 71.6% lower respectively than one-way lanes is 1. When pedestrians "cross the street illegally", the general headway on the road is H ≥ 2.7 s, and the vehicle speed range is V ≤ 7.78 M/s. The results show that the main factors affecting pedestrian "illegal crossing" are the number of lanes, traffic flow, vehicle speed, divider, and pedestrian crossing facilities and so on. In this paper, as show in Fig. 3, priori probability calculation model is established by considering environmental factors such as pedestrian flow ( ω p ), the number of one-way lanes(n l ), divider(di ), crosswalk(cr ) and the speed of obstacles ( v o ), i.e.: Table 1 details the meaning and status values of each node, which are described in detail in this part and "Posteriori probability of risk assessment".
It is not the focus of this paper that the value of the nodes of environmental impact factor should be acquired in the model. We assume that it can be accurately acquired through vehicle sensor and road network information. The probability distribution of ρ l = 1 under the joint distribution of all environmental impact factors is conditional probability. Theoretical and practical studies show that the Poisson model can reasonably predict the probability of random events. Assuming that the number of pedestrians reaching the occluded areas follows the Poisson distribution: Suppose the average arrival rate of pedestrians per hour is ω , we define ω p = ω/3600 , and propose the probability model of pedestrians crossing the road as follows: Prior value of potential risk ω5 5 ω/3600 > 4 ρ l 0 0 represents the probability of pedestrians reaching the occluded area within 1 s, ρ is the environmental impact coefficient, indicating the impact of various environmental factors on pedestrian crossing the road.
where P c represents the probability of pedestrians crossing the street illegally without being affected by other environmental factors. K d and K v represent the "crosswalk influence coefficient" and "obstacle speed influence coefficient" respectively. cr, di ∈ {0, 1} represent whether the pedestrian crossing area is a crosswalk and whether there is a divider in the center of the road respectively. v o represents the moving speed of the obstacle, and n l ∈ N * represents the number of one-way lanes. Based on some resemble research 37,38 , in this paper we set P c = 0.4 , Bayesian reasoning is applicable to situations where the nature of nodes is discrete, so ω p and v o is measured at the stage level. The values of various states are shown in Table 1. According to Eqs. (5) and (6), when ω p ,di,cr ,v o and n l are at different levels, P(ρ l = 1) has different values. Taking a non-signalized control section n l = 1 , di = 0 , cr = 0,v o = 0 as an example, ω detected by on-board sensor is 1 < ω/3600 ≤ 2 , the probability P(ρ l = 1) reasoning result is 0.346, as shown in Fig. 3. Similarly, when ω p = 0 , then P(ρ l = 1) = 0 ; when ω p = 5 , n l = 1 , Posteriori probability of risk assessment P(Z = 1|O). Taking P(ρ l ) as a priori probability, the potential risk value of the AV passing through the interaction area is inferred according to Bayesian theory based on the observation data of the road section by the onboard sensor. Z t i ∈ {0, 1} is used to indicate whether there is TP on road cell i at time t . Z t i = 1 indicates yes, and Z t i = 0 indicates no. Then, P(Z t=0 i ) = P(ρl) , and the occupancy probability of road cell i after t sec is updated to P(Z t+1 i ) . According to Markov property, there is: where K z denotes the state transition matrix in the DBN, and its value can be allocated based on empirical data, in this paper we set Z t i−1 = Z t−1 i . As shown in Fig. 1, the road segment represented by the dotted line cells can be divided into two parts: the road cells observable by AV and those occluded by obstacle. According to Eq. (2), the left side of x t is observable road cells, while the right side is occluded road cells. O t i ∈ {0, 1} is used to represent the observation result of the AV to cell i at time t , where O t i = 0 indicates that no TP is observed on cell i , and O t i = 1 indicates that TP is observed on cell i . According to Bayesian theory, the occupancy probability of cell i is inferred following observation as: where P(O t i |Z t i = 1) represents the observed probability value when Z t i = 1 . For visible road cells, the observation result is correlated with the road structure and performance of onboard visual equipment 15 . In the experiment, can be obtained by Eq. (7).

Risk distance coefficient.
Most existing studies concerning the risk of collisions caused by occluded fields of view have regarded the potential virtual TPs within occluded areas as independent physical entities with motion functions only, which failed to consider their perceptual decision-making functions or their virtual interactions with AVs. In fact, regardless of whether they are pedestrians or vehicles, TPs usually pay attention to the surrounding environment when they entering an intersection area out from behind occluding object, especially at places with high traffic volumes. According to relevant research 4,15 , when the AV reaches the interaction area, the magnitude of risk with the magnitudes of the distance between the TP and the interaction area is nonlinearly correlated because most TPs are also agents with the ability to adjust their states. At larger distances, TPs have more time to judge the dangers and make timely adjustments to speed or direction, thereby reducing the risk. In contrast, a smaller distance indicates greater risk.
McGill was developed a "conditional risk" model integrating drivers' attention, which describes the nonlinear relationship of vehicle distance from intersection areas with risk 15 . Actually, the magnitude of potential risks is linked not only to the pedestrian speed and the distance between TPs and AVs but also to factors such as speed direction and perceptual capability of TPs. On the basis of the "conditional risk" model, this paper proposes a risk function that integrates the attention, speed, and perceptual interaction capabilities of TPs, which is called the risk distance coefficient and denoted by K d .
where d i represents the vertical distance from the TP on cell i to the intersection area. d s is the minimum safe distance, in the case of pedestrians, d s = 0.8 m in the experimentation; σ d is the standard deviation. Parameter indicates the perceptual capability of TPs. k = 0 indicates perceptually incapable, such as children or moving objects, and k = 1 indicates perceptually capable. In addition, θ is defined as the angle between the movement direction of TPs and the line connecting the intersection area and TPs, π 2 ≤ θ ≤ 3π 2 . Figure 4 depicts the risk distance coefficient between the pedestrian and the intersection area.
If the time taken for pedestrians to arrive at the intersection area is longer than for the vehicle to pass through the intersection area, i.e., at farther distances, the area can be regarded as less dangerous. When the distance from pedestrians to the intersection area is less than the safe distance d s , the risk distance coefficient K d = 1.
Potential risk assessment. Finally, the potential risk on road cell i is constituted by the product of P(Z t i = 1|O t i ) and the K d , which can be obtained by Eqs. (8) and (9) 14 , we use γ c to adjust the speed and heading angle in the AV motion planning, which can improve the safety and comfort of the Vehicle speed. When there is no risk, or the risk is less than the threshold γ go , the desired vehicle speed v des e is set as the speed limit v l of the current road section. When the potential risk is predicted, the desired vehicle speed should be adjusted, we set the desired speed as: Deceleration. At time t , the AV runs on the predetermined route at the speed of v t e , due to the obstruction of the front vision, the AV starts the potential risk assessment model. When the vehicle observes danger or the sudden emergence of pedestrians, first consider the safety and decelerate with the deceleration a − e . The greater the absolute value of a − e , the shorter the braking distance, and the higher safety, but the lower the comfort, the vice versa. In motion planning, the security cost L saf (a − e ) associated with a − e should be considered first as: where a − e,max represents the maximum deceleration in the AV performance index. When the AV decelerates with a − e,max , the braking distance is the shortest and the lowest safety cost. σ saf is the standard deviation at which the nonlinear least-squares optimization method can be solved based on the collected driving experience data.s a − e and s a − e,max are represent the forward distance of the vehicle under deceleration a − e and a − e,max in unit time t a respectively. When the deceleration is a.
In addition to the safety cost L saf (a − e ) , the comfort cost L com (a − e ) is also considered to drive the AV to meet the required speed v t e .
where v represents the speed change in unit time t a , a − e,min is the minimum deceleration (the highest comfort), and σ com is the standard deviation. According to Eq. (12) and Eq. (14), the optimal deceleration threshold a − e during braking can be found by solving the optimization problem of Eq. (15): where v e,min and v e,max are the minimum speed and the maximum speed of the AV respectively, and the values can be seen in Table 2. − represents the security cost weight, and a larger − means that the vehicle has higher requirements for safety. Passing on − , the "maximum comfortable deceleration"a − e can be obtained, represents the maximum deceleration selected by the AV in the view of comfort properties. When the real-time speed v t e (11) v des e =v l (1−γ c ) s.t. v e,min ≤ v t e + a e · t a ≤ v e,max ;|a − e,min | ≤ |a − e | ≤ |a − e,max | www.nature.com/scientificreports/ of the AV exceeds the expected speed v des e or there is a potential danger, the vehicle will adjust the speed in realtime according to the distance s t e and the speed v t e , v t+1 e = v t e − a t e · dt , where the real-time deceleration a t e is: Acceleration. Acceleration operation is generally operated when there is no risk or the risk is low, so the acceleration planning in this study mainly considers the comfort and conservatism of vehicle driving. The calculation model of acceleration is similar to the calculation of deceleration, which only needs to change the weight (use + represents conservative cost weight). The comfortable acceleration a + e during acceleration can be obtained by reducing the cost weight of conservative, the specific description is shown in Eqs. (17)(18)(19). and Based on the foregoing analysis, the speed of AVs is formulated as follows: where d t e represents the braking distance of the AV at v t e speed and acceleration a t e : where τ represents the vehicle operation delay time, which in this paper is τ = 0.2 seconds.
Heading angle. According to Eq. (2), the farther the lateral distance between the vehicle and the occluded area, the more conducive it is to increasing visibility and reducing risk, therefore, change the heading angle θ ego when the AV is running to achieve lateral motion control. Inspired by the potential field method, a threat model of potential risk in the occluded area to surrounding vehicles was established as: where 0 ≤ Ŵ e ≤ γ c , indicates the threat degree of potential risk to AV. σ e is the standard deviation of the potential field, and d eo is the distance between the AV and the edge of the occluded area (the junction of the visible area and occluded area). By mapping risk γ c to the threat degree Ŵ e through Eq. (22), the vehicle's heading is deflected θ ego : where e is the risk repulsion factor used to optimize θ ego . " ± " indicates left or right deflection. In part V, this paper takes e = 1. When the AV is allowed to lateral movement, it is possible to reduces the risk or increases visibility by lateral motion. According to Eq. (23), θ ego is inversely proportional to d eo , and real-time risk γ c is in direct proportion. Similarly, after the vehicle bypasses the occluded areas, it takes the initially planned target route as the final attraction target and horizontally returns to the target route.
The potential risk assessment model-based velocity planning algorithm for intermittently occluded areas is shown in Algorithm II.   Fig. 5, the length of the test road we selected is 60 m, and the road structure is shown in Fig. 6a. There are three occluded areas (1#, 2#, 3#) on the right side of the road. The situation of pedestrians crossing the road randomly in the occluded area, and to verify the model, we simulated the random pedestrian model by uses random numbers simulation in the experiment.
Intersection scene. To further validate the effectiveness of the proposed method, we verified the left turn experiment of AVs at intersections without signal lights, as shown in Fig. 6b, in this traffic scene we set n l = 2 n l = 2 , di = 0,cr = 1 , v o = 0 , ω p = 1 . We recreated the scene for verification by using the same map, initial speed of 9 m/s, expected speed of 9 m/s, maximum acceleration, and minimum acceleration provided in the some works 14,21 , the traffic participants are pedestrians and other vehicles on the road, and their maximum speed is set to 10 m/s. To simulate scenarios with heavy occlusion, the occlusions added in this scene are the building which in roadside and bulky vehicles running on roads.

Results and analysis
Straight road scene. In all simulation test, there was no collision occurred. Even if there is no sudden pedestrian, the AV will be moderately slow down at high speed and be as far away from the risk area as possible laterally to increase the visual range and reduce the risk, as shown in Fig. 7. Figure 7 describes only one type case.
When AV passes through the occluded area 1#. When pedestrians suddenly rush out, the vehicle can also slow down or stop smoothly without fierce shaking to avoid accidents, and performs well in ride comfort. In area 2#, the pedestrian changes his direction for safety (as shown in the green track), so the AV keeps moving at a constant speed without slowing down. Compared with the profiles generated by AEB, that is adopted in the works of Schratter et al. 29 , the AV can safely also pass through the risk areas with the distance is 60 m using  www.nature.com/scientificreports/ the proposed method. Experimental in the same environment showed the AV's acceleration of our method is generally lower and the AV is more comfortable. However, to the best of our knowledge, there is no common computational metric in the literature for ride comfort. We using the following Discomfort Score ( DS ), which defined by Yu et al. 14 to represent a continuous range of discomfort: where T is the duration to reach the goal, and a thresh is the threshold of comfortable deceleration. In the same situation under this scene as show in Fig. 7, when the AV crosses the 60 m test road, the AEB method DS = 2.67, taking 9.0 s, and the proposed method DS = 0.63, taking 9.5 s. But when no pedestrian darting out, both methods are the same in terms of DS and time consuming. Due to the randomness of pedestrian crossing the street illegally, we simulated 1000 cases of pedestrian crossing randomly when AV adopted two methods to pass through the scene shown in Fig. 6a. The average discomfort score, DS aver , and the average time consuming, T aver , as follows: The result shows that, when facing the scene with occluded area, using our method will make the ego-vehicle behavior more cautious. In the emergency of darting out of TPs, the proposed method makes the ego-vehicle more preventive and comfortable than the method of AEB.
Intersection scene. Figure 8c, d describe the speed and acceleration profiles generated by using the ethod proposed by Yu et al. 14 and using our proposed method to realize the left turn of the AV in two cases respectively. Figure 8e shows various potential risk curves predicted by the AV using the method proposed in this paper in scenarios (a) and (b) respectively. In the figure, " R − right " is the potential risk arising from the right road at crossroad which occluded by the building, " R − left " is the potential risk arising from the left road at crossroad, " R − opposite " is the potential risk arising from the oncoming lane while AV crossing the intersection, they are represent the risk probability of collision between the AV and the other vehicles if AV's speed adjustment is not made. Both the method of Yu et al. 14 and ours are make the AV turn left safely without collision, for two scenarios, our method has obtained the speed profile which is basically consistent with the speed profile obtained by Yu et al. 14 .  www.nature.com/scientificreports/ For the first scenario, our method obtains zero acceleration, this is because the initial speed of the vehicle is low. When the initial speed of the AV was increased from 9 to 11 m/s, the test found that the speed of the forward movement of AV would drop somewhat, this due to the potential risk of the road blocked by the building is detected by AV. Under the condition of higher speed, the AV will appear more cautious, which is more in line with natural human driving habits, a cautious approach is reasonable when the full state of the world cannot be observed. So, our method is also suitable for similar intersection scenes with occlusions.
In order to verify that our method is applicable to traffic scenes with visual occlusion, we add traffic vehicles in the Fig. 9 scenario. When the other vehicle suddenly appears from the area which occluded by the building, the AV can observe the surrounding environmental factors, change the speed according to the potential risks of reasoning from all directions, and complete the turn left safely without collision, as show in Fig. 9a and Fig. 9b. There are two fluctuations in the " R − opposite " in Fig. 9c, which are represent the potential risks of the oncoming lane being occluded by the two vehicles when they crossing the intersection.
With good generalization adaptability, our method is suitable for most traffic scenes with visual occlusion. By verification, this method is suitable for scenes on road and intersection with single occlusion or intermittent occlusion, such as pedestrian crossings, bus stops and intersections. Complex scenes in real traffic, such as roundabout or suburban roads, can be decomposed into several road or intersection with small curvature. In addition, since the proposed method is inseparable from the observation of the surroundings by on-board sensors, if the weather is bad, such as fog or night with low visibility, our work may face a challenge, which is also a problem that we need to solve in our future research. For more complex and special scenes, we will explore more factors affecting potential risks.

Conclusion
This paper proposed a potential risk assessment model for AVs under occluded vision. Utilizing Bayesian theory, the model can infer the potential risks arising from TPs in occluded areas based on factors such as the surrounding environment of the occluded area, thereby allowing AVs to safely and efficiently pass through the dangerous interactive areas under limited perceptual data. Under the assumption of the AV's travel path has been planned, from the security, comfort and careful driving aspects, the AV movement was improved by the speed and heading angle control. Through the simulation test, the presented approach allows AV advancing into the road in a comfortably and cautious manner, successively increasing the observable area. The model is suitable for scenes with intermittent occlusion, and also applicable to scene of intersection without signal light and visual field occlusion. However, since the proposed method is inseparable from the observation of the surroundings by on-board sensors, if the weather is bad, such as fog or night with low visibility, our work may face a challenge, which is also a problem that we need to solve in our future research. For more complex and special scenes, we will explore more factors affecting potential risks. In addition, the environmental factors considered in the model are not comprehensive enough, especially when there are many dynamic obstacles, the model needs to be further optimized. On the basis of the current work, we will conduct future research in the following three directions: (1) Further study the potential risk assessment of complex traffic scenes, especially with many dynamic obstructions.
(2) Combined with the road risk analysis of traffic engineering, we will systematically establish a collision risk analysis (DBN inference) model under occluded vision and integrate the established risk assessment model into the complete motion planning method of AVs. (3) Explore more factors affecting the potential risk and discuss how to adjust the road structure and natural data used in the model and learn the best model parameters.