CRAWLING: a crowdsourcing algorithm on wheels for smart parking

We present the principled design of CRAWLING: a CRowdsourcing Algorithm on WheeLs for smart parkING. CRAWLING is an in-car service for the routing of connected cars. Specifically, cars equipped with our service are able to crowdsource data from third-parties, including other cars, pedestrians, smart sensors and social media, in order to fulfill a given routing task. CRAWLING relies on a solid control-theoretical formulation and the routes it computes are the solution of an optimal data-driven control problem where cars maximize a reward capturing environmental conditions while tracking some desired behavior. A key feature of our service is that it allows to consider stochastic behaviors, while taking into account streams of heterogeneous data. We propose a stand-alone, general-purpose, architecture of CRAWLING and we show its effectiveness on a set of scenarios aimed at illustrating all the key features of our service. Simulations show that, when cars are equipped with CRAWLING, the service effectively orchestrates the vehicles, making them able to react online to road conditions, minimizing their cost functions. The architecture implementing our service is openly available and modular with the supporting code enabling researchers to build on CRAWLING and to replicate the numerical results.


Introduction
By 2050, it is expected 1 that 66% of the World population will live in urban environments, with many of our cities set to become megacities.Hence, also driven by the Internet of Things (IoT) revolution and the explosion in the amount of available data, many governments and local authorities have adopted (or are in the process of adopting) concepts based on Smart Cities to tackle the key challenges related to the growing intensive urbanization 2 .As a result, much research and development effort is currently devoted to the design of IoT and data-driven services with the goal of making Smart Cities sustainable 3 , capable of improving the quality of life of their Citizens with subsequent societal, economic and environmental impacts.The design of Intelligent Transportation Systems (ITSs) is central to achieve each of these goals and an exciting opportunity is offered by the possibility of designing, and deploying, services leveraging the increasing pervasiveness of connected cars.In this context, we propose the principled design of a data-driven service based on crowdsourcing for the routing of connected cars, CRAWLING (CRowdsourcing Algorithm on WheeLs for smart parkING).Connected cars equipped with CRAWLING are able to crowdsource the skills/services they need in order to fulfill a given routing task.The service is also designed to allow cars equipped with CRAWLING to gather data from other sources, such as other cars, pedestrians, smart sensors or social media.As a result, vehicles equipped with CRAWLING become an active part of a sharing economy 4 ecosystem, where vehicles share, and create, skills that can be used within the service.A key feature of our service is that it is built around a framework that intrinsically allows to consider stochastic behaviors, thus explicitly accounting for users' stochasticity and to capture their privacy requirements.Specifically, CRAWLING exploits a recent data-driven control algorithm 5,6 that returns a randomized behavior for the car (i.e., a probability function) by solving a sequential decision-making problem 7 that formalizes the tracking of a desired behavior expressed in probabilistic terms.That is, our service systematically minimizes a cost function consisting of: (i) a reward capturing road/traffic conditions; (ii) a regularizer that biases the solution towards some target behavior, allowing CRAWLING to keep into account the possible preferences of passengers.While CRAWLING can be tailored towards generic routing problems, for concreteness we focus here on the problem of routing cars to find parking spaces.Smart parking allocation is attracting much research attention [8][9][10][11] , as roaming for parking is known to be a major source of pollution, traffic and user stress [12][13][14] .In a broader context, lack of coordination in transportation systems is believed to be correlated to an increase in circulating vehicles 15 .To mitigate this, in the context of parking management, urban sensors can be used to monitor the availability of parking spaces and the key challenge becomes that of designing algorithms that are able to optimally route the flow of vehicles 16 .Currently, many such algorithms rely on deterministic decision-making 17 .For example, methods 18 for solving multi-depot vehicle routing frame the problem via deterministic optimization, using binary decision variables.On the other hand, the use of stochastic frameworks has been gaining traction in the Intelligent Transportation Systems (ITSs) literature, for example in the context of differential privacy where privacy is achieved by injecting noise in the system 19 , or in driver 20 or pedestrian 21 intent prediction.Efforts to leverage users' data in Smart City contexts are also the focus of much attention 22,23 , for example for energy 24 or taxi 25 demand prediction, or congestion analysis 26 .While such efforts often focus on modelling 27 and/or artificial intelligence methods 28,29 , some specialized stochastic optimization frameworks have also arisen, for example in parking assignment 30 .Finally, data-driven ITSs, and the data-driven analysis of ITSs 31 have also gained traction, although heterogeneous datasets, emerging from different information streams that need to be merged, have been singled out as a challenge 32 .As an example of one such stream, social media monitoring has been recently used to identify trends related to traffic obstructions 33 and monitor traffic in real time 34 .In summary, the key contributions of the paper can be summarized as follows: after formalizing the smart parking problem as a data-driven control problem, we propose the principled design of CRAWLING and present a stand-alone, general-purpose, implementation of the service.The service provably returns the optimal behavior for cars on which it is equipped.Then, we evaluate its computation times and discuss how these are suitable for routing applications.We also show, via simulations leveraging the microscopic traffic simulator SUMO 35 , the effectiveness of CRAWLING in orchestrating the use of parking spaces in a University Campus during rush hours in the morning.Simulations show that CRAWLING effectively directs the connected cars, confirming that cars equipped with our service improve their average time-to-parking, avoiding unfavourable areas of the map and reacting online to environmental changes (reported, in the scenarios we consider, via social media).The paper is organized as follows.In Section 2, we begin by introducing the data-driven control framework exploited by CRAWLING, and subsequently describing its design, the implementation and the scenarios used to assess its performance.Then, in Section 3, we report the results obtained with the service.We do so by first evaluating the computational efficiency and then discussing the performance obtained on the reference scenarios, which were specifically chosen to validate all the key features of the service.Finally, we give concluding remarks, and discuss the relevance of the results, Section 4.

Notation
We note the set of road links, or lanes in what follows, as X .Time is discrete and, at time step k, the connected car occupies link x k ∈ X .The probability of a connected car of going from x k−1 to x k is denoted by π(x k |x k−1 ).The car has available a number of services (or sources) and the i-th service/source is denoted by π (i) (x k |x k−1 ).The target behavior of the car represents the behavior that passengers would like to track in the absence of road (or environmental) disruptions: this is denoted by p(x k |x k−1 ).Finally, the reward obtained by the car at time k for being on x k is denoted by r k (x k ).As we shall see, the reward is used in our service to capture road/environmental disruptions that might not be known to the vehicle passengers.Also, we denote by π(x 1 , . . ., x N |x 0 ) and p(x 1 , . . ., x N |x 0 ) the product of the transition probabilities π(x k |x k−1 ) and p(x k |x k−1 ), respectively.These joint probability functions clearly depend on the initial condition of the car, x 0 , i.e. the starting link/lane of the trip.Finally, we let: (i) E p [h(X)] := ∑ h(x)p(x) be the expectation of h(•) with the sum taken over the support of the probability function p(x); (ii) the Kullback-Leibler (KL) divergence 36 of p(x) with respect to q(x), with p(x) absolutely continuous with respect to q(x) is D KL (p(x) || q(x)) := ∑ p(x) log p(x) q(x) and measures the proximity of the pair p(x), q(x).We use the Matlab-like notation k 1 : k 2 to denote the set of integers between the integers k 1 and k 2 (included).Consequently, {w k } k 1 :k 2 denotes the set {w k 1 , . . ., w k 2 }.

Materials and methods
We now introduce CRAWLING.We first present the algorithm (Section 2.1) and then discuss its implementation (Section 2.2) and the reference scenarios used to assess its performance (Section 2.3).The key functional components of CRAWLING, and its interactions with the surrounding road environment, are schematically illustrated in Figure 1.As shown in such a figure, CRAWLING is an in-car service: it gathers online data from the environment to feed a suitably defined cost function and has available a number of source services (or simply sources in what follows) to route the car.The sources are e.g., specialized routing services as well as directions collected from other cars in the area also equipped with CRAWLING.Based on the cost and the available sources, CRAWLING dynamically determines the next direction the car should take.This is done via a decision-making (DM) algorithm that exploits recent results in the context of data-driven control 5,6 .The DM algorithm returns a probability function (i.e., a policy) capturing the turning probabilities of the car.The next road link that the car should take is then obtained by sampling form the policy.

CRAWLING: description of the service
CRAWLING seeks to determine the optimal turning probabilities, say π(x k |x k−1 ), of the car by orchestrating the use of a set of available sources.The goal is to track a target behavior for the car, p(x k |x k−1 ), while simultaneously maximizing the car-specific reward r k (x k ).The target behavior can be used to capture a preferred set of directions for the car passengers.This can be computed from e.g., historical data 37,38 and does not depend on the road conditions.These can be instead captured through the car-specific reward r k (x k ).Within the service, the reward signal is used to capture data that are not included in the target behavior: typically, these are related to the state of the network, including hazardous conditions, traffic and roadworks Figure 1.CRAWLING functional architecture and its interactions with the environment.CRAWLING is an in-car service and consists of a decision-making (DM) algorithm, a cost module and a sampling mechanism.These components compute the route the car should take based on environmental data and on the available services.
(Scenario 1 described in Section 2.3).Such data can be gathered from a network of urban sensors and the reward can also be fed by interactions with social media (Scenario 2 in Section 2.3).For example, in the simulations from Section 3, a component of the reward favours road links adjacent to an available parking space, encouraging agents to seek such spaces and park.This structure allows the reward to encompass specifications linked to various tasks, meaning CRAWLING is not per se limited to parking management.To introduce our service, say S the number of available sources, i.e., π (i) (x k |x k−1 ) with i = 1, . . ., S (see Section 1.1 for the definitions).These are assumed to be such that < +∞.This is a standing assumption for our service that has been shown to be non-restrictive in practice 6 .Given this set-up, the problem tackled by CRAWLING to determine the optimal route for the car can be formalized as follows 5,6 : and π(x 0 ), p(x 0 ) be priors on the initial conditions.

Find the sequence of weights α
k ] being an S-dimensional vector at time step k, solving min In Problem 1 the cost function formalizes the fact that the goal of CRAWLING is to track the target behavior while maximizing the car-specific reward.The constraints capture the fact that the solution of the problem is a probability function and, in turn, this probability function is determined by picking one of the sources available to CRAWLING.The problem is tackled 5,6 by Algorithm 1, which embeds both the cost and DM algorithm components of the service in Figure 1.The key macro-steps of the algorithm, which outputs the optimal solution to Problem 1, are described next.
The inputs to the algorithm (lines 1 − 6) are, besides reward and sources, a time horizon, N, and the current state of the agent, x k−1 .The target behavior is an optional input parameter.Indeed, if not provided to CRAWLING, this is set to the uniform distribution (lines 7 − 9).We note that, when this happens, the first term in the cost of the problem in (1) becomes an entropic Algorithm 1 CRAWLING in-car service 1: Inputs: for i = 1 : S do 18: end for 20: k , . . ., α end for regularizer, also widely used in the literature on reinforcement learning 39 .The output of the algorithm (lines 10 − 11) are the optimal turning probabilities throughout the time horizon.This is essentially the plan that the car will follow.Following the initialization phase, the for loop computes the reward-to-go for the car (this is computed by the cost module in Figure 1) which represents how promising a given direction plan is for the car.The reward-to-go is computed via a backward recursion that starts from the last time step in the time horizon.Intuitively, the time horizon is a measure of how far in the future CRAWLING can look to make a decision (see Section 3.1 for the corresponding computational aspect).Specifically, in Algorithm 1, the reward signal r k (•) received by CRAWLING is updated with information from the future time steps through the signal rk (•), which is obtained via backward recursion.Then, the cost of each source's policy is calculated (lines 17 − 18) and the source incurring the smallest cost is selected, and used as the agent's policy (i.e., turning probabilities) -this is returned in line 23.Finally, the next link that the car needs to take is determined by sampling from the Sampling component illustrated in Figure 1: this component, given the current state of the car, samples from the next turning probability determined within Algorithm 1.
Before reporting on the implementation of CRAWLING, we also highlight here a benefit in computing (and using as input) probability functions.First, the sources used by CRAWLING are intrinsically stochastic and this can be useful when the agents generating these data want to e.g.guarantee some desired level of differential privacy.This privacy mechanism consists in corrupting the information sent by the source with some (typically) Laplacian noise.Hence, it can be captured with a source having a Laplacian probability function.Moreover, Algorithm 1 does not determine the next state of the car but rather a probability function.This aspect can be used to guarantee some level of privacy for drivers using CRAWLING, making it possible to both privately share their policy (which can become a source for other vehicles equipped with CRAWLING) and determine the next direction by sampling from this probability.

CRAWLING implementation and validation set-up
We now discuss the proposed implementation for CRAWLING.The code can be found at https://tinyurl.com/wcwvcwaj.The key functional components of the validation set-up are illustrated in Figure 2. As shown in such a figure, we validate our service via the microscopic traffic simulator SUMO 35 .This serves as environment (in which real maps can be imported and traffic demand can be generated) with which CRAWLING interacts.Interactions with SUMO happen via the Python interface TraCI: this is used to both gather the environmental data from SUMO and to control the vehicles based on the directions provided by Algorithm 1.Moreover, the Social Interface module gives CRAWLING the possibility of monitoring certain social media (in our scenarios of Section 2.3, Twitter).In the set-up we propose, this is done by using the Python library Tweepy (https://www.tweepy.org/),which allows to retrieve a given user's tweet history.
In the CRAWLING implementation, the discrete time steps are not uniform but are rather associated to the vehicle changing link (i.e., when the vehicle transitions state).This design solution removes the need for the cars to have a synchronized clock.Algorithm 1 is then implemented via a receding horizon strategy, thus allowing to handle possible changes in the environment.Specifically, every time (say, t − 1) a given car equipped with CRAWLING transitions to a new link the algorithm: • gathers the available data and sources (in the setting of Figure 2 this is done by parsing the data from SUMO through TraCI and from Twitter via Tweepy); • builds the reward based on new information that might have become available during the transition to the new link; • computes the optimal plan of actions for the next N links according to Algorithm 1; • determines the next direction by sampling from π(x t |x t−1 ).The car is then controlled through TraCI by setting the direction obtained from the sampling.
In our code (see our gitub repository for the details) connected cars are encoded by an agent Python class, which contains information about the user's goal (i.e., the route/destination the driver vehicle would like to follow/attain).Algorithm 1 used by CRAWLING also has a Python class, which takes an agent as argument (i.e., a vehicle).This class regroups all the methods used for the algorithm, namely the computation of the KL divergence, a receding horizon control loop and a sampling mechanism.A simulation file is also provided to manage the key interfaces between the agent, CRAWLING and the simulator used to assess its performance (i.e, SUMO).Specifically, the simulation file includes functions for updating the state of the CRAWLING vehicle in the simulation, updating the reward and performing simulation steps through the SUMO-Python interface TraCI.The simulation file for Scenario 1 (see Section 2.3) also logs results when the simulation is over, while the simulation file for Scenario 2 (see Section 2.3) contains the necessary code to interface CRAWLING with social media, namely Twitter.The file implements a parsing function that, monitoring specific users (one of the authors in this case), is called at every step.If the tweet parsed by our code has the hashtag '#sumo_experiment' it is considered for parsing.In this case, the function splits the string into individual words and searches for the word 'blocked' (we leave the design of a more refined parser for future research) and for a road link identifier.Then, the date of the tweet is examined to ensure the tweet has an impact on the current trip: if the tweet was posted before the current date it is ignored.Finally, if all the conditions are met, the blocked road link is assigned a very negative reward as described in Section 3.3.

Scenarios description
We now describe the scenarios (Scenario 1 and Scenario 2 in what follows) that are used to validate CRAWLING.The two scenarios were selected to validate different aspects of our service.Namely, with Scenario 1 we wanted to verify the effectiveness of CRAWLING in smoothly managing a flow of cars that, without our service, could possibly create road blocks.As such, this scenario is centered around the management of a fleet of cars entering the University of Salerno campus during morning rushes.Scenario 2 was instead aimed at verifying how CRAWLING would adapt to incoming information about road closures made available via a Twitter account.Scenario 1.This scenario simulates a morning rush hour at the campus of the University of Salerno.The campus of the University of Salerno, shown in Figure 3 is served by two highway exits, and has three parking lots (in our simulations, each parking lot can accommodate up to 50 cars).The highway exits and corresponding campus entrances will be referred to as southern and northern entrance and the parking lots as Multipiano, Terminal and Biblioteca parking lots (see Figure 3 for the position of such parking lots on the campus map).In this scenario, a fleet of 150 cars (i.e., the maximum number of cars that can be accommodated in the simulation) arrives on campus before the teaching begins, with students and faculty staff searching for parking spaces.In particular, 100 cars arrive from the northern highway exit and seek to park on either Terminal or Biblioteca, while the 50 remaining cars arrive from the south and reach the Multipiano parking lot.The cars arrive on campus one by one at 15-second intervals, with the order of arrival being shuffled.That is, when each car arrives, its destination on campus and whether it is equipped with CRAWLING or not, are randomly selected.While cars arrive to campus, roadworks are being carried out on the main ramp leading from the northern highway exit to the campus, rendering traffic on the corresponding links (highlighted in blue in Figure 5) difficult (the vehicles using such links have their speed restricted to less than one kilometer per hour).This ramp also corresponds to the preferred route of uncontrolled cars (that is, cars not equipped with CRAWLING) that seek to park in Terminal.In Section 3 we use this scenario to benchmark the effectiveness of CRAWLING in properly managing the flow of cars.Simulations with varying amounts of controlled cars are performed to quantify performance.Scenario 2. In the second scenario, only two cars are considered.They both enter the campus from the northern highway exit, at an interval of 30 seconds, and seek to reach the Biblioteca parking lot.Both cars are equipped with CRAWLING.Within the scenario, the ramp leading directly from the highway exit to the parking lot is closed.Specifically, after the first car passes through the exit, the ramp is shut off completely and this information is shared through Twitter (by e.g., pedestrians in the area or the local public authority).This information is gathered by CRAWLING and leveraged to re-route the second car entering in the simulation, to find an alternative route that would properly allow it to park in a suitable parking space.

Results
We now describe the results obtained with CRAWLING on the two scenarios of Section 2.3.To do so, we first empirically investigate the computational load incurred by CRAWLING (and discuss our design choices to make the algorithm suitable for routing applications) and then we describe the main settings of our simulation framework.The code can be accessed at https://tinyurl.com/wcwvcwaj.Note that, to replicate our simulations, a working installation of SUMO, TraCI, Tweepy and the main scientific computing Python libraries is required, along with a Twitter account with developer access.

Computational performance
To make CRAWLING suitable for in-car operation, we leverage the fact that cars being on a given link (i.e., state x k−1 ) can only transition to an outgoing link (that is, x k can only be a link in the set of outgoing neighbors of x k−1 ).This means that the support of the pfs π(x k |x k−1 ), p(x k |x k−1 ) and π(x k |x k−1 ) is restricted to the outgoing neighboring links of x k−1 .Hence, only the neighbors of x k−1 need to be considered in the computations of Algorithm 1.Thus, in the algorithm implementation: (i) the set of outgoing neighbors associated to the current link of the car is determined; (ii) computations are subsequently performed only over the neighborhood (which depends on the time horizon) rather than over the full state space (which would amount to the full map).Using this approach, we empirically investigated if the service is fast enough for in-car applications.We did this by running CRAWLING on each link of the University Campus map in Figure 3 and logging the average running time.On this map, each road link is connected, on average, to 7 other links.We repeated these simulations by varying, across them, both the amount of data sources available to the decision-maker (i.e., S was gradually increased between 1 and 6) and the time horizon (i.e., N was gradually increased between 0 and 5).The results are summarized in Figure 4, leftward panel.The figure illustrates that the computation time is linear with respect to the number of sources (Figure 4, middle panel).This is due to the fact that Algorithm 1 iterates once per source at each time step.More interestingly, the computation time is approximately exponential with respect to the time horizon (Figure 4, rightward panel).This is aspect is essentially due to the exponential increase in the state space (even if reduced as described above) as the time horizon, N, increases.

Simulation settings
We now report the settings that are used to implement the scenarios of Section 2.3 and to obtain the results of Section 3.3.Scenarios settings.The parameters are set before running the simulation.In what follows we simply say that vehicles equipped with CRAWLING are controlled and vehicles without the service are uncontrolled.First, we created lists containing: (i) the starting points and target parking lots for uncontrolled vehicles; (ii) the target behavior for controlled vehicles.These lists are then shuffled to randomize the departing order of the vehicles in the simulation.For simulations involving both controlled and uncontrolled vehicles, the indexes of controlled and uncontrolled vehicles are randomly assigned.All this information is stored 7/14 Figure 5. Northern section of the campus with obstructions highlighted in blue for Scenario 1. Map imported from OpenStreetMap in SUMO via its netconvert tool as .npyfiles.Namely: for Scenario 1, the files agent.npyand foe.npy contain this information.These files are generated before each simulation to avoid biased results.This can be done using the notebook Simulation launcher.ipynb.The relevant maps were imported from OpenStreetMap (https://www.openstreetmap.org/)using the netconvert software and cleaned up using netedit.A full tutorial for this procedure can be found in the SUMO online documentation.At the beginning of each simulation, SUMO loads the map, while TraCI adds each car from the .npyfiles to the simulation.The sources, stored as numpy arrays, are loaded when CRAWLING is launched.While SUMO runs the simulation, information including vehicle data and network conditions is obtained in TraCI and transmitted to CRAWLING, which builds the reward.Car settings.All cars are implemented as TraCI agents.Controlled cars query CRAWLING for directions each time they transition to a new link, while uncontrolled cars are assigned a pre-defined trajectory at the beginning of the simulation (the trajectory is determined by the built-in SUMO routing function that determines the shortest path to the destination).Uncontrolled cars are set to automatically park at the end of their course if possible.On the other hand, controlled cars are assigned a parking space if they arrive to a non-full parking lot.If the parking lot is full, both controlled and uncontrolled cars are rerouted by assigning them a new, non-full parking lot (for uncontrolled cars this is done by assigning them a new destination in SUMO, while controlled agents are assigned a new target behavior).

Results
We now describe the simulation specifics and report the results.Reward and behaviors settings.The reward encompasses environmental disruptions.In our use-case, this is computed from the state of the road links and is related to the availability of parking spaces.Specifically, at each time step, each link adjacent to a parking lot is assigned a reward of 100 if the parking lot has vacant spaces, and −10 if not.Road links on which traffic is perturbed are assigned an additional negative reward: (i) −20 if the traffic is heavily slowed down on the link.This has been simulated, as in Scenario 1, by enforcing a speed limit of less than one kilometer per hour on the perturbed road links; (ii) −100 if the road link is blocked altogether (as in Scenario 2).Before running the simulations, a set of sources is compiled by using SUMO's routing function: each source is designed to represent directions leading to a different point in the network.We build 6 sources, using the highway exits, the parking lots and the north-western side of the campus as destinations.This ensures that the directory of behaviors available to the decision-maker provides a wide coverage of directions from each lane.For each road link, the probability functions π (i) (x k | x k−1 ) of the sources are built by assigning a high probability to the lane indicated by the routing while the other neighbor links have a small, uniform, probability of being selected.We also built a last source by merging behaviors of cars routed towards the Biblioteca and Terminal parking lot.This source can be thought of as emulating the traces from previous vehicles having navigated the campus.In the following, the target behavior of each connected car is selected among the sources.Scenario 1 specifics.Our first set of simulations follows the first scenario outlined in Section 2.3.We implemented this scenario by simulating 150 cars arriving at the campus at 15-second intervals (with a simulation lasting approximately one hour and twenty minutes of simulated time) and seeking to park.All cars entering the campus via the Southern exit are directed towards the Multipiano parking lot.Uncontrolled cars entering through the Northern exit all seek to park in the Terminal parking lot.On the other hand, the target behavior of controlled cars entering through the Northern exit leads to either Biblioteca Figure 6.Confidence interval plot of the simulation results.Solid lines represent the mean value, while the shaded area is the standard deviation.Fully controlled fleets manage to fully park within half of the simulation time, while fleets with less controlled cars take longer, failing to find parking spaces for every car before the simulation ends in the extreme case.Note that fleets with just one third of the cars using CRAWLING still significantly outperform the baseline of uncontrolled cars.
or Terminal (this can be interpreted as a target behavior being built from traces of previous CRAWLING cars having similar goals), highlighting the interest of obtaining directions through crowdsourcing stochastic policies.However, the road leading from the northern entrance to the roundabout immediately to its East is obstructed, causing traffic to significantly slow down (as described above, the speed limit on such road links is reduced to less than one kilometer per hour).As specified in Section 2.3, this obstruction occurs on the preferred route of uncontrolled cars directed towards Terminal, that is, uncontrolled cars within the 100 cars entering through the Northern highway exit.This is illustrated in Figure 5, which captures a section of the full campus map.Such obstructions can originate from road work, or model a limited capacity of the road link related to e.g.severe climatic conditions.In the results shown next, we run different set of simulations with 0, 50, 100 and 150 controlled cars.This was done to evaluate the penetration rate of the service, showing its effectiveness as the number of cars equipped with CRAWLING increases.Results for Scenario 1.As shown in Figure 6, the fully uncontrolled fleet (no vehicles equipped with CRAWLING) fails to park every agent in the allocated time.Indeed, all uncontrolled cars arriving from the northern highway exit are routed to the obstructed lanes.On the other hand, when the fleet is fully controlled (with all vehicles being equipped with our service), CRAWLING only needs approximately half the duration of the simulation to park every car.Namely, CRAWLING allows each controlled cars to adapt its route to avoid the obstruction, effectively orchestrating the use of the available parking spaces.Specifically, we observed that connected cars entering from the Northern exit were routed to the Biblioteca parking lot until it became full.When this happened, CRAWLING rerouted the remaining cars to the Terminal parking lot by a detour route avoiding the obstructed road links.Both controlled and uncontrolled cars entering from the southern highway exit reach the Multipiano parking lot, as it is the closest.To complement our analysis, we also recorded the average time spent by the average car on an obstructed link and the average time-to-parking over all simulations.These are shown in Table 1, together with the results from Welch's t-test 40 , which we used to verify the statistical significance of our simulations.The test is aimed at checking whether two random samples originate from distributions having different means.In other words, a p-value less than 0.05 would mean that a set of simulations is statistically insignificant and needs to be rejected.Specifically, we use the p-value to compare the average parking time obtained in the five first simulations performed with a set of parameters with the five last simulations following the same parameters.Table 1 confirms the qualitative behaviors shown in Figure 6.In particular, the increase in the number of cars equipped with CRAWLING decreases both the time spent on obstruction and the time-to-parking.This is due to the fact, with more cars being controlled via CRAWLING, these were able to coordinate to avoid the unfavorable road link, while being directed towards a parking with space with effective capacity to accommodate the incoming cars.Scenario 2 specifics.For the second scenario in Section 2.3 we considered two controlled cars entering in the campus from the North entrance, both with the goal of reaching the western parking lot.However, after the first car exits the highway, the road is being blocked.This information, which causes the reward associated to the corresponding road link to sharply decrease, is twitted.Specifically, from one of the authors' Twitter profile, the following tweet is made: North highway ramp blocked #sumo_experiment after the first car's passage.Periodically, our implementation of CRAWLING checks the Twitter account's post history.The last tweet is parsed and, if an accident is detected -see Section 2.2 for a description of the parsing mechanism -CRAWLING assigns a very negative (i.e., −100) reward to the road link where the disrupting event is signalled.
Results for Scenario 2. The results are summarized in Figure 7.Such a figure illustrates, in panel (a), the key campus areas for the scenario.At the beginning of the scenario, no road obstruction is detected and indeed, as shown in panel (b), the first car entering in the simulation normally transitions through the Northern exit, following the shortest path through the ramp.After the car passes the ramp, the road obstruction occurs (the obstructed link is highlighted in blue in panel (c)) and the obstruction is reported via a tweet (in this experiment, from one of the authors, see panel (d)).Panel (d) showcases the corresponding tweet that was issued upon detection of the road blocking.Then, panel (e) shows the result of CRAWLING adapting to the new information.Indeed, the route for the second car is re-computed by our service to avoid that the second car enters in the obstructed road link.This is a result of CRAWLING detecting the reward change due to the tweet.An interesting question is to determine a measure of trust for the tweets to be used by CRAWLING.We leave this topic for our future research.At the end of the simulation -panel (f) -both cars achieve their goal by reaching their desired parking lot.A video of the corresponding simulation is also available at https://tinyurl.com/wcwvcwaj.Inspired by these studies, possible future directions of our research might involve both embedding tutoring mechanisms in CRAWLING and exploring the links between the approach we presented and certain recent neuroscience theories.We will also embed strategies to filter social media information for their effective use in CRAWLING.Finally, we aim at investigating whether CRAWLING can represent an effective mechanism to fairly regulate possible competitive behaviors between vehicles equipped with our service and legacy, manually-driven, vehicles when these compete for the same, limited, resource.

Figure 2 .
Figure 2. CRAWLING validation set-up.CRAWLING is implemented in Python and interacts with SUMO.This provides the back-end simulation from which CRAWLING gathers the relevant data through TraCI.The directions obtained via Algorithm 1 are fed back to the simulation environment via the interface TraCI.For example, in the validation scenarios described in Section 2.3, CRAWLING receives data such as parking availability or obstructed road links, which are used to compute directions to connected cars equipped with our service.CRAWLING also interfaces via a Social Interface module with information on social media (in our second scenario, Twitter).

Figure 3 .
Figure 3. Campus road network.Highways and parking lots ate represented by their respective international symbols.

Figure 4 .
Figure 4. CRAWLING execution time as a function of available sources and time horizon.Leftward panel: computation time (in seconds) as a function of both time horizon and number of services.Middle panel: computation time (in seconds) as a function of the time horizon with 6 available sources.Rightward panel: computation time (in seconds) as a function of the amount of sources for a time horizon of 5 steps ahead.

Figure 7 .
Figure 7. Snapshots taken from the simulation for Scenario 2. Panel (a): areas of the campus interested by the use-case.Specifically, the red rectangles denote the position of the snapshots on the map.Panel (b): first car taking the fastest route from the Northern entrance when there is no obstruction.Panel (c): the route is blocked after the first car has passed the Northern entrance.Panel (d): the information about the disrupting event is shared via a tweet.Panel (e): CRAWLING adapts to handle the disruption and the second car takes an alternate route.Panel (f): both cars reach the desired parking lot.

Table 1 .
Summary table for the numerical experiments.All times in seconds.