2015 Intl. Conference on Computing and Network Communications (CoCoNet'15), Dec. 16-19, 2015, Trivandrum, India

Modeling and Predicting Fault Tolerance in Vehicular Cloud Computing Puya Ghazizadeh

Ravi Mukkamala

Reza Fathi

Department of Computer Science Millersville University Millersville, PA 17551 Email: [email protected]

Department of Computer Science Old Dominion University Norfolk, VA 23529 Email: [email protected]

Department of Computer Science University of Houston Houston, TX 77204 Email: [email protected]

[6]. It is very likely that, given the right incentives, the owner of a vehicle will decide to rent out their onboard capabilities on-demand or on a per-hour or perday basis, just as owners of large computing or storage facilities find it economically appealing to rent out their excess capacity. For example, we anticipate that in the near future air travelers will park and plug their cars in airport long-term parking lot. In return for free parking and other perks, they will allow their vehicle to participate, during their absence, in the airport datacenter. While these VCs may be used much like traditional clouds, their true novelty and technical challenge lies in the fact that the vehicles come and go, making the assignment of computing resources to jobs challenging.

Abstract—Statistics show that most vehicles spend many hours per day in a parking garage, parking lot, or driveway. At the moment, the computing resources of these vehicles are untapped. Inspired by the success of conventional cloud services, a group of researchers have recently introduced the concept of a Vehicular Cloud. The defining difference between vehicular and conventional clouds lie in the distributed ownership and, consequently, the unpredictable availability of computational resources. As cars enter and leave the parking lot, new computational resources become available while others depart creating a dynamic environment where the task of efficiently assigning jobs to cars becomes very challenging. In this paper we propose a fault-tolerant job assignment strategy, based on redundancy, that mitigates the effect of resource volatility of resource availability in vehicular clouds. We offer a theoretical analysis of the mean time to failure of this strategy. A comprehensive set of simulations have confirmed the accuracy of our theoretical prediction.

I.

A fundamental way in which VCs differ from conventional clouds is in the ownership of resources. In VCs, the ownership of the computational resources is distributed over a large population as opposed to a single owner as in the conventional clouds run by Amazon, Google, IBM, etc. A corollary of this is that the resources of the VC are highly dynamic as cars may leave unexpectedly. Thus, any job assignment will have to feature some form of fault tolerance.

I NTRODUCTION AND MOTIVATION

Recently, we have witnessed the emergence of Cloud Computing, a paradigm shift adopted by information technology (IT) companies with a large installed infrastructure base that often goes under-utilized [10], [11], [21]. The unmistakable appeal of cloud computing is that it provides scalable access to computing resources and to a multitude of IT services. Cloud computing and cloud IT services have seen and continue to see a phenomenal adoption rate around the world [4], [5], [14], [22], [23].

Fault tolerance was a perennial concern in distributed systems and many solutions have been proposed over the years. Fault tolerance remains a crucial issue in cloud computing and a substantial number of papers have proposed solutions for static environments [3], [12]. One of the recognized metrics of fault tolerance is the mean time to failure (MTTF) defined as the expectation of the random variable that keeps track of the occurrence of a system failure. In this paper, we investigate the MTTF for a natural job assignment strategy in VCs. Our main contribution is an analytical derivation of the MTTF for the assignment strategy. Extensive simulation results have confirmed the accuracy of our theoretical prediction.

Inspired by the success and promise of conventional cloud computing, [1], [17]–[19] have introduced the concept of a Vehicular Cloud, (VC, for short), a nontrivial extension of the cloud computing paradigm. Their key insight was that present-day cars are endowed with powerful on-board computers and that most of these vehicles spend many hours per day parked in a parking garage, parking lot, or driveway. At the moment, the computing and storage resources of these vehicles are untapped and the opportunity for their use is wasted.

The remainder of this paper is organized as follows. In Section II we briefly survey relevant work on vehicular clouds; in Section III we state the main contributions

Based on statistics 90% of Americans drive to work

978-1-4673-7309-8/15/$31.00 ©2015 IEEE

395

2015 Intl. Conference on Computing and Network Communications (CoCoNet'15), Dec. 16-19, 2015, Trivandrum, India

rollback recovery can be used in the case where the car currently running the job leaves. While simple and intuitively satisfying, this first solution is very difficult to implement efficiently. Because making a checkpoint of an entire VMs state at runtime, creates a heavy overhead [16] and most of these checkpoints are, in fact, unnecessary reducing dramatically the efficiency of the vehicular cloud.

of this work. Section IV presents the vehicular and the VC models assumed in this work. Next, Section V offers the details of our fault-tolerant job assignment. Section VI states our basic assumptions about the probability distribution of car residency times and the distribution of duration of the recruiting operation. Section VII discusses the MTTF of the job assignment strategy. Section VIII introduces our simulation model and presents an empirical validation of our theoretical predictions. Finally, Section IX offers concluding remarks. II.

In this work we explore an alternative distributed approach. Specifically, we propose a fault tolerant job assignment strategy that does not require a central server to store the state of the computation.

R ELEVANT PREVIOUS WORK ON VEHICULAR CLOUDS

IV.

In order to be able to schedule resources and to assign computational tasks to the various cars in the VC, a fundamental prerequisite is to have an accurate picture of the number of vehicles that are expected to be present in the parking lot as a function of time. What makes the problem difficult is the time-varying nature of the arrival and departure rates. Quite recently, Arif et al. [1] have proposed a stochastic prediction model for the parking occupancy given time-varying arrival and departure rates. They provided closed forms for the probability distribution of the parking lot occupancy as a function of time, for the expected number of cars in the parking lot and its variance, and for the limiting behavior of these parameters as time increases. In addition to analytical results, they have obtained a series of empirical results that confirm the accuracy of our analytical predictions. III.

S YSTEM ASSUMPTIONS

The goal of this section is to review the basic assumptions we make about the capabilities of cars and the type of services the VC provides. A. The vehicular model A typical car or truck today is likely to contain at least some of the following devices: an on-board computer, a GPS device coupled with a digital map, a radio transceiver, a short-range rear collision radar device, and a camera. These are supplemented, in highend models, by sophisticated sensing devices that can alert the driver to all manner of mechanical malfunctions and road conditions [20]. We assume that each car has a virtualizable on-board computer similar, but not necessarily identical, to the Intel Itanium [15], AMD Athlon, or Opteron [13] lines of processors. Because of their sophisticated compute capabilities and ample storage, our cars are perfect candidates for servers in a Warehouse-Scale Computer [2], [9].

O UR CONTRIBUTIONS

As mentioned before, the distributed ownership of the computational resources makes it very challenging to assign, with any degree of confidence, jobs to resident vehicles.

B. Our datacenter and VC model

Ghazizadeh et al. [7], [8] have studied the problem of task scheduling in VCs and presented a near-optimal solution based on mixed integer linear programming. This model is designed for deterministic environment in terms of vehicles arrival and departure and does not consider the random arrival and departure of vehicles.

In this paper we deal with a datacenter supported by a VC built on top of cars parked in a sufficiently large parking lot, similar to a typical parking lot at a major airport that contain thousands of cars [1]. We assume that, due to high demand, the parking lot is almost always nearly full and, in particular, it is always possible to find at least two cars that can be assigned to an incoming user job.

To get a feel for the problem, assume that we just assigned a job to a car currently in the parking lot. If the car stays in the parking lot until the job terminates, all is well.

We assume that the datacenter has implemented a mechanism that identifies available cars in the parking lot. Such a mechanism can be implemented by assigning to each car a status bit. When a car is about to leave the parking lot, the datacenter alerts all the other cars assigned to the same user job of the imminent departure. Finally, the datacenter can select, using some form of locality criteria, one or several available cars that can be assigned to a job.

The difficulty arises when the car departs before job completion. In such a case, unless we take special precautions, the entire work done is lost and we have the restart the entire process again, taking chances on another car, and so on until eventually the job is completed. One possible solution is to set up a checkpointing strategy: this is a policy that prescribes making periodic copies of the state of the computation in such a way that

The cars in the parking lot are connected by the network fabric provided by the datacenter. This could

396

2015 Intl. Conference on Computing and Network Communications (CoCoNet'15), Dec. 16-19, 2015, Trivandrum, India

A

User application

job terminates X1

B

X2

Guest OS

X3

Guest VM

D

C E

Fig. 2.

VMM

Illustrating the job assignment strategy.

Car hardware

VI. Fig. 1.

RESIDENCY TIMES AND RECRUITING

Illustrating the virtualization model employed in the paper.

take the form of various flavors of wired or wireless connections, as discussed in [1].

Throughout this paper, we assume that car residency times are independent, exponentially distributed random variables with a common parameter λ . We assume the durations of recruiting operations to be independent, identically distributed (iid) random variables U with distribution function G. We further assume that the duration of recruiting is independent of the residency time of cars.

For simplicity, we assume that the VC offers only Infrastructure as a Service (IaaS) cloud services. In IaaS, the users request a hardware platform and specify their preferred operating system support. As illustrated in Figure 1, the VC offers the user a virtualized instance of the desired hardware platform and operating system bundled as a Virtual Machine (VM) and guest OS hosted by one of the cars in the parking lot. For faulttolerance purposes, each user job is assigned to multiple cars. When the VM running the user job in a specific car terminates execution, the result is uploaded to the datacenter.

V.

A SSUMPTIONS ABOUT THE DISTRIBUTION OF

We will use the following classic result throughout this paper.

Lemma 6.1: Let X1 , X2 , · · · , Xn , (n ≥ 2), be independent, exponentially distributed random variables with parameters λ1 , λ2 , · · · , λn , respectively. The random variable Z = min{X1 , X2 , · · · , Xn } is exponentially distributed with parameter λ1 + λ2 + · · · + λn .

T HE JOB ASSIGNMENT STRATEGY

Each job is assigned to two cars selected, at random, from the available cars in the parking lot. When one of the cars departs, the remaining car begins a recruiting operation during which the following tasks are performed: First, the car saves the state of its guest VM. Second, with this completed, the resulting image is copied to an available car and the job is restarted on both cars using the saved VM image. The job terminates when the first such instance of job execution terminates.

Often times we will reason about the residual residency time of a car, intended to capture the remaining time the car will spend in the parking lot. It is well known that for exponentially distributed random variables, the residual life is itself exponentially distributed with the same parameter λ . However, we need a stronger version of this classic result. Specifically, we need to show that the residual residency time of a car, beyond a recruiting operation, is still exponentially distributed. For this purpose, we prove the following technical result.

Refer to Figure 2 for an illustration of the job assignment strategy. The job is started, initially, on cars A and B. When car A leaves, the state of car B’s guest VM is saved, a new car C is recruited, the saved image is copied to C and the job is restarted on cars B and C. At some point, car C leaves. Again, the state of the VM running on car B is saved, a new car D is recruited and the job is restarted on cars B and D. When car B leaves, the state of the VM running on D is saved, a new car E is recruited and the job is restarted on cars D and E. Finally, when one of D or E terminates its instance of job execution it sends its results to the datacenter controller, who will inform the remaining car that the job has been completed.

Lemma 6.2: Let X be an exponentially distributed random variable with parameter λ and let U be a nonnegative random variable with distribution function G. If U is independent of X then, for all t ≥ 0, Pr[X > U + t | X > U] = Pr[X > t].

397

2015 Intl. Conference on Computing and Network Communications (CoCoNet'15), Dec. 16-19, 2015, Trivandrum, India

Proof: We write Pr[X > U + t | X > U] = = = = = =

one to leave. Thus, the process makes a transition from state C1C2 to either of states C1 and C2 with probability 1 2.

Pr[{X > U + t} ∩ {X > U}] Pr[X > U] Pr[X > U + t] Next, assume that the process is in state C1 . The process returns to state C1C2 if the recruiting operation Pr[X > U] R∞ terminates before C1 leaves. Let the probability of this 0R Pr[X > U + t|U = u] dG(u) event be p. In this notation, the transition from state C1 ∞ Pr[X > U|U = u] dG(u) to state F occurs with probability 1 − p. R ∞0 0R Pr[X > u + t] dG(u) ∞ 0 Pr[X > u] dG(u) Z R ∞ −λ (u+t) dG(u) 0 e U R∞ −λ u dG(u) 0 e R e−λt 0∞ e−λ u dG(u) R∞ −λ u dG(u) 0 e recruiting begins recruiting ends

= e−λt = Pr[X > t].

Fig. 4.

This completes the proof of the lemma. VII.

Illustrating the proof of Lemma 7.1.

Lemma 7.1: The probability of the transition from state C1 (resp. C2 ) to state C1C2 is

T HE MTTF OF THE JOB ASSIGNMENT

Z ∞

p=

STRATEGY

Imagine a long-lived job running on two cars, C1 and C2 . We adopt the convention that a newly recruited car takes the identity of the departed car. For example, imagine that car C2 just left and that car C1 is running a recruiting operation. If the recruiting completes successfully, the newly recruited car will be referred to as C2 . Likewise for C1 .

Z ∞

Pr[Z > U] = =

Pr[Z > u] dG(u) Z0 ∞

This completes the proof of Lemma 7.1. p

1 2

Let W12 , W1 , W2 denote the sojourn times in the states C1C2 , C1 and C2 , respectively. As before, let U be the duration of a recruiting operation and let Z be the residual residency time of, say, C1 . Observe that both W1 and W2 are distributed as min{U, Z}. In particular,

C2 1−p

e−λ u dG(u). [by Lemma 6.2]

0

C1 C2

C1

Pr[Z > U | U = u] dG(u)

Z0 ∞

=

p

(1)

Proof: Consider what happens when a recruiting operation is started and refer to Figure 4. Let U be the random variable that denotes the duration of a recruiting operation and let Z be the residual residency time of the car running the recruiting operation. For the recruiting to be successful, the event {Z > U} must occur. Conditioning on the duration of the recruiting operation, we write

Referring to Figure 3, we model the problem as a semi-Markov process with states C1C2 , C1 , C2 and F such that the process is in state C1C2 when both cars are in the parking lot. C1 when car C2 has left, but car C1 is still in the parking lot. C2 when car C1 has left, but car C2 is still in the parking lot. F when C1 and C2 have left without completing the job. We assume that

1 2

e−λ u dG(u).

0

1−p F

Pr[W1 > t] = = = =

Fig. 3. Illustrating the semi-Markov process for the job assignment strategy.

the semi-Markov process is initially in state C1C2 . Of interest are the transition probabilities between states. Since the car residency times are iid, each of the two cars C1 and C2 have an equal likelihood of being the first

Pr[min{U, Z} > t] Pr[{U > t} ∩ {Z > t}] Pr[U > t] Pr[Z > t] [by independence] e−λt [1 − G(t)]. (2)

Therefore, the expectation E[W1 ] of sojourn time W1 can be evaluated as follows.

398

2015 Intl. Conference on Computing and Network Communications (CoCoNet'15), Dec. 16-19, 2015, Trivandrum, India

Z ∞

E[W1 ] =

between 21 and 18 . Since the expected residency time is given by λ1 , our choice corresponds to cars spending, on the average, between two and eight hours in the parking lot. Also, we assumed that the number of available cars sufficiently large, so that upon one car departing, a substitute could be found without delay.

e−λt [1 − G(t)] dt

0

 Z ∞ 1 1− e−λ u dG(u) λ 0 1− p . [by Lemma 7.1] λ

= =

(3)

Our model was developed in MATLAB 7.14. In each of the experiments, a job with one of the parameters described above was submitted to the Vehicular Cloud.

An identical argument, with C2 instead of C1 , shows that 1− p E[W2 ] = . λ

B. Simulation results This section presents our simulation results performed in order to validate the analytical results obtained in Sections VII. In order to evaluate the impact of the various problem parameters discussed in Subsection VIII-A we have conducted a number experiments corresponding to the scenarios that we detail next.

Let T12 , T1 and T2 denote, respectively, the time it takes the semi-Markov process to reach state F when the process in the states C1C2 , C1 and C2 , respectively. In this notation, the MTTF of the job assignment strategy is simply T12 . By Lemma 6.1, specialized to n = 2 and to λ1 = λ2 = λ , the expected sojourn time E[W12 ] in state C1C2 1 is 2λ . Next, recall that from C1C2 the process is equally likely to transit to either C1 or C2 . Further symmetry considerations tell us that E[T1 ] = E[T2 ]. Thus, using the Law of Total Expectation we write     1 1 1 1 E[T12 ] = + E[T1 ] + + E[T2 ] 2λ 2 2λ 2 1 = + E[T1 ]. [since E[T1 ] = E[T2 ]] (4) 2λ

Each job was assigned to two cars. Recruiting durations have a uniform distribution with mean duration of 10, 15 and 20 minutes. Car residency times were taken to be exponentially distributed with parameter λ . In our experiments we varied the expected car residency time varies between three and eight hours. The simulation results are featured in Figure 5(a), 5(b), and 5(c); A glance at Figures 5(a), 5(b), and 5(c) confirms that the simulation results confirm the accuracy of our theoretical predictions.

The Law of Total Expectation, in conjunction with (3), allows us to write E[T1 ] = E[W1 ](1 − p) + [E[W1 ] + E[T12 ] p   1− p (1 − p)2 + + E[T12 ] p = λ λ 1− p = + p E[T12 ]. λ

IX.

Vehicular clouds are motivated by the abundant computational resources in present-day vehicles and the fact that most of these vehicles are parked every day, for hours on end, while their owner is working, shopping, travelling, etc. Given the huge number of vehicles on our roads and city streets, vehicular clouds are expected to have a huge societal impact.

(5)

Now, (4) and (5) yield the following system of equations  1  −E[T1 ] + E[T12 ] = 2λ 

E[T1 ] − p E[T12 ] =

In this paper we have proposed a job assignment strategy in vehicular clouds and have analyzed its performance both analytically and by simulation.

1−p λ

R EFERENCES

that can be solved for E[T12 ] to yield 1 1 MT T F = E[T12 ] = + . λ 2λ (1 − p) VIII.

C ONCLUDING REMARKS

[1] S. Arif, S. Olariu, J. Wang, G. Yan, W. Yang, and I. Khalil. Datacenter at the airport: Reasoning about time-dependent parking lot occupancy. IEEE Transactions on Parallel and Distributed Systems, 23(11):2067–2080, 2012. [2] L. A. Barroso and U. H¨olzle. The datacenter as a computer: An introduction to the design of warehouse-scale machines. Morgan & Claypool, San Rafael, California, 2009. [3] J. Deng, S.-H. Huang, Y. Han, and J. Deng. Fault tolerant and reliable computation in cloud computing. In GLOBECOM Workshops (GC Wkshps), 2010 IEEE, pages 1601–1605, Dec 2010. [4] M. Eltoweissy, S. Olariu, and M. Younis. Towards autonomous vehicular clouds. In Proceedings of AdHocNets’2010, Victoria, BC, Canada, August 2010.

(6)

S IMULATION R ESULTS

A. Simulation model In this subsection, we present the detail of our simulation model. We assume a large parking lot, similar to a typical parking lot in the downtown area of a large city. Car residency time are assumed to be exponentially distributed with parameter λ . In our simulation λ varied

399

2015 Intl. Conference on Computing and Network Communications (CoCoNet'15), Dec. 16-19, 2015, Trivandrum, India

[5] I. Foster, Z. Y., I. Raicu, and S. L. Cloud computing and grid computing 360-degree compared. In Grid Computing Environments Workshop, 2008. GCE ’08, pages 1–10, Nov 2008. [6] Fred Dews. Ninety percent of americans drive to work. http://www.http://www.brookings.edu/, 2013. [7] P. Ghazizadeh. Resource allocation in vehicular cloud computing. PhD Thesis, Old Dominion University, July 2014. [8] P. Ghazizadeh, R. Mukkamala, and S. El-Tawab. Scheduling in vehicular cloud using mixed integer linear programming. In Proceedings of the First International Workshop on Mobile Sensing, Computing and Communication, MSCC ’14, pages 7– 12, New York, NY, USA, 2014. ACM. [9] J. L. Hennessy and D. A. Patterson. Computer Architecture a Quantitative Approach. Morgan Kaufman, Elsevier, 2012. [10] R. Jhawar, V. Piuri, and M. Santambrogio. Fault tolerance management in cloud computing: A system-level perspective. IEEE Systems Journal, 7(2):288–297, June 2013. [11] W. Kim. Cloud computing: today and tomorrow. Journal of Object Technology, 8(1):65–72, January-February 2009. [12] S. Malik and F. Huet. Adaptive fault tolerance in real time cloud computing. In Services (SERVICES), 2011 IEEE World Congress on, pages 280–287, July 2011. [13] D. C. Marinescu. Cloud Computing, Theory and Applications. Morgan Kaufman, Elsevier, 2013. [14] National Institute of Standards and Technology (NIST). DRAFT cloud computing synopsis and recommendationsdefinition of cloud computing. http://csrc.nist.gov/publications/drafts/ 800-146/Draft-NIST-SP800-146.pdf, May 2011. [15] G. Neiger, A. Santoni, F. Leung, D. Rodgers, and R. Uhlig. Intel virtualization technology: Hardware support for efficient processor virtualization. Intel Technology Journal, 10(3):167– 178, 2006. [16] B. Nicolae and F. Cappello. Blobcr: Efficient checkpoint-restart for hpc applications on iaas clouds using virtual disk image snapshots. In High Performance Computing, Networking, Storage and Analysis (SC), 2011 International Conference for, pages 1–12, Nov 2011. [17] S. Olariu, M. Eltoweissy, and M. Younis. Towards autonomous vehicular clouds. ICST Transactions on Mobile Communications and Computing, 11(7-9):1–11, July-September 2011. [18] S. Olariu, T. Hristov, and G. Yan. The next paradigm shift: From vehicular networks to vehicular clouds. In Stojmenovic I. and Basagni, S., (Eds), Mobile Computing, Wiley and Sons, New York, 2014. [19] S. Olariu, I. Khalil, and M. Abuelela. Taking vanet to the clouds. International Journal of Pervasive Computing and Communication, 7(1):7–21, 2011. [20] Texas Transportation Institute. 2012 urban mobility report. http: //mobility.tamu.edu/ums/, December 2013. [21] A. Younge, G. von Laszewski, L. Wang, S. Lopez-Alarcon, and W. Carithers. Efficient resource management for cloud computing environments. In Green Computing Conference, 2010 International, pages 357–364, Aug 2010. [22] Y. Zhang, Z. Zheng, and M. R. Lyu. Bftcloud: A byzantine fault tolerance framework for voluntary-resource cloud computing. In Cloud Computing (CLOUD), 2011 IEEE International Conference on, pages 444–451, July 2011. [23] W. Zhao, P. Melliar-Smith, and L. Moser. Fault tolerance middleware for cloud computing. In Cloud Computing (CLOUD), 2010 IEEE 3rd International Conference on, pages 67–74, July 2010.

Checkpoint Expected Duration = 10 minutes 250 Simulation Result Analytical Result

Mean Time To Failure (hours)

200

150

100

50

0

2

3

4 5 6 Average Residency Time of Car (hours)

7

8

7

8

7

8

(a) Illustrating Scenario 1. Checkpoint Expected Duration = 15 minutes 140 Simulation Result Analytical Result

Mean Time To Failure (hours)

120

100

80

60

40

20

0

2

3

4 5 6 Average Residency Time of Car (hours)

(b) Illustrating Scenario 2. Checkpoint Expected Duration = 20 minutes 120 Simulation Result Analytical Result

Mean Time To Failure (hours)

100

80

60

40

20

0

2

3

4 5 6 Average Residency Time of Car (hours)

(c) Illustrating Scenario 3. Fig. 5. Predicted and simulated MTTF of the our job assignment strategy.

400