CloudMap: Workload-aware Placement in Private Heterogeneous Clouds Balaji Viswanathan

Akshat Verma

Sourav Dutta

IBM Research-India Abstract—Cloud computing has emerged as an exciting hosting paradigm to drive up server utilization and reduce data center operational costs. Even though clouds present a single unified homogeneous resource pool view to end users, the underlying server landscape may differ in terms of functionality and reconfiguration capabilities (e.g., support for shared processors, live migration). In a private cloud setting where information on the resources as well as workloads are available, the placement of applications on clouds can leverage it to achieve better consolidation with performance guarantees. In this work, we present the design and implementation of CloudMap, a provisioning system for private clouds. Given an application’s resource usage patterns, we match it with a server cluster with the appropriate level of reconfiguration capability. In this cluster, we place the application on a server that has existing workloads with complementary resource usage profile. CloudMap is implemented using a hybrid architecture with a global server cluster selection module and local cluster-specific server selection modules. Using production traces from live data centers, we demonstrate the effectiveness of CloudMap over existing placement methodologies.

I. I NTRODUCTION Cloud computing has emerged as one of the most disruptive technologies significantly improving data center efficiency and reducing operational costs. Clouds bring a service flavor to infrastructure, platform, and software and enable on demand resource fulfillment to an enterprise customer. The two key differentiators of an IaaS cloud is the ability to provision new virtual machines or applications in the matter of minutes and significantly increase the utilization of data center resources. Public IaaS clouds (e.g., the Amazon Elastic Compute Cloud EC2 [5]) were the first clouds to emerge on the technology landscape. EC2 hides the complexity of the data center landscape from the end user and allocates resources to the virtual machine without sharing any details with the user. We term such a cloud model as a Closed Cloud model, where the end user does not have any visibility inside the cloud. A Closed Cloud model is not very useful for applications that have specific network, storage or clustering requirements. The OpenCirrus [1] cloud platform addresses this issue and allows end users to make more informed choices on the physical servers and data center networking employed for the hosted applications. However, the cloud provider is not aware of the workloads running on the leased server instances and their application performance. We term such a cloud model as a OneWay-Open Cloud model. A third model for IaaS cloud c 2012 IEEE 978-1-4673-0269-2/12/$31.00

has emerged quickly in large shared data centers that host multiple large customers or even in data centers of large enterprises. A cloud running inside a data center catering to private customers is often called a private cloud. A private IaaS cloud also inherits the salient advantages of IaaS clouds, namely on demand provisioning and high resource utilization. However, in a private cloud, the end customer has complete visibility on the deployment of its workloads and the cloud provider has visibility on the performance of the applications in the cloud. We term such a cloud model with 2-way visibility as an Open Cloud model. The placement of applications in a typical public cloud environment is fairly straightforward due to the closed nature of the cloud. Hence, the cloud provider has no information on the nature of application and its workload profile. On the other hand, an Open Cloud model provides rich information on the application characteristics as well as the data center resources. This information can be leveraged to drive much higher resource efficiency and guarantee application performance in the shared cloud environment. A. Research Challenges Pool 1 VM1

Pool 2

VM2

Server 1

Pool 3

VM3

VM4

Pool 4

Server 2

Free Space

Fig. 1.

Pool 5

Static

Dynamic

Resource Pools in a Private Cloud

Placement of applications on a shared private cloud following an Open Cloud model poses significant research challenges. Even though clouds expose a homogeneous view of the underlying resources, typical data centers consist of hardware procured over the years with varying capabilities and performance profiles. Fig. 1 describes a typical data center layout. A data center would consist of multiple resource pools, where all the servers in a resource pool are in the same subnet and have similar properties. Servers in some resource pool may allow dynamic resource reconfiguration. These pools are labeled as ‘Dynamic’ pools. Resource pools containing legacy servers that do not support dynamic reconfiguration are labeled as ‘Static’ pools. Each server in either pool may host one or more virtual machine images. The spare capacity on each server may be used to host new applications.

network and disk I/O used by the virtual machine. More details about the workload are available in an earlier work [20]. Our first goal was to understand the variability of a workload. Since server pools in a cloud have varying degrees of dynamic resource configuration capabilities, we wanted to investigate if cloud workloads also have varying degree of variability. Fig. 2(a), 2(b), 2(c) capture the Standard Deviation and Coefficient of Variation (CoV = Standard Deviation/Mean) for the CPU usage of virtual machines in 3 representative application suites. Standard Deviation captures the absolute variation in workload whereas CoV captures the relative variation in the workload from its average intensity. We observe that the CoV of the workloads vary from 0.06 to 3 or more and the standard deviation also varies from 2% to 35%. This leads to our first observation. Observation 1: There are both low covariance and high covariance applications. Our first observation implies that we should match workloads to server pools based on the variability of the workload and the dynamic reconfiguration capabilities of the server pool. We next investigate if the zoning restrictions between applications may prevent us from following such an approach. We observe that there is a strong correlation between variability of an application and the variability of its components. We observe that AppCluster3 has a large number of virtual machines with high variability (Fig. 2(c)). On the other hand, AppCluster2 has low aggregate variability and most of its virtual machines also have very small standard deviation (Fig. 2(b)). Similarly, most virtual machines in AppCluster1 have similar degree of variability. This leads to our next observation. Observation 2: Components of an application typically have similar workload variance. 0.7 VM1 VM2 VM3 VM4 VM5

0.6 0.5 0.4 Correlation

Placement in a private cloud poses some fundamental challenges. A cloud is elastic by nature and moves resources in response to workload variations. However, the elasticity in various resource pools in the cloud vary significantly. Similarly, the workloads on the cloud may have very different resource usage profiles. A fundamental problem that a placement strategy needs to solve is to match the workload requirements to the server pool capability. For example, a steady workload may be a good fit for a server pool that does not allow live migration or resizing. On the other hand, a highly variable workload needs to be either sized for peak workload intensity (leading to low resource utilization) or it should be placed on a server pool with dynamic reconfiguration capabilities. In a cloud placement scenario, one needs to identify workload complementarity between a new workload and existing workloads. Further, the online nature of the problem dictates taking into account the profile of workloads likely to arrive in future. Incorporating the impact of existing and future workloads into placement is a significant challenge. Finally, even though a cloud provides dynamic reconfiguration capabilities, there are costs associated with reconfiguration. Hence, a placement scheme needs to minimize the amount of dynamic reconfiguration that may be required on the cloud during steady operation. B. Contributions In this work, we investigate the placement of applications running in a captive data center environment to a shared private cloud infrastructure. We identify Online placement and Heterogeneous Reconfiguration Capability as the key differences between data center consolidation and placement in private clouds. Using an analysis of production workloads, we characterize the importance of incorporating the resource profile of existing workloads, while placing new applications in a private cloud. Drawing from our insights, we design the CloudMap cloud provisioning framework and algorithm. CloudMap is based on a hybrid architecture with a global and a local placement manager. The global placement manager identifies an appropriate pool based on workload variability and reconfiguration capabilities of the server pool. The local placement manager identifies the target server in the pool based on the resource profile of the new workload and existing workloads placed in the pool. We present a detailed experimental evaluation using production traces, which highlights the importance of various design choices made in CloudMap and establish its effectiveness.

0.3 0.2 0.1 0 -0.1 -0.2 0

Fig. 3.

10

20

30 40 VirtualMachines

50

60

70

Intra-cluster and Inter-cluster Correlation (VMs 1 to 5)

0.7 VM6 VM7 VM8 VM9 VM10

0.6 0.5

We conducted a trace study to understand the implications of the new aspects of cloud placement. For this trace study, we simulated a cloud placement scenario with application suites running in the production data center of a Fortune 500 enterprise. An application suite consisted of a set of virtual machines, each hosting one component of the large enterprise application. The traces contained the resource utilization for each virtual machine over a period of 2 months. Each entry in the trace file captured 5-minute averages for the CPU, memory,

Correlation

0.4

II. U NDERSTANDING C LOUD P LACEMENT

0.3 0.2 0.1 0 -0.1 -0.2 0

Fig. 4.

10

20

30 40 VirtualMachines

50

60

70

Intra-cluster and Inter-cluster Correlation (VMs 6 to 10)

Our second observation implies that matching applications to pools based on aggregate application variability may work well in practice. Our next goal was to understand the impact of the

1

15

0.5

3

30

1.5 1

15

2

0 0

1

2

3

4

5 6 7 VirtualMachine

8

9

10

11

30

1.5 1

0.5

0

45 CoV Std. Deviation

2.5

15

0.5

0

0 0

(a) AppCluster 1 Fig. 2.

2

4

6 8 VirtualMachine

10

12

0

14

0 0

2

4

(b) AppCluster 2 CoV and Standard Deviation

online aspect of cloud placement. We compute the correlation of one component of an application against other components of the same application as well as other applications. The comparison results are shown in Fig. 3 and Fig. 4. The first 10 points in the figure show the correlation of a VM from AppCluster1 with other VMs in AppCluster1. The remaining points show the correlation between the VM and VMs in other clusters. We observe that the correlation between VMs in same cluster is higher than correlation between VMs in different clusters. However, even though a VM is more likely to be correlated with a VM in the same application suite, there are many instances of high correlation with a VM of a different application suite (e.g., VM2 with VM 40 in Fig. 3). This leads to the following observations. Observation 3: Components of an application are positively correlated with each other. Observation 4: There may be correlation between VMs across different application suites. The above observations have very important implications for a placement strategy. A placement strategy can not be oblivious of the placement of existing applications in the cloud. If a new VM is placed with existing complementary workloads, it may lead to lower resource consumption. Hence, a placement strategy needs to be aware of the existing VM placement on the cloud. A second important implication of our observations is that VMs from one application suite are usually positively correlated and hence should be distributed across a candidate server pool. Hence, we do not really need to find complementary patterns between the components of an application. We use this observation to design an efficient placement strategy that can place each component of the application independently. III. C LOUD M AP A RCHITECTURE AND F LOW A. Architecture We present the CloudMap placement architecture in Fig. 5. A private cloud consists of heterogeneous server clusters with varying capabilities and the placement problem needs to map a workload to (i) an appropriate server cluster in the cloud and (ii) a target server in the selected cluster. In CloudMap, we have opted for a hybrid design, where the server pool selection logic is implemented by a Global Placement Manager and the target server selection logic is implemented by Local Placement Managers for each cluster. Our hybrid placement framework allows CloudMap to be applicable in large federated clouds, where precise information of the servers in each cluster may be difficult to obtain. The other significant design

8 10 12 VirtualMachine

14

16

18

20

(c) AppCluster 3 Monitored Data (Exisiting Applications)

Placement Request

Pool Selection

Global Placement Manager

Automation Manager

6

VM Image

Image Library

Cloud Cluster 1 Local Placement Manager Server Selection

Server

Automation Agent

Server

VM Creation

Image Manager

Monitoring Engine

Virtualization Manager

Server Server

Monitored Data (New Application)

Cloud Cluster 2 Local Placement Manager

Monitoring Engine

Monitoring Engine Server Server

Server

Server

Server

Server

Server

Server

Server

Server

Server

Server

Server

Server

Server

Server

Automation Agent

Server Server

Virtualization Manager

Server

Server

Captive Data Center

Fig. 5.

CloudMap Placement Architecture

decision taken in CloudMap is to support the Open Cloud model, where the placement logic has information on available resources as well as workload performance. The Local Placement Managers use the correlation between existing workloads on various servers and the resource profile of the workload being placed to identify an optimal target server. A third design choice in CloudMap is to make all placement decisions based on a dominant resource. A dominant resource is defined as the resource with the highest utilization in the cloud [7]. Use of a single dominant resource greatly simplifies the placement problem and allows us to design an efficient placement algorithm. The overall flow of CloudMap consists of the following steps. When a placement request is submitted, the Global Placement Manager uses the monitored data for the workload and the capabilities of various server pools to select a target server pool. It then instructs an Automation Manager to create an image backup of the workload in the Image Library using the Image Manager. Finally, it instructs the Local Placement Manager to place the workload in its managed cluster. The Local Placement Manager identifies an appropriate target server in the pool and instructs the Automation Agent to place the workload along with its resource requirements. The Automation Agent uses the Virtualization Manager to create one or more virtual machines on the target server and assign resources to meet its performance goals. Once the virtual machine is created, the Automation Agent restores the backed up image from the Image Library on the target virtual machine. Once the virtual machine is restored, the user is notified that the virtual machine is available for use.

Standard Deviation (%)

45 CoV Std. Deviation

2 CoV

30

1.5

Standard Deviation (%)

2 CoV

3 2.5

CoV

45 CoV Std. Deviation

Standard Deviation (%)

3 2.5

B. Global Placement Manager For each workload, we monitor the load generated by it and compute the CoV . Workloads associated with a high degree of CoV are identified as unstable and deemed to require dynamic consolidation. As observed earlier, an application and its components have similar behavior with respect to their variance. Hence, we use the CoV for the aggregate application workload to identify the target pool for all components of the application. We consider a heterogeneous cloud environment with different server types. Based on the dynamic re-consolidation capabilities of the existing servers in a data center, the servers are clustered into distinct pools, namely ‘Static’ or ‘Dedicated’, ‘Dynamic’ or ‘Shared’ and ‘Live Migration’. The parameter p is used to define CoV thresholds, which are used to map workloads to pools. To elaborate further for a cloud with p = 3, we define thresholds CoVH and CoVL . All applications with CoV ≥ CoVH are placed in a ’Live Migration’ pool. Applications with CoVL < CoV < CoVH are placed in a ’Shared’ pool whereas applications with CoV ≤ CoVL are placed in ’Dedicated’ server pool. C. Local Placement Manager: Estimating Workload Affinity 100 %

Total Capacity (TC) Unused Capacity UC(t) Pnew

UC(t=T) Increase in Peak (P)

Pold

Utilization

Wastage (W)

x-percentile usage New VM Peak (P2)

New Envelop E

new

Existing Envelop E

old

Time Interval

Fig. 6.

and Ejold (Pnew − Pold ). Also, assume that P 2 is the tallest peak of the new workload alone. Peak Ratio (P R) is defined as the ratio between the increase in the peak P and the peak of the new workload P 2 (e.g., Fig. 6). When P R is 1, the peaks completely overlap and thus the server will be a bad placement candidate for the new workload. A lower value of P R indicates a weaker correlation and a better candidate server for the new workload. Buffer Weight (BW) is computed as the overlap between all the peaks of Ejold and the peaks of Ejnew . The metric also gives different weightage to each of the peaks based on the likelihood that the peak will prevent future workloads from being placed on the server. Formally, we use a parameter U C(T ) to capture the amount of un-utilized capacity of the server for the peak level. A low U C(T ) implies that the server may become bottlenecked due to the peak and new workloads cannot be placed. For any peak Ti and server j, we estimate Pi as the increase in the peak due to the new workload. Buffer Weight BWj (Ti ) is computed as the ratio between Pi and the unused capacity U C(Ti ) for the peak, i.e., Pi /U C(Ti ). The sum of the buffer weight of all the peaks is the Buffer Weight for each candidate server for the workload. One of the goals of cloud is to increase the resource utilization. BW captures this aspect of cloud placement as it is inversely proportional to Unused Capacity. Hence, even if the there is an overlap of the peaks, a server with low utilization has a low BW value making the server a good candidate for placing the new workload. Hence, a placement based on BW improves resource utilization as well.

Local Placement parameters

The Local Placement Manager estimates the affinity of a new workload with existing workloads in the pool. We use the idea of creating a resource envelop, first proposed in [20], to easily identify complementary workloads. The envelop of a workload profile identifies a body value CB to capture the typical resource usage (e.g., 90 or 80 percentile) and a peak value CP to capture peak usage (e.g., max usage). The envelop is a 2-level time series, where all resource usage below CB is replaced by CB and all usage above CB is replaced by CP . Since a server in a cloud may host multiple workloads, we refine the 2-level envelop to a k-level envelop with multiple peaks (e.g, peaks corresponding to 100, 98, 96, 94 and 92 etc percentile usage). For placing a new workload, we create two envelops Ejold and Ejnew per target server j. Ejold is the envelop for server j with existing workloads and Ejnew is the envelop assuming the new workload is also placed on server j (dotted and solid lines in Fig. 6). We use the envelop to estimate the following affinity metrics. Peak Ratio (PR) captures the overlap between the tallest peak of the existing workloads on the server and the new workload. Consider an old envelop E old with a tallest peak of Pold and the placement of the new workload leads to a new peak of Pnew . Let P be the increase in the tallest peak between Ejnew

D. Local Placement Manager: Server Selection For each workload, we first attempt to find a complementary or completely non-overlapping server (Case 1). If no such server exists, we try to find a candidate server with correlation or PeakRatio P R below a certain threshold (Case 2). If we do not find any candidates below the threshold, we place the workload on a server with lowest Buffer Weight (Case 3). For the 2nd and 3rd cases, we use the expected future demand to decide if new servers should also be considered for placement. Case 1: We first check for servers in the pool with uncorrelated workload. If such a server exists, then the new workload is allocated to it. However, if there are no such complimentary servers for the new workload, we aim to optimize the tradeoff between placing the workload on a non-complimentary server (to improve resource utilization) and placing the workload on a new server (to improve the chances of finding compatible workloads later). We use the insight that if new servers would be required to cater to future workloads, we can place the workload on new servers (since server utilization would increase with future workloads). On the other hand, if we have a large unused capacity on existing servers, starting new servers would increase W astage. Hence,Pwe define a parameter Global Unused Capacity (GU C = ∀Sj U C(T ) (Fig. 6) to capture the total free resource available on currently running servers. If the predicted future resource requirement is greater than the current GUC, we start a new server

Input: Workloads Wi (i = {1, N }), Server Sj (j = {1, M }, Envelop Factor = k Output: Placement xi,j For each Server Sj j j Create Env(Sj , k), Cmax , CB End-For UnCorr = Φ, Candidate = 0, MinPR = P Rth , MinBW = ∞ Cluster workloads and create a proportional round robin order For each workload Wi For each server Sj NumOverlap = Number of peaks overlapping between Wi and Sj If (NumOverlap = 0) U nCorri = U ncorri ∪ Sj End-If End-For If (|U nCorri | > 0) /* Case 1 */ Sj0 = Sj ∪ Wi Candidate = j, such that Sj0 has the minimum wastage Else If (GU C < F utureRequest) Start new server S Sj = Sj ∪ S End-If For each server Sj /* Evaluate Case 2 */ Sj0 = Sj ∪ Wi j0 j P R = (Cmax − Cmax )/P eakWi If M inP R > P R Candidate = j End-If End-For If (Candidate = 0) /* Case 3 */ For each server Sj Sj0 = Sj ∪ Wi j0 j j0 BW = (Cmax − Cmax )/(Capacityj − Cmax ) If M inBW > BW Candidate = j End-If End-For End-If End-If xi,Candidate = 1 End-For Fig. 7.

Local Placement Manager Algorithm

corresponding to the pool selected and add it to the available server list of the pool. Case 2: In this case, we find a server that has some degree of complementarity. For all the candidate servers j, we compute the Peak Ratio (P Rj ) assuming that the new workload is placed on server j. If there is a candidate server with Peak Ratio P Rj less than a threshold P Rth , we place the VM on the candidate server with the minimum P Rj value. Case 3: If no server exists with the required degree of complementarity, we identify a server that is most likely to find complementary workloads later. We calculate the Buffer Weight (BWj ) for all servers that have capacity to host the workload. We then allocate the new workload to the one having the least BW amongst these servers. If no server in the mapped pool can handle the request, we then place the new workload in the other server pools accordingly. The pseudo code of the algorithm is detailed in Fig. 7.

IV. E XPERIMENTAL E VALUATION A. Implementation We have implemented CloudMap to provide the provisioning logic for a private cloud running in a Fortune 500 enterprise. CloudM ap is implemented as a java-based web application and closely follows the overall architecture proposed in Fig. 5. The Automation manager in our implementation is Tivoli Provisioning Manager, which allows creation of provisioning workflows to automate various system management tasks. The Image Manager is an IBM proprietary image management solution. The Virtualization Manager in our implementation is VMWare vSphere for Intel servers and IBM Hardware Management Console for Power-series servers. B. Competing Algorithms We compare the performance of CloudMap with the following methodologies, which capture existing cloud placement algorithms and incremental improvements using various CloudMap ideas. • Closed Cloud or Round-Robin Placement (RR): RR first selects a target pool in a round robin fashion and then selects a target server in a round robin fashion. This is the procedure generally used in most cloud environments following the Closed Cloud model. • One-way Open or Heterogeneity-Aware Placement (HA): The Heterogeneity aware (HA) algorithm computes the CoV of the workload, its dynamic consolidation needs and appropriately places the new workload in the corresponding pool of server. However, within the selected pool, this new workload is placed in a round-robin manner. • Workload-Aware Placement (WA): This scheme captures a round-robin pool selection followed by the workload aware server selection implemented by the Local Placement Manager in CloudMap. C. Experimental Setup Due to privacy reasons, we could not report the actual placement of virtual machines from the real data center. Hence, for the purpose of the comparative evaluation, we created an experimental setup in our development environment. We used a baseline experimental scenario to closely simulate the real production cloud, where CloudMap is deployed. We consider 59 virtual machines to be placed on a cloud with 16 midrange servers. The virtual machines were part of 4 workloads, and needed to be placed on the cloud in 4 batches. We used the monitored data for the 59 workloads from the live data center and used a 7 day period for our evaluation. We used the first 6 days of workload data to run the various algorithms and come up with a cloud placement. Once the placement was computed, we use the monitored data for the 7th day as an evaluation period to create a load profile for the cloud. The load profile was then used to estimate the per-server utilization and resource violations. The separation of the analysis period from the evaluation period allowed our evaluation to also factor any prediction errors in the methodologies. The cloud consisted of 3 server pools with varying reconfiguration capabilities. The ’Dedicated’ server pool allows

Parameter Baseline Range # Workloads 59 N.A Server Size Multiplier 1 0.33 - 1.67 Batch Size 15 5 - 59 Static/Dynamic Pool Ratio 1 0.5 - 1 TABLE I E XPERIMENTAL PARAMETERS

fixed size virtual machines to be created, which could not be resized without shutting down the virtual machines. The ’Shared’ server pool allowed sharing of resources between virtual machines and dynamic resizing of the virtual machines in response to workload variations. The ’Live Migration’ pool allowed both sharing as well as live migration of virtual machines within the pool. We varied the number of physical cores in the servers to vary the size of the physical servers used in our study and captured it using the Server Size Multiplier parameter. We varied the batch size as well as the number of servers in each pool. The baseline experimental settings and the ranges used in this work are captured in Table. I. D. Experimental Results 1) Comparative Evaluation: In this experiment, we placed all the 4 batches of virtual machines using all the techniques and study the servers used and their utilization. Fig. 8(a) reports the overall amount of cloud resources used by the various algorithms to host the workloads. We observe that CloudM ap uses fewer servers in all the pools as opposed to the other algorithms. An intelligent categorization of the workloads allows CloudMap to place bursty workloads in dynamic pools. By intelligently placing them with uncorrelated workloads, we could size the workloads based on average utilization and reduce wastage. On the other hand, steady workloads were sized based on peak and placed on static servers. Since the difference between peak and average is small for steady workloads, the wastage is small for static pools as well. We find that RR exhibits the worst performance in all the server pools as it does not take into account the workload characteristics when deciding the server pool and the target server in the pool. Hence, it incurs maximum wastage of resources. W A blindly tries to place some bursty workloads into the Dedicated pool due to its round-robin pool selection approach, thereby wasting resources and using more servers. However, within the Shared and LiveM igration pools, W A uses fewer servers as it considers the workload characteristics when selecting the target server within a pool. The HA algorithm performs well in the dedicated pool as it identifies the workloads with low covariance and places them in the corresponding pool, but fails to select complimentary workloads for the target servers and hence gets a performance hit by using a large number of servers in the dynamic pools as compared to CloudM ap. In order to understand the performance better, we study the cloud placement in Fig. 8(b). RR and W A select the server pools in a round-robin fashion and end up placing the same number of workloads in the various pools. Similarly, HA and CloudM ap being aware of the various dynamic consolidation constraints of the data center use CoV of the workloads to

pick the appropriate server pools and hence show the same allocation. We next investigate the ability of the algorithms to improve the utilization of the cloud. The algorithms that are aware of pool capabilities match stable workloads to dedicated pool and bursty workloads to dynamic pools. CloudMap shows a higher server utilization in dedicated pool as it places workloads with low covariance into dedicated servers sizing them for their peak (Fig. 8(c)). The other algorithms place bursty workloads in the dedicated pool, increasing the number of servers required and decreasing their utilization. In dynamic pools, CloudMap places a greater number of high variance and complementary nature workloads on fewer servers to achieve better utilization. W A also achieves high utilization in these pools as it uses workload complementarity. However, this is achieved with lower variance workloads. Hence, it has to place some high variance workloads in Dedicated server pool and exhibits the lowest utilization in this pool. We observe that CloudM ap shows the best overall average utilization for the entire data center. We now investigate the goodness of placement or the number of resource violations in the proposed placement. Fig. 9(a) captures the resource violation observed in the different pools. We see that violation is minimum for both CloudMap and HA highlighting the importance of incorporating the heterogeneity of the data center during workload placement. This is especially true in the dedicated server pool as bursty workloads cannot take resources from other workloads, leading to huge violations. In the shared pool, CloudM ap exhibits some violation due to prediction errors between the analysis and evaluation period. 2) Sensitivity Analysis: We now vary the batch size from the baseline setup to understand its effect on the overall objective of server utilization. Fig. 9(b) shows the total cloud resources used with varying batch sizes. A batch size equal to the total number of virtual machines allows each workload to use perfect knowledge to determine the placement. We note that smaller batch sizes also do not lead to a significant increase in number of servers. For the entire range of batch sizes, CloudMap is the best performing algorithm followed by W A. This highlights the importance of incorporating workloadawareness in cloud placement. In fact, our study suggests that incorporating workload awareness is more important than incorporating heterogeneity-awareness in private clouds. To understand the performance of CloudMap placement in Datacenters with high-end/mid-range servers against those where commodity servers predominate, we vary the server sizes by varying the number of processors in the servers in each pool. CloudM ap uses far fewer servers for catering to the resource demands of the data center than the other algorithms, irrespective of the capacities of the servers (Fig. 9(c)). It is interesting to note that CloudMap outperforms other algorithms even more for small servers. In large servers, the relative size of individual workloads is fairly small. Hence, it is more likely for some complementary workloads to be colocated even using random allocation. In small servers, the

CloudMap HA WA RR

25 # Workloads

8

4

20 15 10

2

0

0

(a) Number of Servers used in each pool

0.2

0.1

0

Live Migration

10 8 6 4

20 18 16 14

25 20 15 10 5 0

0 Dedicated

Shared

10

20

Live Migration

(a) Percentage violations in each pool 60

Reassigned Violations

35 8

20 15

20

# Servers

25

30

Violation(%)

30 40

6

4

10 10 0

5 CloudMap HA WA 50% Less

RR

CloudMap HA WA 60% Less

RR

30 Batch Size

40

50

60

0.2

0.4

0.6

0.8 1 1.2 Server Size Multipler (X)

1.4

1.6

1.8

(b) Total Servers Used with change in Batch Size (c) Total Servers Used with change in Server Size Fig. 9. 10 0.5

40

50

Live Migration

CloudMap HA WA RR

30

10

0

Shared

35

12

2

Dedicated

(c) Average Server Utilization in each pool 40

CloudMap HA WA RR

22 Number of Server Used

Violation (%)

12

Shared

24

CloudMap HA WA RR

14

Workload Reassigned(%)

0.3

(b) Number of Workloads allocated to each pool Fig. 8.

Number of Server Used

16

Dedicated

CloudMap HA WA RR

0.4

5

2

CloudMap HA WA RR

Average Server Utilization

# Servers

6

0.5

CloudMap HA WA RR

30

Average Server Utilization

10

CloudMap HA WA RR

0.4

0.3

0.2

0.1

0 0

0

Dedicated

Shared

Live Migration

(a) Reassignment and violations with fewer servers in (b) Number of Servers used in each pool for Corre- (c) Average Server Utilization in each pool for Cordynamic pool lated Batches related Batches Fig. 10.

impact of intelligent workload profiling is more pronounced, leading to higher savings. In our next experiment, we reduce the number of servers in dynamic pools and observe its effect on placement of workloads. All algorithms when unable to place a workload in the desired pool, overflow to the next available pool. Workloads get reassigned from live migration capable pool to shared pool to dedicated pool. CloudMap and HA varies the number of workloads assigned to any pool by computing CoV thresholds for each pool from the CoV of the all future requests and the total capacity of each pool. WA and RR, on the other hand, move any workload that does not fit in a pool to next pool. Fig. 10(a) shows 2 runs with 50% and 60% less servers in both the dynamic pools (Shared and Live Migration). CloudMap successfully varies the CoV bounds and avoids reassigning any workloads, seeing marginal increase in violations. HA is also able to assign workloads to server pools correctly. However, since it is not workload-aware, it wastes resources and runs out of dynamic pools, requiring few reassignments. The other algorithms are not heterogeneity-aware and reassign many workloads to other pools. Hence, they may allocate highly variable workloads to dedicated pools and risk penalty of increased violations. In our algorithm, we place each virtual machine in a batch independent of other workloads. This allows our algorithm to scale well with increase in the number of virtual machines. We depend on the profiling of a virtual machine with existing

workloads to implicitly spread out correlated workloads in a batch over distinct servers. It is important to understand if ignoring the correlation between new workloads that form a batch impacts the performance of CloudMap, if the correlations are very high. In our next experiment, we modify the original batches of workloads to study if placing each virtual machine independently works in a worst-case scenario. Hence, in each batch, we pick workloads that are most positively correlated. This ensures that each batch has positively correlated workloads and acts as a worst-case grouping for our technique. Figs. 10(b) and 10(c) show the performance of the competing algorithms in this worst case scenario. However, we find that CloudM ap still outperforms the other algorithms by using fewer servers in the different pools leading to a higher utilization of the data center. This experiment highlights the online nature of CloudMap, which allows it to incorporate the resource profile of a workload immediately after it is placed. Hence, the performance of CloudMap is not sensitive to correlation between workloads of an application suite or batch. Our experimental study thus conclusively establishes the effectiveness of CloudMap to optimize cloud resources under a wide variety of settings. CloudMap by virtue of using correlation information of all workloads and knowledge of server pool and server capabilities achieves better workload placement. V. R ELATED W ORK Virtual Machine Placement: Public clouds do not disclose

the exact placement algorithm used publicly but use some variant of round robin placement strategy on their server pool. There are also products that help create cloud offerings like the IBM Tivoli Service Automation Manager (TivSAM)which use round robin or First Fit [13]. Server consolidation in data centers using virtualization had emerged as a powerful functionality to minimize data center costs. Virtualization vendors as well as third party providers have consolidation planning tools that aid in coming up with a placement of the workloads in the virtualized data center [3], [4], [6], [15], [17]. Recent placement algorithms are based on detailed temporal analysis of the workloads and place workloads with complementary resource usage patterns together [20]. Dynamic consolidation in a virtualized data center has also been an active research area [10], [9], [14], [18], [21], [23]. Dynamic consolidation differs from static consolidation by working on a smaller time granularity and continuously optimizing the virtualized data center. In [14], the authors advocate presenting guest virtual machines with a set of soft power states such that applicationspecific power requirements can be integrated as inputs to the system policies, without application specificity at the virtualization-level. In an earlier work, we have proposed dynamic consolidation mechanisms to minimize power [18] and optimize performance subject to a power budget [21]. In [9], the authors optimize multiple objectives namely performance, power and adaptation costs. However, all these algorithms assume homogeneous servers and do not take correlation with existing workloads into account. Workload Characterization: Data center consolidation needs to profile workloads and ensure that the performance of the applications does not suffer due to consolidation. Workload profiling to minimize resource bottleneck and identify good candidates for static [20], [8] and dynamic consolidation has been studied before[2]. Similarly, researchers have worked on modeling the power drawn by servers in a data center [11], [16], incorporate virtualization overheads due to I/O processing [22], and modeling the impact of shared caches and provisioning the memory resources [19], [12]. The workload characterization work is complimentary to our work and is leveraged by CloudMap to place workloads in an efficient manner without impacting application performance. VI. C ONCLUSION & F UTURE W ORK In this work, we investigate the problem of placing enterprise workloads from a captive data center to a private cloud. We identify the key aspects of placement in private clouds, which differentiate it from public cloud placement or dynamic consolidation. We conduct a study of enterprise workloads to identify the importance of incorporating the resource profile of existing workloads. Using the insights from our workload study, we design CloudMap, a placement engine for private clouds. Through a carefully designed set of experiments, we established the effectiveness of CloudMap to optimally utilize cloud resources, while meeting the resource demands of the workloads.

Migration of enterprise workloads from a captive environment to a cloud happens in an online fashion. Further, over time, usage patterns of workloads may change. These two and similar conditions necessitate a reconfiguration of the placement of existing workloads. A reconfiguration should also consider additional cost of moving applications between servers in the same server pool and across server pools. The placement algorithm should be formulated to minimize the overall reconfiguration cost. As future work, we plan to incorporate this into CloudMap. R EFERENCES [1] A. I. Avetisyan and et al. Open cirrus: A global cloud computing testbed. In IEEE Computer, 2010. [2] N. Bobroff, A. Kochut, and K. Beaty. Dynamic placement of virtual machines for managing sla violations. In IEEE Conf. Integrated Network Management, 2007. [3] Server Consolidation and Virtualization Analysis by CiRBA. http://www.cirba.com/. [4] VMWare Guided Consolidation. http://www.vmware.com/products/vi/vc/features.html. [5] EC2. Amazon elastic compute cloud. http://aws.amazon.com/ec2/. [6] Virtual Iron: True Server Virtualization for Everyone. http://www.virtualiron.com/. [7] A. Ghodsi, M. Zaharia, B. Hindman, A. Konwinski, S. Shenker, and I. Stoica. Dominant resource fairness: fair allocation of multiple resource types. In NSDI, 2011. [8] D. Gmach, J. Rolia, L. Cherkasova, and A. Kemper. Workload analysis and demand prediction of enterprise data center applications. In IISWC, 2007. [9] G. Jung, M. Hiltunen, K. Joshi, R.Schlichting, and Calton Pu. Mistral: Dynamically managing power, performance, and adaptation cost in cloud infrastructures. In Proc. ICDCS, 2010. [10] G. Jung, K. Joshi, M. Hiltunen, R.Schlichting, and Calton Pu. A cost-sensitive adaptation engine for server consolidation of multitier applications. In Proc. Middleware, 2009. [11] R. Koller, A. Verma, and A. Neogi. Wattapp: An application aware power meter for shared data centers. In ICAC, 2010. [12] R. Koller, A. Verma, and R. Rangaswami. Generalized erss tree model: Revisiting working sets. In IFIP Performance, 2010. [13] Tivoli Service Automation Manager. http://www01.ibm.com/software/tivoli/products/service-auto-mgr/. [14] Ripal Nathuji and Karsten Schwan. Virtualpower: coordinated power management in virtualized enterprise systems. In Proc. ACM SOSP, 2007. [15] VMWareCapacity Planner. http://www.vmware.com/products/capacityplanner/. [16] S. Rivoire, P. Ranganathan, and C. Kozyrakis. A comparison of highlevel full-system power models. In HotPower, 2008. [17] HP Capacity Advisor Consolidation software. http://h18013.www1.hp.com/products/servers/ management/capad/index.html. [18] A. Verma, P. Ahuja, and A. Neogi. pmapper: Power and migration cost aware application placement in virtualized systems. In Proc. Middleware, 2008. [19] A. Verma, P. Ahuja, and A. Neogi. Power-aware dynamic placement of hpc applications. In ACM ICS, 2008. [20] A. Verma, G. Dasgupta, T. Nayak, P. De, and R. Kothari. Server workload analysis for power minimization using consolidation. In Proc. Usenix ATC, 2009. [21] A. Verma, P. De, V. Mann, T. Nayak, A. Purohit, G. Dasgupta, and R. Kothari. Brownmap: Enforcing power budget in shared data centers. In Proc. Middleware, 2010. [22] T. Wood, L. Cherkasova, K. Ozonat, and P. Shenoy. Profiling and modeling resource usage of virtualized applications. In Proc. Middleware, 2008. [23] Qian Zhu, Jiedan Zhu, and Gagan Agrawal. Power-aware consolidation of scientific workflows in virtualized environments. In Proc. SC, 2010.