Customer Experiences Adopting Private Cloud Building Elastic, Scalable Applications on Virtualized Infrastructures
Snehal S. Antani
IBM Private Cloud Strategy
[email protected] http://snehalantani.googlepages.com
1
Post-Virtualization Band-aid to improve CPU utilization Observation: after applying virtualization, customers are not achieving the CPU utilization promised because they are quickly running out of memory (RAM) Cause: • • • •
1 GB of app server JVM Heap = 1.4 GB of OS memory (due to OS overhead, varies per OS) Application server best practices recommends JVM min & max heap size to be the same (Min=Max= 1GB) To avoid memory page thrashing by the OS, performance best practices recommends application servers not run on virtualized or overcommitted memory (except on System z) example: 700 Application Server JVM’s each with 1 GB heap requires ~1 TB of dedicated RAM!
Solution: •
Apply Application Virtualization technology (WebSphere Virtual Enterprise) • Automatically start/stop JVM’s based on application demand and priority • Reduces the average # of running JVM’s
•
Apply Elastic Caching technology (WebSphere eXtreme Scale) • Cached application data (using EHCache, Hibernate, etc) is stored in a global cache • HTTP session data is stored in a global cache • Reduces the average JVM size • Global cache can be stored on hardware optimized for memory-intensive workloads (WebSphere XC10, nonvirtualized x86, etc)
Example results: • •
Customer reduce 700 running JVM’s to 400 on average; average JVM size reduced from 1GB to 750 MB Total required RAM reduced from 1TB to 420 MB, CPU utilization increased significantly 2
Understanding Application Virtualization
3
Application Virtualization - Dynamic, policy-based middleware systems - Provides dynamic infrastructure services for WAS and non-WAS (Tomcat, WebLogic, etc) across VMware (linux, windows), Sun zones (solaris), PowerVM (AIX), zVM (z/Linux, z/OS) - Dynamically obtain & release server resources to meet demands - Determine request priorities based on business objectives - Monitor the health of the application infrastructure -
Route requests away from sick application servers,
-
Take actions to repair unhealthy servers
- Driven by policies, reducing admin intervention Classification Policies aligned with business objectives
Policies to manage server & application health
Performance Metrics
Intelligent Routing Tier
HTTP Traffic Web Server Tier
• Request Flow Control • Cluster/Server Optimization Instructions
Pool of Application Servers (WAS, Non-WAS)
4
WebSphere Virtual Enterprise optimizes the middleware system with dynamic clusters - Applications are installed to a Dynamic Cluster - ODR uses policies to determine when to start/stop dynamic cluster members - Dynamic Cluster member = App Server JVM
Frame 1
Frame 2
- You can convert your existing WAS clusters into dynamic clusters App 1 Dynamic Cluster Min = 1, Max=2
HTTP Traffic Web Server Tier
= App Server JVM
On-Demand Router (ODR)
App 2 Dynamic Cluster Min = 1, Max=2 5
Understanding Elastic Caching via Data Grids
6
Architecture Web Server Tier
Application Server Tier
Elastic Data Grid
A
D’
B
C’
C
B’
D
A’
Database Tier
7
WebSphere eXtreme Scale – A Data Grid Application Server WXS Data Grid Client
App BankRecordGrid. get(0123456789);
(Access Services)
Get record 0123456789 from the Bank Record data grid
Bank Record Data Partition Http Session Data Partition Health Record Data Partition WebSphere eXtreme Scale Data Grid (Placement, Replication, Elasticity Services)
JVM
JVM
JVM 8
Reducing the Average JVM footprint with a global cache • Reduce the JVM memory footprint with smarter caching
• Data stored as key-value pairs. (Think: hashmap if ‘infinite’ size) • Can be horizontally partitioned – list-based, hash-based, range-banged partitioning schemes can be applied to the data
• Often transient or referential data: – HTTP sessions, user profile, – Read-only or read-mostly – Can tolerate some staleness Large app JVM’s with duplicate cached data
Smaller app JVM’s that connect to a fault-tolerant, centralized distributed cache
JVM 1
JVM 2
JVM 3
JVM 4
App
App
App
App
Cache
Cache
Cache
Cache
JVM 1
JVM 2
App
App
JVM 3
App
WebSphere eXtreme Scale cache
JVM 4
App
9
Global data cache enables Data Virtualization across the Enterprise & Clouds • Expose cached data to all applications running within the cloud • Improve fault-tolerance, performance, and reduce the application memory footprint • Offer simple access mechanisms to data via:
– Map-based query interface – Entity Manager query interface – ESB Mediations
Cluster
Cluster
App 2
App 3
Cluster
Cluster
App 1
App 4
WebSphere eXtreme Scale Fault-Tolerant, Elastic Data Store
ESB App 5 Cluster
DB
DB
DB 10
The cache spans datacenters for higher availability Datacenter 2 AS WXS WXS AS datacenter 1
Multi-datacenter replication WXS
SoR
AS
datacenter 3
1. Store data in a geographically distributed cache 2. Cache offloads data from the back-end 3. Replication in the cache keeps the back-end from getting overloaded in case of failures 4. Replicated cache improves application availability
11
Combining Application Virtualization & Elastic Caching
12
Web Application Challenges
1. Web Application Infrastructures don’t scale well
2. New application infrastructure designs enable scaling to tolerate huge transaction volumes 1.
Think: Facebook, Google, Amazon, Ebay
3. WebSphere eXtreme Scale delivers technology to build highly scalable web application infrastructures
13
Traditional Application Infrastructures Add items to MY shopping cart
Route Request to JVM that holds MY shopping cart
Add item to a shopping cart My Shopping Cart Contents
Web Server
Application
Application
Conversational State
Conversational State
App Server
App Server
- Business Transactions and Conversational State are located in the same JVM - Conversational State (HTTP Sessions) require session affinity & special failover - App Server required complex HA and failover - As a result, scalability was limited 14
Scalable Web Application Infrastructures Add items to MY shopping cart Simple round-robin load balancing
Add item to a shopping cart Get My Shopping Cart Contents
Web Server
Application
Application
App Server
App Server
Conversational State WebSphere eXtreme Scale Data Grid JVM
JVM
JVM
JVM
JVM
JVM
- Business Transactions and Conversational State are in separate tiers - Business Transactions are stateless - Conversational State is fault tolerant - Each tier can be scaled independently 15
Next-Generation Application Infrastructures
SLA-Driven Load Balancing
SLA’s If URL = “/checkout” • priority = 1, Response time < 250 ms If URL = “/catalog” • priority = 3, Response time < 500 ms
On-Demand On-DemandRouter Router
Health Policies Memory leak, Storm-drain, Excessive Time out, Custom defined
Dynamic Cluster
Application
.....
Application
Cache Policies Synchronous Replication, Async Replication
App Server
App Server
Conversational State WebSphere eXtreme Scale Data Grid JVM
JVM
JVM
JVM
JVM
JVM
- On-demand Router auto-scales the application to meet service-level agreements - Web sessions (conversational state) are fault-tolerant and linearly scalable - Applications & Conversational State are resilient, self-healing, and span datacenters - Significantly increases the “density” of the system (increasing CPU utilization) - No Application Code changes necessary 16
IBM’s private cloud foundation addresses the needs of the most diverse enterprises Hybrid Cloud Service Automation, Provisioning, & Monitoring (IBM Service Delivery Manager)
WebSphere Application Server Dynamic Infrastructure Services (WebSphere Virtual Enterprise)
Oracle Weblogic
Integration (Patterns deployed to IBM SmartCloud)
Apache Tomcat
JBoss, .Net, etc
HTTP Session Cache (WebSphere eXtreme Scale, WebSphere XC10) VMWare (Linux, Win)
KVM (Linux, Win)
PowerVM (AIX)
z/VM (z/Linux, z/OS)
•
Elastic application runtimes
•
Application health management & self-healing
•
Elastic data services
•
Continuously available applications
•
Pattern-based middleware
•
Heterogeneous platform management
•
Dynamic workload management
•
Optimized workload placement
•
Fine-grained SLA enforcement
•
Service automation & governance
•
Fine-grained chargeback 17
Questions?
Snehal S. Antani
IBM Private Cloud Strategy
[email protected] http://snehalantani.googlepages.com
18