A Taste of Chef on AWS Chef-provisioning basics & AWS Integration

1

Introductions

2

Instructor Introduction • Name: Alex Vinyar • Current job role: Solutions Engineer @ Chef • Previous job roles/background: 
 Microsoft - Xbox (systems engineer),
 Expedia(release engineer),
 Chef (Consulting) • Experience with Config Management: I’ve been automating since batch files. 3

Instructor Introduction • GITHUB repo: !

https://github.com/vinyar/tokyo_chef_provisioning

4

Quick Survey • Hands up: who is a developer?

5

Quick Survey • Hands up: who is a operations guru?

6

Quick Survey • Hands up: who is a startup founder?

7

Quick Survey • Hands up: who is new to Chef?

8

What is Chef?

9

Chef is a Language !

• A programming language that lets you describe how you deal with the infrastructure the same way you deal with the applications

10

Chef is a Language • Learning Chef is like learning the basics of a language • 1% fluency will be gained from this quick-start • 70% fluency can be gained from a full multi-day Chef Fundamentals training course • The remaining just takes practice • The best way to learn Chef is to use Chef 11

Quick Overview of Chef

12

A tale of Amazonian growth Application POC

13

Add a database Application

Shiny Database

14

Shard your data Application

Shiny Databases

15

Add application servers

…some time later

16

A tale of Amazonian growth ELB

CloudWatch

Auto Scaling Group SQS

ElastiCache Custom

Metrics

Shiny

Databases

Metrics

Analysis

Data

Replicas 17

Infrastructure has a Topology ELB

CloudWatch

Auto Scaling Group SQS

ElastiCache Custom

Metrics

Shiny

Databases

Metrics

Analysis

Data

Replicas 18

Complexity increases quickly

Supporting Application

ELB

CloudWatch New Hotness

Auto Scaling Group SQS

ElastiCache Supporting Application

Custom

Metrics

Shiny

Databases

Metrics

Analysis

Data

Replicas 19

… and change happens!

r e s u l a c o r l u w o y e l n l a a o d t Ad ount ces acc instan

Supporting Application

ELB

CloudWatch New Hotness

Auto Scaling Group SQS

ElastiCache Supporting Application

Custom

Metrics

Shiny

Databases

Metrics

Analysis

Data

Replicas 20

… and change happens!

Supporting Application

. T o b o o B ‘ b o c b d s d n a w r o use ’ -g cl n w Clo ELB

CloudWatch New Hotness

Auto Scaling Group SQS

ElastiCache Supporting Application

Custom

Metrics

Shiny

Databases

Metrics

Analysis

Data

Replicas 21

Configuration Desperation

. T o b o o B ‘ b o c b d s d n a w r o use ’ -g cl n w Clo

Supporting Application

ELB

CloudWatch New Hotness

Auto Scaling Group SQS

ElastiCache Supporting Application

Custom

Metrics

Shiny

Databases

Metrics

Analysis

Data

Replicas 22

Chef Solves This Problem • But you already guessed that, didn’t you?

23

Chef is Infrastructure as Code • Programmatically provision and configure servers • Treat like any other code base

http://www.flickr.com/photos/louisb/4555295187/

• Reconstruct business from code repository, data backup, and compute resources 24

Policy-based Management • You capture the policy for your infrastructure in code • Chef ensures each node in your infrastructure complies with the policy • Policy describes the desired state • Policies can be statically or dynamically defined

25

Chef Components

26

Chef Server Functions

27

Resources • The fundamental building blocks of Chef • Describes a piece of the system and its desired state • package… • service… • file… • cron job… • user… • etc… 28

Recipes • A collection of resources • Describe configuration policy you want to apply • Recipes can: • Install and configure software components • Manage file content • Deploy applications • Execute other recipes • and more 29

Example Recipe package "apache2" template "/etc/apache2/apache2.conf" do! source "apache2.conf.erb"! owner "root"! group "root"! mode "0644"! variables(:allow_override => "All")! notifies :reload, "service[apache2]"! end service "apache2" do! action [:enable,:start]! supports :reload => true! end

30

Recipes: a collection of resources package "apache2" template "/etc/apache2/apache2.conf" do! source "apache2.conf.erb"! owner "root"! group "root"! mode "0644"! variables(:allow_override => "All")! notifies :reload, "service[apache2]"! end service "apache2" do! action [:enable,:start]! supports :reload => true! end

31

Configuration Drift • Configuration Drift happens when: • Your infrastructure requirements change • The configuration of a server falls out of policy !

• Chef makes it easy to manage • Model the new requirements in your Chef configuration files • Run the chef-client to enforce your policies 32

Resources Fundamental Building Blocks

33

Resources • Piece of the system and its desired state

34

Resources - Package • Package that should be installed

package "vim" do! action :install! end

35

Test and Repair • Resources follow a test and repair model

package "vim"

Test Is vim installed?

36

Test and Repair • Resources follow a test and repair model

package "vim"

Test Is vim installed?

Yes

37

Test and Repair • Resources follow a test and repair model

package "vim"

Test Is vim installed?

Yes Done 38

Test and Repair • Resources follow a test and repair model

package "vim"

Test Is vim installed?

Yes Done 39

No

Test and Repair • Resources follow a test and repair model

package "vim"

Test Is vim installed?

Yes

No

Done

Install it

40

Test and Repair • Resources follow a test and repair model

package "vim"

Test Is vim installed?

Yes Done 41

Repair

No Install it

Resources: Test and Repair • Resources follow a test & repair model !

• Resource currently in the desired state? (test) • Yes - Do nothing • No - Bring the resource into the desired state (repair)

42

Lets look at the code… Example webserver recipe

43

Applying policy to EC2 Chef Provisioning for AWS

44

What just happened? chef-client --local-mode -r 'recipe[webserver::provision]'

Workstation AWS API

New Instance

45

What just happened? chef-client --local-mode -r 'recipe[webserver::provision]'

ec2 server create

Workstation

AWS API

New Instance

46

What just happened? chef-client --local-mode -r 'recipe[webserver::provision]'

ec2 server create

Workstation

AWS API

new instance created

New Instance

47

What just happened? chef-client --local-mode -r 'recipe[webserver::provision]'

ec2 server create

AWS API

Workstation

ssh / scp

install chef-client

configure chef-client

run chef-client

new instance created

New Instance

48

What just happened? chef-client --local-mode -r 'recipe[webserver::provision]'

ec2 server create

AWS API

Workstation

ssh / scp

install chef-client

configure chef-client

run chef-client

download cookbooks

apply recipes

save node object

new instance created

New Instance

49

Policy for other AWS Services Beyond EC2

50

Where is our LB? • Your servers are now load balanced • The easiest way to get the public DNS name for the new load balancer is from the EC2 Management Console • We'll make this process easier in the future • https://github.com/opscode/chef-provisioning-aws/ issues/27

51

AWS Services with Chef Provisioning • AWS Services you can currently manage !

• SQS Queues ! • SNS Topics ! • ELBs ! • VPCs

• Security Groups • SSH Key pairs • EC2 Instances • Launch configs • AMIs ! • Autoscaling Groups

!

• AWS driver is under active development • Expect more supported services 52

Healing your infrastructure Recovering from Disaster

53

Chef is Infrastructure as Code

http://www.flickr.com/photos/louisb/4555295187/

• Remember this slide? • Programmatically provision and configure servers • Treat like any other code base • Reconstruct business from code repository, data backup, and compute resources 54

Simulate a disaster • The Problem: We haven’t practiced reconstructing our business using code we’ve written !

• Success Criteria: We can recover the state of our application after a disaster

55

Wrap Up Recap & Next Steps

56

But wait… • There is so much more! !

• How much time do we have left, I could go on for days!

57

Continued Learning • The LearnChef Site • Guided Tutorials • Chef Fundamentals intro http://learnchef.com • How-To’s, Conference Talks, Webinars, more http://youtube.com/user/getchef • Attend a Chef Fundamentals Class 58

Further Resources • http://chef.io • http://docs.chef.io • http://supermarket.chef.io • http://lists.opscode.com • irc.freenode.net #chef, #chef-hacking, #learnchef • Twitter @chef #getchef, @learnchef #learnchef

59

Food Fight Show • http://foodfightshow.org • The Podcast Where DevOps Chef Do Battle • Regular updates about new Cookbooks, Knifeplugins, and more • Best Practices for working with Chef

60

Chef Fundamentals • https://www.chef.io/training/ !

• 3 Day Chef Fundamentals (Linux/Windows) • February 24-26, 2015, 8am-2:30pm !

• Want 25% OFF? 
 
 email me: [email protected] 61

Time to hack

62

63

Questions? @vinyar1

Thank you!

[email protected]