Kubernetes for lazy people ..lessons learned deploying k8s (in aws)

..but it looks really cool and I'm tired of using 10% of my CPU

simple k8s checklist • will there be persistent storage required? • how will you manage upgrading it? • where will you persist logs • Can I reasonably secure it? • where will long term metrics live if you have no state storage?

Kubernetes supports persistent volumes • ... whats the problem?

OK so I'm lazy.. lets do this without state • Good news is now you can really easily orchestrate your apps • bad(ish) news is that now you need to run your datastores alongside your cluster... • Life-cycle of clusters is simplified.. you can now "blue/green" k8s clusters themselves (unless application load is your typical bottleneck)

Enter kubernetes/kops (ascinema) • kops is a go app that functions similarly to kubectl • It deploys and manages a pretty stable and decently pre-rolled cluster for you • it uses similar principals to terraform • got an s3 bucket and a route53 subdomain? • feature lists will probably be missing something that you want (but just ask, they're super friendly, see #sig-aws on k8s slack)

...some considerations for prod • Do you need namespace segregation within your cluster? (if so.. ensure you enable a CNI plugin) • Do you need to put this into private subnets inside an existing VPC? (if your subnet ranges consume the entire VPC subnet you're gonna need a new VPC or some kinda heroics) • The https://api.{{FQDN}} of your cluster will be on a public IP (last I checked.. you might want to put an IP whitelist on this..) • What about granting RW/RO access to the API to specific users/groups/teams (and to defined resources?)

Birthing your new cluster ./kops-linux-amd64 create cluster \ --channel=alpha --vpc=vpc-a12345bc \ --network-cidr=10.0.123.0/19 \ --zones=eu-west-1a,eu-west-1b,eu-west-1c \ --master-zones=eu-west-1a,eu-west-1b,eu-west-1c \ --master-size=t2.medium \ --node-size=t2.medium \ --ssh-public-key=/home/you/.ssh/id_rsa_key.pub \ --topology=private \ --networking=weave \ new-cluster.example.com

Namespace segregation • On default k8s allows containers in all namespaces to talk to each other • Even when pods live in different namespaces pods can discover each others service ports and connect to each other

Sample k8s namespace policy

Granting containers access to IAM resources

see: https://github.com/jtblin/kube2iam

Check that apps pull at least the default policy

Sample terraform for a real-life policy

Prometheus overview • deploy prometheus within the cluster • Then expose it on an ELB (type: LoadBalancer) •

• Scrape that ELB from your primary prometheus

sample build pipeline in Groovy

doDeploy()

Tip: "kubectl rollout status deployment/xyz"

OK job done...