The Advanced Message Queuing Protocol (AMQP) is an application layer protocol for messaging. This talk introduces RabbitMQ, its prominent multi-lingual open-source implementation.
Emile Joubert December 2010 © 2010 VMware Inc. All rights reserved
Disclaimer This session may contain product features that are currently under development. This session/overview of the new technology represents no commitment from VMware to deliver these features in any generally available product. Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind. Technical feasibility and market demand will affect final delivery. Pricing and packaging for any new technologies or features discussed or presented have not been determined. "THESE FEATURES ARE REPRESENTATIVE OF FEATURE AREAS UNDER DEVELOPMENT. FEATURE COMMITMENTS ARE SUBJECT TO CHANGE, AND MUST NOT BE INCLUDED IN CONTRACTS, PURCHASE ORDERS, OR SALES AGREEMENTS OF ANY KIND. TECHNICAL FEASIBILITY AND MARKET DEMAND WILL AFFECT FINAL."
© 2010 VMware Inc. All rights reserved
About me... Emile studied computer science and mathematics before following a career that included electronic publishing; joining LShift in 2008 and SpringSource in 2010.
© 2010 VMware Inc. All rights reserved
Outline • • • • •
Messaging AMQP Publishing walk-through RabbitMQ and the benefits of Erlang Large scale sample uses
© 2010 VMware Inc. All rights reserved
Messaging Use Cases Decoupling
© 2010 VMware Inc. All rights reserved
Messaging Use Cases Bidirectional Decoupling e.g. Remote Procedure Call
© 2010 VMware Inc. All rights reserved
Messaging Use Cases Pipelining and Decoupling
© 2010 VMware Inc. All rights reserved
Messaging Use Cases Work-distribution and Decoupling e.g. duplication and round-robin
© 2010 VMware Inc. All rights reserved
Messaging Use Cases Work aggregation, distribution and decoupling
© 2010 VMware Inc. All rights reserved
Why Messaging? SQL : : Databases : : Filesystem AMQP : : Messaging : : Network
• Logical decoupling • Physical decoupling • Temporal decoupling
© 2010 VMware Inc. All rights reserved
AMQP • An open, wire-level binary protocol for Message Orientated Middleware • Created by users and technologists working together, to satisfy real needs • A straight-forward and complete solution for business messaging: SMTP for business
© 2010 VMware Inc. All rights reserved
AMQP Model Flexible routing/ queuing parameters Multi-vendor interoperability
Network-optimized
Network-friendly
AMQP Model
AMQP Wire Format
Framing Transport
© 2010 VMware Inc. All rights reserved
AMQP Specification Sample
© 2010 VMware Inc. All rights reserved
AMQP Model • • • • •
Producers Exchanges Bindings Queues Consumers
© 2010 VMware Inc. All rights reserved
Publishing: Setup Exchanges, queues & bindings are created dynamically by producers & consumers
© 2010 VMware Inc. All rights reserved
Publishing
© 2010 VMware Inc. All rights reserved
Publishing
© 2010 VMware Inc. All rights reserved
Publishing Publishing a second message
© 2010 VMware Inc. All rights reserved
Consuming
© 2010 VMware Inc. All rights reserved
Consuming Consuming second message
© 2010 VMware Inc. All rights reserved
Publishing Multiple Messages
© 2010 VMware Inc. All rights reserved
Publishing Multiple Messages ...to several consumers on the same queue
© 2010 VMware Inc. All rights reserved
Publishing to Multiple Queues
© 2010 VMware Inc. All rights reserved
Publishing to Multiple Queues Sending
© 2010 VMware Inc. All rights reserved
Publishing to Multiple Queues Messages go to every queue bond to a fanout exchange
© 2010 VMware Inc. All rights reserved
Publishing to Multiple Queues Queues can be consumed from at different rates
© 2010 VMware Inc. All rights reserved
Publishing: Topic Routing * matches 1 word # matches 0 or more words
© 2010 VMware Inc. All rights reserved
Publishing: Topic Routing
© 2010 VMware Inc. All rights reserved
AMQP Model Elements Exchange Properties • passive • durable • auto-delete Exchange Types • direct • fanout • topic • headers
Queue Properties • passive • durable • exclusive • auto-delete
Message Properties • routing key • immediate • delivery-mode • priority • expiration © 2010 VMware Inc. All rights reserved
Prominent AMQP Implementations RabbitMQ (Erlang) OpenAMQ (C++) Apache Qpid (Java and C++) RedHat MRG (Messaging, Realtime & Grid) (C++)
© 2010 VMware Inc. All rights reserved
Why Erlang? • • • •
Concise (binary matching) Sane concurrency (no shared state) Built-in distributed database (Mnesia) Multi-platform (Windows, Unix)
© 2010 VMware Inc. All rights reserved
Erlang binary matching decode_method('basic.publish', <>) -> F3 = (F3Bits band 1) /= 0, F4 = (F3Bits band 2) /= 0, #'basic.publish'{ticket = F0, exchange = F1, routing_key = F2, mandatory = F3, immediate = F4}; © 2010 VMware Inc. All rights reserved
What is RabbitMQ? l
l l
RabbitMQ consists of the broker (server) and several clients (Java, .net and others) They speak the AMQP protocol to each other Anyone can write other clients which also speak AMQP and work with RabbitMQ l
l
l
e.g. Ruby, Java, Spring, Python, C, C#, Perl, Erlang, Lisp, Haskell, PHP,
It's in many popular Linux distributions and can easily be installed on OS X Released under MPL © 2010 VMware Inc. All rights reserved
What is RabbitMQ? l l l
l
17k lines of Erlang code in the broker Supports clustering for increased scalability Can work with Pacemaker and associated tools to provide High Availability Supports several extension points via plugins, e.g. exchange types and backing queue storage
© 2010 VMware Inc. All rights reserved
AMQP compared to JMS AMQP has a wire-level binary protocol, instead of an API AMQP offers multi-language interoperability AMQP message payloads are opaque Use Spring-AMQP to migrate
© 2010 VMware Inc. All rights reserved
Protocols AMQP (SSL) STOMP XMPP (Rabbiter) 0MQ REST
© 2010 VMware Inc. All rights reserved
Real-world features • Unroutable messages • Client acknowledgements • Persistence • Transactions • Flow control • Security
© 2010 VMware Inc. All rights reserved
Management & Monitoring RabbitMQ plugin •
Inspect and manage resources
•
Load & save configuration
•
HTTP API
•
Commandline tool
© 2010 VMware Inc. All rights reserved
Management & Monitoring
© 2010 VMware Inc. All rights reserved
Management & Monitoring
© 2010 VMware Inc. All rights reserved
Cloud Ready • Messaging service hosted in cloud • Cloud infrastructure services • e.g. Nasa Nebula • Providing a service in the cloud • e.g. SQS on Amazon
© 2010 VMware Inc. All rights reserved
Nebula Architecture
© 2010 VMware Inc. All rights reserved
UIDAI
© 2010 VMware Inc. All rights reserved
UIDAI Unique Identification Authority of India (UIDAI) will issue a12-digit for all residents • • • •
One of the largest IT projects ever Largest online identity DB ever 200 million messages per day peak 600 million residents over 4 years
• • • • • • • •
Spring Tomcat Hibernate MySQL Hadoop Mule Apache httpd RabbitMQ
© 2010 VMware Inc. All rights reserved
The future • AMQP 1.0 • Connect-a-thon
• • Get in touch! • http://www.rabbitmq.com/ This presentation will be available at http://www.springsource.com/webinars from Monday 13 Deccember 2010
© 2010 VMware Inc. All rights reserved
