Spring Cloud Stream

External

• https://cloud.spring.io/spring-cloud-stream/
• https://cloud.spring.io/spring-cloud-stream-app-starters/
• Reference Spring Cloud Stream Core https://docs.spring.io/spring-cloud-stream/docs/Elmhurst.RELEASE/reference/htmlsingle/#_spring_cloud_stream_core

Internal

• Spring Cloud

Overview

Spring Cloud Stream is a framework for building highly scalable event-driven microservices connected with shared messaging systems. Spring applications use Spring Cloud Stream libraries to bind to a messaging middleware. Spring Cloud Stream builds upon Spring Boot to create standalone production-grade applications and uses Spring Integration to provide connectivity to message brokers. The applications built with Spring Cloud Stream are middleware-neutral. Instead, the application communicates with the outside world through input and output channels injected into it by Spring Cloud Stream runtime. Channels are connected to brokers through middleware-specific binder implementations.

Concepts

Binder

The component responsible to provide integration with external messaging systems. Spring Cloud Stream automatically detects and uses a binder found on the classpath. The actual destination (such as Kafka topics or RabbitMQ exchanges) it is selected at boot based on configuration (application arguments, environment variables, application.yml or application.properties).

Available binders:

• Rabbit Binder
• Kafka Binder
• AWS Kinesis Binder
• Google Cloud PubSub Binder
• TestSupportBinder

Persistent Publish/Subscribe

Communication between applications follows a publish-subscribe model, where data is broadcast through shared destinations (topics). Communicating through shared topics rather than point-to-point queues reduces coupling between microservices.

Consumer Group

The microservice model relies on the ability of an application to scale horizontally by increasing the number of "equivalent" components that are supposed to perform the same function and thus spread the load. However, when doing so, different instances of the service are placed in a competing consumer relationship, where only one of the instances is expected to handle a given message. Spring Cloud Stream models this behavior through the concept of a consumer group. Each consumer binding can use the spring.cloud.stream.bindings.<channelName>.group property to specify a group name. All groups that subscribe to a given destination receive a copy of published data, but only one member of each group receives a given message from that destination. By default, when a group is not specified, Spring Cloud Stream assigns the application to an anonymous and independent single-member consumer group that is in a publish-subscribe relationship with all other consumer groups. In general, it is preferable to always specify a consumer group when binding an application to a given destination. When scaling up a Spring Cloud Stream application, you must specify a consumer group for each of its input bindings. Doing so prevents the application’s instances from receiving duplicate message.

Consumer group subscriptions are durable: a binder implementation ensures that group subscriptions are persistent and that, once at least one subscription for a group has been created, the group receives messages, even if they are sent while all applications in the group are stopped.

Consumer Types

Message-Driven

Message-driven consumer are also known as asynchronous consumers.

Polled

Polled consumer are also known as synchronous consumers. A synchronous controller allows controlling the rate at which messages are consumed.

Partition

Spring Cloud Stream offers the possibility to partition the physical communication medium provided by a broker topic into multiple partitions. Partitioning can be used whether the broker itself is naturally partitioned (Kafka) or not (RabbitMQ). Partitioning is a critical concept in stateful processing, where it is critical (for either performance or consistency reasons) to ensure that all related data is processed together. To set up a partition, both the data-producing and the data-consuming ends must be configured similarly.

Programming Model

Destination Binder

A destination binder is the implementation behind a binder abstraction: the component responsible with providing integration with external messaging systems.

Destination Binding

Bridge between the external messaging system and application-provided Producers and Consumers of messages. The Producers and Consumers are created by the Destination Binders.

Message

The canonical data structure used by Producers and Consumers to communicate with Destination Binders, and thus other applications via external messaging systems.

Interfaces

Sink

Input and no output.

public interface Sink {
  String INPUT = "input";

  @Input(Sink.INPUT)
  SubscribableChannel input();
}

Channel

Input Channel

An input channel funnels received messages into the application. The implementation is transparently created by Spring Cloud Stream and injected into the application.

Output Channel

Published messages leave the application via output channels. An input channel funnels received messages into the application. The implementation is transparently created by Spring Cloud Stream and injected into the application.

Destination

The destination can be a queue, topic, or others.

Message Handler

Is this the same thing as the handler method?

@StreamListener(Sink.INPUT)
public void handle(...) {
...
}

Also see @StreamListener.

Annotations

• @EnableBinding - provides immediate connectivity to a message broker.
• @StreamListener - annotates a handler method to receive events for stream processing.
• @Input
• @Output

Configuration

spring.cloud.stream.bindings.input.destination

Note that in function of the dynamic binding at boot, the destination can be a Kafka topic, a RabbitMQ exchange, etc.