Spring Cloud Stream: Difference between revisions

From NovaOrdis Knowledge Base
Jump to navigation Jump to search
(Replaced content with "=External= * https://cloud.spring.io/spring-cloud-stream/ * https://cloud.spring.io/spring-cloud-stream-app-starters/ * Reference Spring Cloud Stream Core https://docs.sp...")
Line 16: Line 16:


=Configuration=
=Configuration=
------------------------------------
=Overview=
Spring Cloud Stream is a framework for building highly scalable event-driven microservices connected with shared messaging systems. Spring applications use Spring Cloud Stream libraries to bind to a messaging middleware. Spring Cloud Stream builds upon [[Spring Boot]] to create standalone production-grade applications. The applications built with Spring Cloud Stream are middleware-neutral. Instead, the application communicates with the outside world through [[#Input_Channel|input]] and [[#Input_Channel|output]] channels injected into it by Spring Cloud Stream runtime. Channels are connected to brokers through middleware-specific [[#Binder|binder]] implementations.
[[Spring Initializr]] lists Spring Cloud Stream under "Cloud Messaging"
=Spring Cloud Stream and Spring Integration=
Spring Cloud Stream is built on the concepts and patterns defined by [[EIP|Enterprise Integration Patterns]] and relies on the [[Spring Integration]] implementation to provide connectivity to message brokers, so it supports the foundation, semantics, and configuration options that are already established by [[Spring Integration]].
<font color=darkgray>TO DO: https://docs.spring.io/spring-cloud-stream/docs/Elmhurst.RELEASE/reference/htmlsingle/#_spring_integration_support</font>
=Concepts=
==Binder==
The component responsible to provide integration with external messaging systems. Spring Cloud Stream automatically detects and uses a binder found on the classpath. For more details on binder detection see [[#Binder_Detection|Binder Detection]] below. The actual destination (such as Kafka topics or RabbitMQ exchanges) it is selected at boot based on configuration ([[Spring_Application_Configuration_Concepts#Application_Arguments|application arguments]], [[Spring_Application_Configuration_Concepts#Environment_Variables|environment variables]], [[application.yml]] or [[application.properties]]).
Available binders:
* [[Spring Cloud Stream Rabbit Binder|Rabbit Binder]]
* [[Spring Cloud Stream Kafka Binder|Kafka Binder]]
* [[Spring Cloud Stream AWS Kinesis Binder|AWS Kinesis Binder]]
* [[Spring Cloud Stream Google Cloud PubSub Binder|Google Cloud PubSub Binder]]
* [[Spring Cloud Stream TestSupportBinder|TestSupportBinder]]
The binder implementation ([[#Destination_Binder|destination binder]]) is responsible for connectivity, message routing to and from producers and consumers, data type conversion, etc. A producers is any component that sends messages to a channel, which in turn is bound to an external message broker with the binder implementation for that broker.
==Persistent Publish/Subscribe==
Communication between applications follows a publish-subscribe model, where data is broadcast through shared destinations (topics).  Communicating through shared topics rather than point-to-point queues reduces coupling between microservices.
==Consumer Group==
The microservice model relies on the ability of an application to scale horizontally by increasing the number of "equivalent" components that are supposed to perform the same function and thus spread the load. However, when doing so, different instances of the service are placed in a competing consumer relationship, where only one of the instances is expected to handle a given message. Spring Cloud Stream models this behavior through the concept of a consumer group. Each consumer binding can use the <tt>spring.cloud.stream.bindings.<channelName>.group</tt> property to specify a group name. All groups that subscribe to a given destination receive a copy of published data, but only one member of each group receives a given message from that destination. By default, when a group is not specified, Spring Cloud Stream assigns the application to an anonymous and independent single-member consumer group that is in a publish-subscribe relationship with all other consumer groups. In general, it is preferable to always specify a consumer group when binding an application to a given destination. When scaling up a Spring Cloud Stream application, you must specify a consumer group for each of its input bindings. Doing so prevents the application’s instances from receiving duplicate message.
Consumer group subscriptions are durable: a binder implementation ensures that group subscriptions are persistent and that, once at least one subscription for a group has been created, the group receives messages, even if they are sent while all applications in the group are stopped.
==Consumer Types==
===Message-Driven===
Message-driven consumer are also known as asynchronous consumers.
===Polled===
Polled consumer are also known as synchronous consumers. A synchronous controller allows controlling the rate at which messages are consumed. <font color=darkgray>TODO: https://docs.spring.io/spring-cloud-stream/docs/Elmhurst.RELEASE/reference/htmlsingle/#spring-cloud-streams-overview-using-polled-consumers</font>
==Partition==
Spring Cloud Stream offers the possibility to partition the physical communication medium provided by a broker topic into multiple partitions. Partitioning can be used whether the broker itself is naturally partitioned (Kafka) or not (RabbitMQ). Partitioning is a critical concept in stateful processing, where it is critical (for either performance or consistency reasons) to ensure that all related data is processed together. To set up a partition, both the data-producing and the data-consuming ends must be configured similarly.
<font color=darkgray>TODO https://docs.spring.io/spring-cloud-stream/docs/Elmhurst.RELEASE/reference/htmlsingle/#spring-cloud-stream-overview-partitioning</font>
==Content Type Negotiation==
<font color=darkgray>TODO https://docs.spring.io/spring-cloud-stream/docs/Elmhurst.RELEASE/reference/htmlsingle/#content-type-management</font>
==Schema Evolution Support==
<font color=darkgray>TODO https://docs.spring.io/spring-cloud-stream/docs/Elmhurst.RELEASE/reference/htmlsingle/#schema-evolution</font>
=Programming Model=
==Destination Binder==
A destination binder is the implementation behind a [[#Binder|binder]] abstraction: the component responsible with providing integration with external messaging systems. The destination binders are responsible for connectivity, message routing to and from producers and consumers, data type conversion, etc. While binders handle most of their responsibility in a transparent manner, they still require some minimal configuration.
===Binder Detection===
<font color=darkgray>Spring Cloud Stream automatically detects and uses a binder found on the classpath. More: https://docs.spring.io/spring-cloud-stream/docs/Elmhurst.RELEASE/reference/htmlsingle/#_binder_detection, multiple binders on the classpath: https://docs.spring.io/spring-cloud-stream/docs/Elmhurst.RELEASE/reference/htmlsingle/#multiple-binders</font>
==Destination Binding==
The destination binding is the bridge between the external messaging system and application-provided Producers and Consumers of messages. The Producers and Consumers are created by the [[#Destination_Binder|destination binders]]. A destination binding is defined by the [[@EnableBinding]] annotations. A typical Spring Cloud Stream application declares a configuration class annotated with [[@EnableBinding]] (note that [[@EnableBinding]] is also a [[@Configuration]]).
==Message==
The canonical data structure used by Producers and Consumers to communicate with Destination Binders, and thus other applications via external messaging systems.
==Example==
<syntaxhighlight lang='java'>
@SpringBootApplication
public class ExampleApplication {
  ...
  @StreamListener(Processor.INPUT)
  @SendTo(Processor.OUTPUT)
  public String handle(String value) {
      System.out.println("Received: " + value);
      return value.toUpperCase();
  }
}
@EnableBinding(Processor.class)
public class ExampleConfiguration {
}
</syntaxhighlight>
==Interfaces==
Spring Cloud Stream provides binding interfaces for typical message exchange contracts. These interfaces can be used to parameterize the [[@EnableBinding]] annotation.
===Sink===
Identifies the contract for message consumer: input and no output. Provides a destination from which the message is consumed.
<syntaxhighlight lang='java'>
public interface Sink {
  String INPUT = "input";
  @Input(Sink.INPUT)
  SubscribableChannel input();
}
</syntaxhighlight>
===Source===
Identifies the contract for message producer. Provides a destination to send messages to.
<syntaxhighlight lang='java'>
public interface Source {
  String OUTPUT = "output";
  @Output(Source.OUTPUT)
  MessageChannel output();
}
</syntaxhighlight>
===Processor===
Both a [[#Sink|sink]] and a [[#Source|source]].
===Custom Binding Interface===
Custom binding interfaces can be defined and bindable components annotated with [[@Input]] and [[@Output]].
==Channel==
===Input Channel===
An input channel funnels received messages into the application. The implementation is transparently created by Spring Cloud Stream and injected into the application.
===Output Channel===
Published messages leave the application via output channels. An input channel funnels received messages into the application. The implementation is transparently created by Spring Cloud Stream and injected into the application.
==Destination==
The destination can be a queue, topic, or others.
==Message Handler==
<font color=darkgray>Is this the same thing as the handler method?</font>
<syntaxhighlight lang='java'>
@StreamListener(Sink.INPUT)
public void handle(...) {
...
}
</syntaxhighlight>
Also see [[@StreamListener]].
==Annotations==
* [[@EnableBinding]] - provides immediate connectivity to a message broker.
* [[@StreamListener]] - annotates a [[#Message_Handler|handler method]] to receive events for stream processing.
* [[@Input]]
* [[@Output]]
==Error Handling==
In presence of errors, Spring Cloud Stream attempts to retry message handling a number of times with [[Spring Retry#RetryTemplate|Spring Retry RetryTemplate]]. Over a certain error threshold, the exceptions thrown by the message handlers are propagated back to the [[#Binder|binder]]. The binder either forwards the error to the application, by invoking a custom error handler ([[#Application_Error_Handling|application error handling]]), or to the messaging system ([[#System_Error_Handling|system error handling]]).
===Application Error Handling===
<font color=darkgray>TODO: https://docs.spring.io/spring-cloud-stream/docs/Elmhurst.RELEASE/reference/htmlsingle/#_application_error_handling</font>
===System Error Handling===
<font color=darkgray>TODO https://docs.spring.io/spring-cloud-stream/docs/Elmhurst.RELEASE/reference/htmlsingle/#_system_error_handling</font>
=Examples=
{{External|[https://github.com/spring-cloud/spring-cloud-stream-samples GitHub Spring Cloud Stream Sample Applications]}}
=Configuration=
<font color=darkgray>Binder configuration: https://docs.spring.io/spring-cloud-stream/docs/Elmhurst.RELEASE/reference/htmlsingle/#_binder_configuration_properties
org.springframework.cloud.stream.config.type
org.springframework.cloud.stream.config.inheritEnvironment
org.springframework.cloud.stream.config.environment
org.springframework.cloud.stream.config.defaultCandidate
spring.cloud.stream.bindings.input.destination
Note that in function of the dynamic binding at boot, the destination can be a Kafka topic, a RabbitMQ exchange, etc.
https://docs.spring.io/spring-cloud-stream/docs/Elmhurst.RELEASE/reference/htmlsingle/#_configuration_options
</font>
=Binder Visualization and Control, Health and Metrics=
Spring Cloud Stream supports visualization and control of the Bindings through Actuator endpoints. <font color=darkgray>TODO https://docs.spring.io/spring-cloud-stream/docs/Elmhurst.RELEASE/reference/htmlsingle/#_binding_visualization_and_control
TODO https://docs.spring.io/spring-cloud-stream/docs/Elmhurst.RELEASE/reference/htmlsingle/#_health_indicator
TODO https://docs.spring.io/spring-cloud-stream/docs/Elmhurst.RELEASE/reference/htmlsingle/#spring-cloud-stream-overview-metrics-emitter
</font>
=Reactive Programming Support=
<font color=darkgray>TODO: https://docs.spring.io/spring-cloud-stream/docs/Elmhurst.RELEASE/reference/htmlsingle/#spring-cloud-stream-overview-reactive-programming-support</font>
=Testing=
<font color=darkgray>TODO: https://docs.spring.io/spring-cloud-stream/docs/Elmhurst.RELEASE/reference/htmlsingle/#_testing</font>

Revision as of 02:05, 24 October 2018

External

Internal

Overview

Concepts

Programming Model

Configuration

Retrieved from "https://kb.novaordis.com/index.php?title=Spring_Cloud_Stream&oldid=50629"