Resilience4j Circuit Breaker: Ensuring System Stability

Explore how Resilience4j Circuit Breaker guarantees system reliability, stability, and uptime. Gain insights into how this fault-tolerance pattern prevents cascading failures through clear examples and configurable settings.

1. Introduction

In our previous articles, we explored Resilience4j, a lightweight fault-tolerance library for Java applications. We covered the Retry pattern and the Rate Limiter pattern and how we can implement the same in Spring Boot applications.

In this article, we’ll dive into another essential fault-tolerance pattern: the Circuit Breaker pattern, and how we can implement the same using the Resilience4j circuit breaker library.

2. What is a Circuit Breaker?

A circuit breaker is a crucial pattern in distributed systems that helps protect downstream services from being overwhelmed by faulty or slow upstream services. It is a design pattern that helps prevent cascading failures and improve the overall resilience of an application. It acts as a protective layer between a client and a remote service or component, preventing the client from repeatedly sending requests to a failing service.

When a service or component fails or becomes unresponsive, it can lead to a cascade of failures throughout the system, resulting in degraded performance or complete system outages. The circuit breaker pattern mitigates this issue by temporarily isolating the failing service, allowing it to recover, and preventing the client from wasting resources on requests that are likely to fail.

Imagine a circuit breaker in your home – when there’s an electrical overload, it trips to prevent damage. Similarly, in software, a circuit breaker prevents further calls to a failing service until it recovers.

3. Types of Circuit Breakers

There are two primary types of circuit breakers:

• Time-based Circuit Breaker: Opens the circuit if failures occur within a specified time window.
• Count-based Circuit Breaker: Opens the circuit if a certain number of failures occur within a specified window.

4. What is Resilience4j Circuit Breaker?

Resilience4j Circuit Breaker is a lightweight Java library that implements the Circuit Breaker pattern. It helps developers build resilient applications by managing failures and preventing them from affecting the entire system.

5. Key Features of Resilience4j Circuit Breaker

• State Management: Resilience4j Circuit Breaker manages the three states of the circuit breaker pattern: Closed, Open, and Half-Open.
• Failure Rate Monitoring: It monitors the failure rate of calls to the protected service within a sliding time window or based on the number of consecutive failures/successes.
• Configurable Thresholds: You can configure the failure rate threshold, wait duration in the open state, permitted number of calls in the half-open state, and sliding window size.
• Exception Handling: You can specify which exceptions should be recorded as failures.
• Metrics and Monitoring: Resilience4j provides monitoring capabilities for tracking circuit breaker state transitions, failure rates, and other metrics.
• Event Listeners: You can register event listeners to be notified of circuit breaker state changes and other events.

6. How Does Resilience4j Circuit Breaker Work?

The Circuit Breaker operates based on a finite state machine with three primary states: CLOSED, OPEN, and HALF_OPEN.

How does resilience4j circuit breaker work?

1. CLOSED State:
   • In this state, the Circuit Breaker allows calls to the downstream service as usual.
   • It continuously monitors the failure rate of these calls.
   • If the failure rate remains below a predefined threshold, the Circuit Breaker stays in the CLOSED state, indicating that the downstream service is healthy and operational.
2. OPEN State:
   • If the failure rate of calls to the downstream service exceeds the configurable threshold, the Circuit Breaker transitions to the OPEN state.
   • In the OPEN state, the Circuit Breaker stops allowing any calls to the downstream service.
   • This prevents further strain on the failing service and helps in avoiding cascading failures throughout the system.
   • The OPEN state acts as a protective mechanism, giving the downstream service time to recover without being overwhelmed by requests.
3. HALF_OPEN State:
   • After a specified time, the Circuit Breaker transitions from the OPEN state to the HALF_OPEN state.
   • In the HALF_OPEN state, the Circuit Breaker allows a limited number of test calls to the downstream service.
   • These test calls serve to check if the downstream service has recovered and is now able to handle requests without failing.
   • If these test calls are successful and the service is deemed healthy again, the Circuit Breaker transitions back to the CLOSED state, allowing normal operation.
   • However, if the test calls continue to fail, the Circuit Breaker returns to the OPEN state, indicating that the downstream service is still experiencing issues and needs more time to recover.

Additional States:

• DISABLED State: This state disables the Circuit Breaker, allowing all calls to pass through regardless of the failure rate.
• FORCED_OPEN State: This state forces the Circuit Breaker into the OPEN state, bypassing the normal failure rate threshold checks. It’s useful for manually triggering the Circuit Breaker during maintenance or testing scenarios.

7. Setting Up Resilience4j Circuit Breaker in Spring Boot

To use Resilience4j Circuit Breaker in your Spring Boot application, follow these steps:

Step 1: Add Resilience4j Dependency

Include the Resilience4j dependency in your Spring Boot project.

For Spring Boot 3, add resilience4j-spring-boot3 dependency in pom.xml of your application. The latest version of dependency can be obtained from here.

<dependency>
    <groupId>io.github.resilience4j</groupId>
    <artifactId>resilience4j-spring-boot3</artifactId>
    <version>{resilience4j-spring-boot3-version}</version>
</dependency>

For Spring Boot 2, add resilience4j-spring-boot2 dependency in pom.xml of your application. The latest version of dependency can be obtained from here.

<dependency>
    <groupId>io.github.resilience4j</groupId>
    <artifactId>resilience4j-spring-boot2</artifactId>
    <version>{resilience4j-spring-boot2-version}</version>
</dependency>

Also, add Spring Boot Actuator and Spring Boot AOP dependencies. Spring Boot Actuator dependency is optional but it can be useful for viewing the circuit breaker metrics and Spring Boot AOP dependency is mandatory or else the circuit breaker will not work.

<dependency>
	<groupId>org.springframework.boot</groupId>
	<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
	<groupId>org.springframework.boot</groupId>
	<artifactId>spring-boot-starter-aop</artifactId>
</dependency>

Step 2: Configure Circuit Breaker Instances

Define circuit breaker instances with specific configurations.

Example Configuration:

resilience4j.circuitbreaker:
  instances:
    backendA:
      registerHealthIndicator: true
      slidingWindowType: COUNT_BASED
      slidingWindowSize: 10
      minimumNumberOfCalls: 10
      permittedNumberOfCallsInHalfOpenState: 3
      automaticTransitionFromOpenToHalfOpenEnabled: true
      waitDurationInOpenState: 5s
      failureRateThreshold: 50
      eventConsumerBufferSize: 10
      recordExceptions:
        - org.springframework.web.client.HttpServerErrorException
      ignoreExceptions:
        - com.bootcamptoprod.circuit.breaker.exception.MovieNotFoundException
      recordFailurePredicate: com.bootcamptoprod.circuit.breaker.predicate.RecordFailurePredicate
      ignoreExceptionPredicate: com.bootcamptoprod.circuit.breaker.predicate.IgnoreExceptionPredicate

Let’s understand these configuration options:

Step 3: Annotate Methods with @CircuitBreaker Annotation

Annotate the methods that require circuit breaker logic with the @CircuitBreaker annotation and specify the circuit breaker instance name created in Step 2.

@CircuitBreaker(name = "backendA")
public Movie getMovieDetails(String movieId) {
    return fetchMovieDetails(movieId);
}

8. Understanding Sliding Window Size vs Minimum Number of Calls

In Resilience4j’s CircuitBreaker configuration, the sliding window size and minimum number of calls serve distinct roles. Here’s a simple breakdown:

• Sliding Window Size: This defines how many recent calls are considered when calculating metrics like failure rate and slow call rate. For example, with a sliding window size of 5 and a COUNT_BASED circuit breaker, the circuit breaker looks at the last 5 calls.
• Minimum Number of Calls: This sets the threshold for when the Circuit Breaker starts considering the metrics. For instance, if set to 3, the circuit breaker begins assessing metrics after 3 calls have been made within the sliding window.

So, if you make 3 calls where 2 are successful and 1 fails within the sliding window of 5 calls, the Circuit Breaker will start monitoring and potentially trigger a state transition based on the failure rate and other configured thresholds.

9. Spring Boot Resilience4j Circuit Breaker Examples

Imagine building a movie service website where customers can search for and discover movies. To ensure reliable and responsive movie searches, we’ll explore the various configuration options and practical examples related to the Resilience4j Circuit Breaker module.

a. Count Based Circuit Breaker

resilience4j.circuitbreaker:
  instances:
    countBasedCircuitBreaker:
      registerHealthIndicator: true
      slidingWindowType: COUNT_BASED
      slidingWindowSize: 5
      minimumNumberOfCalls: 5
      permittedNumberOfCallsInHalfOpenState: 3
      automaticTransitionFromOpenToHalfOpenEnabled: true
      waitDurationInOpenState: 5000
      failureRateThreshold: 60
      

@CircuitBreaker(name = "countBasedCircuitBreaker")
public Movie getMovieDetailsWithCountBasedCircuitBreaker(String movieId) {
    return fetchMovieDetails(movieId);
}

This example sets up a Count-Based Circuit Breaker named “countBasedCircuitBreaker.” It monitors the last 5 calls and requires a minimum of 5 calls to start evaluating failures. If the failure rate surpasses 60%, the Circuit Breaker opens, preventing additional calls and throwing a CallNotPermittedException. It also automatically transitions from the open to the half-open state after 5 seconds due to the automaticTransitionFromOpenToHalfOpenEnabled setting.

Here’s a breakdown of the configuration parameters and what they mean:

1. registerHealthIndicator: true – Indicates whether to register a health indicator for the Circuit Breaker instance.
2. slidingWindowType: COUNT_BASED – Specifies that the sliding window type is count-based.
3. slidingWindowSize: 5 – Sets the size of the sliding window to 5, meaning it will monitor the last 5 calls.
4. minimumNumberOfCalls: 5 – Specifies that at least 5 calls must be made before the Circuit Breaker starts monitoring for failures.
5. permittedNumberOfCallsInHalfOpenState: 3 – Sets the maximum number of permitted calls in the half-open state to 3 before transitioning back to open.
6. automaticTransitionFromOpenToHalfOpenEnabled: true – Enables automatic transition from open to half-open state once the waitDurationInOpenState has passed.
7. waitDurationInOpenState: 5000 – Sets the wait duration in open state to 5000 milliseconds (5 seconds) before transitioning to half-open.
8. failureRateThreshold: 60 – Sets the failure rate threshold to 60%, meaning if the failure rate reaches or exceeds 60%, the Circuit Breaker will open.

The @CircuitBreaker annotation in the code snippet marks the method getMovieDetailsWithCountBasedCircuitBreaker to be wrapped by the Circuit Breaker named “countBasedCircuitBreaker.” This means that the Circuit Breaker will monitor and manage calls to the fetchMovieDetails method based on the configuration provided.

When the Count-Based Circuit Breaker named “countBasedCircuitBreaker” opens and does not allow further calls, it throws an io.github.resilience4j.circuitbreaker.CallNotPermittedException with the message CircuitBreaker 'countBasedCircuitBreaker' is OPEN and does not permit further calls. This exception indicates that the Circuit Breaker has reached its open state due to exceeding the configured failure rate threshold, and it is temporarily blocking any new calls to the protected method or service.

Count Based Circuit Breaker Example

b. Time Based Circuit Breaker

resilience4j.circuitbreaker:
  instances:
    timeBasedCircuitBreaker:
      registerHealthIndicator: true
      slidingWindowType: TIME_BASED
      slidingWindowSize: 60
      minimumNumberOfCalls: 5
      permittedNumberOfCallsInHalfOpenState: 3
      automaticTransitionFromOpenToHalfOpenEnabled: true
      waitDurationInOpenState: 5000
      failureRateThreshold: 60
      

@CircuitBreaker(name = "timeBasedCircuitBreaker")
public Movie getMovieDetailsWithTimedBasedCircuitBreaker(String movieId) {
    return fetchMovieDetails(movieId);
}

This example sets up a Time-Based Circuit Breaker named “timeBasedCircuitBreaker” to monitor calls over a 60-second sliding window. It opens if the failure rate exceeds 60%, blocking new calls and triggering a CallNotPermittedException. It also automatically transitions from the open to the half-open state after 5 seconds due to the automaticTransitionFromOpenToHalfOpenEnabled setting.

Here’s a breakdown of the configuration parameters and what they mean:

1. registerHealthIndicator: true – Indicates whether to register a health indicator for the Circuit Breaker instance.
2. slidingWindowType: TIME_BASED – Specifies that the sliding window type is time-based, meaning it monitors calls made within the last specified time period.
3. slidingWindowSize: 60 – Sets the size of the sliding window to 60 seconds, monitoring calls made within the last minute.
4. minimumNumberOfCalls: 5 – Specifies that at least 5 calls must be made before the Circuit Breaker starts monitoring for failures.
5. permittedNumberOfCallsInHalfOpenState: 3 – Sets the maximum number of permitted calls in the half-open state to 3 before transitioning back to open.
6. automaticTransitionFromOpenToHalfOpenEnabled: true – Enables automatic transition from open to half-open state once the waitDurationInOpenState has passed.
7. waitDurationInOpenState: 5000 – Sets the wait duration in open state to 5000 milliseconds (5 seconds) before transitioning to half-open.
8. failureRateThreshold: 60 – Sets the failure rate threshold to 60%, meaning if the failure rate reaches or exceeds 60%, the Circuit Breaker will open.

The @CircuitBreaker annotation in the code snippet marks the method getMovieDetailsWithTimeBasedCircuitBreaker to be wrapped by the Time-Based Circuit Breaker named “timeBasedCircuitBreaker.” This configuration implies that the Circuit Breaker will monitor and manage calls to the fetchMovieDetails method based on the specified time window and failure rate threshold.

When the Time-Based Circuit Breaker named “timeBasedCircuitBreaker” opens and does not allow further calls, it throws an io.github.resilience4j.circuitbreaker.CallNotPermittedException with the message CircuitBreaker 'timeBasedCircuitBreaker' is OPEN and does not permit further calls. This exception indicates that the Circuit Breaker has reached its open state due to exceeding the configured failure rate threshold, and it is temporarily blocking any new calls to the protected method or service.

Time Based Circuit Breaker Example

c. Circuit Breaker on Specific Exceptions and Ignoring Others

resilience4j.circuitbreaker:
  instances:
    circuitBreakerOnException:
      registerHealthIndicator: true
      slidingWindowType: COUNT_BASED
      slidingWindowSize: 5
      minimumNumberOfCalls: 5
      permittedNumberOfCallsInHalfOpenState: 3
      automaticTransitionFromOpenToHalfOpenEnabled: true
      waitDurationInOpenState: 5000
      failureRateThreshold: 60
      recordExceptions:
        - org.springframework.web.client.HttpClientErrorException
      ignoreExceptions:
        - com.bootcamptoprod.circuit.breaker.exception.MovieNotFoundException
        

@CircuitBreaker(name = "circuitBreakerOnException")
public Movie getMovieDetailsWithCircuitBreakerOnException(String movieId) {
    return fetchMovieDetails(movieId);
}

This configuration establishes a Circuit Breaker named “circuitBreakerOnException.” It monitors the last 5 calls and starts evaluating failures after 5 calls. If the failure rate exceeds 60% or if there are 3 or more failures due to org.springframework.web.client.HttpClientErrorException, the Circuit Breaker opens, preventing additional calls and throwing a CallNotPermittedException. Additionally, it automatically transitions from the open to the half-open state after 5 seconds due to the automaticTransitionFromOpenToHalfOpenEnabled setting.

The Circuit Breaker configuration specifies that org.springframework.web.client.HttpClientErrorException is recorded as a failure, while com.bootcamptoprod.circuit.breaker.exception.MovieNotFoundException is ignored. Any other exceptions not listed in these configurations will also be considered as a success by the Circuit Breaker.

Here’s a breakdown of the configuration parameters and what they mean:

1. registerHealthIndicator: true – Indicates whether to register a health indicator for the Circuit Breaker instance.
2. slidingWindowType: COUNT_BASED – Specifies that the sliding window type is count-based.
3. slidingWindowSize: 5 – Sets the size of the sliding window to 5, meaning it will monitor the last 5 calls.
4. minimumNumberOfCalls: 5 – Specifies that at least 5 calls must be made before the Circuit Breaker starts evaluating failures.
5. permittedNumberOfCallsInHalfOpenState: 3 – Sets the maximum number of permitted calls in the half-open state to 3 before transitioning back to open.
6. automaticTransitionFromOpenToHalfOpenEnabled: true – Enables automatic transition from open to half-open state once the waitDurationInOpenState has passed.
7. waitDurationInOpenState: 5000 – Sets the wait duration in open state to 5000 milliseconds (5 seconds) before transitioning to half-open.
8. failureRateThreshold: 60 – Sets the failure rate threshold to 60%, meaning if the failure rate reaches or exceeds 60%, the Circuit Breaker will open.
9. recordExceptions: Specifies the exceptions that are recorded as failures and increase the failure rate.
10. ignoreExceptions: Specifies the exceptions that are ignored and do not count as failures.

The @CircuitBreaker annotation in the code snippet applies to the method getMovieDetailsWithCircuitBreakerOnException. It is managed by the Circuit Breaker named “circuitBreakerOnException.” This means the Circuit Breaker oversees calls to fetchMovieDetails, checking the last 5 calls for failures. If the failure rate exceeds 60% or if there are 3 or more failures due to org.springframework.web.client.HttpClientErrorException, the Circuit Breaker opens to prevent additional calls.

When the Circuit Breaker named “circuitBreakerOnException” opens and does not allow further calls, it throws an io.github.resilience4j.circuitbreaker.CallNotPermittedException with the message CircuitBreaker 'circuitBreakerOnException' is OPEN and does not permit further calls. This exception indicates that the Circuit Breaker has reached its open state due to exceeding the configured failure rate threshold, and it is temporarily blocking any new calls to the protected method or service.

Circuit Breaker On Exception Example

d. Circuit Breaker with Record Failure Predicate

resilience4j.circuitbreaker:
  instances:
    circuitBreakerWithRecordFailurePredicate:
      registerHealthIndicator: true
      slidingWindowType: COUNT_BASED
      slidingWindowSize: 5
      minimumNumberOfCalls: 5
      permittedNumberOfCallsInHalfOpenState: 3
      automaticTransitionFromOpenToHalfOpenEnabled: true
      waitDurationInOpenState: 5000
      failureRateThreshold: 60
      recordFailurePredicate: com.bootcamptoprod.circuit.breaker.predicate.RecordFailurePredicate
      

public class RecordFailurePredicate implements Predicate<Throwable> {
    @Override
    public boolean test(Throwable throwable) {
        System.out.println("Record failure predicate is called.");
        return throwable instanceof MovieNotFoundException;
    }
}

@CircuitBreaker(name = "circuitBreakerWithRecordFailurePredicate")
public Movie getMovieDetailsWithCircuitBreakerRecordFailurePredicate(String movieId) {
    return fetchMovieDetails(movieId);
}

This configuration sets up a Circuit Breaker named “circuitBreakerWithRecordFailurePredicate.” It monitors the last 5 calls and starts evaluating failures after 5 calls. If the failure rate exceeds 60% or if there are 3 or more failures due to MovieNotFoundException, the Circuit Breaker opens, preventing additional calls and throwing a CallNotPermittedException. Additionally, it automatically transitions from the open to the half-open state after 5 seconds due to the automaticTransitionFromOpenToHalfOpenEnabled setting. The RecordFailurePredicate class specifies the conditions under which failures are recorded by checking if the thrown exception is an instance of MovieNotFoundException.

Here’s a breakdown of the configuration parameters and what they mean:

1. slidingWindowType: COUNT_BASED – Specifies that the sliding window type is count-based.
2. slidingWindowSize: 5 – Sets the size of the sliding window to 5, meaning it will monitor the last 5 calls.
3. minimumNumberOfCalls: 5 – Specifies that at least 5 calls must be made before the Circuit Breaker starts evaluating failures.
4. permittedNumberOfCallsInHalfOpenState: 3 – Sets the maximum number of permitted calls in the half-open state to 3 before transitioning back to open.
5. automaticTransitionFromOpenToHalfOpenEnabled: true – Enables automatic transition from open to half-open state once the waitDurationInOpenState has passed.
6. waitDurationInOpenState: 5000 – Sets the wait duration in open state to 5000 milliseconds (5 seconds) before transitioning to half-open.
7. failureRateThreshold: 60 – Sets the failure rate threshold to 60%, meaning if the failure rate reaches or exceeds 60%, the Circuit Breaker will open.
8. recordFailurePredicate: com.bootcamptoprod.circuit.breaker.predicate.RecordFailurePredicate – Specifies the predicate that determines which exceptions are recorded as failures.

The @CircuitBreaker annotation in the code snippet marks the method getMovieDetailsWithCircuitBreakerRecordFailurePredicate to be managed by the Circuit Breaker named “circuitBreakerWithRecordFailurePredicate.” This configuration means that the Circuit Breaker monitors calls to fetchMovieDetails, checking the last 5 calls for failures. If the failure rate exceeds 60% or if there are 3 or more failures due to MovieNotFoundException, the Circuit Breaker opens to prevent additional calls.

The RecordFailurePredicate class specifies the conditions under which failures are recorded by checking if the thrown exception is an instance of MovieNotFoundException.

Circuit Breaker With Record Failure Predicate Example

e. Circuit Breaker with Ignore Exception Predicate

resilience4j.circuitbreaker:
  instances:
    circuitBreakerWithIgnoreExceptionPredicate:
      registerHealthIndicator: true
      slidingWindowType: COUNT_BASED
      slidingWindowSize: 5
      minimumNumberOfCalls: 5
      permittedNumberOfCallsInHalfOpenState: 3
      automaticTransitionFromOpenToHalfOpenEnabled: true
      waitDurationInOpenState: 5000
      failureRateThreshold: 60
      ignoreExceptionPredicate: com.bootcamptoprod.circuit.breaker.predicate.IgnoreExceptionPredicate
      

public class IgnoreExceptionPredicate implements Predicate<Throwable> {
    @Override
    public boolean test(Throwable throwable) {
        System.out.println("Exception predicate is called.");
        return throwable instanceof MovieNotFoundException;
    }
}

@CircuitBreaker(name = "circuitBreakerWithIgnoreExceptionPredicate")
public Movie getMovieDetailsWithCircuitBreakerIgnoreExceptionPredicate(String movieId) {
    return fetchMovieDetails(movieId);
}

The configuration example sets up a Circuit Breaker named “circuitBreakerWithIgnoreExceptionPredicate” to monitor calls using a count-based sliding window with a size of 5. It requires a minimum of 5 calls before evaluating failures and permits up to 3 calls in the half-open state. The Circuit Breaker automatically transitions from the open to the half-open state after 5000 milliseconds (5 seconds) of being open. If the failure rate exceeds 60%, the Circuit Breaker opens.

The IgnoreExceptionPredicate class specifies conditions under which exceptions are ignored. In this case, if the thrown exception is an instance of MovieNotFoundException, it will not be counted as a failure when evaluating the failure rate threshold. However, other exceptions will still be considered for failure rate calculations.

Here’s a breakdown of the configuration parameters and what they mean:

1. slidingWindowType: COUNT_BASED – Specifies that the sliding window type is count-based.
2. slidingWindowSize: 5 – Sets the size of the sliding window to 5, meaning it will monitor the last 5 calls.
3. minimumNumberOfCalls: 5 – Specifies that at least 5 calls must be made before the Circuit Breaker starts evaluating failures.
4. permittedNumberOfCallsInHalfOpenState: 3 – Sets the maximum number of permitted calls in the half-open state to 3 before transitioning back to open.
5. automaticTransitionFromOpenToHalfOpenEnabled: true – Enables automatic transition from open to half-open state once the waitDurationInOpenState has passed.
6. waitDurationInOpenState: 5000 – Sets the wait duration in open state to 5000 milliseconds (5 seconds) before transitioning to half-open.
7. failureRateThreshold: 60 – Sets the failure rate threshold to 60%, meaning if the failure rate reaches or exceeds 60%, the Circuit Breaker will open.
8. ignoreExceptionPredicate: com.bootcamptoprod.circuit.breaker.predicate.IgnoreExceptionPredicate – Specifies the predicate that determines which exceptions are ignored by the Circuit Breaker and do not count as failures.

Overall, this configuration ensures that MovieNotFoundException exceptions are ignored when calculating the failure rate, providing more control over how exceptions are handled within the Circuit Breaker.

Circuit Breaker With Ignore Exception Predicate Example

f. Circuit Breaker for Slow Calls

resilience4j.circuitbreaker:
  instances:
    circuitBreakerForSlowCalls:
      registerHealthIndicator: true
      slidingWindowType: COUNT_BASED
      slidingWindowSize: 5
      minimumNumberOfCalls: 5
      permittedNumberOfCallsInHalfOpenState: 3
      automaticTransitionFromOpenToHalfOpenEnabled: true
      waitDurationInOpenState: 5000
      slowCallRateThreshold: 60
      slowCallDurationThreshold: 1000
      

@CircuitBreaker(name = "circuitBreakerForSlowCalls")
public Movie getMovieDetailsWithCircuitBreakerForSlowCalls(String movieId) {
    return fetchMovieDetails(movieId);
}

This configuration sets up a Circuit Breaker named “circuitBreakerForSlowCalls” to monitor slow calls based on the specified criteria. The Circuit Breaker considers a call as slow when the call duration exceeds 1000 milliseconds (slowCallDurationThreshold). If the slow call rate exceeds 60%, the Circuit Breaker will open, throwing a CallNotPermittedException for subsequent calls. Additionally, it automatically transitions from the open to the half-open state after 5 seconds due to the automaticTransitionFromOpenToHalfOpenEnabled setting.

Here’s a breakdown of the configuration parameters and what they mean:

1. slidingWindowType: COUNT_BASED – Specifies that the sliding window type is count-based.
2. slidingWindowSize: 5 – Sets the size of the sliding window to 5, meaning it will monitor the last 5 calls.
3. minimumNumberOfCalls: 5 – Specifies that at least 5 calls must be made before the Circuit Breaker starts evaluating failures.
4. permittedNumberOfCallsInHalfOpenState: 3 – Sets the maximum number of permitted calls in the half-open state to 3 before transitioning back to open.
5. automaticTransitionFromOpenToHalfOpenEnabled: true – Enables automatic transition from open to half-open state once the waitDurationInOpenState has passed.
6. waitDurationInOpenState: 5000 – Sets the wait duration in open state to 5000 milliseconds (5 seconds) before transitioning to half-open.
7. slowCallRateThreshold: 60 – This property sets the threshold for the slow call rate at 60%. If the percentage of slow calls goes above this threshold, the Circuit Breaker will open.
8. slowCallDurationThreshold: 1000 – This property defines the duration threshold for considering a call as slow. Calls with a duration greater than 1000 milliseconds will be categorized as slow, potentially contributing to the opening of the Circuit Breaker.

Overall, these configuration properties provide precise control over slow call monitoring within the Circuit Breaker. The slowCallRateThreshold property sets the threshold at 60%, determining when the Circuit Breaker should open based on the percentage of slow calls. On the other hand, the slowCallDurationThreshold property specifies the duration threshold of 1000 milliseconds, defining what constitutes a slow call. Together, they enhance the management of slow calls and contribute to the effective operation of the Circuit Breaker.

Circuit Breaker For Slow Calls Example

10. Creating and Reusing Default Circuit Breaker Configurations

You can simplify circuit breaker configuration by creating a default template that can be easily reused across multiple circuit breaker instances for consistent and efficient resilience.

Example:

resilience4j.circuitbreaker:
  configs:
    default:
      registerHealthIndicator: true
      slidingWindowType: TIME_BASED
      slidingWindowSize: 60
      minimumNumberOfCalls: 5
      permittedNumberOfCallsInHalfOpenState: 3
      automaticTransitionFromOpenToHalfOpenEnabled: true
      waitDurationInOpenState: 5s
      failureRateThreshold: 50
  instances:
    backendA:
      baseConfig: default
    backendB:
      baseConfig: default
      slidingWindowType: COUNT_BASED

The circuit breaker configuration provided above uses a default template that can be reused for multiple instances with the ability to override specific properties. Here’s a breakdown of the configuration:

• default: The name of the default circuit breaker configuration template.
• instances: This section defines the circuit breaker instances.
• backendA and backendB: The names of the circuit breaker instances that will be configured.
• baseConfig: Specifies the base configuration template to inherit from. In this case, both backendA and backendB use the default template.
• slidingWindowType: Overrides the slidingWindowType property for the backendB instance, setting it to the count-based circuit breaker.

This configuration allows you to define a default template with common circuit breaker settings and reuse it for multiple instances. You can also customize specific properties for individual instances as needed, such as overriding the sliding window type for the backendB instance to count based.

11. Resilience4j Circuit Breaker Fallback Method

Resilience4j circuit breaker enables the definition of a fallback method that allows developers to define fallback behaviors for methods associated with Circuit Breakers. When a method protected by a Circuit Breaker fails due to exceptions or exceeds predefined thresholds, the fallback method is invoked as an alternative response mechanism. This ensures graceful handling of errors and provides resilience to applications by offering a backup plan when primary operations encounter issues.

@CircuitBreaker(name = "countBasedCircuitBreaker", fallbackMethod = "fetchMovieDetailsFallbackMethod")
public Movie getMovieDetailsWithFallbackMethod(String movieId) {
    return fetchMovieDetails(movieId);
}

public Movie fetchMovieDetailsFallbackMethod(String movieId, CallNotPermittedException callNotPermittedException) {
    log.info("Fallback method called.");
    log.info("CallNotPermittedException exception message: {}", callNotPermittedException.getMessage());
   return new Movie("Default", "N/A", "N/A", 0.0);
}

In this Resilience4j Circuit Breaker fallback example, the @CircuitBreaker annotation is used to apply circuit breaker functionality to the getMovieDetailsWithFallbackMethod. If the Circuit Breaker is open and a CallNotPermittedException occurs, the fallback method fetchMovieDetailsFallbackMethod is invoked.

The fetchMovieDetailsFallbackMethod is a separate method defined in the same class as the circuit breaker-protected method. It takes the same parameters as the original method, along with the CallNotPermittedException.

Inside the fallback method, you can define custom logic to handle the circuit breaker scenario. In this example, the fallback method logs a message indicating that the fallback has been triggered and logs the exception message. It then returns a default Movie object to provide a fallback response.

This way, when the Circuit Breaker is open, the application gracefully handles the situation by executing the fallback method, allowing you to customize the behavior and provide an appropriate response to the user.

Circuit Breaker With Fallback Method Example

12. Event Listeners for Resilience4j Circuit Breaker

Now, we will understand how to register event listeners for the Resilience4j Circuit Breaker in a Spring application. The event listeners capture different events that occur during the circuit breaking and perform specific actions, such as logging or handling the events accordingly.

Example:

@Autowired
private CircuitBreakerRegistry registry;

....

@CircuitBreaker(name = "countBasedCircuitBreaker")
public Movie getMovieDetailsWithCountBasedCircuitBreaker(String movieId) {
    return fetchMovieDetails(movieId);
}

....

@PostConstruct
public void postConstruct() {
    var eventPublisher = registry.circuitBreaker("countBasedCircuitBreaker").getEventPublisher();
    eventPublisher.onEvent(event -> System.out.println("Count Based Circuit Breaker - On Event. Event Details: " + event));
}

Within the postConstruct method, the eventPublisher is obtained from the CircuitBreakerRegistry for the “countBasedCircuitBreaker” circuit breaker instance.

Next, an event listener is registered using the onEvent() method of the eventPublisher. The onEvent() method takes a lambda expression that defines the action to be performed when any event occurs within the circuit breaker. It provides details about the event that occurred.

For each event, a message is printed to the console, including the event details obtained from the event parameter.

By registering this event listener, you can customize the behavior of your circuit breaker operations and gain visibility into the circuit breaker’s behavior by capturing and handling events.

Resilience4j Circuit Breaker Event Details

13. Programmatically Creating Circuit Breaker Instances

In Resilience4j, you have the flexibility to create circuit breaker instances programmatically. This means you can dynamically configure and customize circuit breakers based on your application’s requirements, giving you fine-grained control over how failures are handled and service availability is managed.

Example:

@Configuration
public class CircuitBreakerConfiguration {

    @Autowired
    private CircuitBreakerRegistry circuitBreakerRegistry;

    @Bean
    public CircuitBreaker circuitBreakerWithCustomConfig() {
        CircuitBreakerConfig customConfig = CircuitBreakerConfig.custom()
                .slidingWindowType(CircuitBreakerConfig.SlidingWindowType.COUNT_BASED)
                .slidingWindowSize(5)
                .minimumNumberOfCalls(5)
                .failureRateThreshold(50)
                .enableAutomaticTransitionFromOpenToHalfOpen()
                .maxWaitDurationInHalfOpenState(Duration.ofMillis(5000))
                .permittedNumberOfCallsInHalfOpenState(3)
                .build();

        return circuitBreakerRegistry.circuitBreaker("customCircuitBreaker", customConfig);
    }
}

@CircuitBreaker(name = "customCircuitBreaker")
public Movie getMovieDetailsWithCustomCircuitBreaker(String movieId) {
    return fetchMovieDetails(movieId);
}

In this code snippet, a Circuit Breaker instance is programmatically created with a custom configuration for fetching movie details. By defining the Circuit Breaker configuration using specific parameters such as the sliding window type, window size, failure rate threshold, and transition settings, you have the flexibility to fine-tune the circuit-breaking behavior according to your application’s requirements.

This approach enables you to create and configure Circuit Breakers programmatically, giving you greater control over how failures are managed and handled. It allows you to adapt the Circuit Breaker behavior based on factors like the nature of the movie details fetching operation and the desired reliability and resilience levels for your application.

Resilience4j Circuit Breaker Custom Config Example

14. Dynamically Modifying Circuit Breaker State

You can update the state of an existing circuit breaker instance in Resilience4j by modifying the configuration dynamically.

@Autowired
private CircuitBreakerRegistry registry;

......

    void updateCircuitBreakerState(String name) {
        io.github.resilience4j.circuitbreaker.CircuitBreaker circuitBreaker = registry.circuitBreaker(name);
        if(name.equalsIgnoreCase("closed")) {
            circuitBreaker.transitionToClosedState();
        } else if(name.equalsIgnoreCase("disabled")) {
            circuitBreaker.transitionToDisabledState();
        } else if(name.equalsIgnoreCase("forced-open")) {
            circuitBreaker.transitionToForcedOpenState();
        } else if(name.equalsIgnoreCase("open")) {
            circuitBreaker.transitionToOpenState();
        } else if(name.equalsIgnoreCase("metrics-only")) {
            circuitBreaker.transitionToMetricsOnlyState();
        } else if(name.equalsIgnoreCase("half-open")) {
            circuitBreaker.transitionToHalfOpenState();
        } else if(name.equalsIgnoreCase("open-for-5-minutes")) {
            circuitBreaker.transitionToOpenStateFor(Duration.ofMinutes(5));
        }
    }

This code snippet contains a method called updateCircuitBreakerState, which takes a name parameter to update the state of a Circuit Breaker dynamically based on the provided name. The method retrieves the Circuit Breaker instance from the CircuitBreakerRegistry based on the given name and transitions it to the specified state, such as Closed, Disabled, Forced Open, Open, Metrics-Only, Half-Open, or Open for a specific duration.

You can create a controller endpoint and call this method to dynamically update the state of a Circuit Breaker based on the provided name, allowing you to control the behavior of the Circuit Breaker through external requests.

15. Actuator Monitoring and Management Endpoints

The actuator endpoints provide valuable information and metrics that can help you monitor and analyze the behavior and performance of your Resilience4j circuit breakers.

1. /actuator/circuitbreakers: This endpoint provides information about all the Circuit Breakers in your application, including their names, states (OPEN, CLOSED, HALF_OPEN), failure rates, slow call rates, and other metrics related to circuit breaking.
2. /actuator/health: This endpoint gives an overview of the health status and other related metrics of all registered circuit breakers, indicating whether they are functioning correctly or experiencing any issues.
3. /actuator/circuitbreakerevents: The endpoint provides a log of events related to circuit breakers, including errors, state transitions, and duration information, allowing developers to monitor and analyze circuit breaker behavior and performance.
4. /actuator/metrics/resilience4j.circuitbreaker.state: This endpoint provides metrics specifically related to the state of Circuit Breakers in your application, showing the number of OPEN, CLOSED, and HALF_OPEN Circuit Breakers. It also allows filtering based on specific circuit breaker names and states such as “closed,” “disabled,” “half_open,” “forced_open,” “open,” and “metrics_only.”
5. /actuator/metrics/resilience4j.circuitbreaker.slow.calls:
   This endpoint displays the count of slow failed calls that exceeded a predefined threshold in Resilience4j circuit breakers.
6. /actuator/metrics/resilience4j.circuitbreaker.slow.call.rate: This endpoint provides the rate of slow calls as measured by Circuit Breakers, giving insights into the frequency and pattern of slow calls occurring in your application.
7. /actuator/metrics/resilience4j.circuitbreaker.not.permitted.calls: This endpoint shows metrics related to calls that were not permitted by Circuit Breakers, usually due to the Circuit Breaker being in the OPEN state.
8. /actuator/metrics/resilience4j.circuitbreaker.failure.rate: This endpoint provides the failure rate metric for different Resilience4j circuit breakers. The failure rate is calculated based on the number of failed calls within the circuit breaker.
9. /actuator/metrics/resilience4j.circuitbreaker.calls:
   This endpoint provides metrics related to the total number of calls made to Circuit Breakers, helping you track the overall usage and load on your Circuit Breakers. You can also filter the details based on the circuit breaker name and call status like ignored, failed and successful.
10. /actuator/metrics/resilience4j.circuitbreaker.buffered.calls: This endpoint provides information about the number of calls that are currently buffered and stored in the ring buffer. These buffered calls are categorized by their status, such as “failed” or “successful,” and are associated with specific circuit breakers.

For circuit breaker health and events information to be displayed properly in actuator endpoints consider adding the below-mentioned properties in application.yml file.

management:
  endpoint:
    health:
      show-details: ALWAYS
  health:
    circuitbreakers:
      enabled: true
  endpoints:
    web:
      exposure:
        include: "*"

resilience4j.circuitbreaker:
  instances:
    countBasedCircuitBreaker:
      registerHealthIndicator: true
      slidingWindowType: COUNT_BASED
      slidingWindowSize: 5
      minimumNumberOfCalls: 5
      permittedNumberOfCallsInHalfOpenState: 3
      automaticTransitionFromOpenToHalfOpenEnabled: true
      waitDurationInOpenState: 5000
      failureRateThreshold: 60
      eventConsumerBufferSize: 10

16. Source Code

The complete source code of the above example can be found here.

Additionally, the Postman collection is available in the GitHub repository under the resources folder.
[src > main > resources > postman > Spring-Boot-Resilience4j-Circuit-Breaker.postman_collection.json]

Things to Consider

When working with Resilience4j CircuitBreaker, it’s important to keep the following considerations in mind:

1. Threshold Configuration: Set appropriate thresholds for failure rate, slow call rate, and other metrics based on your application’s requirements. This includes configuring values like failureRateThreshold, slowCallRateThreshold, slidingWindowSize, and minimumNumberOfCalls.
2. Fallback Strategies: Implement effective fallback strategies to gracefully handle failures and degraded performance. Use fallback methods or default values to provide a meaningful response to users when a service is unavailable.
3. Monitoring and Metrics: Utilize monitoring tools and metrics endpoints provided by Resilience4j (e.g., Actuator endpoints) to monitor the health and behavior of Circuit Breakers in real time. Monitor key metrics such as failure rate, slow call rate, and circuit breaker state transitions.
4. Exception Handling: Define proper exception handling mechanisms to capture and handle specific exceptions that may occur during method execution. Customize exception handling based on the type of errors encountered.
5. Testing and Tuning: Thoroughly test Circuit Breaker behavior under various scenarios, including normal operation, failure conditions, and high load situations. Continuously tune and adjust Circuit Breaker configurations based on performance testing results and production feedback.
6. Documentation and Communication: Maintain clear documentation about Circuit Breaker configurations, fallback strategies, and error handling practices. Communicate Circuit Breaker behavior and expected response patterns to other teams and stakeholders involved in system integration.

FAQs

Conclusion

In conclusion, implementing Circuit Breakers is crucial for building resilient microservices that can withstand failures and maintain system stability. By configuring parameters effectively, monitoring metrics, and integrating with other resilience patterns, developers can enhance fault tolerance and improve overall system reliability.

Learn More