Spring Boot Parallel Calls with RestTemplate and CompletableFuture

1. Introduction

In modern web applications, making parallel HTTP calls is a common requirement to improve performance and efficiency. In this guide, we’ll explore how to make parallel calls using Spring Boot RestTemplate and CompletableFuture. We’ll cover making parallel calls, handling exceptions, configuring timeouts for each task, and setting a global timeout for all tasks. Additionally, we’ll discuss how this approach can be used to implement the scatter-gather design pattern.

2. Why Parallel Calls Are Required?

Parallel calls are essential in web applications to reduce the overall response time by performing multiple tasks simultaneously. When your application needs to fetch data from multiple sources or perform several independent operations, making parallel calls can significantly improve performance. Instead of waiting for each call to complete sequentially, parallel execution allows you to aggregate the results faster.

3. Does Spring Boot Provide Any Out-of-the-Box Solution?

Spring Boot, while powerful, does not provide an out-of-the-box solution specifically for making parallel calls using RestTemplate. However, it does support asynchronous processing with CompletableFuture, which we can leverage to make Spring Boot parallel calls.

4. Understanding the Scatter-Gather Pattern

The scatter-gather pattern involves sending multiple requests (scatter) and gathering the responses to process them together. This pattern is useful when you need to perform several tasks concurrently and then combine the results. Using CompletableFuture.allOf, we can implement this pattern effectively in a Spring Boot application.

5. Setting Up the Spring Boot Application

5.1 Add Required Dependency

First, let’s set up a basic Spring Boot application. Ensure you have the following dependencies in your pom.xml:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-web</artifactId>
</dependency>

5.2 Creating Mock Endpoints and Controller

Instead of calling an external service, we will create mock endpoints within our Spring Boot application and return mock data. This will help us focus on the implementation without relying on external services.

Mock Data POJO

This class represents the structure of the data we will return from our mock endpoints.

public class MockResponse {

    private String id;
    private String message;

    // Getters and setters
}

Mock Controller

This controller simulates external endpoints by returning mock data with a delay and exception.

import com.bootcamptoprod.mock.dto.MockResponse;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
@RequestMapping("/mock")
public class MockController {

    private static final Logger logger = LoggerFactory.getLogger(MockController.class);

    // Simulates mock respose with delay
    @GetMapping("/data/{id}")
    public MockResponse getMockData(@PathVariable String id) throws InterruptedException {
        logger.info("Received a request for mock data with id: {}", id);

        Thread.sleep(3000); // Adding a delay to simulate real-world scenarios

        MockResponse response = new MockResponse();
        response.setId(id);
        response.setMessage("Mock data for ID " + id);
        logger.info("Response sent for mock data with id: {}", id);
        return response;
    }
    
    // Simulates exception response
    @GetMapping("/data/{id}/exception")
    public MockResponse getMockDataException(@PathVariable String id) {
        logger.info("Received a request for mock data with id: {}", id);

        throw new RuntimeException("Something went wrong");
    }
    
}

6. Making Parallel Calls with RestTemplate

We’ll create a service that makes parallel HTTP calls using RestTemplate and CompletableFuture.

6.1 Defining the Service

The ParallelService class is where we define the methods for making parallel HTTP calls. This service uses RestTemplate for the HTTP requests and CompletableFuture to handle asynchronous execution.

import com.bootcamptoprod.mock.dto.MockResponse;
import org.springframework.stereotype.Service;
import org.springframework.web.client.RestTemplate;

import java.util.concurrent.CompletableFuture;
import java.util.concurrent.Executor;
import java.util.concurrent.Executors;

@Service
public class ParallelService {

    private RestTemplate restTemplate;

    public ParallelService() {
        this.restTemplate = new RestTemplate();
    }

    private final Executor executor = Executors.newFixedThreadPool(10);

    // Fetches data from a given URL asynchronously
    public CompletableFuture<MockResponse> fetchData(String url) {
        return CompletableFuture.supplyAsync(() -> restTemplate.getForObject(url, MockResponse.class), executor);
    }
    
}

Explanation: fetchData method

• Purpose: Fetches data asynchronously from the given URL.
• Implementation: Uses CompletableFuture.supplyAsync to perform the HTTP GET request using RestTemplate in a separate thread, leveraging an executor to manage the thread pool.
• Returns: A CompletableFuture that will contain the MockResponse object once the call is complete.

6.2 Making Multiple Parallel Calls

To make multiple parallel HTTP calls, we can define a method in the ParallelService class that uses CompletableFuture.allOf.

public CompletableFuture<List<MockResponse>> fetchAllData() {
        CompletableFuture<MockResponse> call1 = fetchData("http://localhost:8080/mock/data/1");
        CompletableFuture<MockResponse> call2 = fetchData("http://localhost:8080/mock/data/2");
        CompletableFuture<MockResponse> call3 = fetchData("http://localhost:8080/mock/data/3");

        return CompletableFuture.allOf(call1, call2, call3)
                .thenApply(v -> Stream.of(call1, call2, call3)
                        .map(CompletableFuture::join)
                        .filter(Objects::nonNull)
                        .toList());
    }

Explanation: fetchAllData method

• Purpose: Fetches data from multiple URLs in parallel and aggregates the results.
• Implementation: Creates multiple CompletableFuture instances for each URL and uses CompletableFuture.allOf to wait for all parallel tasks to complete. Aggregates the results using the join method on each future.
• Returns: A CompletableFuture<List<MockResponse>> that completes when all parallel tasks finish, containing the list of aggregated responses.

6.3 Handling Exceptions

To handle exceptions that may occur during the asynchronous execution, we can use the handle method of CompletableFuture.

public CompletableFuture<MockResponse> fetchDataWithHandling(String url) {
        return CompletableFuture.supplyAsync(() -> restTemplate.getForObject(url, MockResponse.class), executor)
                .handle((result, ex) -> {
                    if (ex != null) {
                        System.err.println("Error fetching data from " + url + ": " + ex.getMessage());
                        return null;
                    }
                    return result;
                });
    }
    

Explanation: fetchDataWithHandling method

• Purpose: Handles exceptions gracefully during asynchronous execution.
• Implementation: Uses the handle method of CompletableFuture to handle exceptions for each individual task. This method allows handling the result and exception together, providing an opportunity to return a fallback value or log the error.
• Returns: A CompletableFuture<MockResponse> that either completes with the HTTP response or handles an exception by returning a fallback value or logging the error.

6.4 Configuring Timeouts for Individual Task

To configure timeouts for each task, we can use the orTimeout method of CompletableFuture.

// Fetches data asynchronously with a specified timeout
public CompletableFuture<MockResponse> fetchDataWithTimeout(String url, long timeout) {
        CompletableFuture<MockResponse> future = CompletableFuture.supplyAsync(() -> restTemplate.getForObject(url, MockResponse.class), executor);
        return future.orTimeout(timeout, TimeUnit.MILLISECONDS).exceptionally(ex -> {
            if (ex instanceof TimeoutException) {
                System.err.println("Timeout while fetching data from " + url);
            } else {
                System.err.println("Some other exception encountered while fetching data from " + url);
            }
            return null;
        });
    }
    
    

Explanation: fetchDataWithTimeout method

• Purpose: Fetches data asynchronously with a specified timeout.
• Implementation: Uses the orTimeout method to set a timeout for the task.
• Returns: A CompletableFuture that completes within the specified timeout or handles a timeout exception.

6.5 Global Timeout for All Tasks

To configure a global timeout for all tasks, we can use the orTimeout method with CompletableFuture.allOf.

// Fetches data from multiple URLs in parallel with a global timeout and aggregates the results
public CompletableFuture<List<MockResponse>> fetchAllDataWithTimeout(long globalTimeout, long individualTaskTimeout) {
        CompletableFuture<MockResponse> call1 = fetchDataWithTimeout("http://localhost:8080/mock/data/1", individualTaskTimeout);
        CompletableFuture<MockResponse> call2 = fetchDataWithTimeout("http://localhost:8080/mock/data/2", individualTaskTimeout);
        CompletableFuture<MockResponse> call3 = fetchDataWithTimeout("http://localhost:8080/mock/data/3", individualTaskTimeout);

        return CompletableFuture.allOf(call1, call2, call3)
                .orTimeout(globalTimeout, TimeUnit.MILLISECONDS)
                .handle((result, ex) -> {
                    if (ex != null) {
                        if (ex instanceof TimeoutException) {
                            System.err.println("Global timeout exception encountered");
                        } else {
                            System.err.println("Some other exception encountered at global level");
                        }
                        return Collections.emptyList();
                    }
                    return Stream.of(call1, call2, call3)
                            .map(CompletableFuture::join)
                            .filter(Objects::nonNull)
                            .toList();
                });
    }
    

Explanation: fetchAllDataWithTimeout method

• Purpose: Fetches data asynchronously from multiple URLs with specified timeouts for individual tasks and a global timeout for all tasks combined.
• Implementation:
  • Asynchronous Call: Uses CompletableFuture.supplyAsync to perform HTTP GET requests using RestTemplate in separate threads.
  • Timeout Handling: Applies individual timeouts to each task using the orTimeout method and a global timeout for all tasks using CompletableFuture.allOf.
  • Exception Handling: Utilizes the handle method to manage global timeouts and other exceptions. Logs appropriate error messages and returns an empty list in case of a global timeout or other exceptions.
• Returns: A CompletableFuture<List<MockResponse>> that either completes with the aggregated HTTP responses or handles a global timeout or other exceptions by returning an empty list.

7. Demonstrating Parallel Calls with RestTemplate

To demonstrate the different methods we’ve implemented in the ParallelService class, we’ve created a ParallelController with various endpoints. These endpoints showcase how to fetch data asynchronously, handle exceptions, and manage timeouts using CompletableFuture.

7.1 Single Data Fetching

This endpoint triggers a single data fetching operation.

Endpoint: /api/fetchData

@GetMapping("/fetchData")
public CompletableFuture<MockResponse> getData() {
    logger.info("Request received for 'fetchData' endpoint.");

    CompletableFuture<MockResponse> responseCompletableFuture = parallelService.fetchData("http://localhost:8080/mock/data/1");

    logger.info("Request processing completed for 'fetchData' endpoint.");
    return responseCompletableFuture;
}

Flow:

• The method fetchData in ParallelService is called to fetch data from a single URL.
• It returns a CompletableFuture<MockResponse> that completes with the HTTP response.

Output:

A single MockResponse object with the fetched data will be returned.

{
    "id": "1",
    "message": "Mock data for ID 1"
}

7.2 Parallel Data Fetching

This endpoint triggers parallel data fetching from multiple URLs.

Endpoint: /api/parallel

@GetMapping("/parallel")
public CompletableFuture<List<MockResponse>> getParallelData() {
    logger.info("Request received for 'parallel' endpoint.");

    CompletableFuture<List<MockResponse>> listCompletableFuture = parallelService.fetchAllData();

    logger.info("Request processing completed for 'parallel' endpoint.");
    return listCompletableFuture;
}

Flow:

• The method fetchAllData in ParallelService is called to fetch data from multiple URLs in parallel.
• It returns a CompletableFuture<List<MockResponse>> that completes when all parallel tasks finish.

Output:

A list of MockResponse objects with the fetched data from all URLs will be returned.

[
    {
        "id": "1",
        "message": "Mock data for ID 1"
    },
    {
        "id": "2",
        "message": "Mock data for ID 2"
    },
    {
        "id": "3",
        "message": "Mock data for ID 3"
    }
]

7.3 Parallel Data Fetching with Exception in One Task

This endpoint triggers parallel data fetching, where one of the tasks throws an exception.

Endpoint: /api/parallelWithExceptionInOneTask

@GetMapping("/parallelWithExceptionIOneTask")
public CompletableFuture<List<MockResponse>> getParallelWithExceptionIOneTask() {
    logger.info("Request received for 'parallelWithExceptionIOneTask' endpoint.");

    CompletableFuture<List<MockResponse>> listCompletableFuture = parallelService.fetchAllDataWithErrorInOneCall();

    logger.info("Request processing completed for 'parallelWithExceptionIOneTask' endpoint.");
    return listCompletableFuture;
}

Flow:

• The method fetchAllDataWithErrorInOneCall in ParallelService is called to fetch data from multiple URLs, with one URL designed to throw an exception.
• It uses fetchDataWithHandling to handle exceptions, ensuring the process continues even if one task fails.

Output:

A list of MockResponse objects with data from the tasks that were completed successfully. The data from the task(task id 2) that encountered an error is excluded.

[
    {
        "id": "1",
        "message": "Mock data for ID 1"
    },
    {
        "id": "3",
        "message": "Mock data for ID 3"
    }
]

7.4 Parallel Data Fetching with a Global Timeout

This endpoint triggers parallel data fetching with a global timeout.

Endpoint: /api/parallelWithGlobalTimeout

@GetMapping("/parallelWithGlobalTimeout")
public CompletableFuture<List<MockResponse>> getParallelDataWithGlobalTimeout() {
    logger.info("Request received for 'parallelWithTimeout' endpoint.");

    CompletableFuture<List<MockResponse>> listCompletableFuture = parallelService.fetchAllDataWithTimeout(1000, 5000);

    logger.info("Request processing completed for 'parallelWithTimeout' endpoint.");
    return listCompletableFuture;
}

Flow:

• The method fetchAllDataWithTimeout in ParallelService is called to fetch data from multiple URLs with individual and global timeouts.
• It uses CompletableFuture.allOf with a global timeout and handles global timeout exceptions by returning an empty list.

Output:

If the global timeout is reached, an empty list is returned. Otherwise, it returns a list of MockResponse objects with the fetched data from all URLs.

// Gloabl timeout of 1000 milliseconds was reached due to which an empty list was returned in output
[]

7.5 Parallel Data Fetching with Individual Task Timeout

This endpoint triggers parallel data fetching with individual task timeouts.

Endpoint: /api/parallelWithIndividualTaskTimeout

@GetMapping("/")
public CompletableFuture<List<MockResponse>> getParallelWithIndividualTaskTimeout() {
    logger.info("Request received for 'parallelWithIndividualTaskTimeout' endpoint.");

    CompletableFuture<List<MockResponse>> listCompletableFuture = parallelService.fetchAllDataWithTimeout(10000, 2000);

    logger.info("Request processing completed for 'parallelWithIndividualTaskTimeout' endpoint.");
    return listCompletableFuture;
}

Flow:

• The method fetchAllDataWithTimeout in ParallelService is called to fetch data from multiple URLs with individual task timeouts.
• Each task is assigned a specified timeout, and any task that exceeds its timeout will handle the exception and return null.

Output:

Since the individual task timeout was set to 2000 milliseconds and each task takes at least 3000 milliseconds to complete (due to Thread.sleep configured in the mock controller), all tasks exceeded their timeouts, were filtered out, and the final response was an empty array.

[]

8. Source Code

The complete source code of the above examples can be found here.

9. Things to Consider

Here are some important considerations to keep in mind when working with parallel calls in Spring Boot:

• Thread Pool Size: Ensure the thread pool size configured for CompletableFuture is sufficient to handle the parallel tasks. A too-small thread pool can lead to tasks waiting for execution, affecting performance.
• Timeouts: Appropriately set timeouts based on the expected response times from external services. Too short timeouts can result in frequent timeouts.
• Exception Handling: Properly handle exceptions to ensure that failures in one or more tasks do not affect the overall application stability. Use logging to capture and analyze errors.
• Data Consistency: Ensure that the data returned from parallel tasks is consistent and that partial failures are handled gracefully. Return meaningful responses even when some tasks fail.
• Performance: Test the performance of your application under load to ensure that it can handle the expected volume of parallel requests. Monitor and optimize as needed.
• Resource Management: Be mindful of the resources consumed by parallel tasks, such as CPU and memory. Optimize the number of parallel tasks based on the available resources to prevent resource exhaustion.
• Mock Data: In a real-world scenario, replace mock data and endpoints with actual service endpoints. Ensure that the external services are reliable and can handle concurrent requests.

10. FAQs

11. Conclusion

In this guide, we explored how to make parallel calls using RestTemplate and CompletableFuture in a Spring Boot application. We covered handling exceptions, configuring timeouts, and implementing the scatter-gather design pattern. By following these steps, you can improve the performance and reliability of your Spring Boot applications when making multiple HTTP calls.

12. Learn More