Library

Apache HttpClient

Overview

Apache HttpClient is developed as "a comprehensive HTTP client library for Java" - a long-established open-source project by the Apache Software Foundation. Designed for robust HTTP communication in enterprise environments, it provides extensive authentication mechanisms, proxy support, SSL/TLS processing, connection management, and caching functionality. With over 20 years of proven track record as a Java HTTP client library, it has been adopted by numerous enterprise systems and large-scale applications.

Details

Apache HttpClient 2025 edition has achieved further maturity as the HTTP communication foundation for enterprise systems, maintaining its important position in the Java ecosystem. Through complete HTTP/1.1 support, advanced SSL/TLS processing, diverse authentication protocols (Basic, Digest, NTLM, Kerberos), and detailed connection pool management, it ensures stability in mission-critical systems. With thread-safe design, extensive configuration options, and extensible architecture, it continues to evolve as a feature-rich HTTP client capable of handling complex enterprise requirements.

Key Features

• Enterprise-Grade Functionality: Extensive authentication and security features required by enterprise systems
• Connection Management: Advanced connection pooling and lifecycle management
• Diverse Authentication: Wide range of authentication methods including Basic, Digest, NTLM, Kerberos
• SSL/TLS Support: Detailed SSL configuration and client certificate support
• Proxy Support: Complete support for HTTP, HTTPS, and SOCKS proxies
• Thread Safety: Safe usage in concurrent processing environments

Pros and Cons

Pros

• High reliability and stability with over 20 years of proven track record
• Extensive authentication and security features required in enterprise environments
• Flexible customization through detailed configuration options
• Thread-safe design for secure concurrent processing
• Continuous maintenance and support by Apache Foundation
• Connection pool functionality for efficient management of large numbers of simultaneous connections

Cons

• High learning cost and complexity for beginners
• Verbose API design compared to modern Java libraries
• Limited HTTP/2 support with slower adoption of latest specifications
• Performance may be inferior to modern libraries like OkHttp in some cases
• Limited asynchronous processing capabilities compared to other libraries
• Difficulty in proper configuration due to excessive configuration options

Reference Pages

• Apache HttpClient Official Documentation
• Apache HttpComponents Project
• Apache HttpClient GitHub Repository

Code Examples

Installation and Basic Setup

<!-- Maven (pom.xml) -->
<dependency>
    <groupId>org.apache.httpcomponents.client5</groupId>
    <artifactId>httpclient5</artifactId>
    <version>5.3</version>
</dependency>

<!-- Logging functionality (recommended) -->
<dependency>
    <groupId>org.apache.httpcomponents.client5</groupId>
    <artifactId>httpclient5-fluent</artifactId>
    <version>5.3</version>
</dependency>

<!-- Caching functionality (optional) -->
<dependency>
    <groupId>org.apache.httpcomponents.client5</groupId>
    <artifactId>httpclient5-cache</artifactId>
    <version>5.3</version>
</dependency>

// Gradle (build.gradle)
dependencies {
    implementation 'org.apache.httpcomponents.client5:httpclient5:5.3'
    implementation 'org.apache.httpcomponents.client5:httpclient5-fluent:5.3'
    implementation 'org.apache.httpcomponents.client5:httpclient5-cache:5.3' // Optional
}

Basic Requests (GET/POST/PUT/DELETE)

import org.apache.hc.client5.http.classic.methods.*;
import org.apache.hc.client5.http.impl.classic.*;
import org.apache.hc.core5.http.*;
import org.apache.hc.core5.http.io.entity.*;
import org.apache.hc.core5.http.io.support.ClassicRequestBuilder;

public class ApacheHttpClientExample {
    
    // Basic GET request
    public void basicGetRequest() throws Exception {
        try (CloseableHttpClient httpClient = HttpClients.createDefault()) {
            HttpGet request = new HttpGet("https://api.example.com/users");
            
            // Set headers
            request.setHeader("Accept", "application/json");
            request.setHeader("User-Agent", "MyApp/1.0 (Apache HttpClient)");

            try (CloseableHttpResponse response = httpClient.execute(request)) {
                System.out.println("Status: " + response.getCode());
                System.out.println("Content-Type: " + response.getFirstHeader("Content-Type"));
                
                // Get response body
                HttpEntity entity = response.getEntity();
                if (entity != null) {
                    String responseBody = EntityUtils.toString(entity);
                    System.out.println("Response: " + responseBody);
                }
            }
        }
    }

    // POST request (sending JSON)
    public void postJsonRequest() throws Exception {
        try (CloseableHttpClient httpClient = HttpClients.createDefault()) {
            HttpPost request = new HttpPost("https://api.example.com/users");
            
            // Set JSON data
            String jsonData = "{\"name\": \"John Doe\", \"email\": \"[email protected]\"}";
            StringEntity entity = new StringEntity(jsonData, ContentType.APPLICATION_JSON);
            request.setEntity(entity);
            
            // Set headers
            request.setHeader("Authorization", "Bearer your-jwt-token");
            request.setHeader("Accept", "application/json");

            try (CloseableHttpResponse response = httpClient.execute(request)) {
                if (response.getCode() >= 200 && response.getCode() < 300) {
                    String responseBody = EntityUtils.toString(response.getEntity());
                    System.out.println("Created user: " + responseBody);
                } else {
                    System.out.println("POST request failed: " + response.getCode());
                }
            }
        }
    }

    // PUT request (update)
    public void putRequest() throws Exception {
        try (CloseableHttpClient httpClient = HttpClients.createDefault()) {
            HttpPut request = new HttpPut("https://api.example.com/users/123");
            
            String jsonData = "{\"name\": \"Jane Doe\", \"email\": \"[email protected]\"}";
            StringEntity entity = new StringEntity(jsonData, ContentType.APPLICATION_JSON);
            request.setEntity(entity);
            
            request.setHeader("Authorization", "Bearer your-jwt-token");

            try (CloseableHttpResponse response = httpClient.execute(request)) {
                if (response.getCode() == 200) {
                    System.out.println("User updated successfully");
                }
            }
        }
    }

    // DELETE request
    public void deleteRequest() throws Exception {
        try (CloseableHttpClient httpClient = HttpClients.createDefault()) {
            HttpDelete request = new HttpDelete("https://api.example.com/users/123");
            request.setHeader("Authorization", "Bearer your-jwt-token");

            try (CloseableHttpResponse response = httpClient.execute(request)) {
                if (response.getCode() == 204) {
                    System.out.println("User deleted successfully");
                }
            }
        }
    }

    // Form data submission
    public void submitFormData() throws Exception {
        try (CloseableHttpClient httpClient = HttpClients.createDefault()) {
            HttpPost request = new HttpPost("https://api.example.com/login");
            
            // Set form parameters
            List<NameValuePair> parameters = new ArrayList<>();
            parameters.add(new BasicNameValuePair("username", "testuser"));
            parameters.add(new BasicNameValuePair("password", "secret123"));
            parameters.add(new BasicNameValuePair("remember", "true"));
            
            UrlEncodedFormEntity entity = new UrlEncodedFormEntity(parameters);
            request.setEntity(entity);

            try (CloseableHttpResponse response = httpClient.execute(request)) {
                String responseBody = EntityUtils.toString(response.getEntity());
                System.out.println("Login response: " + responseBody);
            }
        }
    }

    // Using Fluent API (concise syntax)
    public void fluentApiExample() throws Exception {
        import org.apache.hc.client5.http.fluent.Request;
        
        // Concise GET request
        String response = Request.get("https://api.example.com/users")
                .addHeader("Accept", "application/json")
                .execute()
                .returnContent()
                .asString();
        
        System.out.println("Fluent API response: " + response);
        
        // Concise POST request
        String postResponse = Request.post("https://api.example.com/users")
                .bodyString("{\"name\": \"John Doe\"}", ContentType.APPLICATION_JSON)
                .addHeader("Authorization", "Bearer token")
                .execute()
                .returnContent()
                .asString();
        
        System.out.println("Fluent POST: " + postResponse);
    }
}

Advanced Configuration and Customization (Headers, Authentication, Timeout, etc.)

import org.apache.hc.client5.http.auth.*;
import org.apache.hc.client5.http.config.*;
import org.apache.hc.client5.http.impl.auth.*;
import org.apache.hc.client5.http.impl.classic.*;
import org.apache.hc.client5.http.impl.io.*;
import org.apache.hc.client5.http.socket.*;
import org.apache.hc.client5.http.ssl.*;
import org.apache.hc.core5.http.config.*;
import org.apache.hc.core5.pool.*;
import org.apache.hc.core5.util.Timeout;

public class AdvancedHttpClientExample {
    
    // Create custom HttpClient instance with custom settings
    public CloseableHttpClient createCustomHttpClient() {
        // Connection pool configuration
        PoolingHttpClientConnectionManager connectionManager = 
                PoolingHttpClientConnectionManagerBuilder.create()
                .setMaxConnTotal(100)           // Maximum total connections
                .setMaxConnPerRoute(20)         // Maximum connections per route
                .setConnectionTimeToLive(TimeValue.ofMinutes(10)) // Connection time to live
                .build();

        // Timeout configuration
        RequestConfig requestConfig = RequestConfig.custom()
                .setConnectionRequestTimeout(Timeout.ofSeconds(5))  // Connection request timeout
                .setConnectTimeout(Timeout.ofSeconds(10))           // Connection timeout
                .setResponseTimeout(Timeout.ofSeconds(30))          // Response timeout
                .setCookieSpec(StandardCookieSpec.STRICT)           // Cookie specification
                .setRedirectsEnabled(true)                          // Allow redirects
                .setMaxRedirects(5)                                 // Maximum redirects
                .build();

        // Build HTTP client
        return HttpClients.custom()
                .setConnectionManager(connectionManager)
                .setDefaultRequestConfig(requestConfig)
                .setRetryStrategy(DefaultHttpRequestRetryStrategy.INSTANCE)
                .setRedirectStrategy(DefaultRedirectStrategy.INSTANCE)
                .setUserAgent("MyApp/1.0 (Apache HttpClient 5.3)")
                .build();
    }

    // Basic authentication setup
    public void basicAuthenticationExample() throws Exception {
        // Store authentication credentials
        CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
        credentialsProvider.setCredentials(
                new AuthScope("api.example.com", 443),
                new UsernamePasswordCredentials("username", "password".toCharArray())
        );

        try (CloseableHttpClient httpClient = HttpClients.custom()
                .setDefaultCredentialsProvider(credentialsProvider)
                .build()) {
            
            HttpGet request = new HttpGet("https://api.example.com/protected");
            
            try (CloseableHttpResponse response = httpClient.execute(request)) {
                String responseBody = EntityUtils.toString(response.getEntity());
                System.out.println("Authenticated response: " + responseBody);
            }
        }
    }

    // Digest authentication setup
    public void digestAuthenticationExample() throws Exception {
        CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
        credentialsProvider.setCredentials(
                AuthScope.ANY,
                new UsernamePasswordCredentials("username", "password".toCharArray())
        );

        // Digest authentication configuration
        AuthSchemeProvider digestSchemeProvider = new DigestSchemeProvider();
        Registry<AuthSchemeProvider> authSchemeRegistry = RegistryBuilder.<AuthSchemeProvider>create()
                .register(StandardAuthScheme.DIGEST, digestSchemeProvider)
                .build();

        try (CloseableHttpClient httpClient = HttpClients.custom()
                .setDefaultCredentialsProvider(credentialsProvider)
                .setDefaultAuthSchemeRegistry(authSchemeRegistry)
                .build()) {
            
            HttpGet request = new HttpGet("https://api.example.com/digest-auth");
            
            try (CloseableHttpResponse response = httpClient.execute(request)) {
                System.out.println("Digest auth response: " + response.getCode());
            }
        }
    }

    // Proxy configuration
    public void proxyExample() throws Exception {
        // HTTP proxy configuration
        HttpHost proxy = new HttpHost("proxy.example.com", 8080, "http");
        
        // Proxy authentication (if required)
        CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
        credentialsProvider.setCredentials(
                new AuthScope(proxy),
                new UsernamePasswordCredentials("proxy-user", "proxy-pass".toCharArray())
        );

        RequestConfig requestConfig = RequestConfig.custom()
                .setProxy(proxy)
                .build();

        try (CloseableHttpClient httpClient = HttpClients.custom()
                .setDefaultCredentialsProvider(credentialsProvider)
                .setDefaultRequestConfig(requestConfig)
                .build()) {
            
            HttpGet request = new HttpGet("https://api.example.com/data");
            
            try (CloseableHttpResponse response = httpClient.execute(request)) {
                String responseBody = EntityUtils.toString(response.getEntity());
                System.out.println("Response via proxy: " + responseBody);
            }
        }
    }

    // SSL/TLS configuration
    public void sslConfiguration() throws Exception {
        // Custom SSL context
        SSLContext sslContext = SSLContexts.custom()
                .loadTrustMaterial(null, (certificate, authType) -> true) // Trust all certificates (development only)
                .build();

        // SSL configuration
        SSLConnectionSocketFactory sslSocketFactory = new SSLConnectionSocketFactory(
                sslContext,
                new String[]{"TLSv1.2", "TLSv1.3"},
                null,
                SSLConnectionSocketFactory.getDefaultHostnameVerifier()
        );

        Registry<ConnectionSocketFactory> socketFactoryRegistry = RegistryBuilder.<ConnectionSocketFactory>create()
                .register("https", sslSocketFactory)
                .register("http", new PlainConnectionSocketFactory())
                .build();

        PoolingHttpClientConnectionManager connectionManager = 
                PoolingHttpClientConnectionManagerBuilder.create()
                .setConnectionSocketFactoryRegistry(socketFactoryRegistry)
                .build();

        try (CloseableHttpClient httpClient = HttpClients.custom()
                .setConnectionManager(connectionManager)
                .build()) {
            
            HttpGet request = new HttpGet("https://secure-api.example.com/data");
            
            try (CloseableHttpResponse response = httpClient.execute(request)) {
                System.out.println("SSL communication success: " + response.getCode());
            }
        }
    }

    // Cookie handling
    public void cookieHandling() throws Exception {
        // Create cookie store
        CookieStore cookieStore = new BasicCookieStore();
        
        // Add custom cookie
        BasicClientCookie cookie = new BasicClientCookie("session_id", "abc123");
        cookie.setDomain("api.example.com");
        cookie.setPath("/");
        cookieStore.addCookie(cookie);

        try (CloseableHttpClient httpClient = HttpClients.custom()
                .setDefaultCookieStore(cookieStore)
                .build()) {
            
            // Cookies are automatically sent
            HttpGet request = new HttpGet("https://api.example.com/session-data");
            
            try (CloseableHttpResponse response = httpClient.execute(request)) {
                String responseBody = EntityUtils.toString(response.getEntity());
                System.out.println("Response with cookies: " + responseBody);
                
                // Check received cookies
                for (Cookie receivedCookie : cookieStore.getCookies()) {
                    System.out.println("Received cookie: " + receivedCookie.getName() + "=" + receivedCookie.getValue());
                }
            }
        }
    }
}

Error Handling and Retry Functionality

import org.apache.hc.client5.http.HttpRequestRetryStrategy;
import org.apache.hc.client5.http.impl.DefaultHttpRequestRetryStrategy;
import org.apache.hc.core5.http.HttpRequest;
import org.apache.hc.core5.http.HttpResponse;
import org.apache.hc.core5.http.protocol.HttpContext;
import org.apache.hc.core5.util.TimeValue;

public class ErrorHandlingExample {
    
    // Comprehensive error handling
    public String safeHttpRequest(String url) {
        try (CloseableHttpClient httpClient = createRobustHttpClient()) {
            HttpGet request = new HttpGet(url);
            
            try (CloseableHttpResponse response = httpClient.execute(request)) {
                int statusCode = response.getCode();
                
                if (statusCode >= 200 && statusCode < 300) {
                    return EntityUtils.toString(response.getEntity());
                } else {
                    handleHttpError(response);
                    return null;
                }
                
            } catch (ConnectTimeoutException e) {
                System.err.println("Connection timeout: " + e.getMessage());
                System.err.println("Please check your network connection");
            } catch (SocketTimeoutException e) {
                System.err.println("Read timeout: " + e.getMessage());
                System.err.println("Server response is delayed");
            } catch (UnknownHostException e) {
                System.err.println("Host not found: " + e.getMessage());
                System.err.println("Please check URL or network connection");
            } catch (IOException e) {
                System.err.println("Network error: " + e.getMessage());
            }
            
        } catch (Exception e) {
            System.err.println("Unexpected error: " + e.getMessage());
            e.printStackTrace();
        }
        
        return null;
    }

    private void handleHttpError(CloseableHttpResponse response) throws IOException {
        int statusCode = response.getCode();
        String reasonPhrase = response.getReasonPhrase();
        String errorBody = EntityUtils.toString(response.getEntity());

        System.err.println("HTTP Error: " + statusCode + " " + reasonPhrase);
        
        switch (statusCode) {
            case 400:
                System.err.println("Bad Request: Please check parameters");
                break;
            case 401:
                System.err.println("Unauthorized: Please check authentication");
                break;
            case 403:
                System.err.println("Forbidden: Please check permissions");
                break;
            case 404:
                System.err.println("Not Found: Resource does not exist");
                break;
            case 429:
                String retryAfter = response.getFirstHeader("Retry-After").getValue();
                System.err.println("Rate Limited: " + (retryAfter != null ? "Retry after " + retryAfter + " seconds" : "Try again later"));
                break;
            case 500:
                System.err.println("Internal Server Error");
                break;
            case 502:
                System.err.println("Bad Gateway: Server temporarily unavailable");
                break;
            case 503:
                System.err.println("Service Unavailable: Server overloaded");
                break;
            default:
                System.err.println("HTTP Error details: " + errorBody);
        }
    }

    // Custom retry strategy
    public static class CustomRetryStrategy implements HttpRequestRetryStrategy {
        private final int maxRetries;
        private final TimeValue retryInterval;

        public CustomRetryStrategy(int maxRetries, TimeValue retryInterval) {
            this.maxRetries = maxRetries;
            this.retryInterval = retryInterval;
        }

        @Override
        public boolean retryRequest(HttpRequest request, IOException exception, int execCount, HttpContext context) {
            if (execCount > maxRetries) {
                return false;
            }

            // Retry on specific exceptions
            if (exception instanceof ConnectTimeoutException ||
                exception instanceof SocketTimeoutException ||
                exception instanceof NoHttpResponseException) {
                
                System.out.println("Retrying (" + execCount + "/" + maxRetries + "): " + exception.getMessage());
                return true;
            }

            return false;
        }

        @Override
        public boolean retryRequest(HttpResponse response, int execCount, HttpContext context) {
            if (execCount > maxRetries) {
                return false;
            }

            int statusCode = response.getCode();
            
            // Retry on specific status codes
            if (statusCode == 429 || // Too Many Requests
                statusCode == 500 || // Internal Server Error
                statusCode == 502 || // Bad Gateway
                statusCode == 503 || // Service Unavailable
                statusCode == 504) { // Gateway Timeout
                
                System.out.println("HTTP Status retry (" + execCount + "/" + maxRetries + "): " + statusCode);
                return true;
            }

            return false;
        }

        @Override
        public TimeValue getRetryInterval(HttpResponse response, int execCount, HttpContext context) {
            // Check Retry-After header
            Header retryAfterHeader = response.getFirstHeader("Retry-After");
            if (retryAfterHeader != null) {
                try {
                    int retryAfterSeconds = Integer.parseInt(retryAfterHeader.getValue());
                    return TimeValue.ofSeconds(retryAfterSeconds);
                } catch (NumberFormatException e) {
                    // Use default value if parsing fails
                }
            }

            // Exponential backoff
            long delayMillis = retryInterval.toMilliseconds() * (long) Math.pow(2, execCount - 1);
            return TimeValue.ofMilliseconds(Math.min(delayMillis, 30000)); // Maximum 30 seconds
        }
    }

    // Create robust HttpClient
    private CloseableHttpClient createRobustHttpClient() {
        return HttpClients.custom()
                .setRetryStrategy(new CustomRetryStrategy(3, TimeValue.ofSeconds(1)))
                .setDefaultRequestConfig(RequestConfig.custom()
                        .setConnectTimeout(Timeout.ofSeconds(10))
                        .setResponseTimeout(Timeout.ofSeconds(30))
                        .build())
                .build();
    }

    // Asynchronous error handling
    public void asyncErrorHandling() throws Exception {
        CloseableHttpAsyncClient asyncClient = HttpAsyncClients.createDefault();
        asyncClient.start();

        try {
            HttpGet request = new HttpGet("https://api.example.com/data");
            
            Future<SimpleHttpResponse> future = asyncClient.execute(
                    SimpleRequestBuilder.copy(request).build(),
                    new FutureCallback<SimpleHttpResponse>() {
                        @Override
                        public void completed(SimpleHttpResponse response) {
                            System.out.println("Async request completed: " + response.getCode());
                        }

                        @Override
                        public void failed(Exception ex) {
                            System.err.println("Async request failed: " + ex.getMessage());
                        }

                        @Override
                        public void cancelled() {
                            System.out.println("Async request was cancelled");
                        }
                    }
            );

            // Wait for result (with timeout)
            SimpleHttpResponse response = future.get(30, TimeUnit.SECONDS);
            System.out.println("Async response: " + response.getBodyText());
            
        } finally {
            asyncClient.close();
        }
    }
}

Concurrent Processing and Asynchronous Requests

import org.apache.hc.client5.http.async.methods.*;
import org.apache.hc.client5.http.impl.async.*;
import org.apache.hc.core5.concurrent.FutureCallback;
import java.util.concurrent.*;
import java.util.stream.Collectors;

public class ConcurrentRequestsExample {
    
    // Parallel synchronous requests
    public List<String> fetchMultipleUrlsSync(List<String> urls) {
        try (CloseableHttpClient httpClient = HttpClients.createDefault()) {
            // Parallel processing using ExecutorService
            ExecutorService executor = Executors.newFixedThreadPool(10);
            
            try {
                List<Future<String>> futures = urls.stream()
                        .map(url -> executor.submit(() -> {
                            try {
                                HttpGet request = new HttpGet(url);
                                try (CloseableHttpResponse response = httpClient.execute(request)) {
                                    if (response.getCode() == 200) {
                                        return EntityUtils.toString(response.getEntity());
                                    } else {
                                        return "Error: " + response.getCode();
                                    }
                                }
                            } catch (Exception e) {
                                return "Exception: " + e.getMessage();
                            }
                        }))
                        .collect(Collectors.toList());

                // Collect results
                return futures.stream()
                        .map(future -> {
                            try {
                                return future.get(30, TimeUnit.SECONDS);
                            } catch (Exception e) {
                                return "Timeout or error: " + e.getMessage();
                            }
                        })
                        .collect(Collectors.toList());
                        
            } finally {
                executor.shutdown();
            }
        } catch (IOException e) {
            throw new RuntimeException("HTTP client creation error", e);
        }
    }

    // Asynchronous parallel requests
    public CompletableFuture<List<String>> fetchMultipleUrlsAsync(List<String> urls) {
        CloseableHttpAsyncClient asyncClient = HttpAsyncClients.createDefault();
        asyncClient.start();

        List<CompletableFuture<String>> futures = urls.stream()
                .map(url -> {
                    CompletableFuture<String> future = new CompletableFuture<>();
                    
                    SimpleHttpRequest request = SimpleRequestBuilder.get(url).build();
                    
                    asyncClient.execute(request, new FutureCallback<SimpleHttpResponse>() {
                        @Override
                        public void completed(SimpleHttpResponse response) {
                            try {
                                if (response.getCode() == 200) {
                                    future.complete(response.getBodyText());
                                } else {
                                    future.complete("Error: " + response.getCode());
                                }
                            } catch (Exception e) {
                                future.complete("Response error: " + e.getMessage());
                            }
                        }

                        @Override
                        public void failed(Exception ex) {
                            future.complete("Failed: " + ex.getMessage());
                        }

                        @Override
                        public void cancelled() {
                            future.complete("Cancelled");
                        }
                    });
                    
                    return future;
                })
                .collect(Collectors.toList());

        // Wait for all async processes to complete
        return CompletableFuture.allOf(futures.toArray(new CompletableFuture[0]))
                .thenApply(v -> futures.stream()
                        .map(CompletableFuture::join)
                        .collect(Collectors.toList()))
                .whenComplete((result, throwable) -> {
                    try {
                        asyncClient.close();
                    } catch (IOException e) {
                        System.err.println("Async client shutdown error: " + e.getMessage());
                    }
                });
    }

    // Pagination support
    public List<String> fetchAllPages(String baseUrl, int maxPages) throws Exception {
        try (CloseableHttpClient httpClient = HttpClients.createDefault()) {
            ExecutorService executor = Executors.newFixedThreadPool(5);
            List<Future<String>> futures = new ArrayList<>();

            for (int page = 1; page <= maxPages; page++) {
                final int currentPage = page;
                Future<String> future = executor.submit(() -> {
                    try {
                        String url = baseUrl + "?page=" + currentPage + "&limit=20";
                        HttpGet request = new HttpGet(url);
                        
                        try (CloseableHttpResponse response = httpClient.execute(request)) {
                            if (response.getCode() == 200) {
                                String responseBody = EntityUtils.toString(response.getEntity());
                                System.out.println("Page " + currentPage + " completed");
                                return responseBody;
                            } else {
                                System.err.println("Page " + currentPage + " error: " + response.getCode());
                                return null;
                            }
                        }
                    } catch (Exception e) {
                        System.err.println("Page " + currentPage + " exception: " + e.getMessage());
                        return null;
                    }
                });
                futures.add(future);
            }

            List<String> results = new ArrayList<>();
            for (Future<String> future : futures) {
                try {
                    String result = future.get(60, TimeUnit.SECONDS);
                    if (result != null) {
                        results.add(result);
                    }
                } catch (TimeoutException e) {
                    System.err.println("Page fetch timeout");
                }
            }

            executor.shutdown();
            return results;
        }
    }

    // Streaming file download
    public void downloadLargeFile(String url, String filename) throws Exception {
        try (CloseableHttpClient httpClient = HttpClients.createDefault()) {
            HttpGet request = new HttpGet(url);
            
            try (CloseableHttpResponse response = httpClient.execute(request)) {
                if (response.getCode() != 200) {
                    throw new IOException("Download failed: " + response.getCode());
                }

                HttpEntity entity = response.getEntity();
                long contentLength = entity.getContentLength();
                
                System.out.println("File size: " + contentLength + " bytes");

                try (InputStream inputStream = entity.getContent();
                     FileOutputStream outputStream = new FileOutputStream(filename);
                     BufferedOutputStream bufferedOut = new BufferedOutputStream(outputStream)) {
                    
                    byte[] buffer = new byte[8192];
                    long downloaded = 0;
                    int bytesRead;

                    while ((bytesRead = inputStream.read(buffer)) != -1) {
                        bufferedOut.write(buffer, 0, bytesRead);
                        downloaded += bytesRead;

                        if (contentLength > 0) {
                            int progress = (int) ((downloaded * 100) / contentLength);
                            System.out.print("\rDownload progress: " + progress + "%");
                        }
                    }
                    
                    System.out.println("\nDownload completed: " + filename);
                }
            }
        }
    }

    // Rate-limited parallel processing
    public void processWithRateLimit(List<String> urls, int requestsPerSecond) throws Exception {
        Semaphore rateLimiter = new Semaphore(requestsPerSecond);
        ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(1);
        
        // Replenish semaphore every second
        scheduler.scheduleAtFixedRate(() -> {
            rateLimiter.release(requestsPerSecond - rateLimiter.availablePermits());
        }, 1, 1, TimeUnit.SECONDS);

        try (CloseableHttpClient httpClient = HttpClients.createDefault()) {
            CountDownLatch latch = new CountDownLatch(urls.size());
            
            for (String url : urls) {
                new Thread(() -> {
                    try {
                        rateLimiter.acquire(); // Wait for rate limit
                        
                        HttpGet request = new HttpGet(url);
                        try (CloseableHttpResponse response = httpClient.execute(request)) {
                            System.out.println("Processed: " + url + " (" + response.getCode() + ")");
                        }
                    } catch (Exception e) {
                        System.err.println("Processing error: " + url + " - " + e.getMessage());
                    } finally {
                        latch.countDown();
                    }
                }).start();
            }
            
            // Wait for all processing to complete
            latch.await(300, TimeUnit.SECONDS);
        } finally {
            scheduler.shutdown();
        }
    }
}

Framework Integration and Practical Examples

// Spring Framework integration
@Service
public class HttpClientService {
    private final CloseableHttpClient httpClient;
    
    @Autowired
    public HttpClientService() {
        this.httpClient = HttpClients.custom()
                .setConnectionManager(createConnectionManager())
                .setDefaultRequestConfig(createRequestConfig())
                .build();
    }
    
    private PoolingHttpClientConnectionManager createConnectionManager() {
        return PoolingHttpClientConnectionManagerBuilder.create()
                .setMaxConnTotal(200)
                .setMaxConnPerRoute(50)
                .build();
    }
    
    private RequestConfig createRequestConfig() {
        return RequestConfig.custom()
                .setConnectTimeout(Timeout.ofSeconds(10))
                .setResponseTimeout(Timeout.ofSeconds(30))
                .build();
    }
    
    public String callExternalApi(String url, Map<String, String> headers) {
        try {
            HttpGet request = new HttpGet(url);
            
            // Set headers
            headers.forEach(request::setHeader);
            
            try (CloseableHttpResponse response = httpClient.execute(request)) {
                if (response.getCode() == 200) {
                    return EntityUtils.toString(response.getEntity());
                } else {
                    throw new RuntimeException("API call error: " + response.getCode());
                }
            }
        } catch (IOException e) {
            throw new RuntimeException("HTTP communication error", e);
        }
    }
    
    @PreDestroy
    public void cleanup() {
        try {
            httpClient.close();
        } catch (IOException e) {
            System.err.println("HTTP client shutdown error: " + e.getMessage());
        }
    }
}

// HTTP client with caching functionality
public class CachingHttpClientExample {
    private final CloseableHttpClient cachingClient;
    
    public CachingHttpClientExample() {
        // Cache configuration
        CacheConfig cacheConfig = CacheConfig.custom()
                .setMaxCacheEntries(1000)
                .setMaxObjectSize(8192)
                .setHeuristicCachingEnabled(true)
                .setHeuristicDefaultLifetime(TimeValue.ofMinutes(10))
                .build();
        
        this.cachingClient = CachingHttpClients.custom()
                .setCacheConfig(cacheConfig)
                .setHttpCacheStorage(new BasicHttpCacheStorage(cacheConfig))
                .build();
    }
    
    public String getCachedData(String url) throws Exception {
        HttpGet request = new HttpGet(url);
        request.setHeader("Cache-Control", "max-age=300"); // 5 minute cache
        
        try (CloseableHttpResponse response = cachingClient.execute(request)) {
            Header cacheStatus = response.getFirstHeader("X-Cache-Status");
            if (cacheStatus != null) {
                System.out.println("Cache status: " + cacheStatus.getValue());
            }
            
            return EntityUtils.toString(response.getEntity());
        }
    }
}

// Multipart file upload
public class FileUploadExample {
    
    public String uploadFile(File file, Map<String, String> metadata) throws Exception {
        try (CloseableHttpClient httpClient = HttpClients.createDefault()) {
            HttpPost uploadRequest = new HttpPost("https://api.example.com/upload");
            
            // Build multipart entity
            MultipartEntityBuilder builder = MultipartEntityBuilder.create();
            
            // Add file
            builder.addBinaryBody("file", file, ContentType.DEFAULT_BINARY, file.getName());
            
            // Add metadata
            metadata.forEach((key, value) -> 
                builder.addTextBody(key, value, ContentType.TEXT_PLAIN));
            
            HttpEntity multipartEntity = builder.build();
            uploadRequest.setEntity(multipartEntity);
            
            // Authentication header
            uploadRequest.setHeader("Authorization", "Bearer your-token");
            
            try (CloseableHttpResponse response = httpClient.execute(uploadRequest)) {
                if (response.getCode() == 200) {
                    return EntityUtils.toString(response.getEntity());
                } else {
                    throw new IOException("Upload failed: " + response.getCode());
                }
            }
        }
    }
    
    // Upload with progress tracking
    public void uploadWithProgress(File file, ProgressCallback callback) throws Exception {
        try (CloseableHttpClient httpClient = HttpClients.createDefault()) {
            HttpPost request = new HttpPost("https://api.example.com/upload");
            
            // Entity with progress tracking functionality
            FileEntity fileEntity = new FileEntity(file, ContentType.DEFAULT_BINARY) {
                @Override
                public void writeTo(OutputStream outStream) throws IOException {
                    try (FileInputStream inStream = new FileInputStream(file);
                         ProgressOutputStream progressOut = new ProgressOutputStream(outStream, file.length(), callback)) {
                        
                        byte[] buffer = new byte[8192];
                        int bytesRead;
                        while ((bytesRead = inStream.read(buffer)) != -1) {
                            progressOut.write(buffer, 0, bytesRead);
                        }
                    }
                }
            };
            
            request.setEntity(fileEntity);
            
            try (CloseableHttpResponse response = httpClient.execute(request)) {
                if (response.getCode() == 200) {
                    callback.onComplete(EntityUtils.toString(response.getEntity()));
                } else {
                    callback.onError(new IOException("Upload failed: " + response.getCode()));
                }
            }
        }
    }
    
    public interface ProgressCallback {
        void onProgress(long bytesTransferred, long totalBytes);
        void onComplete(String response);
        void onError(Exception error);
    }
    
    private static class ProgressOutputStream extends OutputStream {
        private final OutputStream target;
        private final long totalBytes;
        private final ProgressCallback callback;
        private long bytesTransferred = 0;
        
        public ProgressOutputStream(OutputStream target, long totalBytes, ProgressCallback callback) {
            this.target = target;
            this.totalBytes = totalBytes;
            this.callback = callback;
        }
        
        @Override
        public void write(int b) throws IOException {
            target.write(b);
            bytesTransferred++;
            callback.onProgress(bytesTransferred, totalBytes);
        }
        
        @Override
        public void write(byte[] b, int off, int len) throws IOException {
            target.write(b, off, len);
            bytesTransferred += len;
            callback.onProgress(bytesTransferred, totalBytes);
        }
    }
}