記一次調用時長優化之路，暨httpclient優化之路

背景

由於公司部分業務是php開發，而我所在的部門是java開發，這之間就需要相互調用，於是就有了代理項目，負責java通過restful方式調用php接口服務， 隨着業務量的增長rt時間越來越慢，於是開始了排查。

定位問題    

通過pingpoint如下圖

 

分析得知接口耗時主要花費在建立連接上。查看項目代碼得知調用http使用的是http-client4.5.5，發現沒用連接池，這個坑也是絕了。

解決方案

    增加資源池管理HttpClientConnectionManager，HttpClientConnectionManager需要配置這幾項

代碼如下

mport lombok.extern.slf4j.Slf4j;
import org.apache.http.*;
import org.apache.http.client.CookieStore;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.client.entity.UrlEncodedFormEntity;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.client.protocol.HttpClientContext;
import org.apache.http.config.Registry;
import org.apache.http.config.RegistryBuilder;
import org.apache.http.conn.ConnectionKeepAliveStrategy;
import org.apache.http.conn.socket.ConnectionSocketFactory;
import org.apache.http.conn.socket.PlainConnectionSocketFactory;
import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
import org.apache.http.conn.ssl.TrustSelfSignedStrategy;
import org.apache.http.entity.StringEntity;
import org.apache.http.impl.client.BasicCookieStore;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.DefaultHttpRequestRetryHandler;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.impl.conn.PoolingHttpClientConnectionManager;
import org.apache.http.message.BasicHeaderElementIterator;
import org.apache.http.message.BasicNameValuePair;
import org.apache.http.protocol.HTTP;
import org.apache.http.protocol.HttpContext;
import org.apache.http.ssl.SSLContextBuilder;
import org.apache.http.util.EntityUtils;
import org.junit.Test;

import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.Set;

/**
 * @author zdd
 * @date 2019/3/4 下午3:58
 */
@Slf4j
public class HttpClientTests {
    /** 全局連接池對象 */
    private static final PoolingHttpClientConnectionManager connManager = new PoolingHttpClientConnectionManager();
    private static CloseableHttpClient httpClient;
    private static ConnectionKeepAliveStrategy myStrategy = new ConnectionKeepAliveStrategy() {

        public long getKeepAliveDuration(HttpResponse response, HttpContext context) {
            // Honor 'keep-alive' header
            HeaderElementIterator it = new BasicHeaderElementIterator(
                    response.headerIterator(HTTP.CONN_KEEP_ALIVE));
            while (it.hasNext()) {
                HeaderElement he = it.nextElement();
                String param = he.getName();
                String value = he.getValue();
                if (value != null && param.equalsIgnoreCase("timeout")) {
                    try {
                        return Long.parseLong(value) * 1000;
                    } catch (NumberFormatException ignore) {
                    }
                }
            }
            return 30 * 1000;
        }

    };
    static final int timeOut = 30 * 1000;

    static {
        try {
            SSLContextBuilder builder = new SSLContextBuilder();
            builder.loadTrustMaterial(null, new TrustSelfSignedStrategy());
            SSLConnectionSocketFactory sslsf = new SSLConnectionSocketFactory(builder.build());
            // 配置同時支持 HTTP 和 HTPPS
            Registry<ConnectionSocketFactory> socketFactoryRegistry = RegistryBuilder.<ConnectionSocketFactory>create().register(
                    "http", PlainConnectionSocketFactory.getSocketFactory()).register(
                    "https", sslsf).build();
            // 設置最大連接數
            connManager.setMaxTotal(200);
            // 設置每個連接的路由數
            connManager.setDefaultMaxPerRoute(200);
            // 初始化httpClient
            httpClient = getHttpClient();
        }catch (Exception e) {
            log.error("connManager Error",e);
        }
    }



    public static CloseableHttpClient getHttpClient() {
        // 創建Http請求配置參數
        RequestConfig requestConfig = RequestConfig.custom()
                // 獲取連接超時時間
                .setConnectionRequestTimeout(timeOut)
                // 請求超時時間
                .setConnectTimeout(timeOut)
                // 響應超時時間
                .setSocketTimeout(timeOut)
                .build();
        CloseableHttpClient httpClient = HttpClients.custom()
                // 把請求相關的超時信息設置到連接客戶端
                .setDefaultRequestConfig(requestConfig)
                // 把請求重試設置到連接客戶端
                .setRetryHandler(new DefaultHttpRequestRetryHandler(0, false))
                // 配置連接池管理對象
                .setConnectionManager(connManager)
                //設置Keep-Alive
                .setKeepAliveStrategy(myStrategy)
                .build();
        return httpClient;
    }

    /**
     * get 方式請求
     * @param url  請求URL
     * @return
     */
    public static String httpGet(String url ) {
        String msg =  "";
        HttpGet httpGet = new HttpGet(url);
        CloseableHttpResponse response = null;
        try {
            response = httpClient.execute(httpGet);
            HttpEntity entity = response.getEntity();
            msg = EntityUtils.toString(entity, "UTF-8");
        } catch (Exception e) {
            log.error("http請求異常,url:{}",url,e);
        }  finally {
            if (null != response) {
                try {
                    EntityUtils.consume(response.getEntity());
                    response.close();
                } catch (IOException e) {
                    log.error("釋放鏈接錯誤",e);
                }
            }
        }
        return msg;
    }


    /**
     * 設置post請求參數
     * @param httpost
     * @param params
     */
    private static void setPostParams(HttpPost httpost, Map<String, Object> params) {
        List<NameValuePair> nvps = new ArrayList<NameValuePair>();
        Set<String> keySet = params.keySet();
        for (String key : keySet) {
            nvps.add(new BasicNameValuePair(key, params.get(key).toString()));
        }
        try {
            httpost.setEntity(new UrlEncodedFormEntity(nvps, "UTF-8"));
        } catch (Exception e) {
            log.info("設置post異常,", e);
        }
    }

    /**
     * 發送post請求
     * @param url
     * @param params
     * @return
     * @throws IOException
     */
    public static String httpPost(String url, Map<String, Object> params) throws IOException {
        //創建post對象
        HttpPost httppost = new HttpPost(url);
        setPostParams(httppost, params);
        CloseableHttpResponse response = null;
        try {
            response = getHttpClient().execute(httppost,
                    HttpClientContext.create());
            HttpEntity entity = response.getEntity();
            String result = EntityUtils.toString(entity, "utf-8");
            EntityUtils.consume(entity);
            return result;
        } catch (Exception e) {
            throw e;
        } finally {
            try {
                if (response != null){
                    response.close();
                }
            } catch (IOException e) {
                log.error("關閉鏈接失敗",e);
            }
        }
    }

    static class GetThread extends Thread {
        private static CloseableHttpClient httpClient;
        private String url;
        private String param;

        public GetThread(CloseableHttpClient client, String url) {
            httpClient = client;
            this.url = url;
        }

        public GetThread(CloseableHttpClient client, String url, String param) {
            httpClient = client;
            this.url = url;
            this.param = param;
        }

        public void run() {
            while (true) {
                HttpPost httpPost = new HttpPost(url);// 創建httpPost
                httpPost.setHeader("Accept", "application/json");
                httpPost.setHeader("Content-Type", "application/json");
                String charSet = "UTF-8";
                StringEntity entity = new StringEntity(param, charSet);
                httpPost.setEntity(entity);
                CloseableHttpResponse response = null;
                try {
                    response = httpClient.execute(httpPost);
                    HttpEntity responseEntity = response.getEntity();
                    String jsonString = EntityUtils.toString(responseEntity);
                    EntityUtils.consume(responseEntity);
                    System.out.println(jsonString);
                    Thread.sleep(1 * 1000);   // 線程休眠時間
                } catch (Exception e) {
                    e.printStackTrace();
                } finally {
                    try {
                        if (response != null) {
                            response.close();
                        }
                        if (httpPost != null) {
                            httpPost.releaseConnection();
                        }
                    } catch (IOException e) {
                        e.printStackTrace();
                    }

                }
            }
        }


        public static void main(String[] args) {
            GetThread[] threads = new GetThread[2];
            for (int i = 0; i < 2; i++) {
                threads[i] = new GetThread(HttpClientTests.httpClient, "http://127.0.0.1:7801/test/hello", "{\"name\":\"張三\"}");
            }
            for (Thread tmp : threads) {
                tmp.start();
            }
        }
    }


}

 

求參數 RequestConfig 含義是：

• connectTimeout 請求超時時間
• socketTimeout 等待數據超時時間
• connectionRequestTimeout 連接不夠用時等待超時時間，一定要設置，如果不設置的話，如果連接池連接不夠用，就會線程阻塞。

PoolingHttpClientConnectionManager，參數的意思

• maxTotal 指的是連接池內連接最大數
• defaultMaxPerRoute 一定要設置，路由指的是請求的系統，如訪問 localhost:8080 和 localhost:8081 就是兩個路由。perRote 指的是每個路由的連接數，因爲我目前只連接一個系統的進行數據同步，所以 perRote 設置和最大連接數一樣

因爲我這都是通過一個域名訪問php服務，所以maxTotal和defaultMaxPerRoute都設置一樣，不設置defaultMaxPerRoute默認是2

創建 HttpClient 有兩種方式，兩種方式都統一管理連接

• 一種是單例創建 httpClient，通過 setConnectionManager() 注入連接池(PoolingHttpClientConnectionManager)， httpClient 是線程安全取決於 connectionManager，而該 pool 是線程安全(源碼註釋)，另外官方文檔上也有如下一段話

  While HttpClient instances are thread safe and can be shared between multiple threads of execution, it is highly recommended that each thread maintains its own dedicated instance of HttpContext .

• 另外是可以創建多個 httpClient 實例，但是必須得注入同一個連接池

爲什麼使用連接池？

使用連接池的好處主要有

• 在 keep-alive 時間內，可以使用同一個 tcp 連接發起多次 http 請求。
• 如果不使用連接池，在大併發的情況下，每次連接都會打開一個端口，使系統資源很快耗盡，無法建立新的連接，可以限定最多打開的端口數。

我的理解是連接池內的連接數其實就是可以同時創建多少個 tcp 連接，httpClient 維護着兩個 Set，leased(被佔用的連接集合) 和 avaliabled(可用的連接集合) 兩個集合，釋放連接就是將被佔用連接放到可用連接裏面。

什麼是 Keep-Alive

HTTP1.1 默認啓用 Keep-Alive，我們的 client(如瀏覽器)訪問 http-server(比如 tomcat/nginx/apache)連接，其實就是發起一次 tcp 連接，要經歷連接三次握手，關閉四次握手，在一次連接裏面，可以發起多個 http 請求。如果沒有 Keep-Alive，每次 http 請求都要發起一次 tcp 連接。
下圖是 apache-server 一次 http 請求返回的響應頭，可以看到連接方式就是 Keep-Alive，另外還有 Keep-Alive 頭，其中

• timeout=5 5s 之內如果沒有發起新的 http 請求，服務器將斷開這次 tcp 連接，如果發起新的請求，斷開連接時間將繼續刷新爲 5s
• max=100 的意思在這次 tcp 連接之內，最多允許發送 100 次 http 請求，100 次之後，即使在 timeout 時間之內發起新的請求，服務器依然會斷開這次 tcp 連接

可以通過 tcpdump -i lo0 -n  port 7801查看tcp鏈接過程

看端口變化，新建tcp會開啓新端口

參考

 

使用 httpclient 連接池及注意事項

HttpClient連接池的使用

高併發場景下的httpClient優化使用