Chapter 8. Advanced topics

8.1. Custom client connections

In certain situations it may be necessary to customize the way HTTP messages get transmitted across the wire beyond what is possible using HTTP parameters in order to be able to deal non-standard, non-compliant behaviours. For instance, for web crawlers it may be necessary to force HttpClient into accepting malformed response heads in order to salvage the content of the messages.

Usually the process of plugging in a custom message parser or a custom connection implementation involves several steps:

  • Provide a custom LineParser / LineFormatter interface implementation. Implement message parsing / formatting logic as required.

    class MyLineParser extends BasicLineParser {
        public Header parseHeader(
                final CharArrayBuffer buffer) throws ParseException {
            try {
                return super.parseHeader(buffer);
            } catch (ParseException ex) {
                // Suppress ParseException exception
                return new BasicHeader("invalid", buffer.toString());
  • Provide a custom OperatedClientConnection implementation. Replace default request / response parsers, request / response formatters with custom ones as required. Implement different message writing / reading code if necessary.

    class MyClientConnection extends DefaultClientConnection {
        protected HttpMessageParser createResponseParser(
                final SessionInputBuffer buffer,
                final HttpResponseFactory responseFactory, 
                final HttpParams params) {
            return new DefaultResponseParser(
                    new MyLineParser(), 
  • Provide a custom ClientConnectionOperator interface implementation in order to create connections of new class. Implement different socket initialization code if necessary.

    class MyClientConnectionOperator extends DefaultClientConnectionOperator {
        public MyClientConnectionOperator(final SchemeRegistry sr) {
        public OperatedClientConnection createConnection() {
            return new MyClientConnection();
  • Provide a custom ClientConnectionManager interface implementation in order to create connection operator of new class.

    class MyClientConnManager extends SingleClientConnManager {
        public MyClientConnManager(
                final HttpParams params,
                final SchemeRegistry sr) {
            super(params, sr);
        protected ClientConnectionOperator createConnectionOperator(
                final SchemeRegistry sr) {
            return new MyClientConnectionOperator(sr);

8.2. Stateful HTTP connections

While HTTP specification assumes that session state information is always embedded in HTTP messages in the form of HTTP cookies and therefore HTTP connections are always stateless, this assumption does not always hold true in real life. There are cases when HTTP connections are created with a particular user identity or within a particular security context and therefore cannot be shared with other users and can be reused by the same user only. Examples of such stateful HTTP connections are NTLM authenticated connections and SSL connections with client certificate authentication.

8.2.1. User token handler

HttpClient relies on UserTokenHandler interface to determine if the given execution context is user specific or not. The token object returned by this handler is expected to uniquely identify the current user if the context is user specific or to be null if the context does not contain any resources or details specific to the current user. The user token will be used to ensure that user specific resources will not be shared with or reused by other users.

The default implementation of the UserTokenHandler interface uses an instance of Principal class to represent a state object for HTTP connections, if it can be obtained from the given execution context. DefaultUserTokenHandler will use the user principle of connection based authentication schemes such as NTLM or that of the SSL session with client authentication turned on. If both are unavailable, null token will be returned.

Users can provide a custom implementation if the default one does not satisfy their needs:

DefaultHttpClient httpclient = new DefaultHttpClient();
httpclient.setUserTokenHandler(new UserTokenHandler() {

    public Object getUserToken(HttpContext context) {
        return context.getAttribute("my-token");

8.2.2. User token and execution context

In the course of HTTP request execution HttpClient adds the following user identity related objects to the execution context:

  • ClientContext.USER_TOKEN='http.user-token':  Object instance representing the actual user identity, usually expected to be an instance of Principle interface

One can find out whether or not the connection used to execute the request was stateful by examining the content of the local HTTP context after the request has been executed.

DefaultHttpClient httpclient = new DefaultHttpClient();
HttpContext localContext = new BasicHttpContext();
HttpGet httpget = new HttpGet("http://localhost:8080/"); 
HttpResponse response = httpclient.execute(httpget, localContext);
HttpEntity entity = response.getEntity();
Object userToken = localContext.getAttribute(ClientContext.USER_TOKEN);
System.out.println(userToken); Persistent stateful connections

Please note that a persistent connection that carries a state object can be reused only if the same state object is bound to the execution context when requests are executed. So, it is really important to ensure the either same context is reused for execution of subsequent HTTP requests by the same user or the user token is bound to the context prior to request execution.

DefaultHttpClient httpclient = new DefaultHttpClient();
HttpContext localContext1 = new BasicHttpContext();
HttpGet httpget1 = new HttpGet("http://localhost:8080/"); 
HttpResponse response1 = httpclient.execute(httpget1, localContext1);
HttpEntity entity1 = response1.getEntity();
Principal principal = (Principal) localContext1.getAttribute(

HttpContext localContext2 = new BasicHttpContext();
localContext2.setAttribute(ClientContext.USER_TOKEN, principal);
HttpGet httpget2 = new HttpGet("http://localhost:8080/"); 
HttpResponse response2 = httpclient.execute(httpget2, localContext2);
HttpEntity entity2 = response2.getEntity();