There are two main type of exceptions that the user of HttpClient may encounter when executing HTTP methods:
Transport exceptions are those caused by input/output failures such as an unreliable connection or an inability to complete the execution of an HTTP method within the given time constraint (socket timeout). Generally transport exceptions are non-fatal and may be recovered from by retrying the failed method. However, special care must be taken when recovering from exceptions in non-idempotent methods (refer to HTTP transport safety for details).
Generic transport exceptions in HttpClient are represented by the standard Java java.io.IOException class or its sub classes such as java.net.SocketException and java.net.InterruptedIOException.
In addition to standard input/output exception classes HttpClient defines several custom transport exceptions that convey HttpClient specific information.
java.io.IOException +- org.apache.commons.httpclient.NoHttpResponseException
In some circumstances, usually when under heavy load, the web server may be able to receive requests but unable to process them. A lack of sufficient resources like worker threads is a good example. This may cause the server to drop the connection to the client without giving any response. HttpClient throws NoHttpResponseException when it encounters such a condition. In most cases it is safe to retry a method that failed with NoHttpResponseException.
java.io.IOException
+- java.io.InterruptedIOException
+- org.apache.commons.httpclient.ConnectTimeoutExceptionThis exception signals that HttpClient is unable to establish a connection with the target server or proxy server within the given period of time.
java.io.IOException
+- java.io.InterruptedIOException
+- org.apache.commons.httpclient.ConnectTimeoutException
+- org.apache.commons.httpclient.ConnectionPoolTimeoutExceptionThis exception can only occur when using the multithreaded connection manager. The exception signals that the connection manager fails to obtain a free connection from the connection pool within the given period of time.
Protocol exceptions generally indicate logical errors caused by a mismatch between the client and the server (web server or proxy server) in their interpretation of the HTTP specification. Usually protocol exceptions cannot be recovered from without making adjustments to either the client request or the server. Some aspects of the HTTP specification allow for different, at times conflicting, interpretations. HttpClient can be configured to support different degrees of HTTP specification compliance varying from very lenient to very strict.
java.io.IOException +- org.apache.commons.httpclient.HttpException
HttpException represents an abstract logical error in HttpClient. Generally this kind of exception cannot be automatically recovered from.
java.io.IOException
+- org.apache.commons.httpclient.HttpException
+- org.apache.commons.httpclient.ProtocolExceptionProtocolException signals a violation of the HTTP specification. It is important to note that HTTP proxies and HTTP servers can have different level of HTTP specification compliance. It may be possible to recover from some HTTP protocol exceptions by configuring HttpClient to be more lenient about non-fatal protocol violations.
java.io.IOException
+- org.apache.commons.httpclient.HttpException
+- org.apache.commons.httpclient.ProtocolException
+- org.apache.commons.httpclient.auth.MalformedChallengeExceptionINTERNAL
MalformedChallengeException signals that an authentication challenge is in some way invalid or illegal in the given authentication context.
java.io.IOException
+- org.apache.commons.httpclient.HttpException
+- org.apache.commons.httpclient.ProtocolException
+- org.apache.commons.httpclient.auth.AuthenticationExceptionINTERNAL
AuthenticationException signals a failure in the authentication process. Usually authentication exceptions are handled internally when executing HTTP methods and are not propagated to the caller.
java.io.IOException
+- org.apache.commons.httpclient.HttpException
+- org.apache.commons.httpclient.ProtocolException
+- org.apache.commons.httpclient.auth.AuthenticationException
+- org.apache.commons.httpclient.auth.AuthChallengeExceptionINTERNAL
AuthenticationException is thrown when HttpClient is unable to respond to any of the authentication challenges sent by the server.
java.io.IOException
+- org.apache.commons.httpclient.HttpException
+- org.apache.commons.httpclient.ProtocolException
+- org.apache.commons.httpclient.auth.AuthenticationException
+- org.apache.commons.httpclient.auth.CredentialsNotAvailableExceptionINTERNAL
CredentialsNotAvailableException indicates that credentials required to respond to the authentication challenge are not available.
java.io.IOException
+- org.apache.commons.httpclient.HttpException
+- org.apache.commons.httpclient.ProtocolException
+- org.apache.commons.httpclient.auth.AuthenticationException
+- org.apache.commons.httpclient.auth.InvalidCredentialsExceptionINTERNAL
InvalidCredentialsException indicates that the credentials used to respond to the authentication challenge have been rejected by the server.
java.io.IOException
+- org.apache.commons.httpclient.HttpException
+- org.apache.commons.httpclient.ProtocolException
+- org.apache.commons.httpclient.cookie.MalformedCookieExceptionINTERNAL
MalformedCookieException signals that the cookie is in some way invalid or illegal in the given HTTP session context.
There are several cookie specifications that are often incompatible. Thus the validity of a cookie is established within a context of a specific cookie specification used to parse and validate the cookie header(s) sent by the server. If the application needs to process cookies differently from the commonly used cookie specifications, it may choose to provide a custom cookie policy or extend the existing one. Please see cookies for more details.
java.io.IOException
+- org.apache.commons.httpclient.HttpException
+- org.apache.commons.httpclient.ProtocolException
+- org.apache.commons.httpclient.RedirectExceptionRedirectException signals violation of the HTTP specification caused by an invalid redirect response. If the application that uses HttpClient needs to be more lenient about redirect responses, it may choose to disable automatic redirect processing and implement a custom redirect strategy.
It is important to understand that the HTTP protocol is not well suited for all types of applications. HTTP is a simple request/response oriented protocol which was initially designed to support static or dynamically generated content retrieval. It has never been intended to support transactional operations. For instance, the HTTP server will consider its part of the contract fulfilled if it succeeds in receiving and processing the request, generating a response and sending a status code back to the client. The server will make no attempts to roll back the transaction if the client fails to receive the response in its entirety due to a read timeout, a request cancellation or a system crash. If the client decides to retry the same request, the server will inevitably end up executing the same transaction more than once. In some cases this may lead to application data corruption or inconsistent application state.
Even though HTTP has never been designed to support transactional processing, it can still be used as a transport protocol for mission critical applications provided certain conditions are met. To ensure HTTP transport layer safety the system must ensure the idempotency of HTTP methods on the application layer.
HTTP/1.1 specification defines idempotent method as
Methods can also have the property of "idempotence" in that (aside from error or expiration issues) the side-effects of N > 0 identical requests is the same as for a single request.
In other words the application ought to ensure that it is prepared to deal with the implications of multiple execution of the same method. This can be achieved, for instance, by providing a unique transaction id and by other means of avoiding execution of the same logical operation.
Please note that this problem is not specific to HttpClient. Browser based applications are subject to exactly the same issues related to HTTP methods non-idempotency.
By default HttpClient attempts to automatically recover from exceptions. The default auto-recovery mechanism is limited to just a few exceptions that are known to be safe.
HttpClient will make no attempt to recover from any logical or HTTP protocol error (those derived from HttpException class).
HttpClient will automatically retry up to 5 times those methods that fail with a transport exception while the HTTP request is still being transmitted to the target server (i.e. the request has not been fully transmitted to the server).
HttpClient will automatically retry up to 5 times those methods that have been fully transmitted to the server, but the server failed to respond with an HTTP status code (the server simply drops the connection without sending anything back). In this case it is assumed that the request has not been processed by the server and the application state has not changed. If this assumption may not hold true for the web server your application is targeting it is highly recommended to provide a custom exception handler.
In order to enable a custom exception recovery mechanism one should provide an implementation of the HttpMethodRetryHandler interface.
HttpClient client = new HttpClient();
HttpMethodRetryHandler myretryhandler = new HttpMethodRetryHandler() {
public boolean retryMethod(
final HttpMethod method,
final IOException exception,
int executionCount) {
if (executionCount >= 5) {
// Do not retry if over max retry count
return false;
}
if (exception instanceof NoHttpResponseException) {
// Retry if the server dropped connection on us
return true;
}
if (!method.isRequestSent()) {
// Retry if the request has not been sent fully or
// if it's OK to retry methods that have been sent
return true;
}
// otherwise do not retry
return false;
}
};
GetMethod httpget = new GetMethod("http://www.whatever.com/");
httpget.getParams().
setParameter(HttpMethodParams.RETRY_HANDLER, myretryhandler);
try {
client.executeMethod(httpget);
System.out.println(httpget.getStatusLine().toString());
} finally {
httpget.releaseConnection();
}