Using HTTP client timeouts in PHP

Timeouts are a rarely discussed topic and neglected in many applications, even though they can have a huge effect on your site during times of high load and when dependent services are slow. For microservice architectures timeouts are important to avoid cascading failures when a service is down.

The default socket timeout in PHP is 60 seconds. HTTP requests performed with for example file_get_contents, fopen, SOAPClient or DOMDocument::load are using this timeout INI setting to decide how long to wait for a response.

Since the socket/stream wait time is not included in PHPs max execution time you are in line for a surprise when requests take up to 30+60=90 seconds of your webservers precious processing queue before getting aborted.

Timeouts are scary, because you don’t know in what state you have left an operation when aborting. But in case of HTTP requests it is usually easy as a client to decide what to do:

  1. Abort my own request and show the user an error
  2. Ignore the timeout and just don’t display the data
  3. Retry (a maximum of 3-5 times for example)

And configuring timeouts has other upsides:

  • If the target server currently has load problems, then aborting with a timeout early could reduce the servers load enough to automatically recover at some point.
  • You can show users an error quickly and they don’t have to wait several seconds only to be taunted by an error page.

Setting the default timeout

Before doing anything else, you should decrease the default timeout early in your code to a value between 5 and 10 seconds:

<?php

ini_set("default_socket_timeout", 10);

You should check your monitoring system (for example Tideways) for clues what a regular HTTP call duration is for calls to internal or third party systems.

If you have many different internal or third-party services in place, then it makes sense to configure individual timeouts based on their usual latency.

A thorough setup defines maximum limits for each internal and third party service and then encodes them in HTTP calls.

Configuring the timeout for individual file_get_contents/fopen calls

In stream based calls you can configure individual timeouts using stream contexts. This applies to for example file_get_contents and fopen:

<?php

// timeout of one second 
$context = stream_context_create(array('http' => array(
     'timeout' => 1.0,
     'ignore_errors' => true,
)));

$data = @file_get_contents("http://127.0.0.1/api", false, $context);

if ($data === false && count($http_response_header) === 0) {
     // request timed out, because $data is false even though we ignore
     // errors and we dont have response headers 
}

Be aware that the timeout handling of streams seems not very accurate and requests can still take longer than you configured, especially when you go below 1 second.

Configuring the timeout for DOMDocument::load

In general you should not use the DOMDocument::load function to load remove XML or HTML. To allow for better error handling code it’s easier to use a HTTP client directly and then load with loadXML. But if you must use it or some library does, you can configure timeouts with stream contexts again in this way:

<?php

// timeout of one second
$context = stream_context_create(array('http' => array('timeout' => 1.0)));

libxml_set_streams_context($context);

$document = new DOMDocument();
$document->load('http://127.0.0.1/xml');

This affects all HTTP calls that PHP does through libxml.

Configuring the timeout for SOAPClient

The SOAPClient class does not use the PHP stream API for historical reasons. This means that timeout configuration works slightly different. Setting the timeout parameter in SOAPClients stream context does not work.

Instead you can either set the connection_timeout setting that handles timeouts before the connection was established or directly set default_socket_timeout in the code using ini_set. The only other way is to overwrite SoapClient::__doRequest method to use cURL.

<?php

ini_set('default_socket_timeout', 1);
$client = new SOAPClient($wsdl, array('connection_timeout' => 1));

try {
     $client->add(10, 10);
 } catch (SOAPFault $e) {
     if (strpos($e->getMessage(), 'Error Fetching http headers') !== false) {
         // this can be a timeout!
     } 
}

Configuring the timeout for cURL extension

If you want to have more control about HTTP requests than with PHPs built in stream support there is no way around the cURL extension.

We have much better timeout control with cURL, with two settings: one for the connection timeout and one for the maximum execution time:

<?php

$ch = curl_init('http://127.0.0.1/api');
curl_setopt($ch, CURLOPT_TIMEOUT, 2); 
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 1);

$response = curl_exec($ch);

if ($response === false) {
     $info = curl_getinfo($ch);

    if ($info['http_code'] === 0) {
         // timeout
     } 
}

If you want to be more strict and abort on a millisecond level, then you can alternatively use CURLOPT_TIMEOUT_MS and CURLOPT_CONNECTTIMEOUT_MS constants, but the manual warns that this might still only be checked every full second depending on how cURL was compiled and setting a value below one second usually requires to ignore signals with curl_setopt($ch, CURLOPT_NOSIGNAL, 1); to avoid an immediate timeout error.

One important thing to remember is cURL has an indefinite timeout by default and does not obey the default_socket_timeout INI setting. This means that you have to configure the timeout in every cURL call in your code. With third party code this might even be complicated or impossible.

Benjamin Benjamin 04.01.2017