HTTP client

Introduction

The HTTP client is how your application calls other servers: a payment provider, a weather API, another of your own services.

Always use it in place of cURL, Guzzle or file_get_contents(). Those hold the whole worker until the other server answers, so one slow API stalls every request the worker has. This client waits on the event loop: while a request is on its way, the worker serves other requests. That holds for every step, finding the server's address, connecting, encrypting, sending and waiting.

It is the PSR-18 client, so a library that asks for Psr\Http\Client\ClientInterface is given the same one.

Configuration

The HTTP client is registered by default by App::configure(), so it needs nothing in app.php.

https needs PHP's openssl extension.

Making requests

Ask for the client in a constructor:

use Crab\Contracts\HttpClient\Client;

final readonly class ShowWeather
{
    public function __construct(private Client $http)
    {
    }

    public function handle(string $city): array
    {
        $weather = $this->http->get('https://api.weather.example/v1/now', query: ['city' => $city]);

        return $weather->ok() ? $weather->json() : ['error' => $weather->getStatusCode()];
    }
}

Route::get('/weather/{city}', ShowWeather::class);

$http->get($url, query: ['page' => 2], headers: ['Accept' => 'application/json']);
$http->post($url, json: ['name' => 'Ada']);
$http->put($url, json: $data);
$http->patch($url, json: ['name' => 'Ada L.']);
$http->delete($url);
$http->request('PUT', $url, body: $csv, headers: ['Content-Type' => 'text/csv']);

Sending data

json: null sends no JSON body, and form: [] without files sends no form body. To send the JSON literal null, use request() with body: 'null', headers: ['Content-Type' => 'application/json'].

Reading the answer

The answer is a PSR-7 response with three additions:

$response->ok();                    // whether the status is 200 to 299
$response->json();                  // the body, decoded
$response->throw();                 // throws unless the status is 2xx; otherwise returns the response
$response->getStatusCode();
$response->getHeaderLine('Content-Type');
(string) $response->getBody();

Errors

An error status is an answer. A 404 or a 500 is returned like any other, because what to do about it is your decision:

$response = $http->get("https://api.example.com/users/{$id}");
if ($response->getStatusCode() === 404) {
    throw new HttpException(404, 'No such user.');
}

When anything but a 2xx is a failure for you, say so with throw(). That includes a 304 and a redirect that was not followed:

$user = $http->get($url)->throw()->json();

No answer always throws. All three are a Psr\Http\Client\ClientExceptionInterface, and no message holds the query string or a header, so they are safe to log.

json() throws JsonException for an empty or malformed JSON body, and LengthException when any JSON object has more than 1,000 members, including repeated names. Lists of smaller objects may contain more than 1,000 items. Exceeding the redirect limit throws Crab\HttpClient\ClientException.

Timeouts

$http->timeout(2.0)->get($url);

timeout() returns a client that allows the whole call that many seconds: finding the address, connecting, waiting for a free connection, redirects, retries and their pauses all share one deadline. Each later try gets the time left. http_client.connect_timeout also limits each connection, within that remaining time. The client you called it on is unchanged, which matters because every request a worker serves shares it.

Give every call to another service a timeout you chose. The default of 30 seconds is longer than most clients of your API will wait.

To give up when your own client has gone, pass its cancellation:

$http->request('GET', $url, cancellation: $request->cancellation());

Trying again

$http->retry(3)->get($url);                 // up to three more times
$http->retry(3, delay: 0.5)->get($url);     // random pauses: 0.375–0.5 s, 0.75–1 s, 1.5–2 s
$http->timeout(2)->retry(3)->get($url);     // two seconds for the whole call

A request is tried again only when that cannot do it twice:

Retrying also requires an empty or seekable body. A request outside the first row is never retried after following a redirect, even if the next hop fails or asks for a retry.

A server's Retry-After replaces a shorter delay and must fit before the whole-call deadline. A 500 is never tried again: a server that failed may fail the same way.

If the next pause cannot fit before the deadline, the call ends at once with the last try's result: its response is returned, or its failure is thrown. No more requests are sent. A try or connection that itself runs out of time throws NetworkException.

For work that must eventually succeed, such as delivering a webhook, do not retry in the request. Push a job, which retries over minutes.

Sending files

use Crab\Contracts\HttpClient\File;

$http->post($url, form: ['title' => 'Q3'], files: [
    'report' => File::fromPath('/tmp/q3.pdf', type: 'application/pdf'),
    'notes' => File::fromContents($text, 'notes.txt', 'text/plain'),
    'avatar' => $request->file('avatar'),               // an upload you received, passed on
    'photos' => [File::fromPath($one), File::fromPath($two)],
]);

Files are copied in chunks into a temporary body stream, which spills to disk when large; small bodies may stay in memory.

Downloading files

Give sink: a path and the body goes straight into that file as it arrives, never into memory or a temporary copy. Use it for anything you keep as a file: a ROM, an export, a backup.

$http->timeout(1200)->request('GET', $romUrl,
    sink: '/srv/roms/7.zip',
    maxBytes: 4 * 1024 * 1024 * 1024,           // 4 GiB; the default is 64 MiB
    decode: false,                              // the bytes as the server has them
    cancellation: $cancellation,
)->throw();

• The file appears only when the body is all there. The body is written to a hidden file beside it and renamed into place at the end. A download that fails, is cut short or is too large leaves nothing behind, and an old file at the path stays until a new one replaces it.
• Only a 2xx answer is written to the file. Any other answer is kept as usual, so $response->json() reads the error the server sent, and throw() reports it. A redirect's own body never reaches the file.
• The response's body is then the file, opened for reading.
• A retry starts over with a new file, so nothing is appended twice.
• The folder must exist and be writable. If it is not, the request is refused before anything is sent.

sink: also takes a writable PSR-7 stream, written from where it is, and the response's body is then that stream. A failure leaves what arrived before it in the stream, and the request is not tried again once any of the body went into it. Give a path when a failed download must leave nothing.

decode: false sends no Accept-Encoding and keeps a compressed body as it came, with its Content-Encoding and Content-Length. Use it when the bytes are checked against a hash or stored as they are. With the default, decode: true, a body sent with gzip or deflate is decompressed and those two headers are removed.

Streaming answers

stream() takes what request() takes, but returns as soon as a successful answer's head arrives, and its body is read as it comes. Use it for an AI model's reply or any other answer that a server sends bit by bit:

$reply = $http->timeout(300)->stream('POST', 'https://api.example.com/v1/chat',
    json: ['messages' => $messages, 'stream' => true],
    cancellation: $request->cancellation(),
)->throw();

foreach ($reply->events() as $event) {       // server-sent events, each once it has arrived
    if ($event->data === '[DONE]') {
        break;
    }
    $text = $event->json()['choices'][0]['delta']['content'] ?? '';
}

• Reading waits, and so does the request. A read waits until more has arrived. The request stops taking more once the reader is a quarter of a megabyte behind, so a slow reader never makes the body pile up in memory.
• Letting go stops it. Break out of the loop and drop the response, or call close() on its body, and the rest is not received. Cancelling cancellation: stops it too.
• An error status is read whole, as request() reads it, so throw() and json() work as usual.
• The timeout covers the whole answer. Raise it for a long one, as above.
• idle: gives up on a server that has stopped sending. With idle: 60.0, the call waits at most 60 seconds for the answer's head, and each read at most 60 seconds for more of the body; then the request is given up and a NetworkException says that nothing arrived. Pair it with a high timeout(), so a long answer that keeps coming is never cut, and one that stalls is: $http->timeout(3600)->stream('POST', $url, json: $prompt, idle: 60.0).
• maxBytes: and decode: hold as for request(). Redirects are followed and an answer that broke off before any of its body arrived is tried again as usual. Once any of the body has been read, the request is not made again.

events() and lines() work on any answer, not only a streamed one. The two together relay a model's reply to a browser: see Streaming.

Limiting the size of an answer

No answer's body may be longer than http_client.max_bytes, 64 MiB by default. That holds for every call, sendRequest() included, so a server that sends more than you asked for, by mistake or on purpose, cannot fill the memory or the disk. Give one request a different limit with maxBytes::

$http->request('GET', $reportUrl, maxBytes: 512 * 1024 * 1024);

The limit is checked as the body arrives. An answer whose Content-Length is already over the limit is refused before a byte of it is read. Otherwise the body is given up the moment it passes the limit. Either way the client throws Crab\HttpClient\TooLargeException, closes the connection or HTTP/2 stream it came on, and leaves no file behind. A compressed body counts both as received and as decompressed, so a small answer that expands to gigabytes is stopped too.

The limit applies to each answer, redirects included. A TooLargeException is not retried, since the same answer would come back.

A client for one API

Make a client with the API's address and credentials, once, in a plugin, and bind it under a name of your own:

// app/Setup.php
use Crab\HttpClient\Client;

$container->singleton(StripeClient::class, static fn (): StripeClient => new StripeClient(
    new Client('https://api.stripe.com/v1', headers: ['Authorization' => 'Bearer ' . config('stripe.key')]),
));

final readonly class StripeClient
{
    public function __construct(private Client $http)
    {
    }

    public function charge(int $amount, string $source): array
    {
        return $this->http->post('/charges', form: ['amount' => $amount, 'source' => $source])->throw()->json();
    }
}

Wrapping the client in a class of your own gives the rest of the application methods that mean something, and one place to fake in tests.

A redirect to another origin drops only Authorization and Cookie; other credential headers, such as X-Api-Key, remain. Send those credentials with redirects: 0 on the client, or follow redirects only from a base URL you trust.

Cookies

A client keeps no cookies unless it is given a jar, because one jar on a client that every request shares would hand one caller's cookies to the next. Give each caller a separate jar for each site. A jar accepts a cookie for a parent domain, such as shop.co.uk from api.shop.co.uk, but never for a public suffix such as co.uk or github.io, under which unrelated sites live: it checks the Public Suffix List, which ships with the framework. A cookie whose domain is the public suffix it came from is kept for that host alone:

use Crab\HttpClient\CookieJar;

$session = $http->cookies(new CookieJar());
$session->post('https://shop.example/login', form: ['user' => 'ada', 'pass' => $secret]);
$session->get('https://shop.example/orders');            // sent with the cookie the login set

A plaintext HTTP response cannot replace, delete or shadow an unexpired Secure cookie of the same name on an overlapping domain and path. This also holds while following redirects. HTTPS responses can update or remove those cookies normally.

A proxy

http_client.proxy, or proxy: on a client of your own, sends every request through an HTTP proxy: http://user:[email protected]:8080. An https request goes through a tunnel the proxy cannot read.

What it does not do

• HTTP/3, HTTP/2 without encryption, and server push.
• Proxies reached over TLS, and SOCKS proxies.
• Transfer codings other than chunked. An HTTP/1.1 answer sent with any other Transfer-Encoding, or with that and a Content-Length too, or with Content-Length values that disagree, is refused as malformed.
• Brotli. With PHP's zlib extension, answers compressed with gzip or deflate are decompressed for you unless you pass decode: false; without it they remain encoded.

HTTP/2 is spoken with every https server that offers it: all requests to one server then share one connection and run at once. With the others it is HTTP/1.1 on connections that are kept and reused, up to http_client.connections to one server.

In tests

FakeClient is the real client with replies written in advance in place of the network:

use Crab\Contracts\HttpClient\Client;
use Crab\HttpClient\FakeClient;
use Psr\Http\Message\RequestInterface;

$http = new FakeClient([
    'GET https://api.example.com/users/7' => ['id' => 7, 'name' => 'Ada'],                       // JSON, 200
    'POST https://api.example.com/charges' => FakeClient::response(402, ['error' => 'declined']),
    'https://slow.example.com/*' => new RuntimeException('it timed out'),                         // thrown
    'https://echo.example.com/*' => fn (RequestInterface $request) => (string) $request->getBody(),
]);
$app->container->instance(Client::class, $http);

// ... run the code under test ...

self::assertCount(1, $http->sent('POST https://api.example.com/charges'));
self::assertSame('{"amount":500}', (string) $http->requests[0]->getBody());

• A reply is found by method and URL, the first that matches. * stands for anything, and the method may be left out. Patterns without ? ignore the query string; include it and the expected query or wildcards to distinguish query values.
• A request nothing matches throws, naming it, so a test never reaches the network.
• Everything else is the real client: base URL, headers, redirects, cookies and throw() behave as they do over the network. So do sink:, maxBytes: and decode:: a reply longer than the limit throws TooLargeException, and a gzip reply is decompressed unless decode: false is given.

use Crab\HttpClient\TooLargeException;

$http = new FakeClient([
    'GET https://roms.example/7' => FakeClient::response(200, str_repeat('x', 4096)),
], maxBytes: 1024);

$this->expectException(TooLargeException::class);
$http->request('GET', 'https://roms.example/7', sink: $path);