Reliable Integration with Third-Party APIs in PHP
BONUS: We have discussed this topic with an expert in the PHP community in our podcast:
It’s a fact of modern software development that aspects of our applications interact with third-party APIs. This could be for any number of reasons, with some common ones being payment processing, telecommunications, logging, and data analysis.
So, since our applications rely upon third-party APIs so much, we need to ensure that we integrate with them as effectively — and defensively — as we can. Otherwise, when the APIs break, so too will our application — and its reputation with our users.
To help avoid a loss of reputation, I’m going to cover five ways in which you can integrate with third-party APIs and ensure a minimum of downtime and stress.
Implement API Monitoring
Cloudflare status page
Let’s start with monitoring. Even though third-party APIs aren’t part of your infrastructure, you still have to know whether they’re working or not. This is especially important when you consider that some of the most notable internet platforms have had outages in recent years.
Here are some of the most notable:
- Facebook has had two outages in 2019 so far, one in March and another in April. These outages, as you would expect, had a knock-on effect on Facebook’s most well-known services, WhatsApp, and Instagram.
- Google, Amazon, and Reddit experienced outages on Jun 24, this year. These impacted a significant number of other services and businesses which rely upon Google and Amazons extensive service offerings. Did you do any meaningful work that day?
- Twitter had an outage in April 2018.
To monitor effectively, you need to integrate monitoring services into your infrastructure and make them a standard part of your development processes. Like any aspect of software development, there are many sites and vendors on the market that provide monitoring services.
A number of the more well-known ones are:
Alternatively, several third-party APIs provide a status endpoint, such as the one below from Cloudflare, which are easy enough to integrate against. They may offer SDKs which make the process relatively trivial. Alternatively, you could roll a homegrown solution, such as the one-liner below, which checks if Cloudflare is currently online, using Selenium with Python.
from selenium import webdriver
executable_path = "path/to/geckodriver" domain = "https://www.cloudflarestatus.com/" xpath = "//div/span[starts-with(@class,'status')]"
options = webdriver.FirefoxOptions() options.headless = True driver = webdriver.Firefox(options=options, executable_path=executable_path) driver.get(domain)
element = driver.find_element_by_xpath(xpath) print(element.text) driver.quit() Respect the APIs Rate Limits
Most APIs — especially when using guest accounts or introductory free tiers — limit the number of requests, by using rate limits. Rate limits restrict clients’ ability to make more than a fixed number of requests to an API within a given time frame.
The time frame can range from requests per minute to requests per day. When rate limits are reached, subsequent requests are either temporarily blocked or throttled, until the time frame expires. One example is [Git Hub’s API], which rate limits requests to 5,000 requests per hour.
There are many reasons why APIs implement rate-limiting. One of the most common is to avoid a malicious script hogging all of an APIs resources.
There are several ways in which APIs communicate this information. Using Git Hub’s API as our working example; the API sends three response headers, these are:
| Header | Description |
|---|---|
X-RateLimit-Limit |
The maximum number of requests you’re permitted to make per hour. |
X-RateLimit-Remaining |
The number of requests remaining in the current rate limit window. |
X-RateLimit-Reset |
The time at which the current rate limit window resets in UTC epoch seconds. |
Here’s an example, taken from their API documentation:
HTTP/1.1 200 OK Date: Mon, 01 Jul 2013 17:27:06 GMT Status: 200 OK X-RateLimit-Limit: 60 X-RateLimit-Remaining: 56 X-RateLimit-Reset: 1372700873 From this information, you can see that:
- Requests are rate limited to a maximum of 60 requests per hour.
- 56 requests remain within the current time frame.
- The rate limit window resets on Mon Jul 1 19:47:53 CEST 2013 (
date -d @1372700873).
Ensure that this information is used in your application’s integration code, to respond appropriately when the rate limit draws near and when it is exceeded.
Applications should also check if servers are overwhelmed by checking a response’s HTTP status code. If an HTTP 429 (Too Many Requests) is returned, you know that the server’s under high load and can’t currently respond.
For an amusing introduction to rate limiting, check out What is API Rate Limiting All About?.
Implement Retries
There are several ways in which code can handle rate limits (and API request failures). The application could choose to ignore exceptions thrown when rate limits are reached, but doing so isn’t altogether helpful. The application could instead be coded to back off when approaching the rate limit.
Alternatively, a third approach is to implement retries. In short, retries attempt to complete a failed API request a set number of times within a set interval.
If one of the requests completes successfully, then the job is removed. If not, then the failed request is logged for later analysis. Retries could likely be handled by a combination of a queueing server and a background process.
If rate limits are regularly being exceeded, it may be time to either revisit your application’s architecture or to consider increasing your rate plan. That said, use monitoring to stay abreast of request volume and rate limit overruns.
Always Have Tests
It should go without saying, but tests are essential for code that integrates against third-party APIs, as well as everything else. I was all too glad that I had them in place some years ago when I was integrating against a third-party API.
While the vendor’s documentation was, for the most part, quite good, the stability of the API wasn’t! As a result, requests to the vendor’s API would intermittently fail. By having tests in place, however, along with the code under version control, we could verify that our code:
- Hadn’t changed
- Still integrated correctly, based on the latest available version of the vendor’s API documentation.
Knowing this was particularly important. This was because the vendor’s first resort was often to say that there must be some mistake or change in our code because the relevant section of their API had not changed. Having tests showed me that our code was unchanged and gave me the confidence to pursue the fact that a recent release on their end was responsible for the outage.
Read the Documentation
Stripe documentation home page
A well-written API has well written and feature-complete documentation, containing the details of, but not limited to:
- Client errors
- Every endpoint
- Example requests
- How to authenticate
- Pagination
- Request parameters (which document every field, its type, and a short description)
- Response headers
- Response objects (which document every field, its type, as well as a short description).
In addition to this, the API documentation is laid out in a well-structured, intuitive, and easy-to-read format. An excellent example of this is Stripe’s API documentation. I’ve used it on several occasions and have found it to be a great example.
Assuming that the API documentation is updated with each API release, you can refer to it with confidence. If you’re looking to integrate with a third-party API and its API is either poorly documented — or worse, not documented — I strongly suggest choosing an alternative vendor.
In Conclusion
Those are five ways that help ensure reliable communication when integrating against third-party APIs. I hope they help make the process simpler, more reliable, and more efficient.
