Testing if Franken PHP Classic Mode is faster and more scalable than PHP-FPM

FrankenPHP bills itself as “The Modern PHP App Server” and provides a single-process solution to running PHP. Its worker mode allows to greatly increase the performance of cleanly developed applications that are compatible with it by saving on framework bootstrapping time.

Unfortunately, not every application is compatible with worker mode. Depending on the amount of global state kept by the application, updating it for worker mode is not always easy.

For completeness, we investigate if FrankenPHP in “Classic Mode” is also able to speed up your application. Prior research by Ivan Vulovic in https://vulke.medium.com/frankenphp-vs-php-fpm-benchmarks-surprises-and-one-clear-winner-173231cb1ad5 suggests it is – but the numbers seem a little too good to be true.

In this blog post, we aim to verify these results with the explicit goal of testing PHP runtime performance itself, and not the performance of PHP script execution, Linux scheduler, web server, or other influences. Our results show that there is essentially no performance or throughput difference between PHP-FPM vs. FrankenPHP Classic-Mode.

Benchmark Setup

As in our previous article comparing the performance of different PHP versions across popular applications, we’re using a Hetzner VPS with 8 dedicated vCores on an AMD EPYC 7003 or EPYC 9004 (Hetzner CCX33). This time, the operating system is the newly released Debian 13 (“Trixie”).

We’ll be comparing FrankenPHP v1.9.1 (PHP 8.4.12 Caddy v2.10.2) against nginx/1.26.3 paired with PHP-FPM 8.4.11. The measurements are performed using Vegeta v12.12.0.

We’re installing FrankenPHP as a Standalone Binary using the recommended curl |sh command. Both nginx and PHP-FPM are installed using Debian Trixie’s official package repositories.

After installation, we only make a small change to nginx’ configuration, adjusting the port to avoid conflicts and to enable the PHP configuration block provided by Debian.

At this point both FrankenPHP and the nginx+FPM combination run with their default configuration, without any performance tuning.

Given that we want to measure the overhead added or removed by selecting a different runtime environment and not the performance of PHP itself, we’re going to test with simple scripts that do not contain any actual business logic.

In addition, we are not testing with a lot of concurrency; instead, we spawn 1 worker per CPU. For the full Vegeta output including more throughput and response percentiles click on the “test label” in the first column of each result table below.

Testing HTML Response Generation

A common server-side rendered HTML landing page contains about 50 KiB of data, so let’s use this one as the basic benchmark:

Response	FPM rps	FrankenPHP Classic rps	FPM 99% ms	FrankenPHP Classic 99%
HTML	7023.11	6934.06	2.022 ms	2.06 ms

Across this 60-second benchmark with 8 clients hammering the server as fast as possible, the nginx+FPM combination was able to serve 7023 requests per second (with overall better latency), whereas FrankenPHP in classic mode could serve 6934 requests (with a little better maximum latency). Nginx+FPM are thus roughly 1.3% faster than FrankenPHP in classic mode. Not enough to really matter.

Case closed? Not yet. A landing page is just one of the cases you’re going to experience in your application. What if, for some reason, the results change for other use cases?

Testing Binary Response Generation

Instead of an HTML response, let’s try a binary response format, for example, PDF:

Response	FPM rps	FrankenPHP Classic rps	FPM 99% ms	FrankenPHP Classic 99%
HTML	7023.11	6934.06	2.022 ms	2.06 ms
PDF	7610.31	5368.85	1.828 ms	4.246 ms

It seems that FPM got faster (with even better latencies) and FrankenPHP got slower, only by changing the content-type from text/html to application/pdf, keeping the response exactly the same:

Let’s try again with an “encrypted” base64-encoded payload being embedded into our HTML. Encryption is simulated by random data:

Response	FPM rps	FrankenPHP Classic rps	FPM 99% ms	FrankenPHP Classic 99%
HTML	7023.11	6934.06	2.022 ms	2.06 ms
PDF	7610.31	5368.85	1.828 ms	4.246 ms
Random	1940.05	5606.22	7.704 ms	3.143 ms

Both FPM and FrankenPHP became slower, but that is expected since the script now does much more work. This time FrankenPHP beats FPM 2.89×. An odd result, that can be explained later in the blog post.

What if we try to do even less work than our landing page example rather than more work? Let’s try with a Hello World example.

Response	FPM rps	FrankenPHP Classic rps	FPM 99% ms	FrankenPHP Classic 99%
HTML	7023.11	6934.06	2.022 ms	2.06 ms
PDF	7610.31	5368.85	1.828 ms	4.246 ms
Random	1940.05	5606.22	7.704 ms	3.143 ms
Hello World	18478.69	18403.81	0.904 ms	0.877 ms

Both FPM and FrankenPHP are serving an impressive 18400 requests per second at a 0.4% difference in RPS.

Testing With Higher Concurrency

The article by Ivan Vulovic compared the performance with higher concurrency. While we believe this not to be particularly useful, since we would be measuring the performance of the operating system’s scheduler more than anything else, perhaps FrankenPHP is doing something to be more friendly to the scheduler. Let’s see with the Hello World example that is comparable to the tests done by Ivan Vulovic:

Response	FPM rps	FrankenPHP Classic rps	FPM 99% ms	FrankenPHP Classic 99%
HTML	7023.11	6934.06	2.022 ms	2.06 ms
PDF	7610.31	5368.85	1.828 ms	4.246 ms
Random	1940.05	5606.22	7.704 ms	3.143 ms
Hello World	18478.69	18403.81	0.904 ms	0.877 ms
Hello World Concurrency 100	21847.61	22675.24	9.742 ms	23.365 ms

FrankenPHP serves roughly 3.7% more requests per second, but at a much worse latency distribution.

Testing with Optimized Configuration

Up until this point, we were running the “default configuration” for both FrankenPHP and nginx+FPM. Let’s look into some performance tuning.

FrankenPHP provides an official Performance Optimization section in the documentation. No actionable FrankenPHP-specific advice is provided, except for “Don’t use Musl”, which we are not.

For nginx and FPM, there’s no official guide that we are aware of, but we’ve made the following changes:

• Disabled access logs in nginx (matching FrankenPHP).
• Enabled fastcgi_keep_conn in nginx.
• Configured pm=static and pm.max_children=16 in FPM (matching FrankenPHP’s num_threads).

Making these changes did not result in any differences worth mentioning.

Accounting for Output Compression Performance

So, how can all the above results be explained, especially the sudden deviations in the PDF and Random test scenarios?

It appears we were not actually measuring the performance or overhead of PHP-FPM vs. FrankenPHP, but rather the quality of the output compression implementation of nginx and Caddy, respectively.

Vegeta is setting a accept-encoding: gzip request header, and both nginx and FrankenPHP will automatically compress responses coming from PHP when the circumstances are right:

Is Webserver compressing?	FrankenPHP	nginx
HTML	yes	yes
PDF	no	no
Random	yes	yes
Hello World	no	yes

In the case of the PDF example, the response will not get compressed with either FrankenPHP or nginx. In the case of nginx, skipping this compression seems to improve performance; FrankenPHP seems to handle this case of a larger response body not particularly well. On the other hand, nginx is not able to handle incompressible random responses well when attempting to compress them.

We could also confirm this by setting an accept-encoding: identity request header with Vegeta to disable the compression.

Using the landing page example with 8 concurrent requests shows numbers comparable to the PDF invoice when compression is disabled:

Response	FPM rps	FrankenPHP Classic rps	FPM 99% ms	FrankenPHP Classic 99%
HTML	7023.11	6934.06	2.022 ms	2.06 ms
PDF	7610.31	5368.85	1.828 ms	4.246 ms
Random	1940.05	5606.22	7.704 ms	3.143 ms
Hello World	18478.69	18403.81	0.904 ms	0.877 ms
Hello World Concurrency 100	21847.61	22675.24	9.742 ms	23.365 ms
HTML No GZIP	7456.47	5504.16	1.786 ms	3.993 ms

We also did some final tests with HAProxy 3.0.11 instead of nginx as the FastCGI gateway in front of PHP-FPM and were able to squeeze out some additional RPS with even better latency compared to nginx, but not really enough to matter in practice.

Conclusion

As a conclusion, we can confidently say the differences in overhead between PHP-FPM and FrankenPHP in classic mode are not relevant enough to suggest moving to FrankenPHP Classic Mode is always better.

As with our previous benchmarks, we can conclude that your application’s architecture has the biggest impact on performance, that there is no shortcut that will magically make things faster.

Shaving off the last microsecond from PHP’s startup performance will not make an impact when spending several hundred milliseconds waiting for the database.

An APM such as Tideways can help find this kind of offender within your application code.

As for choosing a PHP runtime, use whatever your team is comfortable with already.

1. When using Caddy with PHP-FPM, moving to FrankenPHP can reduce operations complexity.
2. When you want to benefit from one of the FrankenPHP exclusive features, like 103 early hints, Go extensions, or Mercure realtime support, a move from PHP-FPM to Classic mode might make sense.
3. In other cases PHP-FPM still has an edge in performance when your application code is already optimized but not yet ready for worker mode.