Table of contents of the article:
Few companies can compete with Cloudflare's expertise in serving massive web traffic.
To be more precise, let's say this: Cloudflare handles over 10% of all HTTP/HTTPS traffic worldwide. Additionally, globally, the Cloudflare network serves over 25 million HTTP requests per second and is used by nearly 80% of all websites using reverse proxy services.
So, one thing is certain: the company has faced colossal traffic loads and pushed the capabilities of modern technology to the limit.
As a reverse proxy that proxy traffic between the Cloudflare network and servers on the Internet, Nginx has been a vital part of CloudFlare's architecture, up until now, even considering the significant improvements and optimizations they have made in the implementation of HTTP/2 and the “new” QUIC or HTTP/3 although still unofficially as official QUIC support has yet to be officially included by NGINX.
With Cloudflare's scaling, we have surpassed NGINX. It has been great for many years, but over time its limitations to our scaling needs have meant building something new that made sense. We could no longer get the performance we needed, nor did NGINX have the functionality we needed for our very complex environment.
Hence, it appears that NGINX's limit (for CloudFlare's very special needs) has also been reached and the company recently unveiled its internal solution in search of a superior option. Introducing Pingora, a new HTTP proxy server developed by Cloudflare.
The announcement tweeted on Twitter on September 14 speaks volumes:
What is Pingora HTTP Proxy Server
Pingora is a new in-house built HTTP proxy server by Cloudflare, written in the Rust programming language. Its development was driven by the need to improve and expand the capabilities Nginx offers for Cloudflare's global network demands.
Why RUST? Because it can achieve the same performance and features as what C can safely do for memory without sacrificing performance. Security issues such as Buffer Overflow, Stack Overflow, Heap Overflow, dynamic memory allocation, and core language limitations such as C, fail with RUST.
As you probably know, some Linux kernel components are also currently being considered for the transition to Rust-based development.
According to data from CloudFlare, Pingora fully meets expectations and surpasses Nginx previously used in its role as a reverse proxy. Here's what the numbers show.
Pingora serves over 1 trillion requests per day across Cloudflare's global network. However, compared to Nginx, in production, it shows a 5 ms reduction in median TTFB (Time to First Byte). The improved performance is due to Pingora's new architecture, which allows all threads to share connections, compared to NGINX, which only allowed connection reuse on the same worker, thus limiting the possibility of connection recycling. This necessarily led to continuous connection renegotiation and significant, computationally expensive One-Way Handshakes, with the cost of also renegotiating SSL for HTTPS and all the additional latency.
Additionally, for this very reason, Pingora consumes approximately 70% less CPU and 67% less memory than Cloudflare's previous solution for the same level of traffic . Furthermore, CloudFlare engineers claim that implementing new features in Pingora is significantly easier than with Nginx thanks to the server's intuitive interface.
These factors lead us to conclude that Pingora has all the features necessary to dethrone Nginx as the most chosen reverse proxy software.
What can we expect from Pingora in the future?
Now comes the time when we need to make the most significant clarification possible. As you know, our media covers only free and open source software. However, unfortunately, Pingora is currently a closed-source project developed internally by Cloudflare.
Therefore, the entire article wouldn't exist without the following statement from the official announcement, which got us excited:
We'll be back with more technical details about the problems we've addressed, the optimizations we've made, and the lessons we've learned from creating Pingora and launching it to power a significant portion of the internet. We'll also be back with our plan to open source it.
We can only add that we believe that moving Pingora's code to an open source approach will help it skyrocket its popularity in both the open source and business segments. So, we can't wait for this to happen and will keep you updated on any changes.
Those interested in learning more about Pingora HTTP Proxy Server can do so by visiting Cloudflare’s official announcement.
Conclusion
Without a doubt, Pingora is an exciting project with the potential to change many aspects of the web. But an analogy keeps popping up in our heads as if history repeats itself.
In 2001, Igor Sysoev , dissatisfied with the performance of the Apache Web Server and the design concept on which it was built, developed his own in-house project, especially for the company he worked for. He gave the project the unfamiliar abbreviation Nginx.
Three years later, in 2004, the project switched to an open source model. The rest is history.
Today, 21 years later, the king of web servers faces the same challenge. Cloudflare's Pingora HTTP proxy server aims to overcome the limitations set by Nginx. Will they open source it and become the new dominant force in web content delivery? We can't wait to find out.
In Managed Server, however, we want to make some clarifications and clarifications regarding the situation presented by CloudFlare which, although it brings a lot of enthusiasm, trust and hope for the future (hopefully near), tries to submit to the reader only and exclusively their very personal story.
First of all, it must necessarily be said that CloudFlare does not provide direct Hosting and Web Server services and therefore it is peaceful and reasonable to develop a reverse proxy from scratch that could overcome the architectural limits of NGINX and its performance. Let's think for example of the limit of the reuse of the connection on a single Worker that cannot be reused on different workers and therefore find a lower "Hit ratio" as the number of Workers increases with all the consequence of the renegotiation of SSL for example.
In a classic and standard environment such as that of a Hosting Provider you will always have to deal with a classic web server such as NGINX, for example, or OpenResty (in turn, however, built on NGINX) taking into account the architectural limits of the same as a server. web itself. For many years now it has been objectively recognized that G-WAN Web Server has much more performance than the still excellent NGINX.
With regard to wanting to rewrite a reverse proxy from Zero, calibrated to the needs of the company, it is certainly a very good thing (especially if it will be released in Open Source mode), however even Reverse Proxy in Open Source production such as Envoy could have easily done the case by going to elegantly solve the problems that Pingora would seem to solve.
For our part, we will fearlessly wait for the Open Source release at least to test it in the field and evaluate a possible production start to replace the Envoy that we already use for customers with enterprise plans.
