Table of contents of the article:
Few companies can compete with Cloudflare's expertise in serving massive web traffic.
To make things more precise, we'll say this: Cloudflare handles over 10% of all global HTTP / HTTPS Internet traffic. In addition, globally, the Cloudflare network serves over 25 million HTTP requests per second and it's used by almost 80% of all websites that use reverse proxy services.
So, one thing is certain: the company has faced colossal traffic loads and pushed the capabilities of modern technology to the limit.
As a reverse proxy Proxying the traffic between the Cloudflare network and servers on the Internet, Nginx has been a vital part of CloudFlare's architecture, until now, also taking into account the important improvements and optimizations they have made in implementing HTTP / 2 and the "new" QUIC or HTTP / 3 although still unofficially as official QUIC support has yet to be officially included by NGINX.
With Cloudflare's scaling, we have surpassed NGINX. It has been great for many years, but over time its limitations to our scaling needs have meant building something new that made sense. We could no longer get the performance we needed, nor did NGINX have the functionality we needed for our very complex environment.
Hence, it appears that NGINX's limit (for CloudFlare's very special needs) has also been reached and the company recently unveiled its internal solution in search of a superior option. Introducing Pingora, a new HTTP proxy server developed by Cloudflare.
The announcement tweeted on Twitter on September 14 speaks volumes:
Today we are excited to talk about Pingora, a new HTTP proxy we've built in-house using Rust that serves over 1 trillion requests a day, boosts our performance, and enables many new features for Cloudflare customers. Read all the details: https://t.co/PvUG42oFje
- Cloudflare (@Cloudflare) September 14, 2022
What is Pingora HTTP Proxy Server
Pingora is a new in-house built HTTP proxy server by Cloudflare, written in the Rust programming language. Its development was driven by the need to improve and expand the capabilities Nginx offers for Cloudflare's global network demands.
Why RUST? Because it can achieve the same performance and features as what C can safely do for memory without sacrificing performance. Security issues such as Buffer Overflow, Stack Overflow, Heap Overflow, dynamic memory allocation, and core language limitations such as C, fail with RUST.
As you probably know, some Linux kernel components are also currently being considered for the transition to Rust-based development.
According to data from CloudFlare, Pingora fully meets expectations and surpasses Nginx previously used in its role as a reverse proxy. Here's what the numbers show.
Pingora serves over 1 trillion requests per day through Cloudflare's global network. However, compared to Nginx, in production, it shows a reduction of 5 ms on the median TTFB (Time to First Byte). The improved performance is due to the new Pingora architecture, which allows all threads to share connections compared to NGINX which allowed the reuse of connections only on the same Worker and therefore limiting the possibilities of recycling connections which necessarily led to continuous renegotiation of connections and computationally expensive important Thee Way Handshake with the cost of also renegotiating SSL for HTTPS and all the additional latency.
Furthermore, precisely because of this reason, Pingora consumes about 70% less CPU and 67% less memory than the previous Cloudflare solution with the same level of traffic. Additionally, CloudFlare engineers say that implementing new features in Pingora is considerably easier than in Nginx thanks to the server's intuitive interface.
These factors lead us to conclude that Pingora has all the features necessary to dethrone Nginx as the most chosen reverse proxy software.
What can we expect from Pingora in the future?
Now comes the time when we need to make the most significant clarification possible. As you know, our media covers only free and open source software. However, unfortunately, Pingora is currently a closed-source project developed internally by Cloudflare.
Therefore, the entire article wouldn't exist without the following statement from the official announcement, which got us excited:
We'll come back with more technical details on the issues we've faced, the optimizations we've applied, and the lessons we've learned from creating Pingora and launching it to power a significant portion of the internet. We will also come back with our plan to make it open source.
We can only add that we believe that moving Pingora's code to an open source approach will help it skyrocket its popularity in both the open source and business segments. So, we can't wait for this to happen and will keep you updated on any changes.
Those interested in learning more about the Pingora HTTP Proxy Server can do so by visiting theCloudflare's official announcement .
Conclusion
Without a doubt, Pingora is an exciting project with the potential to change many aspects of the web. But an analogy keeps popping up in our heads as if history repeats itself.
In 2001, Igor Sysoev , dissatisfied with performance of the Apache Web Server and of the design concept on which it was built, he developed his internal project, especially for the company where he worked. He gave the project the strange abbreviation Nginx .
Three years later, in 2004, the project switched to an open source model. The rest is history.
Today, 21 years later, the king of web servers faces the same challenge. Cloudflare's Pingora HTTP proxy server aims to exceed the limits set by Nginx. Will they open source it and become the new dominant force in web content distribution? We can't wait to find out.
In Managed Server, however, we want to make some clarifications and clarifications regarding the situation presented by CloudFlare which, although it brings a lot of enthusiasm, trust and hope for the future (hopefully near), tries to submit to the reader only and exclusively their very personal story.
First of all, it must necessarily be said that CloudFlare does not provide direct Hosting and Web Server services and therefore it is peaceful and reasonable to develop a reverse proxy from scratch that could overcome the architectural limits of NGINX and its performance. Let's think for example of the limit of the reuse of the connection on a single Worker that cannot be reused on different workers and therefore find a lower "Hit ratio" as the number of Workers increases with all the consequence of the renegotiation of SSL for example.
In a classic and standard environment such as that of a Hosting Provider you will always have to deal with a classic web server such as NGINX, for example, or OpenResty (in turn, however, built on NGINX) taking into account the architectural limits of the same as a server. web itself. For many years now it has been objectively recognized that G-WAN Web Server has much more performance than the still excellent NGINX.
With regard to wanting to rewrite a reverse proxy from Zero, calibrated to the needs of the company, it is certainly a very good thing (especially if it will be released in Open Source mode), however even Reverse Proxy in Open Source production such as Envoy could have easily done the case by going to elegantly solve the problems that Pingora would seem to solve.
For our part, we will fearlessly wait for the Open Source release at least to test it in the field and evaluate a possible production start to replace the Envoy that we already use for customers with enterprise plans.
