Sharing Sessions (or Other Values) in Key/Value Databases with a KeyDB Cluster

Horizontal Scalability and Shared Sessions: A (Non)Trivial Problem

In highly available environments or those that need to support large and distributed loads, such as modern eCommerce or web portals, it is very common to scale the infrastructure horizontally, distributing the load across multiple application servers. This involves a crucial challenge: keep user sessions consistent, regardless of the node the user finds himself on during his navigation.

Imagine, for example, an architecture with three PHP-FPM servers behind a load balancer. Every time a user makes a request, it could end up on a different node. If sessions are managed locally (on file), the user will lose the context as soon as the traffic is diverted to another node. This is where the need for centralize session management.

One of the most common solutions is to use a NoSQL Key/Value database to store sessions or other transient and high-frequency access data. The most used technologies in this area are Memcached e Redis.

Redis vs Memcached: Why Redis is the modern choice

Redis (REmote DIctionary Server) was born in 2009 from an idea of Salvatore Sanfilippo (aka antirez), an Italian developer, who created it to solve performance problems in his startup project. Initially conceived as a simple in-memory key-value store, it quickly evolved into a powerful data structure server, thanks to native support for lists, sets, hashes, sorted sets and more. Redis has been adopted by thousands of companies worldwide for caching, sessions, messaging and real-time analytics. In 2015 the project was donated to the community under the BSD license, and in 2020 Sanfilippo left the leadership of the project. Today Redis is maintained by Redis Inc., which develops both open source and enterprise versions, with advanced features such as active-active replication and multi-region support. Redis has become one of the most widely used tools in the DevOps and cloud-native ecosystem.

While Memcached is still used, Redis has largely surpassed Memcached in many real-world contexts thanks to its advanced features:

• Support for complex data structures (lists, sets, hashes, etc.) Unlike Memcached, which works exclusively with key/value strings, Redis allows you to use much more versatile data structures. These include lists (useful for managing FIFO and LIFO queues), sets (unordered sets without duplicates), sorted sets (with associated scores, ideal for rankings), hashes (perfect for representing objects or associative arrays), as well as advanced types such as bitmaps, hyperloglogs, and streams. This variety allows you to model complex application scenarios directly in the database, without the need for intermediate transformations in the code.
• Optional persistence on disk Redis can operate either as a volatile database, entirely in RAM, or with persistence mechanisms to ensure data durability. The two main modes are RDB (which takes snapshots at configurable intervals) and AOF (which logs all operations performed). This allows you to find a balance between performance and reliability, offering the possibility of restoring data even after a service restart. Memcached, on the other hand, is completely devoid of persistence: when the server is turned off, everything that was stored is lost.
• Native Atomic Operations Redis ensures that every operation is atomic, that is, executed in isolation and completely without the possibility of interference from other clients. This is especially important for concurrent operations on shared data. For example, incrementing a counter or changing a hash is always done safely. Additionally, Redis supports transactions via MULTI/EXEC and the use of Lua scripts that allow you to execute multiple operations in bulk within the server, without the risk of race conditions and with better performance than client-side logic.
• Replication and clustering support Redis offers a number of advanced features for scalability and high availability. You can configure read-only replicas (master/slave replication) to distribute the load, or adopt Redis Cluster to obtain a true horizontal division of data (sharding) across multiple nodes. Added to this is Redis Sentinel, the component dedicated to monitoring, automatic failover and dynamic management of masters. Memcached, on the other hand, does not natively support replication or clustering: each node is isolated and consistency must be managed entirely by the application layer.

For these reasons, Redis is considered the de facto standard for storing shared sessions in PHP, Node.js, Python, Ruby and other web technologies environments.

However, when you get into the details of implementing Redis in a cluster context, non-trivial limitations emerge, especially in the community (free) version.

The Limitation of the Community Version of Redis: The Problem of Asynchronous Replication and Master/Slave Clusters

The open source version of Redis supports clustering, but in a master / slaveThese are:

• Each node accepts writes only for a subset of the keys In a Redis Cluster, keys are divided into 16.384 “hash slots,” distributed across the nodes. This means that each node is only responsible for a portion of the total dataset and will only accept writes for keys that fall into its slots. As a result, if an application attempts to write a key to a node that is not responsible for that key, Redis will return an error of type MOVED and the client will have to retry on the other node. This adds application-side complexity or requires cluster-aware clients.
• Replication occurs asynchronously. Data written to a master node is replicated to its slaves, but the replication process is not synchronous: there is no guarantee that a newly written data will already be available on the slaves when a failover is triggered. This introduces the risk of temporary data loss, especially in high-write scenarios. Out-of-sync replication can be a problem for applications that require strong consistency, such as payment systems, login sessions, or volatile but critical user data.
• There is no transparent Master/Master (multi-writer) replication The open source Redis cluster does not allow multiple nodes to accept writes for the same keys at the same time. It is not possible to have a configuration where every node is active for writing and all changes are propagated in real time to the others. This means that you cannot write the session on any node and expect it to be automatically replicated on all the others, as would happen in a master/master cluster. This limitation becomes a bottleneck in distributed architectures where each application node has its own local or nearby Redis and you want to avoid complex and slow cross-node writes.

This means that if we want to save the session on a Redis node, we cannot expect that such a session will be immediately available on all other nodes. Replication takes time, and if a network or node fails, writes can be lost or inconsistent.

Furthermore, in an architecture where three PHP servers need to be able to write or read sessions at any time, you are often forced to configure multiple Redis addresses, or to use a complex and inefficient “retry” and fallback mechanism.

Example: Redis with PHP-FPM in a High Availability eCommerce

Suppose we have three application servers with PHP-FPM and Magento. Each node has a Redis session configuration similar to this:

session.save_handler = redis
session.save_path = "tcp://redis1:6379,tcp://redis2:6379,tcp://redis3:6379"

This approach has some obvious problems:

• Variable latencies Every TCP/IP connection introduces network latency, which can vary based on physical distance, node load, and network quality. In a distributed environment, not all Redis servers will be equidistant from the various application servers. As a result, the rate at which a session is written to or read from can vary significantly, resulting in inconsistent response times and a less fluid user experience, especially under load.
• Problems in case of fault If one of the Redis nodes listed is slow to respond or completely unavailable, the PHP-FPM process will have to wait for TCP timeout before attempting to connect to the next node. This behavior introduces significant delays in the application response cycle. Additionally, PHP does not always handle the fallibility of the Redis backend very well, especially if an effective retry or failover mechanism is not configured.
• Inconsistent writings A user session could be written to redis1, but if the reply is towards redis2 o redis3 has not yet occurred — or has failed —, subsequent read attempts by other PHP nodes connected to these Redis will not find the session updated. This can lead to session loss, unexpected logout or erratic application behavior. In practice, the application ends up behaving as if the session had never been created or had expired, with all the consequences that this entails in terms of UX and continuity of service.

These scenarios become even more critical when managing authentication sessions, carts, checkouts or custom pages: any loss or inconsistency can result in serious UX damage and, in the eCommerce sector, even direct financial loss.

The ideal solution? A transparent Master/Master cluster with Active Sync

In an ideal world, every PHP application server should be able to write user session to nearest Redis node or directly to localhost, without worrying about which node is the “master” or whether the session will be read correctly by other servers. The backend system should take care of this on its own synchronize changes in real time on all other nodes in the cluster, in a completely transparent, efficient and fault tolerant.

This architecture — known as Master / Master replica and Active Sync o multi-writer active-active — represents the ideal model for scalable and distributed environments: each node is simultaneously a reader and a writer, with all changes propagated via active synchronization to the other peers in the cluster. This has several advantages: no single point of failure, minimum latencies, high availability e Simplification of application logic, eliminating the need to manage complex fallback or retry mechanisms.

Unfortunately, this Active Sync technology It is not supported in the open source version of Redis, which is instead based on a master/slave architecture with asynchronous replication. The only Redis option that allows for a true Master/Master configuration with Active Sync is the Redis Enterprise, commercially distributed by Redis Inc., which includes advanced features such as active-active with CRDT (Conflict-free Replicated Data Types), but whose use is restricted by expensive licenses and controlled infrastructures, often in specific cloud providers or managed environments.

For those who want to stay in the open source ecosystem, or avoid the recurring costs of enterprise versions, this represents a important technical and operational limit, especially in modern architecture where Agility and distributed resilience with active synchronization are now key requirements.

The KeyDB Project is Born: From Snapchat to the Open Source World

Precisely starting from this concrete need - that is, the possibility of write to any node and ensure immediate and consistent replication across the entire cluster — a team of engineers from Snapchat decided to tackle the problem at its root. Snapchat, in fact, is not just a popular social application, but a real infrastructure giant that processes billions of events and user interactions every day, often in real time and with very stringent latency and availability requirements.

What is Snapchat?

Snapchat is an instant messaging app that is very popular among young people, which stands out for the ability to send ephemeral photos and videos, that is, content that self-destructs after being viewed. It is also known for its AR lenses, daily stories and quick chat features. On a technical level, Snapchat is based on High-performance distributed infrastructures, capable of dynamically scaling to respond to global traffic spikes — think weekends, live events, or simple time zones that trigger continuous flows of connected users.

In such a scenario, relying on Redis in its open source version soon represented a limitation: the need to have a asynchronous master/slave replication was too weak to handle ephemeral content and real-time sessions without the risk of inconsistency or data loss. The enterprise version of Redis also didn’t support the open, controlled approach Snapchat wanted for its core infrastructure.

This is how the team decided to Forking Redis starting from the version 5 code, maintaining full compatibility with the existing Redis interface and APIs, but introducing a series of fundamental architectural innovations. The goal was clear: create a more modern, more performing key-value database, capable of supporting active multi-master replication natively and without paid licenses.

Thus was born KeyDB, a database open source which retains all the power and simplicity of Redis, but radically extends its capabilities, making it suitable for contexts mission-critical, low latency and highly available. Today, KeyDB is adopted by numerous companies around the world who are looking for a truly scalable and free alternative to Redis Enterprise, while remaining within the compatible Redis ecosystem.

A Brief History of KeyDB

KeyDB It was initially released in 2019 as a Redis 5 fork, with the aim of overcoming some of the structural limitations of the open source version of Redis, while maintaining its compatibility and ease of use. The project was born within Snap Inc. (Snapchat's parent company) and is today actively maintained by a dedicated team of developers, with regular updates, bugfixes and new features.

The main innovations introduced by KeyDB compared to Redis include:

• Native multi-master replication Unlike Redis OSS, which only supports master/slave configurations and one-way replication, KeyDB allows multiple nodes to accept writes simultaneously, propagating changes to other peers in real time. This allows for the creation of active-active clusters truly distributed, without the need for external coordinators or complex consistency management solutions.
• Multithreaded threading (Redis is single-threaded for core operations) Redis, by design, runs all major operations on a single thread, which is a limitation on modern machines with multicore CPUs. KeyDB breaks this barrier by implementing a multithreaded request handling engine, with significant advantages in terms of throughput, parallelism and optimal use of hardware resources. In high-volume scenarios, the performance differences become tangible.
• Full support for existing Redis commands One of the strengths of KeyDB is its full compatibility with Redis syntax and commands, making it possible to seamlessly replace the Redis backend in any existing application, without having to change any code or client configurations. Advanced commands, transactions, and Lua scripts are also supported.
• Drop-in compatibility with Redis clients All Redis clients — for PHP, Python, Node.js, Go, Java, etc. — work natively with KeyDB, thanks to the maintenance of the RESP network protocol. This means that developers and DevOps can integrate KeyDB into their existing infrastructures easily and immediately, without changes to the code side.
• Superior performance in real-world scenarios Thanks to multithreading, synchronous replication and internal optimizations, KeyDB has demonstrated in numerous benchmarks that be significantly faster than Redis in real-world scenarios, especially under concurrent loads or in the presence of multiple active clients. In particular, the average response time to requests (latency) is more stable and contained even under stressful conditions.
• Open source (BSD 3-Clause license) KeyDB is released under a BSD 3-Clause Permissive License, which also allows commercial use without constraints. This makes the project particularly attractive for companies, startups and cloud providers who want to build high-performance and distributed solutions without having to deal with the recurring costs of enterprise licensing.

Thanks to these features, KeyDB has quickly become one of the most serious and reliable alternatives to Redis Enterprise, allowing maintain all the benefits of the Redis ecosystem, but with one greater architectural flexibility, superior performance and above all an open source model completely free from licensing costs.

KeyDB in Master/Master perspective and Active Sync: a new paradigm

The most significant strength of KeyDB is the ability to configure a Multi-writer cluster, In which each node is simultaneously master and replica of the others, allowing any node to accept writes independently. Unlike the traditional Redis model, here changes are not centralized on a single node, but are automatically propagated in real time to all peers of the cluster, maintaining data consistency without the need for complex application logic.

How does it work?

• Each KeyDB node is able to accept writes autonomously There is no longer a “central point” for writing: each node in the cluster can receive and manage writes in parallel, allowing applications to always interact with the closest instance, dramatically reducing latency and improving performance.
• Changes are immediately replicated to all other nodes. When a node receives a write (e.g. a session update), it is broadcast in real time to other nodes in the cluster, ensuring that they all maintain the same state synchronously. This behavior eliminates the need for polling, asynchronous propagation, or application fallback mechanisms.
• The replication protocol is synchronous and bidirectional, avoiding divergences Active replication between nodes is bidirectional, which means that each node not only sends but also receives changes. Furthermore, the mechanism is designed to be synchronous, significantly reducing the risk of data inconsistency. Any conflicts are handled automatically according to deterministic rules, and there are no time windows in which data can diverge.
• It is possible to connect multiple nodes even in geographically distributed environments KeyDB allows you to build clusters that span different data centers or different cloud regions, maintaining data consistency even over long distances. This feature is particularly useful for implementing disaster recovery strategies, geographic load balancing or high availability on a global scale, all while maintaining acceptable propagation times.

Concrete advantages for PHP sessions

When using KeyDB for session management in PHP environments (e.g. with distributed LAMP or LEMP stacks), the benefits become immediately apparent:

• Single Write: Each application server writes to the local node Instead of having to contact a remote Redis node or manually distribute writes across multiple endpoints, each application server can write on your local instance of KeyDB, thus achieving extremely fast response times and reducing latency to a minimum.
• Automatic replication: KeyDB propagates the session to other nodes Once the session data has been saved on a node, Propagation to the rest of the cluster is automatic and immediate. This means that even if the next user request arrives at another application server, the session will already be available, without the need for external synchronization.
• Transparent failover: if one node fails, the others are already synchronized In case of a KeyDB node failure, the application does not lose the user's session: the other nodes in the cluster are already aligned and ready to respond to requests. This ensures high availability even in the event of accidents or unexpected maintenance.
• Zero session loss: consistency and availability are guaranteed Thanks to the synchronous and multi-master nature of the cluster, There is no window of inconsistency between nodes. The risk of a session being written on one node and not propagated in time to the others (as happens with classic Redis) is completely eliminated, making KeyDB an ideal solution for highly reliable contexts such as eCommerce, portals with authentication, or real-time applications.

Master/Master cluster setup example

Let's imagine three KeyDB nodes: keydb1, keydb2, keydb3.

Configuration on keydb1.conf:

replicaof keydb2 6379
active-replica yes

Configuration on keydb2.conf:

replicaof keydb3 6379
active-replica yes

Configuration on keydb3.conf:

replicaof keydb1 6379
active-replica yes

This setup creates a active replication cycle, where each node is updated in real time by the others. It is natively supported and stable in KeyDB.

Why KeyDB is the right choice today

For distributed, scalable and highly reliable architectures, KeyDB represents a modern, robust and open source solution which fills the gaps left by the community version of Redis.

KeyDB is compatible with all Redis clients and does not require any changes to the applications. For those who work in the world of PHP, Magento, WordPress or Prestashop and manage scalable infrastructures, KeyDB allows higher performance, greater fault tolerance and architectural simplification.

PHP Integration: What's Changing?

On the PHP side, absolutely nothing changes: the application code remains exactly the same, without the need for modifications or adaptations. The only difference will be in the configuration of the PHP-FPM session handler, where it will be enough to point to a single KeyDB endpoint, exactly as you would do with Redis.

The real innovation is that, thanks to KeyDB's multi-master replication, no longer need to configure multiple host lists as was traditionally done with Redis. This completely eliminates the need to specify all comma-separated Redis nodes in your PHP configuration, dramatically simplifying your infrastructure and reducing operational complexity.

session.save_handler = redis
session.save_path = "tcp://keydb1:6379"

Or even more simply, you can point directly to an instance on localhost:

session.save_handler = redis
session.save_path = "tcp://localhost:6379"

With this minimalist setup, each PHP-FPM node communicates exclusively with its own local KeyDB, which takes care of replicating all changes in real time to the other nodes in the cluster. The application layer remains completely isolated and unaware of the underlying complexity of distributed replication, while still getting all the benefits of a highly available system. This approach is incredibly simpler than the traditional Redis multi-host setup:

# Configurazione tradizionale Redis (non più necessaria con KeyDB)
session.save_handler = redis
session.save_path = "tcp://redis1:6379,tcp://redis2:6379,tcp://redis3:6379"

With KeyDB, everything becomes simple, elegant and effective, maintaining total transparency for the PHP application.

Conclusion: KeyDB, the modern answer to scalability challenges

In a technological landscape increasingly oriented towards distribution, resilience and performance, session management (and in general volatile data) can no longer be based on solutions designed for monolithic or centralized contexts. Redis has marked a turning point in the way of conceiving key-value databases, but its open source version, as solid and popular as it is, shows clear limits in complex and mission-critical distributed scenarios.

KeyDB was born precisely to fill this gap, offering a platform open source, performant, compatible and truly oriented towards modern scalability. With the ability to write to any node, synchronous and bidirectional replication, multithreaded support, and full compatibility with Redis clients and commands, KeyDB allows you to build infrastructures simple to manage, robust, and incredibly fast.

For those who manage PHP environments, highly available eCommerce, microservices, or high concurrency systems, KeyDB represents a strategic choice which allows for monitoring the simplify architecture, increase service availability, and reduce the risk of session loss or inconsistency.

With no licenses to pay, no vendor lock-in and full control of the infrastructure, KeyDB is today one of the most concrete and reliable alternatives for those who want the best of Redis, but without compromise.