Cloudflare fixes Containers failure that allowed to read residual data on shared disks

Cloudflare corrected a vulnerability in his Containers service that allowed an authorized client to read residual data left by other customers' containers on shared disks, according to the company and the researchers who reported the failure. The problem did not involve access to ongoing processes of other customers or the ability to choose who belonged to the recovered information, but it did leave storage blocks with file remains - directories, database pages and complete SQLite bases, according to the count - that an attacker with access to the container was able to extract.

Confirmed facts: the researcher Oren Yomtov, of the firm Accomplish, reported the failure to Cloudflare on 4 September; the technical cause was the reconfiguration of disks in "thin provisioning" mode (granular capacity of allocation in blocks of 64 KB) so that, by returning blocks to the shared pool, the system did not clean them before reassigning them; Cloudflare reactivated the deletion of new blocks and removed / recycled the units in production, completing the cleaning on 19 September and disclosing the incident five days later. The investigators verified the patch on September 14 and confirmed that their concept test ceased to work.

How it worked technically (explanation): the disks assigned to containers are created by a device of type thin provision, which assigns space in blocks of 64 KB. When a container is destroyed, its blocks returned to the general pool. Due to a configuration that omitted cleaning (wipe / zero) when reassigning blocks, the next container that received that block could only write 4 KB within the 64 KB and then read the block at a gross level; the rest of the 60 KB remained with the bytes of the previous tenant. The researchers exploited exactly that: they wrote a small fragment and read the entire block from the crude block level to recover residual data. This vector attacks the unsafe reuse of blocks, not the hot memory and integrity of running processes.

Which systems and customers were affected: Cloudflare Containers and its product Cloudflare Sandboxes(a layer designed to run unreliable code, including IA agents) were identified as affected. The researchers also noted that the same disk configuration affected the Browser Run product, although Cloudflare only named Containers and Sandboxes in its release. Cloudflare controls the physical allocation of servers and disks, so the exposure came from the supplier's side and not from a customer's choice.

Scope observed by researchers and by Cloudflare: in controlled production tests, the researchers recovered residual material in 18 of 24 attempts on servers chosen by Cloudflare, and in 20 of 22 physical machines distributed on four continents; the remains included directories, browser profiles (Chromium), .env files, credentials files and SQLite bases, according to the researchers' report and the partial confirmation of Cloudflare. Cloudflare says that his internal inquiries - based on signatures created from the concept test - only found authorized activity (that of the researchers and their own staff) in the retained records.

What is confirmed and what is still uncertain: it is confirmed that there was residual data exposure in reassigned disk blocks and that Cloudflare applied two remediations (reactivate the deletion by assigning new blocks and replace / deploy discs and caches in production). It is a reasonable estimate - although not mathematically publicly proven - that the remains could contain useful secrets (tokens, keys, credentials) as .env and databases appeared. The uncertain is the total exposure window: Cloudflare did not specify since when the unsafe configuration was active or for how long in absolute terms, so it cannot be determined with certainty how many customers were potentially exposed or for what exact period.

Real risks for customers and practical consequences: the recovered information may include credentials, tokens API, configuration secrets or portions of databases that allow subsequent attacks (re-use of credentials, out-of-environment climbing, access to connected services). Although there is no public evidence of third-party abuse, the combination of partially recovered data and automated analysis mechanisms can facilitate the reconstruction of secrets. In addition, for organisations subject to regulatory or compliance requirements, the mere possibility of exfiltration of data in a multi-tenant environment may require internal notifications or reviews.

Specific measures to be taken by affected or at risk customers: first, identify whether to use Containers, Sandboxes or Browser Run and which images or containers could contain secrets on disk or in image layers. Then, immediately roll any credential that might have been in those containers: API tokens, database keys, .env file secrets and SSH keys. Do not wait for the supplier's confirmation alone; proactive rotation is a low-cost risk action. Revocating tokens and regenerating credentials in connected services (data banks, cloud services, third party APIs) is the most direct measure to mitigate possible abuse.

Other recommended technical actions: stop storing secrets in the persistent container file system; use secret management (vaults) or secret management mechanisms that inject values in running time without writing them to disk; mount secrets in memory (tmpfs) when feasible; encrypt sensitive data at application level before writing them to disk so that the block exposure does not allow clear keys to be read. See good practices in secret management: OWASP Secrets Management Cheat Sheet.

Verification and monitoring: review logs and unusual activity in the services that use the affected containers; search for access from PIs or unexpected patterns and set up alerts on abnormal key use. Although Cloudflare reported having executed signature-based detection, it is appropriate to apply its own detection of anomalies in critical services and to audit images and layers of container in search of embedded secrets.

Additional technical context and references: this problem is an example of how performance optimization and low-level storage configuration (thin provision and layer caching) can introduce data leakage vectors in multi-tenant environments. To understand the Thin provisioning mechanism and its technical risks, see the Linux kernel documentation on Device Mapper Thin provisioning: kernel. For practical instructions on how to manage container secrets and reduce this type of risk, documentation of good practice of platforms and orchestrators is useful (e.g. Docker and Kubernetes practices and secret management).

Conclusion: Cloudflare closed the detected route of attack and cleaned up the resources in production, and for now there are no public signs of mass abuse. However, the nature of the failure - reading of residual data on shared disks - implies that any client who has kept secrets on images or container disks should rotate credentials and review secret management practices. The operational lesson is clear: in multi-tenant environments, the defense must include both supplier controls and strict developer practices to not leave secrets on disk or rely exclusively on physical or hyperviewer security.

Additional sources and reading: the communication and follow-up of the supplier are available on the Cloudflare blog https: / / blog.cloud, and the technical explanation of thin provision in Linux can be found in the official documentation of the kernel: https: / / www.kernel.org / doc / html / latest / admin-guide / device-mapper / thin-provising.html.