uffd: add support for UFFD_EVENT_REMOVE events by bchalios · Pull Request #1896 · e2b-dev/infra

bchalios · 2026-02-11T21:13:23Z

Enabling Firecracker free-page-reporting feature requires us to handle remove events (UFFD_EVENT_REMOVE) in our userfaultfd handler. These events are triggered whenever Firecracker calls madvise(MADV_DONTNEED) (or similar) on a range of guest memory addresses.

The main thing that changes on our logic is that page faults in a page that has previously been removed need to be served with a zero page rather than a page from the snapshot file.

This commit changes the page fault serving logic to:

Introduce tracking of the state of every page in the guest's memory mappings.
Add logic to handle the new UFFD_EVENT_REMOVE event
Modify existing logic to take into account current state when deciding how to handle each page fault

This is dependent on the part of #1858 that enables free page reporting on the Firecracker side.

packages/orchestrator/internal/sandbox/uffd/memory/mapping.go

packages/orchestrator/internal/sandbox/uffd/userfaultfd/userfaultfd.go

packages/orchestrator/internal/sandbox/uffd/memory/region.go

packages/orchestrator/internal/sandbox/uffd/userfaultfd/userfaultfd.go

packages/orchestrator/internal/sandbox/fc/process.go

packages/shared/pkg/fc/firecracker.yml

packages/orchestrator/internal/sandbox/uffd/memory/region.go

bchalios · 2026-02-20T18:39:54Z

Rebased this one on top of #1953 (which is based on top of #1937).

I've applied some refactoring suggested by @ValentaTomas here: https://github.com/e2b-dev/infra/tree/refactor-remove and also tried to make it that free page reporting is optional and only possible on templates built with Firecracker v1.14.

Keeping it in Draft until dependency PRs are merged.

Use new v1.10 and v1.12 Firecracker versions with support for getting information about dirty memory calling the /memory/dirty endpoint. Signed-off-by: Babis Chalios <babis.chalios@e2b.dev>

Bump it to version v1.12 which now supports the logic for getting dirty memory via the /memory/dirty endpoint. It also brings a few new features from Firecracker, e.g. NetworkOverrides during loading snapshots. Signed-off-by: Babis Chalios <babis.chalios@e2b.dev>

Use Firecracker's /memory/dirty endpoint for getting information about the guest's dirty memory.

Otherwise calling copy() on the UFFD will clear it. Also fix comment regarding the structure of pagemap. Signed-off-by: Babis Chalios <babis.chalios@e2b.dev>

Bump the swagger file for Firecracker to v1.14 and regenerate APIs/models. Signed-off-by: Babis Chalios <babis.chalios@e2b.dev>

Add support for Firecracker v1.14 and make it the default. Signed-off-by: Babis Chalios <babis.chalios@e2b.dev>

As we're going to handle UFFD_EVENT_REMOVE events triggerred by Firecracker, we need to keep track of the state of all the guest memory pages. Theis commit introduces 3 states: * Unfaulted - A page that has not been faulted yet. * Faulted - A page that we have previously faulted in. * Removed - A page that we have received a remove event for and haven't faulted in since. It also adds the necessary book keeping of page state in all the memory regions of the guest, along with methods for retrieving and setting the state of pages. Signed-off-by: Babis Chalios <babis.chalios@e2b.dev>

Import a few more bindings that we'll need for handling remove events. Signed-off-by: Babis Chalios <babis.chalios@e2b.dev>

Handle UFFD_EVENT_REMOVE events from the file descriptor. These events are triggerred when Firecracker calls madvise() with MADV_DONTNEED on some memory range that we are tracking. This Firecracker behaviour is support with version 1.14.0 onwards using the free page reporting and hinting features of balloon devices. What this means for us is that, we need to track removed pages because subsequent page faults need to be served with a zero page. Signed-off-by: Babis Chalios <babis.chalios@e2b.dev>

Balloon devices provide memory reclamation facilities through free page reporting, as a more efficient mechanism (in terms of latency and CPU time) than traditional ballooning. Free page reporting instructs the guest to periodically report memory that has been freed, so that we can reclaim it in the host side. It is enabled before starting the sandbox and does not require any further host-side orchestration. Enable free page reporting for all new templates using Firecracker versions >= v1.14.0. Also, allow users to optionally disable it for these versions. Older Firecracker versions don't support the feature. Trying to enable it for those, will return an error. Signed-off-by: Babis Chalios <babis.chalios@e2b.dev>

e2b-request-same-site-reviewers bot assigned jakubno Feb 11, 2026

bchalios force-pushed the feat/free-page-reporting branch from aef7cd8 to eab27ea Compare February 11, 2026 21:13

claude bot reviewed Feb 11, 2026

View reviewed changes

packages/orchestrator/internal/sandbox/uffd/memory/mapping.go Outdated Show resolved Hide resolved

claude bot reviewed Feb 11, 2026

View reviewed changes

packages/orchestrator/internal/sandbox/uffd/userfaultfd/userfaultfd.go Outdated Show resolved Hide resolved

claude bot reviewed Feb 11, 2026

View reviewed changes

packages/orchestrator/internal/sandbox/uffd/userfaultfd/userfaultfd.go Outdated Show resolved Hide resolved

claude bot reviewed Feb 11, 2026

View reviewed changes

packages/orchestrator/internal/sandbox/uffd/memory/region.go Outdated Show resolved Hide resolved

bchalios force-pushed the feat/free-page-reporting branch 2 times, most recently from a1d0a02 to 03e2966 Compare February 11, 2026 21:20

jakubno assigned ValentaTomas Feb 11, 2026

bchalios force-pushed the feat/free-page-reporting branch 5 times, most recently from b766eb3 to 4d197e4 Compare February 12, 2026 23:48