Skip to content

Implement soft delete#395

Merged
benbjohnson merged 1 commit intomainfrom
ltx-zero-commit
Aug 24, 2023
Merged

Implement soft delete#395
benbjohnson merged 1 commit intomainfrom
ltx-zero-commit

Conversation

@benbjohnson
Copy link
Collaborator

This pull request adds support for soft deleting databases. Previously, deleting a database would wipe it from the primary and then the primary would send a "drop db" frame to the replicas to have them delete as well.

However, this is problematic when connected to LiteFS Cloud (LFSC) for several reasons. First, LFSC acts as the data authority so when the primary removes the database, LiteFS detects that it exists on LFSC and downloads a snapshot, thus negating the delete. Second, LFSC aims to provide durability even in the face of accidental deletion so a delete should not remove old versions of the database.

Changes to LTX (superfly/ltx#46) were made to have a zero-length database represent a deletion. These changes have been integrated into LFSC and this PR integrates it into LiteFS.

/cc @gc-victor

Fixes #382

Usage

# Create a new database on the LiteFS mount.
$ sqlite3 /litefs/db "CREATE TABLE t (x)"

# The new database has a TXID of 1.
$ cat /litefs/db-pos 
0000000000000001/f83db67898d45151

# Removing the database will delete all associated files and will be TXID 2.
$ rm /litefs/db

# The position file will not be visible via "ls" but can be read via "cat"
$ cat /litefs/db-pos 
0000000000000002/8000000000000000

# Re-creating the database will start a new database file. This is TXID 3.
$ sqlite3 /litefs/db "CREATE TABLE my_new_table (y)"

$ cat /litefs/db-pos 
0000000000000003/e01f61ed0d81644e

@benbjohnson benbjohnson added the enhancement New feature or request label Aug 24, 2023
@benbjohnson benbjohnson merged commit 837e759 into main Aug 24, 2023
@benbjohnson benbjohnson deleted the ltx-zero-commit branch August 24, 2023 22:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Improve deletion via LTX zero commit

1 participant