fjall-rs
Posts Tags

Announcing Fjall 2.2

October 20, 2024
-
4 min. read
Blog post thumbnail

Fjall 2.2 was just released, featuring optimistic concurrency control for multi-writer transaction support, using serializable snapshot isolation.

Transactions before 2.2

Transactions have been supported since version 1.1.2 in the shape of single-writer transactions, which are serializable. Serializable is the strongest isolation level, and is supported by most, if not all, transactional databases. Single writers are trivially serializable, because they literally serialized. For contentious workloads, this offers low overhead because (a) there is no bookkeeping of possible transaction conflicts, (b) transactions never have to be retried because they cannot fail. However, for longer running transactions that do not interfere much, they are… not great.

The transaction API supports pretty much all operations, such as:

and it also features extra transactional helpers for advanced atomic operations, such as fetch-and-update, or take (get-and-remove).

Here is an example to get an idea of the transaction API:

// A default keyspace does not have a transactional layer
// One has to use `open_transactional` to get transactions
let keyspace = Config::default().open_transactional()?;
let partition = keyspace.open_partition("items", Default::default())?;

{
  // We can set the durability of the transaction to Sync*
  // to get ACID transactions
  // That way, we guarantee that once `commit()` returns, the data is
  // stable on disk
  let mut tx = keyspace.write_tx().durability(Some(PersistMode::SyncData));

  tx.insert(&partition, "hello", "world");

  // Read-your-own-write
  let item = tx.get(&partition, "hello")?;
  assert_eq!(Some("world".as_bytes().into()), item);

  // Commit the transaction
  // This can fail because of I/O errors, but not because of
  // transactional shenanigans
  tx.commit()?;

  assert_eq!(b"world", &*partition.get("hello")?.unwrap());
}

{
  let mut tx = keyspace.write_tx();

  tx.insert(&partition, "hello", "welt");

  // If we drop a transaction without committing,
  // all writes are rolled back
  drop(tx);

  assert_eq!(b"world", &*partition.get("hello")?.unwrap());
}

Because the write transaction takes an actual lock, starting a second transaction in the same thread would deadlock the program. So don’t do that.

Serializable snapshot isolation

2.2 now features multi-writer transactions with optimistic concurrency control. To be more precise, serializable snapshot isolation (SSI) is used. This is the same way Postgres’ serializable isolation level works, and is more thoroughly explained in this paper.

The API features all the functions the single-writer transactions offer, as well.

To use SSI transactions, one has to toggle a feature flag:

fjall = { version = "2.2", default-features = false, features = [
  "bloom", # <- this is enabled by default
  "lz4", # <- this is enabled by default
  "ssi_tx", # <- use SSI
] }

It is important to be aware that SSI transactions cannot simply be slotted into existing code without adjusting it a bit. SSI transactions can - and will - fail, and have to be retried. To do so one can simply run the transaction body in a loop, and check for success:

loop {
    let mut tx = keyspace.write_tx();

    let prev = tx.get(&partition, "hello")?;

    tx.insert(&partition, "hello", "welt");

    if tx.commit()?.is_ok() {
      break;
    }
}

// our transaction has succeeded

You may notice that the transaction commit now returns a wrapped Result<Result<()>>. The first result can safely be short-circuited using the ? operator, because it will only trigger when an I/O error occurs, which is pretty fatal. The second result describes whether the transaction succeeded without conflict.

This API is partly inspired by Sled’s compare-and-swap API, you can read more about it here.

Summary

Fjall 2.2 now supports both single-writer and multi-writer transactions. If in doubt, single-writer transactions (the default) are a sensible choice, unless you know your workload has longer running transactions that do not interfere too much.

Credits

Big thanks to Jerome Gravel-Niquet @ https://github.com/jeromegn for making this release possible by implementing serializable snapshot isolation!

Interested in LSM-trees and Rust?

Check out fjall, an MIT-licensed LSM-based storage engine written in Rust.

Tags