Introduce volatile_set implementation

[ranshid]

Overview:

This PR introduces a complete redesign of the volatile_set data structure,
creating an adaptive container for expiring entries. The new design is
memory-efficient, scalable, and dynamically promotes/demotes its internal
representation depending on runtime behavior and volume.

The core concept uses a single tagged pointer (expiry_buckets) that encodes
one of several internal structures:
- NONE (-1): Empty set
- SINGLE (0x1): One entry
- VECTOR (0x2): Sorted vector of entry pointers
- HT (0x4): Hash table for larger buckets with many entries
- RAX (0x6): Radix tree (keyed by aligned expiry timestamps)

This allows the set to grow and shrink seamlessly while optimizing for both
space and performance.

Motivation:

The previous design lacked flexibility in high-churn environments or
workloads with skewed expiry distributions. This redesign enables dynamic
layout adjustment based on the time distribution and volume of the inserted
entries, while maintaining fast expiry checks and minimal memory overhead.

Key Concepts:

• All pointers stored in the structure must be odd-aligned to preserve
  3 bits for tagging. This is safe with SDS strings (which set the LSB).

• Buckets evolve automatically:

  • Start as NONE.
  • On first insert → become SINGLE.
  • If another entry with similar expiry → promote to VECTOR.
  • If VECTOR exceeds 127 entries → convert to RAX.
  • If a RAX bucket's vector fills and cannot split → promote to HT.

• Each vector bucket is kept sorted by entry->getExpiry().

• Binary search is used for efficient insertion and splitting.

• Expiry window alignment is managed by:
  VOLATILESET_BUCKET_INTERVAL_MIN = 16ms
  VOLATILESET_BUCKET_INTERVAL_MAX = 8192ms

API Changes:

Create/Free:
void vsetInit(vset *set);
void vsetClear(vset *set);

Mutation:
bool vsetAddEntry(vset *set, vsetGetExpiryFunc getExpiry, void *entry);
bool vsetRemoveEntry(vset *set, vsetGetExpiryFunc getExpiry, void *entry);
bool vsetUpdateEntry(vset *set, vsetGetExpiryFunc getExpiry, void *old_entry,
void *new_entry, long long old_expiry,
long long new_expiry);

Expiry Retrieval:
void *vsetFirstExpired(vset *set, vsetGetExpiryFunc getExpiry, mstime_t now);
void *vsetPopExpired(vset *set, vsetGetExpiryFunc getExpiry, mstime_t now);

Utilities:
bool vsetIsEmpty(vset *set);

Iteration:
void vsetStart(vset *set, vsetIterator *it);
bool vsetNext(vsetIterator *it, void **entryptr);
void vsetStop(vsetIterator *it);

Entry Requirements:

All entries must conform to the following interface via volatileEntryType:

   sds entryGetKey(const void  entry);         // for deduplication
   long long getExpiry(const void  entry);     // used for bucketing
   int expire(void  db, void  o, void  entry); // used for expiration callbacks

Diagrams:

1. Tagged Pointer Representation

   Lower 3 bits of expiry_buckets encode bucket type:

   +------------------------------+
   | pointer | TAG (3b) |
   +------------------------------+
   ↑
   masked via VSET_PTR_MASK

   TAG values:
   0x1 → SINGLE
   0x2 → VECTOR
   0x4 → HT
   0x6 → RAX

2. Evolution of the Bucket

Volatile set top-level structure:

+--------+     +--------+     +--------+     +--------+
| NONE   | --> | SINGLE | --> | VECTOR | --> |   RAX  |
+--------+     +--------+     +--------+     +--------+

If the top-level element is a RAX, it has child buckets of type:

+--------+     +--------+     +-----------+
| SINGLE | --> | VECTOR | --> | HASHTABLE |
+--------+     +--------+     +-----------+

Vectors can split into multiple vectors and shrink into SINGLE buckets. A RAX with only one element is collapsed by replacing the RAX with its single element on the top level (except for HASHTABLE buckets which are not allowed on the top level).

3. RAX Structure with Expiry-Aligned Keys

   Buckets in RAX are indexed by aligned expiry timestamps:

   +------------------------------+
   | RAX key (bucket_ts) → Bucket|
   +------------------------------+
   | 0x00000020 → VECTOR |
   | 0x00000040 → VECTOR |
   | 0x00000060 → HT |
   +------------------------------+

4. Bucket Splitting (Inside RAX)

   If a vector bucket in a RAX fills:

   • Binary search for best split point.
   • Use getExpiry(entry) + get_bucket_ts() to find transition.
   • Create 2 new buckets and update RAX.

   Original:
   [entry1, entry2, ..., entryN] ← bucket_ts = 64ms

   After split:
   [entry1, ..., entryK] → bucket_ts = 32ms
   [entryK+1, ..., entryN] → bucket_ts = 64ms

   If all entries share same bucket_ts → promote to HT.

5. Shrinking Behavior

   On deletion:

   • HT may shrink to VECTOR.
   • VECTOR with 1 item → becomes SINGLE.
   • If RAX has only one key left, it’s promoted up.

Summary:

This redesign provides:
✓ Fine-grained memory control
✓ High scalability for bursty TTL data
✓ Fast expiry checks via windowed organization
✓ Minimal overhead for sparse sets
✓ Flexible binary-search-based sorting and bucketing

It also lays the groundwork for future enhancements, including metrics,
prioritized expiry policies, or segmented cleaning.

[zuiderkwast]

Yes, I believe this design can work well.

It just needs some polishing regarding naming and snake case / camel case, etc.

I quite like the prefix "vset" for volatile set. I think you should rename the struct to vset, the function prefix of all of these to vset and the file names to vset.{c,h}.

If we want to use this also for the main keyspace later (where the elements are not odd-aligned) we can for example skip single-element encoding if we try to add an element that is not odd-aligned. In that case, we can go directly to vector encoding.

[ranshid]

Yes, I believe this design can work well.

It just needs some polishing regarding naming and snake case / camel case, etc.

+1 I agree, will make order to it.

I quite like the prefix "vset" for volatile set. I think you should rename the struct to vset, the function prefix of all of these to vset and the file names to vset.{c,h}.

wel, it is possible.

If we want to use this also for the main keyspace later (where the elements are not odd-aligned) we can for example skip single-element encoding if we try to add an element that is not odd-aligned. In that case, we can go directly to vector encoding.

So for objects we can still do it by passing the ket sds pointer. since we know that the object will have the expiry we can go to the allocation start. I think the main challenge for generic keys is maintaining the updates (when the object is relocated, but I do not think that is something that happens outside of defrag, since once the object has expiry it will probably not be relocated). Another challenge is making the vset per slot. this will require some implementation ~kvstore (or maybe add it to the kvstore somehow).
regarding your suggestion to go to vector - it might work, but ATM lets keep this restriction.

[ranshid]

@zuiderkwast Thank you sooooooo much for taking the time to review this. I will make sure to work on your comment first thing on Sunday

[zuiderkwast]

Another pass on this. I tend to get stuck on details like naming rather than reviewing the actual logic. :D

[ranshid]

Another pass on this. I tend to get stuck on details like naming rather than reviewing the actual logic. :D

I understand. maybe in this specific review case the naming, style and the external API are the more important thing.

regarding the algorithm, I think I can help focus you on several questions:

1. the design is attempting to keep memory access local. this means that we would prefer not to access the entries expiration time too often. Still I choose to keep vectors sorted. I did that since it simplified the implementation. an alternative would be to just sort the vector before attempt to split it and for the other API cases we can just search the entire vector when we need to (more expensive than just access the minimum)
2. the entire pop/first API was created since the rax iterator is potentially invalidated on mutative actions (like remove and add). Still I think the pop should be improved. I think the API should be able to accept a pop function (i.e hashTypeExpireEntry in our case) and a max_count argument like:

size_t vsetPopExpired(vset *set, vsetGetExpiryFunc getExpiry, vsetExpiryFunc expiryFunc, mstime_t now, size_t count);

so it will internally iterate on buckets and remove entries up to max_count OR when the expiryFunc tels it to stop.
Why? because when expiring entries we would also like to potentially remove the containing object HASH (so the pointer to the set is lost) and perform other mutative actions. This is, however more in order to improve the activeExpiration performance and so I decided to hold with this change till this PR is up in the air.

[zuiderkwast]

Yeah I'm starting to grasp this design.

the design is attempting to keep memory access local. this means that we would prefer not to access the entries expiration time too often. Still I choose to keep vectors sorted. I did that since it simplified the implementation. an alternative would be to just sort the vector before attempt to split it and for the other API cases we can just search the entire vector when we need to (more expensive than just access the minimum)

Just an idea: If you want to use unsorted vectors, use a max size of 8. This makes sure the unsorted vector fits in a cache line, so linear search is still really fast. (It guess it can even be made O(1) using AVX512.)

[zuiderkwast]

In hashtable.{c,h} I put the API overview in the .h file and the internals overview in the .c file. Not sure if it matters...

[ranshid]

Let me try and polish it near the end of it.

[zuiderkwast]

The PR top-comment will maybe not be kept, but just thinking of the Evolution of the Bucket diagram. It makes no distinction between SINGLE and VECTOR on the top-level vs within a rax. These are quite different and I got confused by the diagram at first.

Idea below the line:

---

Volatile set top-level structure:

+--------+     +--------+     +--------+     +--------+
| NONE   | --> | SINGLE | --> | VECTOR | --> |   RAX  |
+--------+     +--------+     +--------+     +--------+

If the top-level element is a RAX, it has child buckets of type:

+--------+     +--------+     +-----------+
| SINGLE | --> | VECTOR | --> | HASHTABLE |
+--------+     +--------+     +-----------+

Vectors can split into multiple vectors and shrink into SINGLE buckets. A RAX with only one element is collapsed by replacing the RAX with its single element on the top level (except for HASHTABLE buckets which are not allowed on the top level).

[ranshid]

The PR top-comment will maybe not be kept, but just thinking of the Evolution of the Bucket diagram. It makes no distinction between SINGLE and VECTOR on the top-level vs within a rax. These are quite different and I got confused by the diagram at first.

Idea below the line:

Volatile set top-level structure:

+--------+     +--------+     +--------+     +--------+
| NONE   | --> | SINGLE | --> | VECTOR | --> |   RAX  |
+--------+     +--------+     +--------+     +--------+

If the top-level element is a RAX, it has child buckets of type:

+--------+     +--------+     +-----------+
| SINGLE | --> | VECTOR | --> | HASHTABLE |
+--------+     +--------+     +-----------+

Vectors can split into multiple vectors and shrink into SINGLE buckets. A RAX with only one element is collapsed by replacing the RAX with its single element on the top level (except for HASHTABLE buckets which are not allowed on the top level).

Yeh. It does hint about it, since the HT is only shown as a rax bucket, But I like your style better. I will change that

[ranshid]

I see there are new memory leaks. will search for it.

[zuiderkwast]

Will you describe somewhere when and where the vector is sorted vs unsorted?

[zuiderkwast]

Nice one-line comparison.

Would return a - b; work too?

[ranshid]

Nice one-line comparison.

Would return a - b; work too?

Well, not if we want to be compatible to returning [-1, 0, 1]

[zuiderkwast]

It's not required IIUC. From man qsort:

The comparison function must return an integer less than, equal to, or greater than zero if the first argument is considered to be respectively less than, equal to, or greater than the second.

[ranshid]

I guess it is possible

[ranshid]

Will you describe somewhere when and where the vector is sorted vs unsorted?

Sure.

[ranshid]

I see there are new memory leaks. will search for it.

nasty bug. should be fixed now

[ranshid]

note for the reviewers:

I think in the current implementation vsetUpdateEntry is very sub-optimal. it always remove and re-insert the entry. I think this performs bad in case of defrag and also in cases were the entry is not switching buckets. I will update the code with change for this.

[ranshid]

note for the reviewers:

I think in the current implementation vsetUpdateEntry is very sub-optimal. it always remove and re-insert the entry. I think this performs bad in case of defrag and also in cases were the entry is not switching buckets. I will update the code with change for this.

done

[ranshid]

So this PR was accidentally merged to the origin branch. I reverted these changes but I cannot reopen the PR and most of the comments here are cleared. lets move to work on #5