feat(storage): add CrdtType to entity metadata #1786

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed

xilosada wants to merge 36 commits into master from feat/storage-add-crdt-type-to-metadata

Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Cargo.toml

-Original file line number
+Diff line change
@@ Expand Up / @@ -84,6 +84,7 @@ members = [ @@
         "./apps/private_data",
         "./apps/blobs",
         "./apps/collaborative-editor",
+        "./apps/comprehensive-crdt-test",
         "./apps/abi_conformance",
         "./apps/state-schema-conformance",
         "./apps/xcall-example",
@@ Expand Down @@

apps/comprehensive-crdt-test/Cargo.toml

-Original file line number
+Diff line change
@@ -0,0 +1,24 @@
+    [package]
+    name = "comprehensive-crdt-test"
+    version.workspace = true
+    authors.workspace = true
+    edition.workspace = true
+    repository.workspace = true
+    license.workspace = true
+    publish = false
+    [lib]
+    crate-type = ["cdylib"]
+    [dependencies]
+    thiserror.workspace = true
+    calimero-sdk.workspace = true
+    calimero-storage.workspace = true
+    hex.workspace = true
+    [build-dependencies]
+    calimero-wasm-abi.workspace = true
+    serde_json.workspace = true
+    [package.metadata.workspaces]
+    independent = true

apps/comprehensive-crdt-test/README.md

-Original file line number
+Diff line change
@@ -0,0 +1,64 @@
+    # Comprehensive CRDT Test Application
+    This application tests **ALL** CRDT types, UserStorage, FrozenStorage, and root-level concurrent modifications.
+    ## Features Tested
+    ### CRDT Types
+    - ✅ **Counter** - Grow-only counter with concurrent increments
+    - ✅ **UnorderedMap** - Field-level merge semantics
+    - ✅ **Vector** - Element-wise merge
+    - ✅ **UnorderedSet** - Union merge semantics
+    - ✅ **RGA (ReplicatedGrowableArray)** - Text CRDT for collaborative editing
+    - ✅ **LwwRegister** - Last-write-wins register
+    ### Storage Types
+    - ✅ **UserStorage (Simple)** - User-owned simple values
+    - ✅ **UserStorage (Nested)** - User-owned nested data structures
+    - ✅ **FrozenStorage** - Content-addressable immutable storage
+    ### Root-Level Merging
+    - ✅ **Concurrent Root Modifications** - Tests that root merge works when different nodes modify different root fields concurrently
+    ## Purpose
+    This app is designed to:
+. Test all CRDT types in a single application
+. Verify root-level concurrent modifications trigger proper merging
+. Test UserStorage and FrozenStorage alongside CRDT types
+. Serve as a comprehensive integration test for the sync protocol
+    ## Usage
+    Build the app:
+    ```bash
+    ./build.sh
+    ```
+    Run the workflow:
+    ```bash
+    merobox bootstrap run workflows/comprehensive-crdt-test.yml
+    ```
+    ## Workflow Tests
+    The `comprehensive-crdt-test.yml` workflow tests:
+. Root Counter - concurrent increments merge correctly
+. Root Map - field-level merge when different nodes modify different keys
+. Root Vector - element-wise merge
+. Root Set - union merge
+. Root RGA - text CRDT merge
+. Root Register - LWW semantics
+. UserStorage Simple - user-owned data sync
+. UserStorage Nested - nested user data with CRDTs
+. FrozenStorage - content-addressable storage
+. Root-Level Concurrent Modifications - different nodes modifying different root fields simultaneously
+    ## Architecture
+    The app state (`ComprehensiveCrdtApp`) contains all CRDT types and storage types as root-level fields. This design allows testing root-level concurrent modifications where:
+    - Node 1 modifies `root_counter`
+    - Node 2 modifies `root_map`
+    - Node 1 modifies `root_set`
+    All concurrently, triggering `merge_root_state` to merge all fields correctly.

apps/comprehensive-crdt-test/build.rs

-Original file line number
+Diff line change
@@ -0,0 +1,38 @@
+    use std::fs;
+    use std::path::Path;
+    use calimero_wasm_abi::emitter::emit_manifest;
+    fn main() {
+        println!("cargo:rerun-if-changed=src/lib.rs");
+        // Parse the source code
+        let src_path = Path::new("src/lib.rs");
+        let src_content = fs::read_to_string(src_path).expect("Failed to read src/lib.rs");
+        // Generate ABI manifest using the emitter
+        let manifest = emit_manifest(&src_content).expect("Failed to emit ABI manifest");
+        // Serialize the manifest to JSON
+        let json = serde_json::to_string_pretty(&manifest).expect("Failed to serialize manifest");
+        // Write the ABI JSON to the res directory
+        let res_dir = Path::new("res");
+        if !res_dir.exists() {
+            fs::create_dir_all(res_dir).expect("Failed to create res directory");
+        }
+        let abi_path = res_dir.join("abi.json");
+        fs::write(&abi_path, json).expect("Failed to write ABI JSON");
+        // Extract and write the state schema
+        if let Ok(mut state_schema) = manifest.extract_state_schema() {
+            state_schema.schema_version = "wasm-abi/1".to_owned();
+            let state_schema_json =
+                serde_json::to_string_pretty(&state_schema).expect("Failed to serialize state schema");
+            let state_schema_path = res_dir.join("state-schema.json");
+            fs::write(&state_schema_path, state_schema_json)
+                .expect("Failed to write state schema JSON");
+        }
+    }

apps/comprehensive-crdt-test/build.sh

-Original file line number
+Diff line change
@@ -0,0 +1,19 @@
+    #!/bin/bash
+    set -e
+    # Add wasm32 target if not already present
+    rustup target add wasm32-unknown-unknown || true
+    # Build the app
+    cargo build -p comprehensive-crdt-test --target wasm32-unknown-unknown --release
+    # Copy WASM file to res directory
+    mkdir -p res
+    cp target/wasm32-unknown-unknown/release/comprehensive_crdt_test.wasm res/comprehensive_crdt_test.wasm
+    # Optimize WASM if wasm-opt is available
+    if command -v wasm-opt &> /dev/null; then
+        wasm-opt -O2 res/comprehensive_crdt_test.wasm -o res/comprehensive_crdt_test.wasm || true
+    fi
+    echo "Build complete: res/comprehensive_crdt_test.wasm"

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(storage): add CrdtType to entity metadata #1786

Uh oh!

Diff view

Diff view

There are no files selected for viewing

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!