-
Notifications
You must be signed in to change notification settings - Fork 0
feat: Edge Runtime & Offline-First Sync Protocol #355
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com>
Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com>
Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com>
…liases for new packages Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com>
Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com>
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
Adds an edge runtime adaptation layer and an offline-first sync protocol to the ObjectQL monorepo, introducing new client/server sync packages, wiring test aliases, and documenting usage for edge + sync workflows.
Changes:
- Added
@objectql/edge-adapterfor runtime detection, capability validation, and edge driver binding resolution. - Added offline-first sync packages: client
@objectql/plugin-sync(mutation log + sync engine) and server@objectql/protocol-sync(handler + change log + version store). - Enabled sync-related driver capabilities for WASM drivers and updated docs/test config to support new packages.
Reviewed changes
Copilot reviewed 33 out of 34 changed files in this pull request and generated 10 comments.
Show a summary per file
| File | Description |
|---|---|
| vitest.config.ts | Adds path aliases for new edge + sync packages. |
| pnpm-lock.yaml | Adds new workspace importers/lock entries for the added packages. |
| packages/protocols/sync/tsconfig.json | Introduces tsconfig for new server sync protocol package. |
| packages/protocols/sync/src/version-store.ts | Adds in-memory server-side record version tracking. |
| packages/protocols/sync/src/sync-handler.ts | Implements push handling, conflict detection, and delta computation. |
| packages/protocols/sync/src/plugin.ts | Adds SyncProtocolPlugin to attach sync handler to the runtime context. |
| packages/protocols/sync/src/index.ts | Exports protocol-sync public API and re-exports sync types. |
| packages/protocols/sync/src/index.test.ts | Adds unit tests for ChangeLog, VersionStore, SyncHandler, and plugin integration. |
| packages/protocols/sync/src/change-log.ts | Adds checkpointed server change log with retention configuration and pruning API. |
| packages/protocols/sync/package.json | Defines new @objectql/protocol-sync package metadata/scripts. |
| packages/protocols/sync/README.md | Documents server-side sync protocol usage and API. |
| packages/foundation/plugin-sync/tsconfig.json | Introduces tsconfig for new client sync plugin package. |
| packages/foundation/plugin-sync/src/sync-engine.ts | Adds client sync engine with batching, checkpointing, and listener hooks. |
| packages/foundation/plugin-sync/src/plugin.ts | Adds SyncPlugin to expose per-object engines on the runtime context. |
| packages/foundation/plugin-sync/src/mutation-logger.ts | Adds in-memory append-only mutation log implementation. |
| packages/foundation/plugin-sync/src/index.ts | Exports plugin-sync public API and re-exports sync types. |
| packages/foundation/plugin-sync/src/index.test.ts | Adds unit tests for logger/resolvers/engine/plugin behaviors. |
| packages/foundation/plugin-sync/src/conflict-resolver.ts | Adds LWW/CRDT/manual conflict resolver implementations + factory. |
| packages/foundation/plugin-sync/package.json | Defines new @objectql/plugin-sync package metadata/scripts. |
| packages/foundation/plugin-sync/README.md | Documents client-side sync plugin usage and configuration. |
| packages/foundation/edge-adapter/tsconfig.json | Introduces tsconfig for new edge adapter package. |
| packages/foundation/edge-adapter/src/plugin.ts | Adds EdgeAdapterPlugin to detect runtime/validate/resolve bindings and attach context. |
| packages/foundation/edge-adapter/src/index.ts | Exports edge-adapter public API. |
| packages/foundation/edge-adapter/src/index.test.ts | Adds unit tests for runtime detection, capability validation, binding resolution, and plugin lifecycle. |
| packages/foundation/edge-adapter/src/detector.ts | Implements edge runtime auto-detection logic. |
| packages/foundation/edge-adapter/src/capabilities.ts | Implements capability lookup + requirement validation. |
| packages/foundation/edge-adapter/src/binding-resolver.ts | Implements default driver selection + binding resolution. |
| packages/foundation/edge-adapter/package.json | Defines new @objectql/edge-adapter package metadata/scripts. |
| packages/foundation/edge-adapter/README.md | Documents edge runtime adapter usage and configuration. |
| packages/drivers/sqlite-wasm/src/index.ts | Enables mutationLog and changeTracking capabilities for sqlite-wasm driver. |
| packages/drivers/pg-wasm/src/index.ts | Enables mutationLog and changeTracking capabilities for pg-wasm driver. |
| content/docs/server/sync.mdx | Adds server docs page describing offline-first sync architecture and usage. |
| content/docs/server/meta.json | Updates server docs navigation to include edge and sync. |
| content/docs/server/edge.mdx | Adds server docs page describing edge runtime support and configuration. |
Files not reviewed (1)
- pnpm-lock.yaml: Language not supported
| private generateId(): string { | ||
| // Simple UUID v4-like generation for environments without crypto.randomUUID | ||
| const hex = '0123456789abcdef'; | ||
| let id = ''; | ||
| for (let i = 0; i < 32; i++) { |
Copilot
AI
Feb 8, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
MutationLogEntry.id is specified in @objectql/types as a UUID v7 (time-ordered). generateId() currently produces a Math.random-based v4-like ID, which violates the protocol contract and can break ordering/dedup expectations. Generate UUID v7 (or use a UUID v7 library) and avoid Math.random for IDs.
| /** Record a mutation and optionally trigger debounced sync */ | ||
| recordMutation(entry: { | ||
| objectName: string; | ||
| recordId: string | number; | ||
| operation: MutationOperation; |
Copilot
AI
Feb 8, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
SyncEngine doesn't honor SyncConfig.enabled. recordMutation() will append mutations and schedule sync even when sync is disabled for an object, contradicting the config contract. Gate recordMutation/scheduleSync/sync() on this.config.enabled (and avoid emitting sync events) when disabled.
| this.logger = new MutationLogger(options.clientId); | ||
| this.transport = options.transport; | ||
| this.config = options.config; | ||
| this.resolver = createResolver(options.config.strategy ?? 'last-write-wins', options.onConflict); | ||
| } |
Copilot
AI
Feb 8, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
SyncEngine constructs a ConflictResolver (this.resolver) but never uses it; conflicts are only collected and surfaced to listeners. This makes the advertised conflict resolution strategies effectively unused. Either apply the resolver to conflicts during sync (and then requeue/patch mutations) or remove the resolver wiring to avoid misleading API behavior.
| async handlePush(request: SyncPushRequest, resolver: RecordResolver): Promise<SyncPushResponse> { | ||
| const maxMutations = this.config.maxMutationsPerRequest ?? 100; | ||
|
|
||
| if (request.mutations.length > maxMutations) { | ||
| const results: SyncMutationResult[] = request.mutations.map(() => ({ |
Copilot
AI
Feb 8, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ChangeLog retention is never enforced unless prune() is called, but handlePush() never invokes prune(). This makes change-log growth unbounded in long-running servers despite having a retention config. Call this.changeLog.prune() periodically (e.g., at the start/end of handlePush) or make pruning automatic in ChangeLog.record().
| // Handle update operations — check for conflicts | ||
| if (mutation.baseVersion !== null && mutation.baseVersion < currentVersion) { | ||
| const serverRecord = await resolver.getRecord(mutation.objectName, mutation.recordId); | ||
| if (!serverRecord) { | ||
| return { status: 'rejected', reason: 'Record not found' }; | ||
| } | ||
|
|
||
| const objectConflictFields = this.conflictFields.get(mutation.objectName) ?? []; | ||
| const conflictingFields = this.detectConflictingFields(mutation, serverRecord, objectConflictFields); | ||
|
|
||
| if (conflictingFields.length > 0) { | ||
| const conflict: SyncConflict = { | ||
| objectName: mutation.objectName, | ||
| recordId: mutation.recordId, | ||
| clientMutation: mutation, | ||
| serverRecord, | ||
| conflictingFields, | ||
| }; | ||
| return { status: 'conflict', conflict }; | ||
| } | ||
| } | ||
|
|
||
| // No conflict — apply the update | ||
| const newVersion = this.versionStore.increment(mutation.objectName, mutation.recordId); | ||
| await resolver.applyMutation(mutation, newVersion); | ||
| this.changeLog.record({ | ||
| objectName: mutation.objectName, | ||
| recordId: mutation.recordId, | ||
| operation: 'update', | ||
| data: mutation.data, | ||
| serverVersion: newVersion, | ||
| }); | ||
| return { status: 'applied', serverVersion: newVersion }; |
Copilot
AI
Feb 8, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updates are applied unconditionally here. If the record doesn't exist on the server (currentVersion === 0), an 'update' mutation can still be applied, which can create phantom records depending on RecordResolver.applyMutation. Consider rejecting update when the record is not found (and validating baseVersion is non-null for update ops).
| // Handle update operations — check for conflicts | |
| if (mutation.baseVersion !== null && mutation.baseVersion < currentVersion) { | |
| const serverRecord = await resolver.getRecord(mutation.objectName, mutation.recordId); | |
| if (!serverRecord) { | |
| return { status: 'rejected', reason: 'Record not found' }; | |
| } | |
| const objectConflictFields = this.conflictFields.get(mutation.objectName) ?? []; | |
| const conflictingFields = this.detectConflictingFields(mutation, serverRecord, objectConflictFields); | |
| if (conflictingFields.length > 0) { | |
| const conflict: SyncConflict = { | |
| objectName: mutation.objectName, | |
| recordId: mutation.recordId, | |
| clientMutation: mutation, | |
| serverRecord, | |
| conflictingFields, | |
| }; | |
| return { status: 'conflict', conflict }; | |
| } | |
| } | |
| // No conflict — apply the update | |
| const newVersion = this.versionStore.increment(mutation.objectName, mutation.recordId); | |
| await resolver.applyMutation(mutation, newVersion); | |
| this.changeLog.record({ | |
| objectName: mutation.objectName, | |
| recordId: mutation.recordId, | |
| operation: 'update', | |
| data: mutation.data, | |
| serverVersion: newVersion, | |
| }); | |
| return { status: 'applied', serverVersion: newVersion }; | |
| // Handle update operations — check for existence and conflicts | |
| if (mutation.operation === 'update') { | |
| if (currentVersion === 0) { | |
| return { status: 'rejected', reason: 'Record not found' }; | |
| } | |
| if (mutation.baseVersion == null) { | |
| return { status: 'rejected', reason: 'Missing baseVersion for update' }; | |
| } | |
| if (mutation.baseVersion < currentVersion) { | |
| const serverRecord = await resolver.getRecord(mutation.objectName, mutation.recordId); | |
| if (!serverRecord) { | |
| return { status: 'rejected', reason: 'Record not found' }; | |
| } | |
| const objectConflictFields = this.conflictFields.get(mutation.objectName) ?? []; | |
| const conflictingFields = this.detectConflictingFields(mutation, serverRecord, objectConflictFields); | |
| if (conflictingFields.length > 0) { | |
| const conflict: SyncConflict = { | |
| objectName: mutation.objectName, | |
| recordId: mutation.recordId, | |
| clientMutation: mutation, | |
| serverRecord, | |
| conflictingFields, | |
| }; | |
| return { status: 'conflict', conflict }; | |
| } | |
| } | |
| // No conflict — apply the update | |
| const newVersion = this.versionStore.increment(mutation.objectName, mutation.recordId); | |
| await resolver.applyMutation(mutation, newVersion); | |
| this.changeLog.record({ | |
| objectName: mutation.objectName, | |
| recordId: mutation.recordId, | |
| operation: 'update', | |
| data: mutation.data, | |
| serverVersion: newVersion, | |
| }); | |
| return { status: 'applied', serverVersion: newVersion }; | |
| } |
| async install(ctx: RuntimeContext): Promise<void> { | ||
| if (!this.config.endpoint.enabled) return; | ||
|
|
||
| this.handler = new SyncHandler({ | ||
| config: this.config.endpoint, |
Copilot
AI
Feb 8, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
SyncEndpointConfig includes fields like path/realtime, but the plugin only registers a handler on ctx.engine and doesn’t use endpoint.path (or register an HTTP/WebSocket route). This makes the config surface and docs read like a full endpoint implementation when it's not. Either wire these fields into the runtime’s HTTP layer (or a server plugin) or remove/rename unused config fields.
|
|
||
| const transport = { | ||
| async push(request) { | ||
| const res = await fetch('/api/sync/push', { |
Copilot
AI
Feb 8, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Client example posts to '/api/sync/push', but later server examples/config use '/api/sync' (endpoint.path) and an Express route at '/api/sync/push'. Align these paths (or explain push vs base path) so readers configure client/server consistently.
| const res = await fetch('/api/sync/push', { | |
| const res = await fetch('/api/sync', { |
| const validation = validateCapabilities(runtime, this.config.requirements); | ||
| if (!validation.valid) { | ||
| throw new Error( | ||
| `[${this.name}] Runtime '${runtime}' missing capabilities: ${validation.missing.join(', ')}`, | ||
| ); |
Copilot
AI
Feb 8, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
EdgeAdapterPlugin throws a generic Error when capability validation fails. The repo commonly uses structured ObjectQLError for user-facing failures; using Error here makes error handling inconsistent for consumers. Throw ObjectQLError (with a code/details) and include the missing capabilities in details rather than only in the message string.
| export function createResolver( | ||
| strategy: string, | ||
| onConflict?: (conflict: SyncConflict) => Record<string, unknown> | undefined | ||
| ): ConflictResolver { |
Copilot
AI
Feb 8, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
createResolver() / ConflictResolver.strategy are typed as plain string even though SyncStrategy is a closed union in @objectql/types. Using string weakens type-safety and permits invalid strategies at compile time. Consider typing strategy as SyncStrategy (and ConflictResolver.strategy as SyncStrategy) so the API stays aligned with the protocol types.
| - Server-side append-only change log with monotonic checkpoints | ||
| - Configurable retention policy (days) | ||
| - Checkpoint-based delta queries for efficient sync | ||
| - Automatic pruning of expired entries |
Copilot
AI
Feb 8, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The README claims "Automatic pruning of expired entries", but the current implementation only exposes prune() and never calls it automatically. Update the README to reflect manual pruning, or implement automatic pruning in the sync handler/change log so the statement is accurate.
| - Automatic pruning of expired entries | |
| - Supports manual pruning of expired entries via the `prune()` API |
Implements the Q3 2026 work plan: edge runtime adaptation layer and offline-first sync protocol with client-side mutation logging, server-side delta computation, and pluggable conflict resolution.
New Packages
@objectql/edge-adapterEdgeAdapterPluginimplementingRuntimePluginlifecycle@objectql/plugin-syncMutationLogger— client-side append-only log with sequence trackingSyncEngine— push/pull orchestration with debounced batching and checkpoint managementSyncPluginwith lazy per-object engine creation@objectql/protocol-syncChangeLog— server-side append-only change log with checkpoint-based delta computationVersionStore— per-record optimistic concurrency controlSyncHandler— processes push requests, detects conflicts, returns server deltasOther Changes
mutationLogandchangeTrackingcapabilities onsqlite-wasmandpg-wasmUsage
81 new tests across 3 packages.
Warning
Firewall rules blocked me from connecting to one or more addresses (expand for details)
I tried to connect to the following addresses, but was blocked by firewall rules:
fastdl.mongodb.org/home/REDACTED/work/_temp/ghcca-node/node/bin/node node ./postinstall.js(dns block)If you need me to access, download, or install something from one of these locations, you can either:
💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.