Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions src/main/asciidoc/core-concepts/chapter.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@ include::indexes.adoc[]

include::full-text-index.adoc[]

include::materialized-views.adoc[]

include::graphs.adoc[]

include::databases.adoc[]
217 changes: 217 additions & 0 deletions src/main/asciidoc/core-concepts/materialized-views.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,217 @@
[[materialized-views]]
=== Materialized Views
image:../images/edit.png[link="https://github.com/ArcadeData/arcadedb-docs/blob/main/src/main/asciidoc/core-concepts/materialized-views.adoc" float=right]

A materialized view is a schema-level object that stores the result of a SQL `SELECT` query as a backing document type.
Unlike regular views (which re-execute the query on every access), a materialized view holds a pre-computed snapshot of data that can be queried directly for fast reads.

A materialized view:

* Wraps a SQL `SELECT` query as its _defining query_
* Stores results in a backing `DocumentType` with standard buckets
* Supports three _refresh modes_: manual, incremental (post-commit), and periodic (scheduled)
* Persists its definition and metadata in `schema.json` alongside other schema objects
* Can be created, dropped, refreshed, and altered via SQL DDL statements or the Java Schema API

==== Refresh Modes

[[refresh-mode-manual]]
===== MANUAL

The view data is never automatically updated.
You must trigger a refresh explicitly via <<sql-refresh-materialized-view,`REFRESH MATERIALIZED VIEW`>> or the Java API.
Use this when you control refresh timing yourself or when source data changes infrequently.

[[refresh-mode-incremental]]
===== INCREMENTAL

After every committed transaction that modifies a source type, ArcadeDB automatically refreshes the view in a post-commit callback.
The refresh is:

* *Transactionally safe*: runs after the source transaction commits successfully; rolled-back transactions do not trigger a refresh
* *Batched per transaction*: multiple record changes in a single transaction result in one refresh, not one per record
* For *simple queries* (single source type, no aggregates, no `GROUP BY`, no subqueries, no `JOIN`s): performs a full refresh
* For *complex queries* (aggregates, `GROUP BY`, etc.): also performs a full refresh (per-record incremental optimization is planned for a future release)

If the refresh fails, the view is marked `STALE` and a warning is logged.
A manual refresh can recover it.

[[refresh-mode-periodic]]
===== PERIODIC

A background scheduler thread runs a full refresh at the specified interval after each successful refresh completes.
Intervals are specified in seconds, minutes, or hours:

[source,sql]
----
REFRESH EVERY 30 SECOND
REFRESH EVERY 5 MINUTE
REFRESH EVERY 1 HOUR
----

The scheduler uses a single daemon thread (`ArcadeDB-MV-Scheduler`) shared across all periodic views.
If the database is closed, all scheduled tasks are cancelled automatically.

==== View Status

Each view tracks a `status` field that reflects its current state:

[%header,cols=2]
|===
| Status | Meaning
| `VALID` | Data is up to date with the last refresh
| `STALE` | A refresh failed or was interrupted; data may be outdated
| `BUILDING` | A refresh is currently in progress
| `ERROR` | The last refresh encountered a fatal error
|===

If the database crashes while a view is `BUILDING`, the status is reset to `STALE` on the next startup to signal that the data may be incomplete.

==== Querying a Materialized View

Query a materialized view exactly like any other document type:

[source,sql]
----
SELECT * FROM ActiveUsers
SELECT name FROM ActiveUsers WHERE name LIKE 'A%'
SELECT count(*) FROM RecentOrders
----

==== Java API

*Creating a view*

[source,java]
----
database.transaction(() -> {
database.getSchema().buildMaterializedView()
.withName("ActiveUsers")
.withQuery("SELECT name, email FROM User WHERE active = true")
.withRefreshMode(MaterializedViewRefreshMode.MANUAL)
.create();
});
----

Builder options:

[%header,cols=2]
|===
| Method | Description
| `withName(String)` | Name for the view (required)
| `withQuery(String)` | Defining SQL `SELECT` query (required)
| `withRefreshMode(MaterializedViewRefreshMode)` | `MANUAL`, `INCREMENTAL`, or `PERIODIC` (default: `MANUAL`)
| `withTotalBuckets(int)` | Number of buckets for the backing type
| `withPageSize(int)` | Page size for the backing type
| `withRefreshInterval(long)` | Interval in milliseconds for `PERIODIC` mode
| `withIgnoreIfExists(boolean)` | When `true`, returns existing view instead of throwing
|===

*Querying schema*

[source,java]
----
Schema schema = database.getSchema();

// Check existence
boolean exists = schema.existsMaterializedView("ActiveUsers");

// Get a specific view
MaterializedView view = schema.getMaterializedView("ActiveUsers");

// List all views
MaterializedView[] views = schema.getMaterializedViews();
----

*Refreshing and dropping*

[source,java]
----
// Programmatic refresh
database.getSchema().getMaterializedView("ActiveUsers").refresh();

// Drop via schema
database.getSchema().dropMaterializedView("ActiveUsers");

// Drop via the view itself
database.getSchema().getMaterializedView("ActiveUsers").drop();
----

*Inspecting a view*

[source,java]
----
MaterializedView view = database.getSchema().getMaterializedView("HourlySummary");

view.getName(); // "HourlySummary"
view.getQuery(); // the defining SQL query
view.getRefreshMode(); // MaterializedViewRefreshMode.PERIODIC
view.getStatus(); // "VALID", "STALE", "BUILDING", or "ERROR"
view.getLastRefreshTime(); // epoch millis of last successful refresh
view.isSimpleQuery(); // true if eligible for per-record optimization
view.getSourceTypeNames(); // list of source type names parsed from the query
view.getBackingType(); // the underlying DocumentType
----

==== Behavior and Constraints

* *Backing type protection*: You cannot `DROP TYPE` on a type that backs a materialized view. Drop the materialized view first.
* *Name uniqueness*: The view name must not match any existing type or materialized view.
* *Source type validation*: All types referenced in the `FROM` clause must exist when the view is created.
* *Persistence*: View definitions are stored in `schema.json` under a `"materializedViews"` key and survive database restarts. Listener registration for `INCREMENTAL` views and scheduler tasks for `PERIODIC` views are re-established on startup.
* *Transaction safety*: The initial full refresh and all subsequent refreshes run inside their own transactions.
* *Query result columns*: Only non-internal properties (those not starting with `@`) are copied into the backing type during refresh.
* *No schema on backing type*: The backing document type is schema-less; property types are not enforced.

==== Error Handling

* If a post-commit refresh fails (`INCREMENTAL` mode), the view is marked `STALE` and a `WARNING` is logged. The source transaction is unaffected.
* If a periodic refresh fails, the view is marked `ERROR` and a `SEVERE` log entry is written. The scheduler continues running and will retry on the next interval.
* Callback errors in the transaction callback system are logged at `WARNING` level and do not affect the triggering transaction or other callbacks.

==== Limitations

* <<sql-alter-materialized-view,`ALTER MATERIALIZED VIEW`>> is not yet implemented.
* Per-record incremental refresh (tracking `_sourceRID` to update individual view rows) is a planned future optimization. Currently, all refresh operations perform a full truncate-and-reload.
* No support for cross-database queries in the defining query.
* Server replication: materialized view data lives in the local backing type and is replicated like any other document type in an HA cluster, but refresh triggering is local to the node that executes the write.

==== Example: Sales Dashboard

[source,sql]
----
-- Source type
CREATE DOCUMENT TYPE Sale;

-- A periodic summary refreshed every minute
CREATE MATERIALIZED VIEW SalesByProduct
AS SELECT product, sum(amount) AS total, count(*) AS count
FROM Sale
GROUP BY product
REFRESH EVERY 1 MINUTE;

-- An incremental view of recent activity (simple query)
CREATE MATERIALIZED VIEW RecentSales
AS SELECT product, amount, date
FROM Sale
WHERE date >= '2026-01-01'
REFRESH INCREMENTAL;

-- Query the views
SELECT * FROM SalesByProduct ORDER BY total DESC;
SELECT product, amount FROM RecentSales WHERE amount > 1000;

-- Manual refresh after a bulk import
REFRESH MATERIALIZED VIEW SalesByProduct;

-- Teardown
DROP MATERIALIZED VIEW SalesByProduct;
DROP MATERIALIZED VIEW RecentSales;
----

For more information, see:

* <<sql-create-materialized-view,`CREATE MATERIALIZED VIEW`>>
* <<sql-drop-materialized-view,`DROP MATERIALIZED VIEW`>>
* <<sql-refresh-materialized-view,`REFRESH MATERIALIZED VIEW`>>
* <<sql-alter-materialized-view,`ALTER MATERIALIZED VIEW`>>
14 changes: 12 additions & 2 deletions src/main/asciidoc/query-languages/sql/chapter.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,10 @@ image:../images/edit.png[link="https://github.com/ArcadeData/arcadedb-docs/blob/
| <<sql-delete,DELETE>> | <<sql-truncate-type,TRUNCATE TYPE>> | <<sql-import-database,IMPORT DATABASE>> |
| <<sql-create-vertex,CREATE VERTEX>> | <<sql-create-bucket,CREATE BUCKET>> | <<sql-align-database,ALIGN DATABASE>> |
| <<sql-create-edge,CREATE EDGE>> | ALTER BUCKET (_not implemented_) | <<sql-check-database,CHECK DATABASE>> |
| <<sql-move-vertex,MOVE VERTEX>> | <<sql-drop-bucket,DROP BUCKET>> | |
| <<sql-match,MATCH>> | <<sql-truncate-bucket,TRUNCATE BUCKET>> | |
| <<sql-move-vertex,MOVE VERTEX>> | <<sql-drop-bucket,DROP BUCKET>> | <<sql-create-materialized-view,CREATE MATERIALIZED VIEW>> |
| <<sql-match,MATCH>> | <<sql-truncate-bucket,TRUNCATE BUCKET>> | <<sql-drop-materialized-view,DROP MATERIALIZED VIEW>> |
| | | <<sql-refresh-materialized-view,REFRESH MATERIALIZED VIEW>> |
| | | <<sql-alter-materialized-view,ALTER MATERIALIZED VIEW>> (_planned_) |
| <<sql-traverse,TRAVERSE>> | <<sql-create-property,CREATE PROPERTY>> | <<sql-create-index,CREATE INDEX>> |
| | <<sql-alter-property,ALTER PROPERTY>> | <<sql-rebuild-index,REBUILD INDEX>> |
| | <<sql-drop-property,DROP PROPERTY>> | <<sql-drop-index,DROP INDEX>> |
Expand Down Expand Up @@ -126,6 +128,8 @@ include::sql-align-database.adoc[]

include::sql-alter-database.adoc[]

include::sql-alter-materialized-view.adoc[]

include::sql-alter-property.adoc[]

include::sql-alter-type.adoc[]
Expand All @@ -140,6 +144,8 @@ include::sql-create-bucket.adoc[]

include::sql-create-edge.adoc[]

include::sql-create-materialized-view.adoc[]

include::sql-create-index.adoc[]

include::sql-create-property.adoc[]
Expand All @@ -156,6 +162,8 @@ include::sql-drop-bucket.adoc[]

include::sql-drop-index.adoc[]

include::sql-drop-materialized-view.adoc[]

include::sql-drop-property.adoc[]

include::sql-drop-trigger.adoc[]
Expand All @@ -178,6 +186,8 @@ include::sql-profile.adoc[]

include::sql-rebuild-index.adoc[]

include::sql-refresh-materialized-view.adoc[]

include::sql-select.adoc[]

include::sql-select-execution.adoc[]
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
[[sql-alter-materialized-view]]
[discrete]
==== SQL - `ALTER MATERIALIZED VIEW`
image:../images/edit.png[link="https://github.com/ArcadeData/arcadedb-docs/blob/main/src/main/asciidoc/query-languages/sql/sql-alter-materialized-view.adoc" float=right]

Changes the refresh mode of an existing materialized view.

NOTE: `ALTER MATERIALIZED VIEW` is planned for a future release and is not yet implemented.

*Syntax*

[source,sql]
----
ALTER MATERIALIZED VIEW <name>
REFRESH MANUAL | REFRESH INCREMENTAL | REFRESH EVERY <n> SECOND | MINUTE | HOUR
----

* *`<name>`* The name of the materialized view to alter.
* *`REFRESH MANUAL`* Switches the view to manual refresh mode.
* *`REFRESH INCREMENTAL`* Switches the view to incremental refresh mode (automatic after each relevant commit).
* *`REFRESH EVERY <n> SECOND | MINUTE | HOUR`* Switches the view to periodic refresh mode at the specified interval.

For more information, see:

* <<materialized-views,Materialized Views>>
* <<sql-create-materialized-view,`CREATE MATERIALIZED VIEW`>>
* <<sql-refresh-materialized-view,`REFRESH MATERIALIZED VIEW`>>
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
[[sql-create-materialized-view]]
[discrete]
==== SQL - `CREATE MATERIALIZED VIEW`
image:../images/edit.png[link="https://github.com/ArcadeData/arcadedb-docs/blob/main/src/main/asciidoc/query-languages/sql/sql-create-materialized-view.adoc" float=right]

Creates a new materialized view in the schema.
A materialized view stores the result of a SQL `SELECT` query in a backing document type for fast reads.
The defining query is executed immediately on creation to perform the initial data load.

*Syntax*

[source,sql]
----
CREATE MATERIALIZED VIEW [IF NOT EXISTS] <name>
AS <select-query>
[REFRESH MANUAL | REFRESH INCREMENTAL | REFRESH EVERY <n> SECOND | MINUTE | HOUR]
[BUCKETS <n>]
----

* *`<name>`* Defines the name of the materialized view. It must not conflict with any existing type or materialized view.
* *`IF NOT EXISTS`* Silently skips creation if a view with the same name already exists, instead of raising an error.
* *`<select-query>`* A SQL `SELECT` statement that defines the data for the view. All source types referenced must exist at creation time.
* *`REFRESH MANUAL`* (default) The view is only refreshed when triggered explicitly.
* *`REFRESH INCREMENTAL`* The view is automatically refreshed after each committed transaction that modifies a source type.
* *`REFRESH EVERY <n> SECOND | MINUTE | HOUR`* The view is refreshed on a schedule at the given interval.
* *`BUCKETS <n>`* Sets the number of buckets for the backing document type.

*Examples*

* Create a simple manual-refresh view:

[source,sql]
----
CREATE MATERIALIZED VIEW ActiveUsers
AS SELECT name, email FROM User WHERE active = true
----

* Create a view that updates automatically after each relevant commit:

[source,sql]
----
CREATE MATERIALIZED VIEW RecentOrders
AS SELECT customer, amount FROM Order WHERE status = 'PENDING'
REFRESH INCREMENTAL
----

* Create a view that refreshes on a schedule:

[source,sql]
----
CREATE MATERIALIZED VIEW HourlySummary
AS SELECT product, sum(amount) AS total FROM Sale GROUP BY product
REFRESH EVERY 1 HOUR
----

* Skip creation if the view already exists:

[source,sql]
----
CREATE MATERIALIZED VIEW IF NOT EXISTS ActiveUsers
AS SELECT name FROM User WHERE active = true
----

* Create a view with a custom bucket count for large result sets:

[source,sql]
----
CREATE MATERIALIZED VIEW BigView
AS SELECT * FROM HugeType
BUCKETS 8
----

For more information, see:

* <<materialized-views,Materialized Views>>
* <<sql-drop-materialized-view,`DROP MATERIALIZED VIEW`>>
* <<sql-refresh-materialized-view,`REFRESH MATERIALIZED VIEW`>>
* <<sql-alter-materialized-view,`ALTER MATERIALIZED VIEW`>>
Loading
Loading