Skip to content

Database.runStream() truncates results when using async iterator #2502

@flovouin

Description

@flovouin

Please make sure you have searched for information in the following guides.

A screenshot that you have tested with "Try this API".

This bug occurs in the client library's Database.runStream() wrapper implementation, not in the Spanner API itself. The underlying Snapshot.runStream() works correctly and returns all results.

Link to the code that reproduces this issue. A link to a public Github Repository or gist with a minimal reproduction.

https://github.com/flovouin/nodejs-spanner-bugs/blob/main/run-stream.ts

A step-by-step description of how to reproduce the issue, based on the linked reproduction.

Run the provided script. You must use a production instance, not the emulator. However the database DDL doesn't matter.

A clear and concise description of what the bug is, and what you expected to happen.

When consuming results from Database.runStream() using an async iterator (for await...of), all 100 rows should be returned. The query SELECT x FROM UNNEST(GENERATE_ARRAY(0, 99)) AS x generates exactly 100 rows.

Instead, Database.runStream() truncates the results and returns fewer rows. Using Snapshot.runStream() with the same query and async iterator correctly returns all 100 rows.

Output of the script:

Query: SELECT x FROM UNNEST(GENERATE_ARRAY(0, 99)) AS x
Expected row count: 100

Database.runStream() row count: 32
Snapshot.runStream() row count: 100

A clear and concise description WHY you expect this behavior, i.e., was it a recent change, there is documentation that points to this behavior, etc. **

Database.runStream() is a convenience wrapper around Snapshot.runStream(). Both methods should behave identically when consuming results via async iteration.

The bug is in the Database.runStream() wrapper implementation. It wraps the underlying snapshot stream but introduces an issue that causes the stream to end prematurely when consumed as an async iterator.

Workaround: Use Snapshot.runStream() directly instead of Database.runStream():

const [snapshot] = await database.getSnapshot();

try {
  const stream = snapshot.runStream({
    sql: "SELECT x FROM UNNEST(GENERATE_ARRAY(0, 99)) AS x",
    json: true,
  });

  for await (const row of stream) {
    // Process row
  }
} finally {
  snapshot.end();
}

Metadata

Metadata

Assignees

No one assigned

    Labels

    api: spannerIssues related to the googleapis/nodejs-spanner API.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions