Skip to content

Evaluate different data format for speeding up reprocessing #59

@hellais

Description

@hellais

Reprocessing is currently not so fast, given that it's feeding from JSONL files. We should consider using something that's more performant like avro, msgpack or similar.

For historical purposes, here are some benchmarks I ran sometime ago while building OONI Data:

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions