Plans for more frequent data releases #453
Replies: 1 comment 1 reply
-
|
Hi @DRMap, thanks for the thoughtful question, and you're right that the lag can feel long. Let me tell you a little bit about what's happening behind the scenes. I was just talking to one of our pipeline engineers about their process for refreshing OSM data. "Always three Saturdays before release for buildings and base. Plus fixes to issues found up to two Saturdays before release," they said. "In other words: we freeze OSM three Saturdays before release. Fix issues for a week. And then we use that freeze plus fixes in the Overture release." The process for divisions (different pipeline, different engineer) is a little looser: "We don't have a clear procedure when and how we freeze. We take the data from OSM on time/date when OSM Nightwatch says it's stable and go from there. If we identify issues we fix OSM and repeat the process." The main constraints are the time we spend cleaning, fixing, and validating the data-- not just OSM, all our feed datasets. We also freeze the Overture schema two weeks before release, and we try to keep our data refreshes in sync so that we can run our schema-data validation processes across datasets. We're actively optimizing those validation processes, and it's possible we'll start releasing some datasets on a submonthly cadence this year. No firm commitments yet, but it's on our radar. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
Hi everyone,
I’ve been following the monthly Overture data releases and appreciate the consistency of the current cadence. I’m wondering whether there are any plans or discussions about increasing the release frequency in the future — for example, moving toward bi‑weekly or even more continuous updates.
As an example, the release 2025-12-17 had OSM data from 2025-11-30. The next release was 2026-01-21. This means OSM contributions that were made on 2025-12-01 were only publiished in Overture ~50 days later.
Are there technical, organizational, or data‑pipeline constraints that make monthly releases the practical limit right now? And is there any roadmap item or long‑term vision for offering more frequent updates as the project evolves?
Any insight into how the team is thinking about release cadence would be really helpful. Thanks for all the work you’re doing on Overture.
Dietmar
Beta Was this translation helpful? Give feedback.
All reactions