Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 15 additions & 15 deletions .github/workflows/lint_queries.yml
Original file line number Diff line number Diff line change
@@ -1,28 +1,28 @@
---
name: SQLFluff lint the queries
run-name: Basic testing of the queries in src/queries
name: SQLFluff lint the queries
run-name: Basic testing of the queries in src/queries

on:
pull_request:
paths:
- 'src/queries/**'
workflow_dispatch:
on:
pull_request:
paths:
- 'src/queries/**'
workflow_dispatch:

permissions:
id-token: write
contents: read
permissions:
id-token: write
contents: read

jobs:
run:
runs-on: ubuntu-latest
steps:
jobs:
run:
runs-on: ubuntu-latest
steps:
- name: Checkout Repo
uses: actions/checkout@v3

- name: Set up Python 3.11
uses: actions/setup-python@v3
with:
python-version: "3.11"
python-version: '3.11'

- name: Install dependencies
run: |
Expand Down
13 changes: 3 additions & 10 deletions .github/workflows/production_deploy_documentation.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,6 @@ on:

# Allow running from the actions tab
workflow_dispatch:
inputs:
schema_branch:
description: 'Schema branch to pull from for /reference'
required: true
default: 'main'

permissions:
id-token: write # required to use OIDC authentication
Expand All @@ -29,24 +24,22 @@ jobs:
uses: actions/checkout@v3

- name: Set up Node.js
uses: actions/setup-node@v4
uses: actions/setup-node@v6
with:
node-version: "20"
node-version: '24'

- name: Install NPM dependencies
run: npm install --prefer-dedupe

- name: Build Docusaurus Pages 🔧
env:
SCHEMA_BRANCH: ${{ inputs.schema_branch }}
run: npm run build

- name: Setup Pages 🗂️
uses: actions/configure-pages@v5

- name: Upload static files as GH-Pages artifact 📦
id: docs-website
uses: actions/upload-pages-artifact@v3
uses: actions/upload-pages-artifact@v4
with:
path: build

Expand Down
9 changes: 4 additions & 5 deletions .github/workflows/staging_deploy.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ run-name: Publish docs to staging website (for PR)

on:
pull_request:
branches: [ main ]
branches: [main]

permissions:
id-token: write
Expand All @@ -19,9 +19,9 @@ jobs:
uses: actions/checkout@v4

- name: Set up Node.js
uses: actions/setup-node@v4
uses: actions/setup-node@v6
with:
node-version: "20"
node-version: '24'

- name: Install NPM dependencies
run: npm ci --omit=dev
Expand All @@ -30,10 +30,9 @@ jobs:
env:
DOCUSAURUS_URL: https://staging.overturemaps.org/
DOCUSAURUS_BASE_URL: /${{ github.event.repository.name }}/pr/${{ github.event.number }}/
SCHEMA_BRANCH: ${{ inputs.schema_branch }}
run: npm run build

- name: Upload docs build as GH-Pages artifact 📦
- name: Upload docs build as an artifact 📦
uses: actions/upload-artifact@v4
with:
path: build
Expand Down
5 changes: 0 additions & 5 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,9 +1,4 @@
.DS_Store
.docusaurus
node_modules/
docs/_examples/
docs/_schema/
docs/schema/
docs/schema
static/_schema/
build/
9 changes: 9 additions & 0 deletions .markdownlint.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
{
"default": true,
"MD013": false,
"MD033": false,
"MD041": false,
"MD024": {
"siblings_only": true
}
}
6 changes: 6 additions & 0 deletions .prettierignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
build/
.docusaurus/
node_modules/
package-lock.json
*.md
*.mdx
10 changes: 10 additions & 0 deletions .prettierrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
{
"semi": true,
"singleQuote": true,
"tabWidth": 2,
"trailingComma": "es5",
"printWidth": 100,
"bracketSpacing": true,
"arrowParens": "always",
"proseWrap": "preserve"
}
2 changes: 1 addition & 1 deletion blog/2024-04-22-beta-release.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ WITH admins AS (
We plan to [deprecate admins](https://docs.overturemaps.org/release-notes/) by the July release. In the meantime, both `admins` and `divisions` will be available to users.

## Bridges, islands, waterfalls, and more!
We added more rich detail to our `base` layer in this release, including an `infrastructure` type with familiar features from [Facebook’s Daylight map distribution](https://daylightmap.org/). We also added new subtypes and classes for the `land`, `land_use`, and `water` feature types. You'll find a comprehensive listing of the subtypes and classes for each feature type in our [schema reference docs](https://docs.overturemaps.org/schema/reference/base/infrastructure). Ready to make your own map? We have a [tutorial to help you get started](https://docs.overturemaps.org/examples/build-a-map/#13/47.6/-122.33/0/45).
We added more rich detail to our `base` layer in this release, including an `infrastructure` type with familiar features from [Facebook’s Daylight map distribution](https://daylightmap.org/). We also added new subtypes and classes for the `land`, `land_use`, and `water` feature types. You'll find a comprehensive listing of the subtypes and classes for each feature type in our schema reference docs. Ready to make your own map? We have a [tutorial to help you get started](https://docs.overturemaps.org/examples/build-a-map/#13/47.6/-122.33/0/45).

## Stay tuned for more highlights
We'll be back soon with more posts that explore our path from the beta release to production. In the meantime, we invite you to [get started with our data](https://docs.overturemaps.org/getting-data/) and share with us your [comments and feedback](https://github.com/OvertureMaps/data/discussions).
Expand Down
64 changes: 32 additions & 32 deletions blog/2024-05-16-land-cover.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -5,64 +5,67 @@ tags:
- base
---

![land cover gif](/img/blog/lc.gif)
import useBaseUrl from '@docusaurus/useBaseUrl';

## Mapmakers rejoice!
<video autoPlay loop muted playsInline style={{ width: '100%' }}>
<source src={useBaseUrl('/img/blog/lc.mp4')} type="video/mp4" />
</video>

The [May release](https://docs.overturemaps.org/release/latest) of Overture Maps includes new high-resolution land cover data *and* new cartographic schema properties.
## Mapmakers rejoice!

The [May release](https://docs.overturemaps.org/release/latest) of Overture Maps includes new high-resolution land cover data _and_ new cartographic schema properties.

<!-- truncate -->

Our `land_cover` layer is vectorized data derived from the [European Space Agency’s 2020 WorldCover (10m) rasters](https://esa-worldcover.org/en). It's similar to the land cover layer in the [Daylight](https://daylightmap.org/2023/10/11/landcover.html) map distribution, but Overture Maps added higher-resolution data (zoom level 15) with more detail and land cover classes. You'll find 10 subtypes in the new data: snow, forest, urban, grass, crop, barren, wetland, moss, mangrove, and shrub.
Our `land_cover` layer is vectorized data derived from the [European Space Agency’s 2020 WorldCover (10m) rasters](https://esa-worldcover.org/en). It's similar to the land cover layer in the [Daylight](https://daylightmap.org/2023/10/11/landcover.html) map distribution, but Overture Maps added higher-resolution data (zoom level 15) with more detail and land cover classes. You'll find 10 subtypes in the new data: snow, forest, urban, grass, crop, barren, wetland, moss, mangrove, and shrub.

Our May release also includes [schema properties](https://docs.overturemaps.org/schema/reference/base/land_cover/) that offer cartographic "hints" for optimal use of Overture Maps data in mapmaking. We added `min_zoom` and `max_zoom` to define the recommended zooms for each resolution of land cover, using the common “slippy maps” zoom level specification. This is a first step toward improving the user experience for mapmakers. We plan to expand these properties in future releases of Overture Maps data.
Our May release also includes schema properties that offer cartographic "hints" for optimal use of Overture Maps data in mapmaking. We added `min_zoom` and `max_zoom` to define the recommended zooms for each resolution of land cover, using the common “slippy maps” zoom level specification. This is a first step toward improving the user experience for mapmakers. We plan to expand these properties in future releases of Overture Maps data.

## Exploring land cover

In the notebook example below, we'll show you how to extract, process, and visualize land cover data for an area of interest using [lonboard](https://developmentseed.org/lonboard/latest/) and the [Overture Maps Python command-line tool](https://github.com/OvertureMaps/overturemaps-py). We recommend that you consult [this example](https://developmentseed.org/lonboard/latest/examples/overture-maps/ ) in the lonboard docs to better understand the methods used here. You can view and download the complete notebook on [Notebook Sharing Space](https://notebooksharing.space/view/b63f6b3dda1da99c45caf53284fbc508aaae0a43480b43bb0316db636c5e6677).

In the notebook example below, we'll show you how to extract, process, and visualize land cover data for an area of interest using [lonboard](https://developmentseed.org/lonboard/latest/) and the [Overture Maps Python command-line tool](https://github.com/OvertureMaps/overturemaps-py). We recommend that you consult [this example](https://developmentseed.org/lonboard/latest/examples/overture-maps/) in the lonboard docs to better understand the methods used here. You can view and download the complete notebook on [Notebook Sharing Space](https://notebooksharing.space/view/b63f6b3dda1da99c45caf53284fbc508aaae0a43480b43bb0316db636c5e6677).

![land cover](/img/blog/notebook-overture-lc.png)

To follow along, you'll need to have [JupyterLab or Jupyter Notebook](https://jupyter.org/) running and the following dependencies installed:
- [lonboard](https://developmentseed.org/lonboard/latest/#install)
- [overturemaps-py](https://github.com/OvertureMaps/overturemaps-py)
- [pandas](https://pandas.pydata.org/)
- [GeoPandas](https://geopandas.org/en/stable/index.html)
- [Shapely](https://shapely.readthedocs.io/en/stable/index.html)
To follow along, you'll need to have [JupyterLab or Jupyter Notebook](https://jupyter.org/) running and the following dependencies installed:

- [lonboard](https://developmentseed.org/lonboard/latest/#install)
- [overturemaps-py](https://github.com/OvertureMaps/overturemaps-py)
- [pandas](https://pandas.pydata.org/)
- [GeoPandas](https://geopandas.org/en/stable/index.html)
- [Shapely](https://shapely.readthedocs.io/en/stable/index.html)

``` python
```python
import pandas as pd
import geopandas as gpd
import overturemaps
import overturemaps
from shapely import wkb
from lonboard import Map, PolygonLayer
from lonboard.colormap import apply_categorical_cmap
```

``` python
```python
# specify bounding box
bbox = -78.6429, 39.463, -73.7806, 41.6242
```

``` python
```python
# read in Overture Maps land_cover data type
table = overturemaps.record_batch_reader("land_cover", bbox).read_all()
table = table.combine_chunks()
```

``` python
```python
# convert to dataframe
df = table.to_pandas()
```

``` python
```python
# filter for higher resolution land_cover features
df_h = df[df.cartography.apply(lambda x: x['min_zoom'] == 8)]
```

``` python
```python
# create color map for land_cover subtypes, loosely based on natural-color palette: https://www.shadedrelief.com/shelton/c.html
color_map = {
"urban": [167, 162, 186],
Expand All @@ -71,37 +74,37 @@ color_map = {
"shrub": [239, 218, 182],
"grass": [254, 239, 173],
"crop": [222, 223, 154],
"wetland": [158, 207, 195],
"wetland": [158, 207, 195],
"mangrove": [83, 171, 128],
"moss": [250, 230, 160],
"snow": [255, 255, 255],
"snow": [255, 255, 255],
}
```

``` python
```python
# apply color map to land_cover subtypes
colors = apply_categorical_cmap(df_h.subtype, color_map)
```

``` python
```python
# dataframe to geodataframe, set crs
gdf = gpd.GeoDataFrame(
df_h,
geometry=df_h['geometry'].apply(wkb.loads),
df_h,
geometry=df_h['geometry'].apply(wkb.loads),
crs="EPSG:4326"
)
```

``` python
# create map layer
```python
# create map layer
layer = PolygonLayer.from_geopandas(
gdf= gdf[['id','subtype', 'cartography', 'geometry']].reset_index(drop=True),
get_fill_color=colors,
get_line_color=colors,
)
```

``` python
```python
#render map
view_state = {
"longitude": -76.2,
Expand All @@ -113,6 +116,3 @@ view_state = {
m = Map(layer, view_state=view_state)
m
```



29 changes: 21 additions & 8 deletions blog/2024-07-24-explore-site.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -4,24 +4,37 @@ tags:
- developer-tools
---

import useBaseUrl from '@docusaurus/useBaseUrl';

Overture has launched a new browser-based tool to give users a no-code option to interact with our data. The Explore website is now the quickest way to get started with Overture, no SQL required: [https://explore.overturemaps.org](https://explore.overturemaps.org/#13.01/39.95692/-75.15583)

| ![Exploring Philadelphia](/img/blog/explore-philadelphia2.gif) |
|:--:|
| *Checking out ["Lion Crushing a Serpent"](https://www.associationforpublicart.org/artwork/lion-crushing-a-serpent/) in Philadelphia’s Rittenhouse Park* |
<figure>
<video autoPlay loop muted playsInline style={{ width: '100%' }}>
<source src={useBaseUrl('/img/blog/explore-philadelphia2.mp4')} type="video/mp4" />
</video>
<figcaption style={{ textAlign: 'center' }}>
<em>
Checking out{' '}
<a href="https://www.associationforpublicart.org/artwork/lion-crushing-a-serpent/">
"Lion Crushing a Serpent"
</a>{' '}
in Philadelphia's Rittenhouse Park
</em>
</figcaption>
</figure>

<!-- truncate -->

## Not a map
## Not a map

The Explore tool might look like a map, but we prefer to think of it as an x-ray data inspector. We’ve punted on making cartographic decisions in favor of displaying all of Overture’s themes, all at once: addresses, base, buildings, divisions, places, and transportation. Our users are coming at the data with different perspectives and goals, and we want to provide as much information as possible. We also want to stoke your curiosity and inspire you to ask new questions of Overture data.
The Explore tool might look like a map, but we prefer to think of it as an x-ray data inspector. We’ve punted on making cartographic decisions in favor of displaying all of Overture’s themes, all at once: addresses, base, buildings, divisions, places, and transportation. Our users are coming at the data with different perspectives and goals, and we want to provide as much information as possible. We also want to stoke your curiosity and inspire you to ask new questions of Overture data.

Seeing all the data all at once might be a bit overwhelming. We recommend zooming to an area of interest, hovering over the layers icon in the upper left corner of the map, and toggling the data themes off and on. You can click on an individual map feature, like the famous ["Lion Crushing a Serpent"](https://youtu.be/rTl2ewUc6bA?feature=shared) statue, to inspect its properties. Want to download the data? In the upper right corner of the Explore site, you can click on the Download Visible button to generate a GeoJSON file containing all the data visible in the browser.
Seeing all the data all at once might be a bit overwhelming. We recommend zooming to an area of interest, hovering over the layers icon in the upper left corner of the map, and toggling the data themes off and on. You can click on an individual map feature, like the famous ["Lion Crushing a Serpent"](https://youtu.be/rTl2ewUc6bA?feature=shared) statue, to inspect its properties. Want to download the data? In the upper right corner of the Explore site, you can click on the Download Visible button to generate a GeoJSON file containing all the data visible in the browser.

The [Explore site](https://explore.overturemaps.org/#13.01/39.95692/-75.15583) is currently in beta, and your feedback will inform the features we add in the coming months. If you have questions about the data, or if you want to report a problem with data quality or a glitch in the website, please click on the bug icon and [file an issue on GitHub](https://github.com/OvertureMaps/io-site/issues). (Note: we are actively working on fixing known issues with the downloaded data.)

## Build with us

This project grew out of the diverse experiences and skill sets of the engineers working on Overture. We’re using [WebAssembly](https://webassembly.org/) and Rust, specifically [geoarrow-rs](https://geoarrow.org/geoarrow-rs/), to query and download [Overture’s GeoParquet files](https://docs.overturemaps.org/getting-data/). The front end is React and [Maplibre](https://maplibre.org/), and we’re generating [PMTiles](https://docs.protomaps.com/pmtiles/) to visualize the data. It’s a serverless site, using all cloud-native data storage formats. We also plan to make [PMTiles available with each Overture release](https://github.com/OvertureMaps/overture-tiles). Special thanks to Kyle Barron from [Development Seed](https://developmentseed.org/) and Brandon Liu from [Protomaps](https://protomaps.com/) for collaborating with Overture's Developer Advocacy team on this project.
This project grew out of the diverse experiences and skill sets of the engineers working on Overture. We’re using [WebAssembly](https://webassembly.org/) and Rust, specifically [geoarrow-rs](https://geoarrow.org/geoarrow-rs/), to query and download [Overture’s GeoParquet files](https://docs.overturemaps.org/getting-data/). The front end is React and [Maplibre](https://maplibre.org/), and we’re generating [PMTiles](https://docs.protomaps.com/pmtiles/) to visualize the data. It’s a serverless site, using all cloud-native data storage formats. We also plan to make [PMTiles available with each Overture release](https://github.com/OvertureMaps/overture-tiles). Special thanks to Kyle Barron from [Development Seed](https://developmentseed.org/) and Brandon Liu from [Protomaps](https://protomaps.com/) for collaborating with Overture's Developer Advocacy team on this project.

[Explore is open source](https://github.com/OvertureMaps/io-site), under an MIT license, and we welcome contributions from the Overture community. Our goal is to give you the basic building blocks for creating your own tools around Overture data. Let us know what you make and we’ll happily share it on our [community projects page](https://docs.overturemaps.org/community/).
[Explore is open source](https://github.com/OvertureMaps/io-site), under an MIT license, and we welcome contributions from the Overture community. Our goal is to give you the basic building blocks for creating your own tools around Overture data. Let us know what you make and we’ll happily share it on our [community projects page](https://docs.overturemaps.org/community/).
Loading