Flexible tool to build planet-scale vector tilesets from OpenStreetMap data in a few hours
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
dependabot[bot] ad567b192a
Bump jts-core from 1.18.2 to 1.19.0 (#282)
5 days ago
.github Also run java 18 in CI (#249) 4 weeks ago
.idea Migrate to eclipse formatter to support multiple IDEs (#122) 4 months ago
.mvn Migrate to eclipse formatter to support multiple IDEs (#122) 4 months ago
.vscode Migrate to eclipse formatter to support multiple IDEs (#122) 4 months ago
diagrams CI and docs 8 months ago
planet-logs Gather v0.5.0 benchmarks (#256) 3 weeks ago
planetiler-basemap Declarative schema from configuration file (#160) 3 weeks ago
planetiler-benchmarks 0.5-snapshot 3 weeks ago
planetiler-core Bump jts-core from 1.18.2 to 1.19.0 (#282) 5 days ago
planetiler-custommap Decrease power sample minzoom from 12 to 7 (#268) 3 weeks ago
planetiler-dist Declarative schema from configuration file (#160) 3 weeks ago
planetiler-examples Bump maven-surefire-plugin in /.github/planetiler-examples-dependabot (#270) 3 weeks ago
scripts Upgrade basemap profile to OpenMapTiles v3.13.1 (#227) 1 month ago
.editorconfig initial commit 1 year ago
.gitignore Migrate to eclipse formatter to support multiple IDEs (#122) 4 months ago
ARCHITECTURE.md Change 'render' terminology (#238) 4 weeks ago
CONTRIBUTING.md Small readme cleanups (#265) 3 weeks ago
LICENSE apache 2.0 license 9 months ago
NOTICE.md Use custom min-heap to improve k-way merge by 30-50% (#217) 2 months ago
PLANET.md 0.4.0 tweaks (#153) 3 months ago
README.md Add instructions for using Planetiler with gradle (#274) 1 week ago
config-example.properties Warn when running with insufficient memory or disk space (#73) 4 months ago
eclipse-formatter.xml Migrate to eclipse formatter to support multiple IDEs (#122) 4 months ago
grafana.json Change name to Planetiler (#40) 6 months ago
mvnw CI and docs 8 months ago
mvnw.cmd CI and docs 8 months ago
pom.xml Bump maven-enforcer-plugin from 3.0.0 to 3.1.0 (#281) 6 days ago
quickstart-from-source.sh download from quickstart (#93) 4 months ago
sonar-project.properties Parallel temp feature writes using `--write-threads` argument (#213) 2 months ago

README.md

Planetiler

Planetiler (pla·nuh·tai·lr, formerly named "Flatmap") is a tool that generates Vector Tiles from geographic data sources like OpenStreetMap. Planetiler aims to be fast and memory-efficient so that you can build a map of the world in a few hours on a single machine without any external tools or database.

Vector tiles contain raw point, line, and polygon geometries that clients like MapLibre can use to render custom maps in the browser, native apps, or on a server. Planetiler packages tiles into an MBTiles (sqlite) file that can be served using tools like TileServer GL or even queried directly from the browser. See awesome-vector-tiles for more projects that work with data in this format.

Planetiler works by mapping input elements to vector tile features, flattening them into a big list, then sorting by tile ID to group into tiles. See ARCHITECTURE.md for more details or this blog post for more of the backstory.

Demo

See the live demo of vector tiles created by Planetiler and hosted by the OpenStreetMap Americana Project.

Planetiler Demo Screenshot Style © OpenMapTiles · Data © OpenStreetMap contributors

Usage

To generate a map of an area using the basemap profile, you will need:

  • Java 16+ (see CONTRIBUTING.md) or Docker
  • at least 1GB of free disk space plus 5-10x the size of the .osm.pbf file
  • at least 0.5x as much free RAM as the input .osm.pbf file size

To build the map:

Using Java, download planetiler.jar from the latest release and run it:

wget https://github.com/onthegomap/planetiler/releases/latest/download/planetiler.jar
java -Xmx1g -jar planetiler.jar --download --area=monaco

Or using Docker:

docker run -e JAVA_TOOL_OPTIONS="-Xmx1g" -v "$(pwd)/data":/data ghcr.io/onthegomap/planetiler:latest --download --area=monaco

This starts off by downloading about 1GB of data sources required by the basemap profile including ~750MB for ocean polygons and ~240MB for Natural Earth Data.

To download smaller extracts just for Monaco:

Java:

java -Xmx1g -jar planetiler.jar --download --area=monaco \
  --water-polygons-url=https://github.com/onthegomap/planetiler/raw/main/planetiler-core/src/test/resources/water-polygons-split-3857.zip \
  --natural-earth-url=https://github.com/onthegomap/planetiler/raw/main/planetiler-core/src/test/resources/natural_earth_vector.sqlite.zip

Docker:

docker run -e JAVA_TOOL_OPTIONS="-Xmx1g" -v "$(pwd)/data":/data ghcr.io/onthegomap/planetiler:latest --download --area=monaco \
  --water-polygons-url=https://github.com/onthegomap/planetiler/raw/main/planetiler-core/src/test/resources/water-polygons-split-3857.zip \
  --natural-earth-url=https://github.com/onthegomap/planetiler/raw/main/planetiler-core/src/test/resources/natural_earth_vector.sqlite.zip

You will need the full data sources to run anywhere besides Monaco.

To view tiles locally:

Using Node.js:

npm install -g tileserver-gl-light
tileserver-gl-light --mbtiles data/output.mbtiles

Or using Docker:

docker run --rm -it -v "$(pwd)/data":/data -p 8080:8080 maptiler/tileserver-gl -p 8080

Then open http://localhost:8080 to view tiles.

Some common arguments:

  • --download downloads input sources automatically and --only-download exits after downloading
  • --area=monaco downloads a .osm.pbf extract from Geofabrik
  • --osm-path=path/to/file.osm.pbf points Planetiler at an existing OSM extract on disk
  • -Xmx1g controls how much RAM to give the JVM (recommended: 0.5x the input .osm.pbf file size to leave room for memory-mapped files)
  • --force overwrites the output file
  • --help shows all of the options and exits

Generating a Map of the World

See PLANET.md.

Examples

See the planetiler-examples project.

Benchmarks

Some example runtimes for the Basemap OpenMapTiles-compatible profile (excluding downloading resources):

Input Version Machine Time mbtiles size Logs
s3://osm-pds/2022/planet-220530.osm.pbf (69GB) 0.5.0 c2d-standard-112 (112cpu/448GB) 37m cpu:48h5m gc:3m45s avg:76.9 79GB logs
s3://osm-pds/2022/planet-220530.osm.pbf (69GB) 0.5.0 c6gd.16xlarge (64cpu/128GB) 53m cpu:41h58m avg:47.1 79GB logs, VisualVM Profile
s3://osm-pds/2022/planet-220530.osm.pbf (69GB) 0.5.0 c6gd.8xlarge (32cpu/64GB) 1h27m cpu:37h55m avg:26.1 79GB logs
s3://osm-pds/2022/planet-220530.osm.pbf (69GB) 0.5.0 c6gd.4xlarge (16cpu/32GB) 2h38m cpu:34h3m avg:12.9 79GB logs
s3://osm-pds/2021/planet-211011.osm.pbf (65GB) 0.1.0 DO 16cpu 128GB 3h9m cpu:42h1m avg:13.3 99GB logs, VisualVM Profile
Daylight Distribution v1.6 with ML buildings and admin boundaries (67GB) 0.1.0 DO 16cpu 128GB 3h13m cpu:43h40m avg:13.5 101GB logs

Merging nearby buildings at z13 is very expensive, when run with --building-merge-z13=false:

Input Version Machine Time mbtiles size Logs
s3://osm-pds/2022/planet-220530.osm.pbf (69GB) 0.5.0 c2d-standard-112 (112cpu/448GB) 26m cpu:27h47m avg:63.9 79GB logs
s3://osm-pds/2022/planet-220530.osm.pbf (69GB) 0.5.0 c6gd.16xlarge (64cpu/128GB) 39m cpu:27h4m avg:42.1 79GB logs, VisualVM Profile
s3://osm-pds/2021/planet-220214.osm.pbf (67GB) 0.3.0 r6g.16xlarge (64cpu/512GB) with ramdisk and write to EFS 1h1m cpu:24h33m avg:24.3 104GB logs
s3://osm-pds/2021/planet-211011.osm.pbf (65GB) 0.1.0 Linode 50cpu 128GB 1h9m cpu:24h36m avg:21.2 97GB logs, VisualVM Profile

Alternatives

Some other tools that generate vector tiles from OpenStreetMap data:

  • OpenMapTiles is the reference implementation of the OpenMapTiles schema that the basemap profile is based on. It uses an intermediate postgres database and operates in two modes:
    1. Import data into database (~1 day) then serve vector tiles directly from the database. Tile serving is slower and requires bigger machines, but lets you easily incorporate realtime updates
    2. Import data into database (~1 day) then pregenerate every tile for the planet into an mbtiles file which takes over 100 days or a cluster of machines, but then tiles can be served faster on smaller machines
  • Tilemaker uses a similar approach to Planetiler (no intermediate database), is more mature, and has a convenient lua API for building custom profiles without recompiling the tool, but takes about a day to generate a map of the world

Some companies that generate and host tiles for you:

If you want to host tiles yourself but have someone else generate them for you, those companies also offer plans to download regularly-updated tilesets.

Features

  • Supports Natural Earth, OpenStreetMap .osm.pbf, and Esri Shapefiles data sources
  • Java-based Profile API to customize how source elements map to vector tile features, and post-process generated tiles using JTS geometry utilities
  • Merge nearby lines or polygons with the same tags before emitting vector tiles
  • Automatically fixes self-intersecting polygons
  • Built-in basemap profile based on OpenMapTiles v3.13.1
  • Optionally download additional name translations for elements from Wikidata
  • Export real-time stats to a prometheus push gateway using --pushgateway=http://user:[email protected] argument (and a grafana dashboard for viewing)
  • Automatically downloads region extracts from Geofabrik using geofabrik:australia shortcut as a source URL
  • Unit-test profiles to verify mapping logic, or integration-test to verify the actual contents of a generated mbtiles file (example)

Limitations

  • It is harder to join and group data than when using database. Planetiler automatically groups features into tiles, so you can easily post-process nearby features in the same tile before emitting, but if you want to group or join across features in different tiles, then you must explicitly store data when processing a feature to use with later features or store features and defer processing until an input source is finished (boundary layer example)
  • Planetiler only does full imports from .osm.pbf snapshots, there is no way to incorporate real-time updates.

Use as a library

Planetiler can be used as a maven-style dependency in a Java project using the settings below:

Maven

Add this dependency to your java project:

<dependency>
  <groupId>com.onthegomap.planetiler</groupId>
  <artifactId>planetiler-core</artifactId>
  <version>0.5.0</version>
</dependency>

Gradle

Set up your repositories block as follows:

mavenCentral()
maven {
    url "https://repo.osgeo.org/repository/release/"
}

Set up your dependencies block as follows:

implementation 'com.onthegomap.planetiler:planetiler-core:<version>'

Contributing

Pull requests are welcome! See CONTRIBUTING.md for details.

Support

For general questions, check out the #planetiler channel on OSM-US Slack (get an invite here), or start a GitHub discussion.

Found a bug or have a feature request? Open a GitHub issue to report.

This is a side project, so support is limited. If you have the time and ability, feel free to open a pull request to fix issues or implement new features.

Acknowledgement

Planetiler is made possible by these awesome open source projects:

See NOTICE.md for a full list and license details.

Author

Planetiler was created by Michael Barry for future use generating custom basemaps or overlays for On The Go Map.

License and Attribution

Planetiler source code is licensed under the Apache 2.0 License, so it can be used and modified in commercial or other open source projects according to the license guidelines.

Maps built using planetiler do not require any special attribution, but the data or schema used might. Any maps generated from OpenStreetMap data must visibly credit OpenStreetMap contributors. Any map generated with the profile based on OpenMapTiles or a derivative must visibly credit OpenMapTiles as well.