Convenience and Speed with GRASS GIS 8.0 and 8.2

Vaclav (Vashek) Petras

NCSU GeoForAll Lab at the Center for Geospatial Analytics
NC State University

May 26, 2022

wenzeslaus.github.io/grass-gis-talks

Vaclav (Vashek) Petras

  • Geospatial Research Software Engineer at NC State
  • BS & MS in Geoinformatics, Czech Technical University in Prague, Czechia
  • PhD in Geospatial Analytics, NC State
  • GRASS GIS: Development Team, Project Steering Committee
  • OSGeo: Charter Member

Community

  • Open community of users and contributors
    • Affiliations: industry, academia, government
  • Contributions: code, documentation, free support, …
  • Project Steering Committee
    • PSC Chair: Verónica Andreo (since February 2021)

Credit goes to the whole community, not just this talk's author.

General Features

  • Open-source and directly user-driven development
  • Processing tools: 400+ in core, 400+ addons
  • Interfaces: graphical, command line, Python, C
  • 3rd party interfaces: actinia (REST API), R, QGIS, OGC WPS
  • Innovation with Stability & All-in-one Software Suite

Long-term Benefits

  • Individuals:
    • Emphasis on backwards compatibility.
      • Old commands, often work in new versions with minimal changes.
    • Employer-independent.
    • Learn once, use forever.
  • Organizations:
    • No usage license fees.
    • Provider-independent, contractor-independent.
    • Financial investments go to a public pool.

Roadmap

  • 7.8.7, Feb 23, 2022 Stability and fixes.
  • 8.0.0, Feb 21, 2022 New startup, data management.
  • 8.0.2, May 12, 2022 Stability and fixes.
  • 8.2.0 Release Candidate 2, May 24, 2022 Available for testing.
  • 8.2.0, June 2022 Jupyter, parallel r.neighbors, …

major.minor.micro – micro (x.y.z) brings fixes, minor (x.y) brings features, major (x) brings features and possibly backward incompatible changes


Questions?

Convenience Improvements

First-time User Experience

Initial project sets up automatically and suggestions for next steps are provided.

by Linda Kladivova & rest of the community (many reviews, video calls, user surveys, …)

No Startup Screen

Centralized Data Management

All data manipulation centered around revamped Data tab.

by Linda Kladivova, Anna Petrasova, Vaclav Petras

Single-Window GUI

One window with optimized layout.

Opt-in preview in 8.2: Settings > Preferences > General

by Linda Kladivova

Integration with Jupyter Notebooks

  • Notebooks: Mix of text, code, results, images, …
  • Usage: Prototyping, reproducibility, tutorials, …
  • Now in GRASS GIS: Python functions for easy integration

by Caitlin Haedrich

Integration with Jupyter Notebooks

Web map with base maps, zooming, customizations, …

tinyurl.com/grass-rc2

Integration with Jupyter Notebooks

Time-series visualization

tinyurl.com/grass-rc2

Integration with Jupyter Notebooks

3D visualization

tinyurl.com/grass-rc2

Semantic Labels for Rasters

  • attach a labels to raster maps (e.g., S2_1 or red)
    • image classification can use sematic labels to identify bands
    • different scenes can then be classified if semantic labels are the same
  • organize bands within one spatio-temporal dataset

by Maris Nartiss, Martin Landa, Markus Metz

Docker Containers


  • Different flavours
  • Built and updated automatically
  • For all branches and tags

actinia

by Markus Neteler, Carmen Tawalika, Anika Weinmann, …

Questions?

Speed Improvements

Multi-threading with OpenMP

  • More tasks done in parallel in the same tool
  • nprocs specifies number of cores (processes, threads)
  • memory specifies RAM in MB
  • Performance section in documentation describes the behavior

r.series (OpenMP parallel)

Cell value is a function of the corresponding cells in the inputs.

by Aaron Saw Min Sern in 8.2

r.neighbors (OpenMP parallel)

Cell value is a function of the cells around it.

by Aaron Saw Min Sern in 8.2

r.mfilter (OpenMP parallel)

Cell value is a function of the cells around it.

by Aaron Saw Min Sern in 8.2

r.slope.aspect (OpenMP parallel)

Slope, aspect, and other derivatives of an elevation raster.

by Aaron Saw Min Sern in 8.2

r.patch (OpenMP parallel)

Mosaic multiple rasters into one.

by Aaron Saw Min Sern in 8.2

More parallel tools

  • Core tools: r.sun, v.surf.rst, r.sim.sediment, r.sim.water
  • Addons tools: r.sun.daily, r.in.usgs, r.mapcalc.tiled, t.rast.what.aggr, r.connectivity.corridors, r.viewshed.exposure, and 14 more
  • Parallelizing custom Python scripts:
    • GridModule (by tile)
    • multiprocessing, ParallelModuleQueue (by task)

Faster External Data Links

  • r.external can link (open) external raster data faster.
    • 2-5× faster computation of min & max
    • almost no time if min & max in metadata
    • almost no time if min & max are not needed later
  • Great for workflows when only portion of the data is processed in GRASS GIS.

by Markus Metz

Improved HPC Readiness

by Vaclav Petras, Markus Neteler, Nicklas Larsson, …

Questions?

More Features

Science and Code

  • What algorithm
    • references to related scientific papers
    • references to papers associated with tools or libraries
  • What exact code
    • source code, its history, and latest change
    • DOI record with a copy of the code for each release

    • → transparency, reproducibility, provenance

Lidar Data Import with PDAL

  • Binning & filtering using any point dimension (including user defined ones, e.g., from PDAL filters)
  • Support for 19 binning methods
  • Linux and macOS only. Help needed for Windows!

by Vaclav Petras and Maris Nartiss

Skeletons and Centerlines

v.voronoi tool can now create area skeletons and centerlines.

by Markus Metz

GitHub-centered Software Development

  • Git
  • PRs
  • Issues
  • GitHub Actions

Improved and Robust Code Quality Checks

  • Python code formatted by Black.
  • Python code checked against Flake8, some even Pylint.
  • Most of warnings in C code removed and now checked.
  • Code tested with CodeQL.

by Nicklas Larsson, Vaclav Petras, Anna Petrasova, Carmen Tawalika, ...

Addons Repository

  • Community-maintained tools (addons aka extensions, plugins)
  • Separate from the main repository, but only one repository
  • A repository with the source code, not just a registry
  • Best of both worlds:
    • Broader community of contributors, including one-time contributors
    • Single repository maintained by the core community

Questions?

FUTURES

r.futures - set of tools for urban growth modeling

by NC State Center for Geospatial Analytics

PoPS

r.pops.spread - Pest or Pathogen Spread simulation

by NC State Center for Geospatial Analytics

Visual Exposure to a Defined Source

r.viewshed.exposure - Weighted cumulative viewshed analysis defining visual exposure to a source

by Zofie Cimburova and Stefan Blumentrath

Weighted Layers for Dasymetric Mapping

r.area.createweight - Weighting layer for dasymetric mapping using a random forest regression model.

by Charlotte Flasse, Tais Grippa, Safa Fennia

Hydrologic Parameters Using a Flow Direction Raster

r.accumulate - Weighted flow accumulation, subwatersheds, stream networks, and longest flow paths using a flow direction

Cho (2020): A recursive algorithm for calculating the longest flow path and its iterative implementation (faster and better than competition)
See also: Physically-based hydrologic modeling using GRASS GIS r.topmodel

by Huidae Cho

Valley Bottom Flatness Index

r.valley.bottom - Multi-resolution Valley Bottom Flatness (MrVBF)

by Helmut Kudrnovsky, Steven Pawley

Sentinel, THREDDS, …, netCDF, CSV

Automated download and import of common datasets i.sentinel, i.modis, i.landsat, r.in.usgs, r.in.nasadem, m.crawl.thredds, t.rast.import.netcdf, …

by Fondazione Edmund Mach, OpenGeoLabs, mundialis GSoC, NC State University, CONICET, Norwegian Institute for Nature Research, …

Resources

Support


GRASS GIS logo

Sponsoring

  • Individuals:
  • Organizations:
    • Time: employee time
    • Money: pay developers (companies) to add features or fix bugs
GRASS GIS logo

grass.osgeo.org

vpetras@ncsu.edu
Twitter: vaclavpetras
LinkedIn: Vaclav Petras

wenzeslaus.github.io/grass-gis-talks