Merge pull request #360 from gwmod/dev

Release v0.8

Author: dbrakenhoff

.github/workflows/ci.yml

@@ -17,13 +17,13 @@ jobs:
     runs-on: ubuntu-latest
     strategy:
       matrix:
-        python-version: [3.9]
+        python-version: [3.11]
 
     steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
 
       - name: Set up Python ${{ matrix.python-version }}
-        uses: actions/setup-python@v4
+        uses: actions/setup-python@v5
         with:
           python-version: ${{ matrix.python-version }}
 
@@ -32,25 +32,15 @@ jobs:
           python -m pip install --upgrade pip
           pip install -e .[ci]
 
-      - name: Lint with flake8
-        run: |
-          # stop the build if there are Python syntax errors or undefined names
-          flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
-          # exit-zero treats all errors as warnings.
-          flake8 . --count --exit-zero --max-complexity=10 --max-line-length=80 --statistics
-
       - name: Download executables needed for tests
         shell: bash -l {0}
         run: |
           python -c "import nlmod; nlmod.util.download_mfbinaries()"
 
-      - name: Run notebooks
-        if: ${{ github.event_name == 'push' }}
-        run: |
-          py.test ./tests -m "not notebooks"
-
       - name: Run tests only
-        if: ${{ github.event_name == 'pull_request' }}
+        env:
+            NHI_GWO_USERNAME: ${{ secrets.NHI_GWO_USERNAME}}
+            NHI_GWO_PASSWORD: ${{ secrets.NHI_GWO_PASSWORD}}
         run: |
           py.test ./tests -m "not notebooks"
 

.github/workflows/python-publish.yml

@@ -9,22 +9,24 @@ on:
 
 jobs:
   deploy:
-
     runs-on: ubuntu-latest
-
     steps:
-    - uses: actions/checkout@v3
+    - uses: actions/checkout@v4
+
     - name: Set up Python
-      uses: actions/setup-python@v4
+      uses: actions/setup-python@v5
       with:
-        python-version: '3.9'
+        python-version: '3.11'
+
     - name: Install dependencies
       run: |
         python -m pip install --upgrade pip
         pip install build setuptools wheel
+
     - name: build binary wheel and a source tarball
       run: |
         python -m build --sdist --wheel --outdir dist/
+
     - name: Publish a Python distribution to PyPI
       uses: pypa/gh-action-pypi-publish@release/v1
       with:

.prospector.yaml

@@ -29,6 +29,8 @@ pylint:
     - too-many-branches
     - too-many-statements
     - logging-fstring-interpolation
+    - import-outside-toplevel
+    - implicit-str-concat
 
 mccabe:
   disable:

README.md

@@ -18,9 +18,13 @@ groundwater models, makes models more reproducible and transparent.
 
 The functions in `nlmod` have four main objectives:
 
-1. Create and adapt the temporal and spatial discretization of a MODFLOW model using an xarray Dataset (`nlmod.dims`).
-2. Download and read data from external sources, project this data on the modelgrid and add this data to an xarray Dataset (`nlmod.read`).
-3. Use data in an xarray Dataset to build modflow packages for both groundwater flow and transport models using FloPy (`nlmod.sim`, `nlmod.gwf` and `nlmod.gwt` for Modflow 6 and `nlmod.modpath` for Modpath).
+1. Create and adapt the temporal and spatial discretization of a MODFLOW model using an 
+   xarray Dataset (`nlmod.dims`).
+2. Download and read data from external sources, project this data on the modelgrid and 
+   add this data to an xarray Dataset (`nlmod.read`).
+3. Use data in an xarray Dataset to build modflow packages for both groundwater flow 
+   and transport models using FloPy (`nlmod.sim`, `nlmod.gwf` and `nlmod.gwt` for 
+   Modflow 6 and `nlmod.modpath` for Modpath).
 4. Visualise modeldata in Python (`nlmod.plot`) or GIS software (`nlmod.gis`).
 
 More information can be found on the documentation-website:
@@ -50,9 +54,10 @@ Install the module with pip:
 * `dask`
 * `colorama`
 * `joblib`
+* `bottleneck`
 
 There are some optional dependecies, only needed (and imported) in a single method.
-Examples of this are `bottleneck` (used in calculate_gxg), `geocube` (used in
+Examples of this are `geocube` (used in
 add_min_ahn_to_gdf), `h5netcdf` (used for hdf5 files backend in xarray), `scikit-image`
 (used in calculate_sea_coverage). To install `nlmod` with the optional dependencies use:
 
@@ -65,11 +70,4 @@ notoriously hard to install on certain platforms. Please see the
 
 ## Getting started
 
-If you are using `nlmod` for the first time you need to download the MODFLOW
-executables. You can easily download these executables by running this Python code:
-
-    import nlmod
-	nlmod.download_mfbinaries()
-
-After you've downloaded the executables you can run the Jupyter Notebooks in the
-examples folder. These notebooks illustrate how to use the `nlmod` package.
+Start with the Jupyter Notebooks in the examples folder. These notebooks illustrate how to use the `nlmod` package.

docs/conf.py

@@ -10,10 +10,11 @@
 # add these directories to sys.path here. If the directory is relative to the
 # documentation root, use os.path.abspath to make it absolute, like shown here.
 #
-from nlmod import __version__
 import os
 import sys
 
+from nlmod import __version__
+
 sys.path.insert(0, os.path.abspath("."))
 
 

docs/examples/00_model_from_scratch.ipynb

@@ -20,28 +20,9 @@
    "outputs": [],
    "source": [
     "import flopy as fp\n",
-    "import matplotlib.pyplot as plt\n",
-    "import nlmod\n",
-    "import numpy as np\n",
-    "import pandas as pd"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "nlmod.util.get_color_logger(\"INFO\");"
-   ]
-  },
-  {
-   "attachments": {},
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Download MODFLOW-binaries\n",
-    "To run MODFLOW, we need to download the MODFLOW-excecutables. We do this with the following code:"
+    "import pandas as pd\n",
+    "\n",
+    "import nlmod"
    ]
   },
   {
@@ -50,8 +31,8 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "if not nlmod.util.check_presence_mfbinaries():\n",
-    "    nlmod.download_mfbinaries()"
+    "nlmod.util.get_color_logger(\"INFO\")\n",
+    "nlmod.show_versions()"
    ]
   },
   {

docs/examples/01_basic_model.ipynb

@@ -18,12 +18,6 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "import logging\n",
-    "import os\n",
-    "\n",
-    "import flopy\n",
-    "import geopandas as gpd\n",
-    "import matplotlib.pyplot as plt\n",
     "import nlmod"
    ]
   },
@@ -33,9 +27,8 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "print(f\"nlmod version: {nlmod.__version__}\")\n",
-    "\n",
-    "nlmod.util.get_color_logger(\"INFO\")"
+    "nlmod.util.get_color_logger(\"INFO\")\n",
+    "nlmod.show_versions()"
    ]
   },
   {
@@ -44,19 +37,20 @@
    "source": [
     "## Create model\n",
     "\n",
-    "With the code below we create a modflow model with the name 'IJmuiden'. This model has the following properties :\n",
+    "With the code below we create a modflow model with the name 'IJmuiden'. This model has the following properties:\n",
+    "\n",
     "- an extent that covers part of the Northsea, Noordzeekanaal and the small port city IJmuiden.\n",
-    "- a structured grid based on the subsurface models [Regis](https://www.dinoloket.nl/regis-ii-het-hydrogeologische-model) and [Geotop](https://www.dinoloket.nl/detaillering-van-de-bovenste-lagen-met-geotop). The Regis layers that are not present within the extent are removed. In this case we use 'MSz1' as the bottom layer of the model. Use `nlmod.read.regis.get_layer_names()` to get all the layer names of Regis. All Regis layers below this layer are not used in the model. Geotop is used to replace the holoceen layer in Regis because there is no kh or kv defined for the holoceen in Regis. Part of the model is in the North sea. Regis and Geotop have no data there. Therefore the Regis and Geotop layers are extrapolated from the shore and the seabed is added using bathymetry data from [Jarkus](https://www.openearth.nl/rws-bathymetry/2018.html).\n",
+    "- a structured grid based on the subsurface models [Regis](https://www.dinoloket.nl/regis-ii-het-hydrogeologische-model) and [Geotop](https://www.dinoloket.nl/detaillering-van-de-bovenste-lagen-met-geotop). The Regis layers that are not present within the extent are removed. In this case we use 'MSz1' as the bottom layer of the model. Use `nlmod.read.regis.get_layer_names()` to get all the layer names of Regis. All Regis layers below this layer are not used in the model. Geotop is used to replace the Holocene layer in Regis because there is no kh or kv defined for the Holocene in Regis. Part of the model is in the North sea. Regis and Geotop have no data there. Therefore the Regis and Geotop layers are extrapolated from the shore and the seabed is added using bathymetry data from [Jarkus](https://www.openearth.nl/rws-bathymetry/2018.html).\n",
     "- starting heads of 1 in every cell.\n",
-    "- the model is a steady state model of a single time step.\n",
+    "- the model is a steady state model with a single time step.\n",
     "- big surface water bodies (Northsea, IJsselmeer, Markermeer, Noordzeekanaal) within the extent are added as a general head boundary. The surface water bodies are obtained from a [shapefile](..\\data\\shapes\\opp_water.shp).\n",
-    "- surface drainage is added using [ahn](https://www.ahn.nl) data and a default conductance of $1000 m^2/d$\n",
-    "- recharge is added using data from the [knmi](https://www.knmi.nl/nederland-nu/klimatologie/daggegevens) using the following steps:~~\n",
-    "    1. Check for each cell which KNMI weather and/or rainfall station is closest.\n",
-    "    2. Download the data for the stations found in 1. for the model period. For a steady state stress period the average precipitation and evaporation of 8 years before the stress period time is used.\n",
-    "    3. Combine precipitation and evaporation data from step 2 to create a recharge time series for each cell\n",
-    "    4. Add the timeseries to the model dataset and create the recharge package.\n",
-    "- constant head boundaries are added to the model edges in every layer. The starting head is used as constant head."
+    "- surface drainage is added using the Dutch DEM ([ahn](https://www.ahn.nl)) and a default conductance of $1000 m^2/d$\n",
+    "- recharge is added using data from [knmi](https://www.knmi.nl/nederland-nu/klimatologie/daggegevens) using the following steps:\n",
+    "   1. Check for each cell which KNMI weather and/or rainfall station is closest.\n",
+    "   2. Download the data for the stations found in 1. for the model period. For a steady state stress period the average precipitation and evaporation of 8 years before the stress period time is used.\n",
+    "   3. Combine precipitation and evaporation data from step 2 to create a recharge time series for each cell,\n",
+    "   4. Add the timeseries to the model dataset and create the recharge package.\n",
+    "- constant head boundaries are added to the model edges in every layer. The starting head is used as the specified head."
    ]
   },
   {
@@ -215,6 +209,7 @@
    "source": [
     "## Write and Run\n",
     "Now that we've created all the modflow packages we need to write them to modflow files. You always have to write the modflow data to the model workspace before you can run the model. You can write the model files and run the model using the function `nlmod.sim.write_and_run)` as shown below. This function has two additional options:\n",
+    "\n",
     "1. Write the model dataset to the disk if `write_ds` is `True`. This makes it easier and faster to load model data if you ever need it. \n",
     "2. Write a copy of this Jupyter Notebook to the same directory as the modflow files if `nb_path` is the name of this Jupyter Notebook. It can be useful to have a copy of the script that created the modflow files, together with the files. "
    ]
@@ -250,7 +245,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Data from a model with a structured grid can be easily visualised using the model dataset. Below some examples"
+    "Data from a model with a structured grid can be easily visualised using the model dataset. Below are some examples:"
    ]
   },
   {

docs/examples/02_surface_water.ipynb

@@ -12,7 +12,7 @@
     "\n",
     "This example notebook shows some how to add surface water defined in a GeoDataFrame to a MODFLOW model using the `nlmod` package.\n",
     "\n",
-    "There are three water boards in the model area, of which we download seasonal data about the stage of the surface water. In this notebook we perform a steady-state run, in which the stage of the surface water is the mean of the summer and winter stage. For locations without a stage from the water board, we delineate information from a Digital Terrain Model, to set a stage. We assign a stage of 0.0 m NAP to the river Lek. to The surface water bodies in each cell are aggregated using an area-weighted method and added to the model as a river-package."
+    "There are three water boards in the model area, and we download seasonal data about the stage of the surface water for each. In this notebook we perform a steady-state run, in which the stage of the surface water is the mean of the summer and winter stage. For locations without a stage from the water board, we obtain information from a Digital Terrain Model near the surface water features, to estimate a stage. We assign a stage of 0.0 m NAP to the river Lek. The surface water bodies in each cell are aggregated using an area-weighted method and added to the model with the river-package."
    ]
   },
   {
@@ -25,12 +25,10 @@
     "import os\n",
     "\n",
     "import flopy\n",
-    "import rioxarray\n",
     "import matplotlib.pyplot as plt\n",
-    "import nlmod\n",
-    "from geocube.api.core import make_geocube\n",
-    "from functools import partial\n",
-    "from geocube.rasterize import rasterize_image"
+    "import rioxarray\n",
+    "\n",
+    "import nlmod"
    ]
   },
   {
@@ -40,9 +38,8 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "print(f\"nlmod version: {nlmod.__version__}\")\n",
-    "\n",
-    "nlmod.util.get_color_logger(\"INFO\")"
+    "nlmod.util.get_color_logger(\"INFO\")\n",
+    "nlmod.show_versions()"
    ]
   },
   {
@@ -94,7 +91,7 @@
     "if not os.path.isfile(fname_ahn):\n",
     "    ahn = nlmod.read.ahn.get_ahn4(extent, identifier=\"AHN4_DTM_5m\")\n",
     "    ahn.rio.to_raster(fname_ahn)\n",
-    "ahn = rioxarray.open_rasterio(fname_ahn, mask_and_scale=True)"
+    "ahn = rioxarray.open_rasterio(fname_ahn, mask_and_scale=True)[0]"
    ]
   },
   {
@@ -103,7 +100,7 @@
    "metadata": {},
    "source": [
     "### Layer 'waterdeel' from bgt\n",
-    "As the source of the location of the surface water bodies we use the 'waterdeel' layer of the Basisregistratie Grootschalige Topografie (BGT). This data consists of detailed polygons, maintained by dutch government agencies (water boards, municipalities and Rijkswatrstaat)."
+    "As the source of the location of the surface water bodies we use the 'waterdeel' layer of the Basisregistratie Grootschalige Topografie (BGT). This data consists of detailed polygons, maintained by dutch government agencies (water boards, municipalities and Rijkswaterstaat)."
    ]
   },
   {
@@ -223,7 +220,7 @@
    "metadata": {},
    "source": [
     "#### Save the data to use in other notebooks as well\n",
-    "We save the bgt-data to a GeoPackage file, so we can use the data in other notebooks with surface water as well"
+    "We save the bgt-data to a GeoPackage file, so we can use the data in other notebooks with surface water as well."
    ]
   },
   {
@@ -275,7 +272,13 @@
     "\n",
     "The `stage` and the `botm` columns are present in our dataset. The bottom resistance `c0` is rarely known, and is usually estimated when building the model. We will add our estimate later on.\n",
     "\n",
-    "*__Note__: the NaN's in the dataset indicate that not all parameters are known for each feature. This is not necessarily a problem but this will mean some features will not be converted to model input.*"
+    "<div class=\"alert alert-info\">\n",
+    "    \n",
+    "<b>Note:</b>\n",
+    "\n",
+    "The NaN's in the dataset indicate that not all parameters are known for each feature. This is not necessarily a problem but this will mean some features will not be converted to model input.\n",
+    "   \n",
+    "</div>"
    ]
   },
   {
@@ -493,7 +496,7 @@
     "xlim = ax.get_xlim()\n",
     "ylim = ax.get_ylim()\n",
     "gwf.modelgrid.plot(ax=ax)\n",
-    "ax.set_xlim(xlim[0], xlim[0] + ds.delr * 1.1)\n",
+    "ax.set_xlim(xlim[0], xlim[0] + nlmod.grid.get_delr(ds)[-1] * 1.1)\n",
     "ax.set_ylim(ylim)\n",
     "ax.set_title(f\"Surface water shapes in cell: {cid}\")"
    ]