Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1,633 changes: 1,633 additions & 0 deletions CHANGELOG.md

Large diffs are not rendered by default.

951 changes: 0 additions & 951 deletions CHANGELOG.rst

This file was deleted.

37 changes: 25 additions & 12 deletions doc/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,36 +19,27 @@
# This broke the landing page `index.rst` as it embeds `index.ipynb` file.
# Adding `.md` to make our life easier in the future.
source_suffix = ['.rst', '.md', '.ipynb']
master_doc = 'index'

html_logo = '_static/logo_horizontal.svg'
html_favicon = '_static/favicon.ico'
html_show_sourcelink = False

html_theme_options.update({
'github_url': 'https://github.com/holoviz/datashader',
'icon_links': [
{
'name': 'Twitter',
'url': 'https://twitter.com/datashader',
'icon': 'fa-brands fa-twitter-square',
},
{
'name': 'Discourse',
'url': 'https://discourse.holoviz.org/c/datashader/',
'icon': 'fa-brands fa-discourse',
},
{
"name": "HoloViz",
"url": "https://holoviz.org/",
"icon": "_static/holoviz-icon-white.svg",
"type": "local",
},
{
"name": "Discord",
"url": "https://discord.gg/AXRHnJU6sP",
"icon": "fa-brands fa-discord",
},
],
"pygment_dark_style": "material"
"pygments_dark_style": "material"
})

html_context.update({
Expand All @@ -67,17 +58,29 @@
]

myst_fence_as_directive = ["mermaid"]
myst_enable_extensions = [
# To also use ::: delimiters to denote directives, instead of ```.
'colon_fence',
# MySt-Parser will attempt to convert any isolated img tags (i.e. not
# wrapped in any other HTML) to the internal representation used in sphinx.
'html_image',
'deflist',
]

nbsite_analytics = {
'goatcounter_holoviz': True,
}

nbbuild_cell_timeout = 2000

nb_execution_mode = "auto"

redirects = {
'topics/index': 'https://examples.holoviz.org',
}

autosummary_generate = True

# Datashader uses sphinx.ext.autodoc (e.g. automodule) for its API reference
# and automatically include a module that contains Image. Image inherits
# from xr.DataArray. Datashader uses numpydoc to parse the docstrings.
Expand All @@ -101,3 +104,13 @@ def patch_error_location(self, msg, error=True):

# Override the Sphinx default title that appends `documentation`
html_title = f'{project} v{version}'

# /Users/runner/work/datashader/datashader/datashader/core.py:docstring of datashader.core.Canvas:21:
# WARNING: autosummary: stub file not found 'datashader.Canvas.area'. Check your autosummary_generate setting.
# See https://stackoverflow.com/a/73294408
numpydoc_class_members_toctree = False

numpydoc_show_inherited_class_members = False
numpydoc_class_members_toctree = False

exclude_patterns = ['governance']
3 changes: 2 additions & 1 deletion doc/releases.rst
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
Releases
========

.. include:: ../CHANGELOG.rst
.. include:: ../CHANGELOG.md
:parser: myst_parser.sphinx_
2 changes: 1 addition & 1 deletion doc/user_guide/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ Contents:
Using reduction to inspect rather than aggregate data.

`13. GeoPandas <Geopandas.html>`_
GeoPandas support in Datashader.
GeoPandas support in Datashader.

.. toctree::
:hidden:
Expand Down
2 changes: 1 addition & 1 deletion examples/FAQ.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@
"source": [
"### What data libraries can I use with Datashader?\n",
"\n",
"See the [Performance user guide](user_guide/Performance.ipynb#Data-objects) for the available options for working with columnar/multidimensional/ragged data on single-core/multi-core/distributed/CPU/GPU resources in or out of core for each available glyph."
"See the [Performance user guide](user_guide/10_Performance.ipynb#data-objects) for the available options for working with columnar/multidimensional/ragged data on single-core/multi-core/distributed/CPU/GPU resources in or out of core for each available glyph."
]
}
],
Expand Down
14 changes: 12 additions & 2 deletions examples/getting_started/1_Introduction.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
"Datashader is an [open-source](https://github.com/holoviz/datashader/) Python library for analyzing and visualizing large datasets. Specifically, Datashader is designed to \"rasterize\" or \"aggregate\" datasets into regular grids that can be analyzed further or viewed as images, making it simple and quick to see the properties and patterns of your data. Datashader can plot a billion points in a second or so on a 16GB laptop, and scales up easily to out-of-core, distributed, or GPU processing for even larger datasets.\n",
"\n",
"This page of the getting-started guide will give a simple example to show how it works, and the following page will show how to use Datashader as a standalone library for generating arrays or images directly\n",
"([Pipeline](2_Pipeline.ipynb)). Next we'll show how to use Datashader as a component in a larger visualization system like [HoloViews](https://holoviews.org) or [Bokeh](https://bokeh.org) that provides interactive plots with dynamic zooming, labeled axes, and overlays and layouts ([3-Interactivity](3-Interactivity.ipynb)). More detailed information about each topic is then provided in the [User Guide](../user_guide/).\n",
"([Pipeline](2_Pipeline.ipynb)). Next we'll show how to use Datashader as a component in a larger visualization system like [HoloViews](https://holoviews.org) or [Bokeh](https://bokeh.org) that provides interactive plots with dynamic zooming, labeled axes, and overlays and layouts ([Interactivity](3_Interactivity.ipynb)). More detailed information about each topic is then provided in the [User Guide](../../doc/user_guide/index.rst).\n",
"\n",
"## Example: NYC taxi trips\n",
"\n",
Expand All @@ -30,6 +30,16 @@
"df.head()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
":::{note}\n",
"The file `nyc_taxi.csv` used above is located at\n",
"[nyc_taxi.csv](https://github.com/holoviz/datashader/blob/main/examples/data/.data_stubs/nyc_taxi.csv) in the Datashader repository. When running this example, make sure the file is available locally and update the path accordingly.\n",
":::\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
Expand Down Expand Up @@ -80,7 +90,7 @@
"source": [
"You can select the \"Wheel Zoom\" tool on the right and then do panning and zooming (with the scroll bar). As long as you have a network connection, the maps will update as you zoom, but the datashaded image will only update if you have a live Python process running. If you do have Python \"live\", each time you zoom in, the data will be re-aggregated at the new zoom level, converted to an image, and displayed embedded on the map data, making it simple to explore and understand the data.\n",
"\n",
"At the most basic level, Datashader can accept scatterplot points (as above), line segments (for time series, and trajectories), areas (for filled-area plots), polygons (for choropleths), or gridded data (rasters, quadmeshes, and trimeshes to be regridded), and can turn each of these into a regularly sampled array or the corresponding pixel-based image. The rest of this getting-started guide shows how to go from your data to either images or interactive plots, as simply as possible. The next [getting-started section](2_Pipeline.ipynb) breaks down each of the steps taken by Datashader, using a synthetic dataset so that you can see precisely how the data relates to the images. The [user guide](../user_guide/) then explains each of the steps in much more detail."
"At the most basic level, Datashader can accept scatterplot points (as above), line segments (for time series, and trajectories), areas (for filled-area plots), polygons (for choropleths), or gridded data (rasters, quadmeshes, and trimeshes to be regridded), and can turn each of these into a regularly sampled array or the corresponding pixel-based image. The rest of this getting-started guide shows how to go from your data to either images or interactive plots, as simply as possible. The next [getting-started section](2_Pipeline.ipynb) breaks down each of the steps taken by Datashader, using a synthetic dataset so that you can see precisely how the data relates to the images. The [user guide](../../doc/user_guide/index.rst) then explains each of the steps in much more detail."
]
}
],
Expand Down
4 changes: 2 additions & 2 deletions examples/getting_started/2_Pipeline.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Datashader can work many different data objects provided by different data libraries depending on the type of data involved, such as columnar data in [Pandas](https://pandas.pydata.org) or [Dask](https://dask.org) dataframes, gridded multidimensional array data using [xarray](https://xarray.dev), columnar data on GPUs using [cuDF](https://github.com/rapidsai/cudf), multidimensional arrays on GPUs using [CuPy](https://cupy.chainer.org/), and ragged arrays using [SpatialPandas](https://github.com/holoviz/spatialpandas) (see the [Performance User Guide](../10_Performance.ipynb) for a guide to selecting an appropriate library). Here, we're using a Pandas dataframe, with 50,000 rows by default:"
"Datashader can work many different data objects provided by different data libraries depending on the type of data involved, such as columnar data in [Pandas](https://pandas.pydata.org) or [Dask](https://dask.org) dataframes, gridded multidimensional array data using [xarray](https://xarray.dev), columnar data on GPUs using [cuDF](https://github.com/rapidsai/cudf), multidimensional arrays on GPUs using [CuPy](https://cupy.chainer.org/), and ragged arrays using [SpatialPandas](https://github.com/holoviz/spatialpandas) (see the [Performance User Guide](../user_guide/10_Performance.ipynb) for a guide to selecting an appropriate library). Here, we're using a Pandas dataframe, with 50,000 rows by default:"
]
},
{
Expand Down Expand Up @@ -166,7 +166,7 @@
" - **Canvas.raster**: the collection of data items is an array specifying regularly spaced axis-aligned rectangles forming a regular grid; each cell in this array is rendered as a filled rectangle.\n",
" - **Canvas.quadmesh**: the collection of data items is an array specifying irregularly spaced quadrilaterals forming a grid that is regular in the input space but can have arbitrary rectilinear or curvilinear shapes in the aggregate grid; each cell in this array is rendered as a filled quadrilateral.\n",
"\n",
"These types are each covered in detail in the [User Guide](../user_guide/). Many other plots like time series and network graphs can be constructed out of these basic primitives. Datashader can also be extended to add additional types here and in each section below; see [Extending Datashader](../user_guide/9-Extending.ipynb) for more details. \n",
"These types are each covered in detail in the [User Guide](../../doc/user_guide/index.rst). Many other plots like time series and network graphs can be constructed out of these basic primitives. Datashader can also be extended to add additional types here and in each section below; see [Extending Datashader](../user_guide/9_Extending.ipynb) for more details. \n",
"\n",
"<!-- (to here) -->"
]
Expand Down
2 changes: 1 addition & 1 deletion examples/getting_started/3_Interactivity.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"As you can see, the ``points`` object visualizes itself as a Bokeh plot, where you can already see many of the [problems that motivate datashader](https://anaconda.org/jbednar/plotting_pitfalls) (overplotting of points, being unable to detect the closely spaced dense collections of points shown in red above, and so on). But this visualization is just the default representation of ``points``, using Jupyter's [rich display](https://anaconda.org/jbednar/rich_display) support; the actual ``points`` object itself is merely a data container:"
"As you can see, the ``points`` object visualizes itself as a Bokeh plot, where you can already see many of the [problems that motivate datashader](../user_guide/1_Plotting_Pitfalls.ipynb) (overplotting of points, being unable to detect the closely spaced dense collections of points shown above, and so on). But this visualization is just the default representation of ``points``; the actual ``points`` object itself is merely a data container:"
]
},
{
Expand Down
10 changes: 4 additions & 6 deletions examples/index.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,6 @@
"\n",
"<a href=\"https://examples.pyviz.org/attractors/attractors.html\"><img src=\"assets/images/sym_attractors.jpg\" alt=\"image\" width=\"1000\" /></a>\n",
"\n",
"See the [topics page](topics) for these and many other examples.\n",
"\n",
"\n",
"\n",
"## Installation\n",
Expand All @@ -60,16 +58,16 @@
"The source code for datashader is maintained on [Github](https://github.com/holoviz/datashader), and\n",
"is documented using the API link on this page.\n",
"\n",
"We recommend the [Getting Started Guide](getting_started) to learn\n",
"We recommend the [Getting Started Guide](../doc/getting_started/index.rst) to learn\n",
"the basic concepts and start using Datashader as quickly as possible.\n",
"\n",
"The [User Guide](user_guide) covers specific topics in more detail.\n",
"The [User Guide](../doc/user_guide/index.rst) covers specific topics in more detail.\n",
"\n",
"The [API](api.html) is the definitive guide to each part of\n",
"The [API](../doc/api.rst) is the definitive guide to each part of\n",
"Datashader, but the same information is available more conveniently via\n",
"the `help()` command as needed when using each component.\n",
"\n",
"Please feel free to report [issues](https://github.com/holoviz/datashader/issues) or [contribute code](https://help.github.com/articles/about-pull-requests). You are also welcome to chat with the developers on [gitter](https://gitter.im/pyviz/pyviz), but please use the official channels for reporting issues or making feature requests so that they are captured appropriately.\n"
"Please feel free to report [issues](https://github.com/holoviz/datashader/issues) or [contribute code](https://help.github.com/articles/about-pull-requests)."
]
}
],
Expand Down
10 changes: 10 additions & 0 deletions examples/tiling.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,16 @@
"## Building Tilesets using Datashader"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
":::{warning}\n",
"This notebook is a work in progress, and the tiling functionality may not be fully implemented yet. \n",
"If you'd like to contribute or report missing features, feel free to [open an issue](https://github.com/holoviz/datashader/issues) in the Datashader repository.\n",
":::"
]
},
{
"cell_type": "markdown",
"metadata": {},
Expand Down
2 changes: 1 addition & 1 deletion examples/user_guide/11_Geography.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
"* [Remote Sensing](https://github.com/makepath/xarray-spatial/blob/master/examples/user_guide/6_Remote_Sensing.ipynb)\n",
"* [Pathfinding](https://github.com/makepath/xarray-spatial/blob/master/examples/user_guide/7_Pathfinding.ipynb)\n",
"\n",
"These functionality is provided in the [xarray-spatial](github.com/makepath/xarray-spatial) library.\n",
"These functionality is provided in the [xarray-spatial](https://github.com/makepath/xarray-spatial) library.\n",
"You can check out its [example notebooks](https://github.com/makepath/xarray-spatial/tree/master/examples/user_guide) to see how to use the functions.\n",
"\n",
"See also [GeoViews](https://geoviews.org), which is designed to work with Datashader to provide a large range of additional geospatial functionality."
Expand Down
4 changes: 2 additions & 2 deletions examples/user_guide/5_Grids.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
"\n",
"In some cases, your data is *already* rasterized, such as data from imaging experiments, simulations on a regular grid, or other regularly sampled processes. Even so, the rasters you have already are not always the ones you need for a given purpose, having the wrong shape, range, or size to be suitable for overlaying with or comparing against other data, maps, and so on. Datashader provides fast methods for [\"regridding\"](https://climatedataguide.ucar.edu/climate-data-tools-and-analysis/regridding-overview)/[\"re-sampling\"](https://gisgeography.com/raster-resampling/)/\"re-rasterizing\" your regularly gridded datasets, generating new rasters on demand that can be used together with those it generates for any other data types. Rasterizing into a common grid can help you implement complex cross-datatype analyses or visualizations. \n",
"\n",
"In other cases, your data is stored in a 2D array similar to a raster, but represents values that are not regularly sampled in the underlying coordinate space. Datashader also provides fast methods for rasterizing these more general rectilinear or curvilinear grids, known as [quadmeshes](#quadmesh-Rasterization) as described later below. Fully arbitrary unstructured grids ([Trimeshes](Trimesh.ipynb)) are discussed separately.\n",
"In other cases, your data is stored in a 2D array similar to a raster, but represents values that are not regularly sampled in the underlying coordinate space. Datashader also provides fast methods for rasterizing these more general rectilinear or curvilinear grids, known as [quadmeshes](#quadmesh-rasterization) as described later below. Fully arbitrary unstructured grids ([Trimeshes](6_Trimesh.ipynb)) are discussed separately.\n",
"\n",
"## Re-rasterization\n",
"\n",
Expand Down Expand Up @@ -276,7 +276,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"The `Canvas.raster` method described above supports re-rasterizing grids that are already regularly spaced. For the more general case of rectilinear and curvilinear grids, which are structured (unlike [Trimesh grids](Trimesh.ipynb)) but not necessarily regularly spaced, Datashader provides the `Canvas.quadmesh` method. Despite its similarities with `raster`, `quadmesh` is currently implemented separately from `raster`, and thus has quite different features and limitations. For instance, `quadmesh` supports [GPU](https://datashader.org/user_guide/Performance.html#Data-objects) arrays (including an optimized code path specifically speeding up regularly spaced rasters if they are provided), while `raster` offers optional interpolation and supports distributed computing using Dask, and many other small differences exist (see the docs for each method). So if you have a regularly spaced array, you will probably want to try both `quadmesh` and `raster` and see which one meets your needs. \n",
"The `Canvas.raster` method described above supports re-rasterizing grids that are already regularly spaced. For the more general case of rectilinear and curvilinear grids, which are structured (unlike [Trimesh grids](6_Trimesh.ipynb)) but not necessarily regularly spaced, Datashader provides the `Canvas.quadmesh` method. Despite its similarities with `raster`, `quadmesh` is currently implemented separately from `raster`, and thus has quite different features and limitations. For instance, `quadmesh` supports [GPU](https://datashader.org/user_guide/Performance.html#Data-objects) arrays (including an optimized code path specifically speeding up regularly spaced rasters if they are provided), while `raster` offers optional interpolation and supports distributed computing using Dask, and many other small differences exist (see the docs for each method). So if you have a regularly spaced array, you will probably want to try both `quadmesh` and `raster` and see which one meets your needs. \n",
"\n",
"### Rectilinear Quadmesh\n",
"A rectilinear quadmesh is specified as a 2-dimensional xarray `DataArray` with two 1-dimensional coordinates. The coordinates specify the center position of the quads in each row or column of the grid, allowing the spacing on x and y to be irregular, but leaving the two dimensions orthogonal (and thus always producing an axis-aligned rectangular image out of the provided rectangular array, with each patch also rectangular and axis aligned)."
Expand Down
Loading