Skip to content

3.0.1.rst

Latest commit

 

History

History
519 lines (406 loc) · 24.9 KB

3.0.1.rst

File metadata and controls

519 lines (406 loc) · 24.9 KB

v3.0.1 (27 Jan 2021)

This document explains the changes made to Iris for this release (:doc:`View all changes <index>`.)

.. dropdown:: :opticon:`alert` v3.0.1 Patches
   :container: + shadow
   :title: text-primary text-center font-weight-bold
   :body: bg-light
   :animate: fade-in
   :open:

   The patches included in this release include:

   💼 **Internal**

   #. `@bjlittle`_ gracefully promote formula terms within :mod:`~iris.aux_factory` that have ``units`` of ``unknown``
      to ``units`` of ``1`` (dimensionless), where the formula term **must** have dimensionless ``units``. Without this
      graceful treatment of ``units`` the resulting :class:`~iris.cube.Cube` will **not** contain the expected auxiliary
      factory, and the associated derived coordinate will be missing. (:pull:`3965`)


.. dropdown:: :opticon:`report` Release Highlights
   :container: + shadow
   :title: text-primary text-center font-weight-bold
   :body: bg-light
   :animate: fade-in
   :open:

   The highlights for this major release of Iris include:

   * We've finally dropped support for ``Python 2``, so welcome to ``Iris 3``
     and ``Python 3``!
   * We've extended our coverage of the `CF Conventions and Metadata`_ by
     introducing support for `CF Ancillary Data`_ and `Quality Flags`_,
   * Lazy regridding is now available for several regridding schemes,
   * Managing and manipulating metadata within Iris is now easier and more
     consistent thanks to the introduction of a new common metadata API,
   * :ref:`Cube arithmetic <cube maths>` has been significantly improved with
     regards to extended broadcasting, auto-transposition and a more lenient
     behaviour towards handling metadata and coordinates,
   * Our :ref:`documentation <whatsnew 3.0.1 docs>` has been refreshed,
     restructured, revitalised and rehosted on `readthedocs`_,
   * It's now easier than ever to :ref:`install Iris <installing_iris>`
     as a user or a developer, and the newly revamped developers guide walks
     you though how you can :ref:`get involved <development_where_to_start>`
     and contribute to Iris,
   * Also, this is a major release of Iris, so please be aware of the
     :ref:`incompatible changes <whatsnew 3.0.1 changes>` and
     :ref:`deprecations <whatsnew 3.0.1 deprecations>`.

   And finally, get in touch with us on `GitHub`_ if you have any issues or
   feature requests for improving Iris. Enjoy!

📢 Announcements

  1. Congratulations to @bouweandela, @jvegasbsc, and @zklaus who recently became Iris core developers. They bring a wealth of expertise to the team, and are using Iris to underpin ESMValTool - "A community diagnostic and performance metrics tool for routine evaluation of Earth system models in CMIP". Welcome aboard! 🎉
  2. Congratulations also goes to @jonseddon who recently became an Iris core developer. We look forward to seeing more of your awesome contributions! 🎉

✨ Features

  1. @MoseleyS greatly enhanced the :mod:`~iris.fileformats.nimrod` module to provide richer meta-data translation when loading Nimrod data into cubes. This covers most known operational use-cases. (:pull:`3647`)
  2. @stephenworsley improved the handling of :class:`iris.coords.CellMeasure`s in the :class:`~iris.cube.Cube` statistical operations :meth:`~iris.cube.Cube.collapsed`, :meth:`~iris.cube.Cube.aggregated_by` and :meth:`~iris.cube.Cube.rolling_window`. These previously removed every :class:`~iris.coords.CellMeasure` attached to the cube. Now, a :class:`~iris.coords.CellMeasure` will only be removed if it is associated with an axis over which the statistic is being run. (:pull:`3549`)
  3. @stephenworsley, @pp-mo and @abooton added support for CF Ancillary Data variables. These are created as :class:`iris.coords.AncillaryVariable`, and appear as components of cubes much like :class:`~iris.coords.AuxCoord`s, with the new :class:`~iris.cube.Cube` methods :meth:`~iris.cube.Cube.add_ancillary_variable`, :meth:`~iris.cube.Cube.remove_ancillary_variable`, :meth:`~iris.cube.Cube.ancillary_variable`, :meth:`~iris.cube.Cube.ancillary_variables` and :meth:`~iris.cube.Cube.ancillary_variable_dims`. They are loaded from and saved to NetCDF-CF files. Special support for Quality Flags is also provided, to ensure they load and save with appropriate units. (:pull:`3800`)
  4. @bouweandela implemented lazy regridding for the :class:`~iris.analysis.Linear`, :class:`~iris.analysis.Nearest`, and :class:`~iris.analysis.AreaWeighted` regridding schemes. (:pull:`3701`)
  5. @bjlittle added logging support within :mod:`iris.analysis.maths`, :mod:`iris.common.metadata`, and :mod:`iris.common.resolve`. Each module defines a :class:`logging.Logger` instance called logger with a default level of INFO. To enable DEBUG logging use logger.setLevel("DEBUG"). (:pull:`3785`)
  6. @bjlittle added the :mod:`iris.common.resolve` module, which provides infrastructure to support the analysis, identification and combination of metadata common between two :class:`~iris.cube.Cube` operands into a single resultant :class:`~iris.cube.Cube` that will be auto-transposed, and with the appropriate broadcast shape. (:pull:`3785`)
  7. @bjlittle added the :ref:`common metadata API <metadata>`, which provides a unified treatment of metadata across Iris, and allows users to easily manage and manipulate their metadata in a consistent way. (:pull:`3785`)
  8. @bjlittle added :ref:`lenient metadata <lenient metadata>` support, to allow users to control strict or lenient metadata equivalence, difference and combination. (:pull:`3785`)
  9. @bjlittle added :ref:`lenient cube maths <lenient maths>` support and resolved several long standing major issues with cube arithmetic regarding a more robust treatment of cube broadcasting, cube dimension auto-transposition, and preservation of common metadata and coordinates during cube math operations. Resolves :issue:`1887`, :issue:`2765`, and :issue:`3478`. (:pull:`3785`)
  10. @pp-mo and @TomekTrzeciak enhanced :meth:`~iris.cube.Cube.collapse` to allow a 1-D weights array when collapsing over a single dimension. Previously, the weights had to be the same shape as the whole cube, which could cost a lot of memory in some cases. The 1-D form is supported by most weighted array statistics (such as :meth:`np.average`), so this now works with the corresponding Iris schemes (in that case, :const:`~iris.analysis.MEAN`). (:pull:`3943`)

🐛 Bugs Fixed

  1. @stephenworsley fixed :meth:`~iris.cube.Cube.remove_coord` to now also remove derived coordinates by removing aux_factories. (:pull:`3641`)
  2. @jonseddon fixed isinstance(cube, collections.Iterable) to now behave as expected if a :class:`~iris.cube.Cube` is iterated over, while also ensuring that TypeError is still raised. (Fixed by setting the __iter__() method in :class:`~iris.cube.Cube` to None). (:pull:`3656`)
  3. @stephenworsley enabled cube concatenation along an axis shared by cell measures; these cell measures are now concatenated together in the resulting cube. Such a scenario would previously cause concatenation to inappropriately fail. (:pull:`3566`)
  4. @stephenworsley newly included :class:`~iris.coords.CellMeasure`s in :class:`~iris.cube.Cube` copy operations. Previously copying a :class:`~iris.cube.Cube` would ignore any attached :class:`~iris.coords.CellMeasure`. (:pull:`3546`)
  5. @bjlittle set a :class:`~iris.coords.CellMeasure`'s measure attribute to have a default value of area. Previously, the measure was provided as a keyword argument to :class:`~iris.coords.CellMeasure` with a default value of None, which caused a TypeError when no measure was provided, since area or volume are the only accepted values. (:pull:`3533`)
  6. @trexfeathers set all plot types in :mod:`iris.plot` to now use :obj:`matplotlib.dates.date2num` to format date/time coordinates for use on a plot axis (previously :meth:`~iris.plot.pcolor` and :meth:`~iris.plot.pcolormesh` did not include this behaviour). (:pull:`3762`)
  7. @trexfeathers changed date/time axis labels in :mod:`iris.quickplot` to now always be based on the epoch used in :obj:`matplotlib.dates.date2num` (previously would take the unit from a time coordinate, if present, even though the coordinate's value had been changed via date2num). (:pull:`3762`)
  8. @pp-mo newly included attributes of cell measures in NETCDF-CF file loading; they were previously being discarded. They are now available on the :class:`~iris.coords.CellMeasure` in the loaded :class:`~iris.cube.Cube`. (:pull:`3800`)
  9. @pp-mo fixed the netcdf loader to now handle any grid-mapping variables with missing false_easting and false_northing properties, which was previously failing for some coordinate systems. See :issue:`3629`. (:pull:`3804`)
  10. @stephenworsley changed the way tick labels are assigned from string coords. Previously, the first tick label would occasionally be duplicated. This also removes the use of the deprecated `matplotlib`_ IndexFormatter. (:pull:`3857`)
  11. @znicholls fixed :meth:`~iris.quickplot._title` to only check units.is_time_reference if the units symbol is not used. (:pull:`3902`)
  12. @rcomer fixed a bug whereby numpy array type attributes on a cube's coordinates could prevent printing it. See :issue:`3921`. (:pull:`3922`)

💣 Incompatible Changes

  1. @pp-mo rationalised :class:`~iris.cube.CubeList` extraction methods:

    The former method iris.cube.CubeList.extract_strict, and the strict keyword of the :meth:`~iris.cube.CubeList.extract` method have been removed, and are replaced by the new routines :meth:`~iris.cube.CubeList.extract_cube` and :meth:`~iris.cube.CubeList.extract_cubes`. The new routines perform the same operation, but in a style more like other Iris functions such as :meth:`~iris.load_cube` and :meth:`~iris.load_cubes`. Unlike strict extraction, the type of return value is now completely consistent : :meth:`~iris.cube.CubeList.extract_cube` always returns a :class:`~iris.cube.Cube`, and :meth:`~iris.cube.CubeList.extract_cubes` always returns an :class:`iris.cube.CubeList` of a length equal to the number of constraints. (:pull:`3715`)

  2. @pp-mo removed the former function iris.analysis.coord_comparison. (:pull:`3562`)

  3. @bjlittle moved the :func:`iris.experimental.equalise_cubes.equalise_attributes` function from the :mod:`iris.experimental` module into the :mod:`iris.util` module. Please use the :func:`iris.util.equalise_attributes` function instead. (:pull:`3527`)

  4. @bjlittle removed the module iris.experimental.concatenate. In v1.6.0 the experimental concatenate functionality was moved to the :meth:`iris.cube.CubeList.concatenate` method. Since then, calling the :func:`iris.experimental.concatenate.concatenate` function raised an exception. (:pull:`3523`)

  5. @stephenworsley changed the default units of :class:`~iris.coords.DimCoord` and :class:`~iris.coords.AuxCoord` from "1" to "unknown". (:pull:`3795`)

  6. @stephenworsley changed Iris objects loaded from NetCDF-CF files to have units='unknown' where the corresponding NetCDF variable has no units property. Previously these cases defaulted to units='1'. This affects loading of coordinates whose file variable has no "units" attribute (not valid, under CF units rules): These will now have units of "unknown", rather than "1", which may prevent the creation of a hybrid vertical coordinate. While these cases used to "work", this was never really correct behaviour. (:pull:`3795`)

  7. @SimonPeatman added attribute var_name to coordinates created by the :func:`iris.analysis.trajectory.interpolate` function. This prevents duplicate coordinate errors in certain circumstances. (:pull:`3718`)

  8. @bjlittle aligned the :func:`iris.analysis.maths.apply_ufunc` with the rest of the :mod:`iris.analysis.maths` API by changing its keyword argument from other_cube to other. (:pull:`3785`)

  9. @bjlittle changed the :meth:`iris.analysis.maths.IFunc.__call__` to ignore any surplus other keyword argument for a data_func that requires only one argument. This aligns the behaviour of :meth:`iris.analysis.maths.IFunc.__call__` with :func:`~iris.analysis.maths.apply_ufunc`. Previously a ValueError exception was raised. (:pull:`3785`)

🔥 Deprecations

  1. @stephenworsley removed the deprecated :class:`iris.Future` flags cell_date_time_objects, netcdf_promote, netcdf_no_unlimited and clip_latitudes. (:pull:`3459`)
  2. @stephenworsley changed :attr:`iris.fileformats.pp.PPField.lbproc` to be an int. The deprecated attributes flag1, flag2 etc. have been removed from it. (:pull:`3461`)
  3. @bjlittle deprecated :func:`~iris.util.as_compatible_shape` in preference for :class:`~iris.common.resolve.Resolve` e.g., Resolve(src, tgt)(tgt.core_data()). The :func:`~iris.util.as_compatible_shape` function will be removed in a future release of Iris. (:pull:`3892`)

🔗 Dependencies

  1. @stephenworsley, @trexfeathers and @bjlittle removed Python2 support, modernising the codebase by switching to exclusive Python3 support. (:pull:`3513`)
  2. @bjlittle improved the developer set up process. Configuring Iris and :ref:`installing_from_source` as a developer with all the required package dependencies is now easier with our curated conda environment YAML files. (:pull:`3812`)
  3. @stephenworsley pinned Iris to require Dask >=2.0. (:pull:`3460`)
  4. @stephenworsley and @trexfeathers pinned Iris to require Cartopy >=0.18, in order to remain compatible with the latest version of `matplotlib`_. (:pull:`3762`)
  5. @bjlittle unpinned Iris to use the latest version of `matplotlib`_. Supporting Iris for both Python2 and Python3 had resulted in pinning our dependency on `matplotlib`_ at v2.x. But this is no longer necessary now that Python2 support has been dropped. (:pull:`3468`)
  6. @stephenworsley and @trexfeathers unpinned Iris to use the latest version of Proj. (:pull:`3762`)
  7. @stephenworsley and @trexfeathers removed GDAL from the extensions dependency group. We no longer consider it to be an extension. (:pull:`3762`)

📚 Documentation

  1. @tkknight moved the :ref:`sphx_glr_generated_gallery_oceanography_plot_orca_projection.py` from the general part of the gallery to oceanography. (:pull:`3761`)
  2. @tkknight updated documentation to use a modern sphinx theme and be served from https://scitools-iris.readthedocs.io/en/latest/. (:pull:`3752`)
  3. @bjlittle added support for the black code formatter. This is now automatically checked on GitHub PRs, replacing the older, unittest-based iris.tests.test_coding_standards.TestCodeFormat. Black provides automatic code format correction for most IDEs. See the new developer guide section on :ref:`code_formatting`. (:pull:`3518`)
  4. @tkknight and @trexfeathers refreshed the :ref:`whats_new_contributions` for the :ref:`iris_whatsnew`. This includes always creating the latest what's new page so it appears on the latest documentation at https://scitools-iris.readthedocs.io/en/latest/whatsnew. This resolves :issue:`2104`, :issue:`3451`, :issue:`3818`, :issue:`3837`. Also updated the :ref:`iris_development_releases_steps` to follow when making a release. (:pull:`3769`, :pull:`3838`, :pull:`3843`)
  5. @tkknight enabled the PDF creation of the documentation on the Read the Docs service. The PDF may be accessed by clicking on the version at the bottom of the side bar, then selecting PDF from the Downloads section. (:pull:`3765`)
  6. @stephenworsley added a warning to the :func:`iris.analysis.cartography.project` function regarding its behaviour on projections with non-rectangular boundaries. (:pull:`3762`)
  7. @stephenworsley added the :ref:`cube_maths_combining_units` section to the user guide to clarify how Units are handled during cube arithmetic. (:pull:`3803`)
  8. @tkknight overhauled the :ref:`developers_guide` including information on getting involved in becoming a contributor and general structure of the guide. This resolves :issue:`2170`, :issue:`2331`, :issue:`3453`, :issue:`314`, :issue:`2902`. (:pull:`3852`)
  9. @rcomer added argument descriptions to the :class:`~iris.coords.DimCoord` docstring. (:pull:`3681`)
  10. @tkknight added two url's to be ignored for the make linkcheck. This will ensure the Iris github project is not repeatedly hit during the linkcheck for issues and pull requests as it can result in connection refused and thus travis-ci job failures. For more information on linkcheck, see :ref:`contributing.documentation.testing`. (:pull:`3873`)
  11. @tkknight enabled the napolean_ package that is used by sphinx_ to cater for the existing google style docstrings and to also allow for numpy docstrings. This resolves :issue:`3841`. (:pull:`3871`)
  12. @tkknight configured sphinx-build to promote warnings to errors when building the documentation via make html. This will minimise technical debt accruing for the documentation. (:pull:`3877`)
  13. @tkknight updated :ref:`installing_iris` to include a reference to Windows Subsystem for Linux. (:pull:`3885`)
  14. @tkknight updated the :ref:`iris_docs` homepage to include panels so the links are more visible to users. This uses the sphinx-panels extension. (:pull:`3884`)
  15. @bjlittle created the :ref:`Further topics <further topics>` section and included documentation for :ref:`metadata`, :ref:`lenient metadata`, and :ref:`lenient maths`. (:pull:`3890`)
  16. @jonseddon updated the CF version of the netCDF saver in the :ref:`saving_iris_cubes` section and in the equivalent function docstring. (:pull:`3925`)
  17. @bjlittle applied Title Case Capitalization to the documentation. (:pull:`3940`)

💼 Internal

  1. @pp-mo and @lbdreyer removed all Iris test dependencies on iris-grib by transferring all relevant content to the iris-grib repository. (:pull:`3662`, :pull:`3663`, :pull:`3664`, :pull:`3665`, :pull:`3666`, :pull:`3669`, :pull:`3670`, :pull:`3671`, :pull:`3672`, :pull:`3742`, :pull:`3746`)
  2. @lbdreyer and @pp-mo overhauled the handling of dimensional metadata to remove duplication. (:pull:`3422`, :pull:`3551`)
  3. @trexfeathers simplified the standard license header for all files, which removes the need to repeatedly update year numbers in the header. (:pull:`3489`)
  4. @stephenworsley changed the numerical values in tests involving the Robinson projection due to improvements made in Proj. (:pull:`3762`) (see also Proj#1292 and Proj#2151)
  5. @stephenworsley changed tests to account for more detailed descriptions of projections in GDAL. (:pull:`3762`) (see also GDAL#1185)
  6. @stephenworsley changed tests to account for GDAL now saving fill values for data without masked points. (:pull:`3762`)
  7. @trexfeathers changed every graphics test that includes Cartopy's coastlines to account for new adaptive coastline scaling. (:pull:`3762`) (see also Cartopy#1105)
  8. @trexfeathers changed graphics tests to account for some new default grid-line spacing in Cartopy. (:pull:`3762`) (see also Cartopy#1117)
  9. @trexfeathers added additional acceptable graphics test targets to account for very minor changes in `matplotlib`_ version 3.3 (colormaps, fonts and axes borders). (:pull:`3762`)
  10. @rcomer corrected the `matplotlib`_ backend in Iris tests to ignore :obj:`matplotlib.rcdefaults`, instead the tests will always use agg. (:pull:`3846`)
  11. @bjlittle migrated the black support from 19.10b0 to 20.8b1. (:pull:`3866`)
  12. @lbdreyer updated the CF standard name table to the latest version: v75. (:pull:`3867`)
  13. @bjlittle added PEP 517 and PEP 518 support for building and installing Iris, in particular to handle the PyKE package dependency. (:pull:`3812`)
  14. @bjlittle added metadata support for comparing :attr:`~iris.cube.Cube.attributes` dictionaries that contain numpy arrays using xxHash, an extremely fast non-cryptographic hash algorithm, running at RAM speed limits.
  15. @bjlittle added the iris.tests.assertDictEqual method to override :meth:`unittest.TestCase.assertDictEqual` in order to cope with testing metadata :attr:`~iris.cube.Cube.attributes` dictionary comparison where the value of a key may be a numpy array. (:pull:`3785`)
  16. @bjlittle added the :func:`~iris.config.get_logger` function for creating a generic :class:`logging.Logger` with a :class:`logging.StreamHandler` and custom :class:`logging.Formatter`. (:pull:`3785`)
  17. @owena11 identified and optimised a bottleneck in FieldsFile header loading due to the use of :func:`numpy.fromfile`. (:pull:`3791`)
  18. @znicholls added a test for plotting with the label being taken from the unit's symbol, see :meth:`~iris.tests.test_quickplot.TestLabels.test_pcolormesh_str_symbol` (:pull:`3902`).
  19. @znicholls made :func:`~iris.tests.idiff.step_over_diffs` robust to hyphens (-) in the input path (i.e. the result_dir argument) (:pull:`3902`).
  20. @bjlittle migrated the CIaaS from travis-ci to `cirrus-ci`_, and removed stickler-ci support. (:pull:`3928`)
  21. @bjlittle introduced nox as a common and easy entry-point for test automation. It can be used both from `cirrus-ci`_ in the cloud, and locally by the developer to run the Iris tests, the doc-tests, the gallery doc-tests, and lint Iris with flake8 and black. (:pull:`3928`)