[NETWORKX] Add MixedEdgeGraph into package (#29)

* Implementation of networkx MixedEdgeGraph for subclassing
* M-separation implementation
* update of poetry and numpydoc
* re-organized examples into subdirectories

Signed-off-by: Adam Li <[email protected]>
Co-authored-by: Jaron Lee <[email protected]>

Author: adam2392

.circleci/config.yml

@@ -70,7 +70,7 @@ jobs:
           command: sudo apt update && sudo apt install -y pandoc optipng
       - python/install-packages:
           pkg-manager: poetry
-          cache-version: "v1" # change to clear cache
+          cache-version: "v2" # change to clear cache
           args: "--with docs"
       - run:
           name: Check poetry package versions
@@ -87,7 +87,7 @@ jobs:
           name: Build documentation
           command: |
             cd docs
-            poetry run make html
+            poetry run poe build_docs
       # Save the example test results
       - store_test_results:
           path: docs/_build/test-results

.gitignore

@@ -6,6 +6,9 @@ __pycache__/
 .vscode
 junit-results.xml
 
+*.gv 
+*.png
+
 # C extensions
 *.so
 .DS_Store

CONTRIBUTING.md

@@ -142,7 +142,11 @@ With poetry installed, we have included a few convenience functions to check you
 
     poetry install --with style, docs, test
 
-Check code formatting with black:
+You can verify that your code will pass certain style, formatting and lint checks by running:
+
+    poetry run poe verify
+
+``verify`` runs a sequence of tests that can also be run individually. For example, you can check code formatting with black:
 
     poetry run poe check_format
 
@@ -158,6 +162,16 @@ Finally, you should run some mypy type checks:
 
     poetry run poe type_check
 
+### Documentation
+
 If you need to build the documentation locally and check for doc errors:
 
     poetry run poe build_docs
+
+### Dependency Changes
+
+If you need to add new, or remove old dependencies, then you need to modify the ``pyproject.toml`` file and then also update the ``poetry.lock`` file, which version-controls all necessary dependencies. If you alter any dependency in the ``pyproject.toml`` file, you must run:
+
+    poetry update
+
+To update the lock file.

Makefile

@@ -3,7 +3,7 @@
 build-docs:
 	@echo "Building documentation"
 	make -C docs/ clean
-	make -C docs/ html-noplot
+	make -C docs/ html
 	cd docs/ && make view
 
 clean-pyc:

docs/Makefile

@@ -169,12 +169,6 @@ changes:
 	@echo
 	@echo "The overview file is in $(BUILDDIR)/changes."
 
-linkcheck:
-	$(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck
-	@echo
-	@echo "Link check complete; look for any errors in the above output " \
-	      "or in $(BUILDDIR)/linkcheck/output.txt."
-
 doctest:
 	$(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest
 	@echo "Testing of doctests in the sources finished, look at the " \

docs/api.rst

@@ -63,6 +63,22 @@ and corresponding causal graphs in pywhy-graphs.
    graph_to_arr
    clearn_arr_to_graph
 
+NetworkX Experimental Functionality
+===================================
+Currently, NetworkX does not support mixed-edge graphs, which are crucial
+for representing causality with latent confounders and selection bias. The
+following represent functionality that we intend to PR eventually into
+networkx. They are included in pywhy-graphs as a temporary bridge. We 
+welcome feedback.
+
+.. currentmodule:: pywhy_graphs.networkx
+.. autosummary::
+   :toctree: generated/
+   
+   MixedEdgeGraph
+   bidirected_to_unobserved_confounder
+   m_separated
+   
 Visualization of causal graphs
 ==============================
 Visualization of causal graphs is different compared to networkx because causal graphs

docs/conf.py

@@ -7,9 +7,12 @@
 import os
 import sys
 from datetime import datetime
+import warnings
 
+import numpy as np
+import networkx as nx
 import sphinx_gallery  # noqa: F401
-from sphinx_gallery.sorting import ExampleTitleSortKey
+from sphinx_gallery.sorting import ExampleTitleSortKey, ExplicitOrder
 
 # -- Path setup --------------------------------------------------------------
 
@@ -48,14 +51,14 @@
     "sphinx.ext.doctest",
     "sphinx.ext.intersphinx",
     "sphinx_issues",
-    "sphinx.ext.mathjax",
     "sphinx.ext.viewcode",
+    # "nbsphinx",  # enables building Jupyter notebooks and rendering
+    "sphinx.ext.mathjax",
     "sphinx_gallery.gen_gallery",
     "sphinxcontrib.bibtex",
     "sphinx_copybutton",
     "numpydoc",
-    "IPython.sphinxext.ipython_console_highlighting",
-    "nbsphinx",
+    # "IPython.sphinxext.ipython_console_highlighting",
 ]
 
 # configure sphinx-copybutton
@@ -106,7 +109,7 @@
     "attributes",
     "dictionary",
     "ArrayLike",
-    "nx.MixedEdgeGraph",
+    "pywhy_nx.MixedEdgeGraph",
     # pywhy-graphs
     "causal",
     "Node",
@@ -161,6 +164,7 @@
     "ADMG": "pywhy_graphs.ADMG",
     "PAG": "pywhy_graphs.PAG",
     "CPDAG": "pywhy_graphs.CPDAG",
+    "pywhy_nx.MixedEdgeGraph": "pywhy_graphs.networkx.MixedEdgeGraph",
     # joblib
     "joblib.Parallel": "joblib.Parallel",
     # pandas
@@ -182,12 +186,13 @@
 # directories to ignore when looking for source files.
 # This pattern also affects html_static_path and html_extra_path.
 exclude_patterns = [
-    "auto_examples/index.rst",
     "_build",
     "Thumbs.db",
-    ".DS_Store",
     "**.ipynb_checkpoints",
-    "auto_examples/*.rst",
+    # "auto_examples/*.rst",
+    # "auto_examples/index.rst",
+    # "auto_examples/mixededge/index.rst",
+    # "auto_examples/mixededge/*.rst",
 ]
 
 source_suffix = [".rst", ".md"]
@@ -214,7 +219,6 @@
 # The master toctree document.
 master_doc = "index"
 
-
 # -- Options for HTML output -------------------------------------------------
 
 # The theme to use for HTML and HTML Help pages.  See the documentation for
@@ -253,24 +257,47 @@
 scrapers = ("matplotlib",)
 
 sphinx_gallery_conf = {
+    # Modules for which function level galleries are created.  In
+    # this case sphinx_gallery and numpy in a tuple of strings.
     "doc_module": "pywhy_graphs",
+    # Insert links to documentation of objects in the examples
     "reference_url": {
         "pywhy_graphs": None,
     },
     "backreferences_dir": "generated",
     "plot_gallery": "True",  # Avoid annoying Unicode/bool default warning
-    "within_subsection_order": ExampleTitleSortKey,
     "examples_dirs": ["../examples"],
     "gallery_dirs": ["auto_examples"],
-    "filename_pattern": "^((?!sgskip).)*$",
+    "within_subsection_order": ExampleTitleSortKey,
+    "subsection_order": ExplicitOrder(
+        [
+            "../examples/mixededge",
+            "../examples/intro",
+            "../examples/visualization",
+        ]
+    ),
+    # "filename_pattern": "^((?!sgskip).)*$",
+    "filename_pattern": r"\.py",
     "matplotlib_animations": True,
     "compress_images": ("images", "thumbnails"),
     "image_scrapers": scrapers,
+    'show_memory': not sys.platform.startswith(('win', 'darwin')),
 }
 
+# Add pygraphviz png scraper, if available
+try:
+    from pygraphviz.scraper import PNGScraper
+
+    sphinx_gallery_conf["image_scrapers"] += (PNGScraper(),)
+except ImportError:
+    pass
+
 # Custom sidebar templates, maps document names to template names.
 html_sidebars = {
     "index": ["search-field.html"],
+    "install": [],
+    "tutorial": [],
+    "auto_examples/index": [],
 }
 
 html_context = {
@@ -286,7 +313,17 @@
 nitpick_ignore = [
     ("py:obj", "nx.MixedEdgeGraph"),
     ("py:obj", "networkx.MixedEdgeGraph"),
+    ("py:obj", "pywhy_graphs.networkx.MixedEdgeGraph"),
+    ("py:obj", "pywhy_nx.MixedEdgeGraph"),
     ("py:class", "networkx.classes.mixededge.MixedEdgeGraph"),
     ("py:class", "numpy._typing._array_like._SupportsArray"),
     ("py:class", "numpy._typing._nested_sequence._NestedSequence"),
 ]
+
+
+# -- Warnings management -----------------------------------------------------
+def setup(app):
+    # Ignore .ipynb files
+    app.registry.source_suffix.pop(".ipynb", None)
+
+warnings.filterwarnings("ignore", category=UserWarning)

docs/index.rst

@@ -20,7 +20,7 @@ Contents
 
    installation
    api
-   use
+   Usage<use>
    whats_new
 
 Team

docs/use.rst

@@ -1,11 +1,8 @@
-:orphan:
-
-Examples using pywhy-graphs
-===========================
+How to use pywhy-graphs with examples and tutorials
+===================================================
 
 To be able to effectively use pywhy-graphs, look at some of the examples here
 to learn everything you need! We give an overview of concepts in causal graphs.
 
-
 .. include:: auto_examples/index.rst
-   :start-after: :orphan:
+    :start-after: :orphan:

docs/whats_new/v0.1.rst

@@ -32,6 +32,7 @@ Changelog
 - |Feature| Implement an array API wrapper to convert between causal graphs in pywhy-graphs and causal graphs in ``causal-learn``, by `Adam Li`_ (:pr:`16`)
 - |Feature| Implement an acyclification algorithm for converting cyclic graphs to acyclic with :func:`pywhy_graphs.algorithms.acyclification`, by `Adam Li`_ (:pr:`17`)
 - |Feature| Adding a layout for the nodes positions in the :func:`pywhy_graphs.viz.draw` function, by `Julien Siebert`_ (:pr:`26`)
+- |Feature| Add :class:`pywhy_graphs.networkx.MixedEdgeGraph` for mixed-edge graphs, by `Adam Li`_ (:pr:`29`)
 
 Code and Documentation Contributors
 -----------------------------------