-
-
Notifications
You must be signed in to change notification settings - Fork 377
Feature/gsk 2172 improve documentation ci [GSK-2186] #1623
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
GSK-2172 Improve documentation CI
Atm, docs are fairly often having broken links, imgs and so on We could add some tests related to that: Also, we should make sphinx warnings behave as error, and split docs CI from main flow |
|
There're currently still these warnings:
/Users/rak/Documents/giskard_repos/giskard_main/giskard/models/__init__.py:docstring of giskard.models:22: WARNING: Bullet list ends without a blank line; unexpected unindent.
WARNING: Failed to get a method signature for giskard.Model.is_classification: is not a callable object
WARNING: Failed to get a method signature for giskard.Model.is_binary_classification: is not a callable object
WARNING: Failed to get a method signature for giskard.Model.is_regression: is not a callable object
WARNING: Failed to get a method signature for giskard.Model.is_text_generation: is not a callable object
/Users/rak/Documents/giskard_repos/giskard_main/giskard/models/base/model.py:docstring of giskard.models.base.model.BaseModel:10: WARNING: Bullet list ends without a blank line; unexpected unindent.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/models/base/model.py:docstring of giskard.models.base.model.BaseModel:76: ERROR: Unexpected indentation.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/scanner/report.py:docstring of giskard.scanner.report.ScanReport.generate_tests:3: ERROR: Unknown interpreted text role "method".
/Users/rak/Documents/giskard_repos/giskard_main/giskard/core/suite.py:docstring of giskard.core.suite.Suite.add_test:6: ERROR: Unexpected indentation.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/core/suite.py:docstring of giskard.core.suite.Suite.add_test:8: WARNING: Block quote ends without a blank line; unexpected unindent.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/core/suite.py:docstring of giskard.core.suite.Suite.add_test:10: ERROR: Unexpected indentation.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/core/suite.py:docstring of giskard.core.suite.Suite.add_test:8: WARNING: Inline strong start-string without end-string.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/drift.py:docstring of giskard.testing.tests.drift.test_drift_psi:23: ERROR: Unexpected indentation.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/drift.py:docstring of giskard.testing.tests.drift.test_drift_psi:24: WARNING: Block quote ends without a blank line; unexpected unindent.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/drift.py:docstring of giskard.testing.tests.drift.test_drift_chi_square:25: ERROR: Unexpected indentation.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/drift.py:docstring of giskard.testing.tests.drift.test_drift_chi_square:26: WARNING: Block quote ends without a blank line; unexpected unindent.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/drift.py:docstring of giskard.testing.tests.drift.test_drift_ks:21: ERROR: Unexpected indentation.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/drift.py:docstring of giskard.testing.tests.drift.test_drift_ks:22: WARNING: Block quote ends without a blank line; unexpected unindent.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/drift.py:docstring of giskard.testing.tests.drift.test_drift_earth_movers_distance:21: ERROR: Unexpected indentation.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/drift.py:docstring of giskard.testing.tests.drift.test_drift_earth_movers_distance:22: WARNING: Block quote ends without a blank line; unexpected unindent.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/drift.py:docstring of giskard.testing.tests.drift.test_drift_prediction_psi:24: ERROR: Unexpected indentation.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/drift.py:docstring of giskard.testing.tests.drift.test_drift_prediction_psi:25: WARNING: Block quote ends without a blank line; unexpected unindent.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/drift.py:docstring of giskard.testing.tests.drift.test_drift_prediction_chi_square:26: ERROR: Unexpected indentation.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/drift.py:docstring of giskard.testing.tests.drift.test_drift_prediction_chi_square:27: WARNING: Block quote ends without a blank line; unexpected unindent.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/drift.py:docstring of giskard.testing.tests.drift.test_drift_prediction_ks:23: ERROR: Unexpected indentation.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/drift.py:docstring of giskard.testing.tests.drift.test_drift_prediction_ks:24: WARNING: Block quote ends without a blank line; unexpected unindent.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/drift.py:docstring of giskard.testing.tests.drift.test_drift_prediction_earth_movers_distance:27: ERROR: Unexpected indentation.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/metamorphic.py:docstring of giskard.testing.tests.metamorphic.test_metamorphic_invariance:29: ERROR: Unexpected indentation.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/metamorphic.py:docstring of giskard.testing.tests.metamorphic.test_metamorphic_invariance:30: WARNING: Block quote ends without a blank line; unexpected unindent.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/metamorphic.py:docstring of giskard.testing.tests.metamorphic.test_metamorphic_increasing:27: ERROR: Unexpected indentation.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/metamorphic.py:docstring of giskard.testing.tests.metamorphic.test_metamorphic_increasing:28: WARNING: Block quote ends without a blank line; unexpected unindent.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/metamorphic.py:docstring of giskard.testing.tests.metamorphic.test_metamorphic_decreasing:29: ERROR: Unexpected indentation.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/metamorphic.py:docstring of giskard.testing.tests.metamorphic.test_metamorphic_decreasing:30: WARNING: Block quote ends without a blank line; unexpected unindent.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/metamorphic.py:docstring of giskard.testing.tests.metamorphic.test_metamorphic_decreasing_t_test:24: ERROR: Unexpected indentation.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/metamorphic.py:docstring of giskard.testing.tests.metamorphic.test_metamorphic_decreasing_t_test:25: WARNING: Block quote ends without a blank line; unexpected unindent.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/metamorphic.py:docstring of giskard.testing.tests.metamorphic.test_metamorphic_increasing_t_test:24: ERROR: Unexpected indentation.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/metamorphic.py:docstring of giskard.testing.tests.metamorphic.test_metamorphic_increasing_t_test:25: WARNING: Block quote ends without a blank line; unexpected unindent.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/metamorphic.py:docstring of giskard.testing.tests.metamorphic.test_metamorphic_invariance_t_test:7: ERROR: Unexpected indentation.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/metamorphic.py:docstring of giskard.testing.tests.metamorphic.test_metamorphic_invariance_t_test:29: ERROR: Unexpected indentation.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/metamorphic.py:docstring of giskard.testing.tests.metamorphic.test_metamorphic_invariance_t_test:30: WARNING: Block quote ends without a blank line; unexpected unindent.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/metamorphic.py:docstring of giskard.testing.tests.metamorphic.test_metamorphic_decreasing_wilcoxon:24: ERROR: Unexpected indentation.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/metamorphic.py:docstring of giskard.testing.tests.metamorphic.test_metamorphic_decreasing_wilcoxon:25: WARNING: Block quote ends without a blank line; unexpected unindent.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/metamorphic.py:docstring of giskard.testing.tests.metamorphic.test_metamorphic_increasing_wilcoxon:24: ERROR: Unexpected indentation.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/metamorphic.py:docstring of giskard.testing.tests.metamorphic.test_metamorphic_increasing_wilcoxon:25: WARNING: Block quote ends without a blank line; unexpected unindent.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/metamorphic.py:docstring of giskard.testing.tests.metamorphic.test_metamorphic_invariance_wilcoxon:29: ERROR: Unexpected indentation.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/metamorphic.py:docstring of giskard.testing.tests.metamorphic.test_metamorphic_invariance_wilcoxon:30: WARNING: Block quote ends without a blank line; unexpected unindent.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/performance.py:docstring of giskard.testing.tests.performance.test_mae:19: ERROR: Unexpected indentation.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/performance.py:docstring of giskard.testing.tests.performance.test_mae:20: WARNING: Block quote ends without a blank line; unexpected unindent.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/performance.py:docstring of giskard.testing.tests.performance.test_rmse:19: ERROR: Unexpected indentation.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/performance.py:docstring of giskard.testing.tests.performance.test_rmse:20: WARNING: Block quote ends without a blank line; unexpected unindent.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/performance.py:docstring of giskard.testing.tests.performance.test_recall:17: ERROR: Unexpected indentation.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/performance.py:docstring of giskard.testing.tests.performance.test_recall:18: WARNING: Block quote ends without a blank line; unexpected unindent.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/performance.py:docstring of giskard.testing.tests.performance.test_auc:17: ERROR: Unexpected indentation.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/performance.py:docstring of giskard.testing.tests.performance.test_auc:18: WARNING: Block quote ends without a blank line; unexpected unindent.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/performance.py:docstring of giskard.testing.tests.performance.test_accuracy:17: ERROR: Unexpected indentation.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/performance.py:docstring of giskard.testing.tests.performance.test_accuracy:18: WARNING: Block quote ends without a blank line; unexpected unindent.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/performance.py:docstring of giskard.testing.tests.performance.test_precision:17: ERROR: Unexpected indentation.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/performance.py:docstring of giskard.testing.tests.performance.test_precision:18: WARNING: Block quote ends without a blank line; unexpected unindent.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/performance.py:docstring of giskard.testing.tests.performance.test_f1:17: ERROR: Unexpected indentation.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/performance.py:docstring of giskard.testing.tests.performance.test_f1:18: WARNING: Block quote ends without a blank line; unexpected unindent.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/performance.py:docstring of giskard.testing.tests.performance.test_r2:19: ERROR: Unexpected indentation.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/performance.py:docstring of giskard.testing.tests.performance.test_r2:20: WARNING: Block quote ends without a blank line; unexpected unindent.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/performance.py:docstring of giskard.testing.tests.performance.test_diff_recall:22: ERROR: Unexpected indentation.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/performance.py:docstring of giskard.testing.tests.performance.test_diff_recall:23: WARNING: Block quote ends without a blank line; unexpected unindent.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/performance.py:docstring of giskard.testing.tests.performance.test_diff_accuracy:22: ERROR: Unexpected indentation.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/performance.py:docstring of giskard.testing.tests.performance.test_diff_accuracy:23: WARNING: Block quote ends without a blank line; unexpected unindent.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/performance.py:docstring of giskard.testing.tests.performance.test_diff_precision:22: ERROR: Unexpected indentation.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/performance.py:docstring of giskard.testing.tests.performance.test_diff_precision:23: WARNING: Block quote ends without a blank line; unexpected unindent.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/performance.py:docstring of giskard.testing.tests.performance.test_diff_rmse:24: ERROR: Unexpected indentation.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/performance.py:docstring of giskard.testing.tests.performance.test_diff_rmse:25: WARNING: Block quote ends without a blank line; unexpected unindent.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/performance.py:docstring of giskard.testing.tests.performance.test_diff_f1:22: ERROR: Unexpected indentation.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/performance.py:docstring of giskard.testing.tests.performance.test_diff_f1:23: WARNING: Block quote ends without a blank line; unexpected unindent.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/statistic.py:docstring of giskard.testing.tests.statistic.test_right_label:23: ERROR: Unexpected indentation.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/statistic.py:docstring of giskard.testing.tests.statistic.test_right_label:24: WARNING: Block quote ends without a blank line; unexpected unindent.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/statistic.py:docstring of giskard.testing.tests.statistic.test_output_in_range:35: ERROR: Unexpected indentation.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/statistic.py:docstring of giskard.testing.tests.statistic.test_output_in_range:36: WARNING: Block quote ends without a blank line; unexpected unindent.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/statistic.py:docstring of giskard.testing.tests.statistic.test_disparate_impact:40: ERROR: Unexpected indentation.
looking for now-outdated files... none found
pickling environment... done
checking consistency... /Users/rak/Documents/giskard_repos/giskard_main/docs/getting_started/quickstart/_quickstart_llm.ipynb: WARNING: document isn't included in any toctree
done
preparing documents... done
copying assets... copying static files... done
copying extra files... done
done
writing output... [100%] tutorials/tabular_tutorials/index
/Users/rak/Documents/giskard_repos/giskard_main/docs/reference/notebooks/LLM_Description_Product.ipynb:: WARNING: Lexing literal_block '!pip install "giskard[llm]" --upgrade' as "python" resulted in an error at token: '!'. Retrying in relaxed mode.
/Users/rak/Documents/giskard_repos/giskard_main/docs/reference/notebooks/LLM_QA_IPCC.ipynb:: WARNING: Lexing literal_block '!pip install "giskard[llm]" --upgrade' as "python" resulted in an error at token: '!'. Retrying in relaxed mode.
/Users/rak/Documents/giskard_repos/giskard_main/docs/reference/notebooks/LLM_QA_IPCC.ipynb:: WARNING: Lexing literal_block '!pip install "langchain<=0.0.301" "pypdf<=3.17.0" "faiss-cpu<=1.7.4" "openai<=0.28.1" "tiktoken<=0.5.1"' as "python" resulted in an error at token: '!'. Retrying in relaxed mode.
/Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/llm/hallucination.py:docstring of giskard.testing.tests.llm.hallucination.test_llm_output_coherency:11: WARNING: undefined label: 'coherencyevaluator'
I'll help with this when I get the time. |
…-2172-improve-documentation-ci
|
but it's not failing still on warnings right @Hartorn ? |
This reverts commit 893943e.
…-2172-improve-documentation-ci
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Phenomenal job @Hartorn! Sorry for the late review, we should have this merged asap.
Just left you two comments, if you have time to look at, if not, we can fix them later.
| ## Building the docs locally (continuously) | ||
| ``` | ||
| cd giskard | ||
| pdm run watch-doc |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why remove it? let's just correct it pdm watch-doc
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it's inside the doc folder source, and so it's watched by sphinx
| :::::{grid-item} | ||
| :child-align: | ||
| ::::{grid} 1 1 1 1 | ||
| :child-align: | ||
| :gutter: 1 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why we should not nest grid? I used the following doc: https://sphinx-design.readthedocs.io/en/furo-theme/grids.html#nesting-grids
Yes I think the order is important here, can we please revert this back?
…-2172-improve-documentation-ci
|
Kudos, SonarCloud Quality Gate passed! |













Description
Related Issue
Type of Change
Checklist
CODE_OF_CONDUCT.mddocument.CONTRIBUTING.mdguide.make codestyle.