Skip to content

Conversation

@Hartorn
Copy link
Member

@Hartorn Hartorn commented Nov 21, 2023

Description

Related Issue

Type of Change

  • 📚 Examples / docs / tutorials / dependencies update
  • 🔧 Bug fix (non-breaking change which fixes an issue)
  • 🥂 Improvement (non-breaking change which improves an existing feature)
  • 🚀 New feature (non-breaking change which adds functionality)
  • 💥 Breaking change (fix or feature that would cause existing functionality to change)
  • 🔐 Security fix

Checklist

  • I've read the CODE_OF_CONDUCT.md document.
  • I've read the CONTRIBUTING.md guide.
  • I've updated the code style using make codestyle.
  • I've written tests for all new methods and classes that I created.
  • I've written the docstring in Google format for all the methods and classes that I used.

@Hartorn Hartorn requested a review from rabah-khalek November 21, 2023 10:58
@Hartorn Hartorn self-assigned this Nov 21, 2023
@linear
Copy link

linear bot commented Nov 21, 2023

GSK-2172 Improve documentation CI

Atm, docs are fairly often having broken links, imgs and so on

We could add some tests related to that:
mlflow/mlflow@e2dd797

Also, we should make sphinx warnings behave as error, and split docs CI from main flow

@Hartorn Hartorn added the Lockfile Temporary label to update pdm.lock label Nov 21, 2023
@github-actions github-actions bot removed the Lockfile Temporary label to update pdm.lock label Nov 21, 2023
@Hartorn Hartorn closed this Nov 21, 2023
@Hartorn Hartorn reopened this Nov 21, 2023
@Hartorn Hartorn closed this Nov 22, 2023
@Hartorn Hartorn reopened this Nov 22, 2023
@rabah-khalek
Copy link
Contributor

There're currently still these warnings:

/Users/rak/Documents/giskard_repos/giskard_main/giskard/models/__init__.py:docstring of giskard.models:22: WARNING: Bullet list ends without a blank line; unexpected unindent. WARNING: Failed to get a method signature for giskard.Model.is_classification: is not a callable object WARNING: Failed to get a method signature for giskard.Model.is_binary_classification: is not a callable object WARNING: Failed to get a method signature for giskard.Model.is_regression: is not a callable object WARNING: Failed to get a method signature for giskard.Model.is_text_generation: is not a callable object /Users/rak/Documents/giskard_repos/giskard_main/giskard/models/base/model.py:docstring of giskard.models.base.model.BaseModel:10: WARNING: Bullet list ends without a blank line; unexpected unindent. /Users/rak/Documents/giskard_repos/giskard_main/giskard/models/base/model.py:docstring of giskard.models.base.model.BaseModel:76: ERROR: Unexpected indentation. /Users/rak/Documents/giskard_repos/giskard_main/giskard/scanner/report.py:docstring of giskard.scanner.report.ScanReport.generate_tests:3: ERROR: Unknown interpreted text role "method". /Users/rak/Documents/giskard_repos/giskard_main/giskard/core/suite.py:docstring of giskard.core.suite.Suite.add_test:6: ERROR: Unexpected indentation. /Users/rak/Documents/giskard_repos/giskard_main/giskard/core/suite.py:docstring of giskard.core.suite.Suite.add_test:8: WARNING: Block quote ends without a blank line; unexpected unindent. /Users/rak/Documents/giskard_repos/giskard_main/giskard/core/suite.py:docstring of giskard.core.suite.Suite.add_test:10: ERROR: Unexpected indentation. /Users/rak/Documents/giskard_repos/giskard_main/giskard/core/suite.py:docstring of giskard.core.suite.Suite.add_test:8: WARNING: Inline strong start-string without end-string. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/drift.py:docstring of giskard.testing.tests.drift.test_drift_psi:23: ERROR: Unexpected indentation. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/drift.py:docstring of giskard.testing.tests.drift.test_drift_psi:24: WARNING: Block quote ends without a blank line; unexpected unindent. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/drift.py:docstring of giskard.testing.tests.drift.test_drift_chi_square:25: ERROR: Unexpected indentation. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/drift.py:docstring of giskard.testing.tests.drift.test_drift_chi_square:26: WARNING: Block quote ends without a blank line; unexpected unindent. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/drift.py:docstring of giskard.testing.tests.drift.test_drift_ks:21: ERROR: Unexpected indentation. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/drift.py:docstring of giskard.testing.tests.drift.test_drift_ks:22: WARNING: Block quote ends without a blank line; unexpected unindent. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/drift.py:docstring of giskard.testing.tests.drift.test_drift_earth_movers_distance:21: ERROR: Unexpected indentation. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/drift.py:docstring of giskard.testing.tests.drift.test_drift_earth_movers_distance:22: WARNING: Block quote ends without a blank line; unexpected unindent. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/drift.py:docstring of giskard.testing.tests.drift.test_drift_prediction_psi:24: ERROR: Unexpected indentation. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/drift.py:docstring of giskard.testing.tests.drift.test_drift_prediction_psi:25: WARNING: Block quote ends without a blank line; unexpected unindent. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/drift.py:docstring of giskard.testing.tests.drift.test_drift_prediction_chi_square:26: ERROR: Unexpected indentation. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/drift.py:docstring of giskard.testing.tests.drift.test_drift_prediction_chi_square:27: WARNING: Block quote ends without a blank line; unexpected unindent. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/drift.py:docstring of giskard.testing.tests.drift.test_drift_prediction_ks:23: ERROR: Unexpected indentation. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/drift.py:docstring of giskard.testing.tests.drift.test_drift_prediction_ks:24: WARNING: Block quote ends without a blank line; unexpected unindent. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/drift.py:docstring of giskard.testing.tests.drift.test_drift_prediction_earth_movers_distance:27: ERROR: Unexpected indentation. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/metamorphic.py:docstring of giskard.testing.tests.metamorphic.test_metamorphic_invariance:29: ERROR: Unexpected indentation. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/metamorphic.py:docstring of giskard.testing.tests.metamorphic.test_metamorphic_invariance:30: WARNING: Block quote ends without a blank line; unexpected unindent. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/metamorphic.py:docstring of giskard.testing.tests.metamorphic.test_metamorphic_increasing:27: ERROR: Unexpected indentation. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/metamorphic.py:docstring of giskard.testing.tests.metamorphic.test_metamorphic_increasing:28: WARNING: Block quote ends without a blank line; unexpected unindent. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/metamorphic.py:docstring of giskard.testing.tests.metamorphic.test_metamorphic_decreasing:29: ERROR: Unexpected indentation. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/metamorphic.py:docstring of giskard.testing.tests.metamorphic.test_metamorphic_decreasing:30: WARNING: Block quote ends without a blank line; unexpected unindent. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/metamorphic.py:docstring of giskard.testing.tests.metamorphic.test_metamorphic_decreasing_t_test:24: ERROR: Unexpected indentation. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/metamorphic.py:docstring of giskard.testing.tests.metamorphic.test_metamorphic_decreasing_t_test:25: WARNING: Block quote ends without a blank line; unexpected unindent. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/metamorphic.py:docstring of giskard.testing.tests.metamorphic.test_metamorphic_increasing_t_test:24: ERROR: Unexpected indentation. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/metamorphic.py:docstring of giskard.testing.tests.metamorphic.test_metamorphic_increasing_t_test:25: WARNING: Block quote ends without a blank line; unexpected unindent. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/metamorphic.py:docstring of giskard.testing.tests.metamorphic.test_metamorphic_invariance_t_test:7: ERROR: Unexpected indentation. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/metamorphic.py:docstring of giskard.testing.tests.metamorphic.test_metamorphic_invariance_t_test:29: ERROR: Unexpected indentation. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/metamorphic.py:docstring of giskard.testing.tests.metamorphic.test_metamorphic_invariance_t_test:30: WARNING: Block quote ends without a blank line; unexpected unindent. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/metamorphic.py:docstring of giskard.testing.tests.metamorphic.test_metamorphic_decreasing_wilcoxon:24: ERROR: Unexpected indentation. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/metamorphic.py:docstring of giskard.testing.tests.metamorphic.test_metamorphic_decreasing_wilcoxon:25: WARNING: Block quote ends without a blank line; unexpected unindent. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/metamorphic.py:docstring of giskard.testing.tests.metamorphic.test_metamorphic_increasing_wilcoxon:24: ERROR: Unexpected indentation. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/metamorphic.py:docstring of giskard.testing.tests.metamorphic.test_metamorphic_increasing_wilcoxon:25: WARNING: Block quote ends without a blank line; unexpected unindent. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/metamorphic.py:docstring of giskard.testing.tests.metamorphic.test_metamorphic_invariance_wilcoxon:29: ERROR: Unexpected indentation. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/metamorphic.py:docstring of giskard.testing.tests.metamorphic.test_metamorphic_invariance_wilcoxon:30: WARNING: Block quote ends without a blank line; unexpected unindent. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/performance.py:docstring of giskard.testing.tests.performance.test_mae:19: ERROR: Unexpected indentation. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/performance.py:docstring of giskard.testing.tests.performance.test_mae:20: WARNING: Block quote ends without a blank line; unexpected unindent. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/performance.py:docstring of giskard.testing.tests.performance.test_rmse:19: ERROR: Unexpected indentation. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/performance.py:docstring of giskard.testing.tests.performance.test_rmse:20: WARNING: Block quote ends without a blank line; unexpected unindent. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/performance.py:docstring of giskard.testing.tests.performance.test_recall:17: ERROR: Unexpected indentation. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/performance.py:docstring of giskard.testing.tests.performance.test_recall:18: WARNING: Block quote ends without a blank line; unexpected unindent. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/performance.py:docstring of giskard.testing.tests.performance.test_auc:17: ERROR: Unexpected indentation. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/performance.py:docstring of giskard.testing.tests.performance.test_auc:18: WARNING: Block quote ends without a blank line; unexpected unindent. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/performance.py:docstring of giskard.testing.tests.performance.test_accuracy:17: ERROR: Unexpected indentation. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/performance.py:docstring of giskard.testing.tests.performance.test_accuracy:18: WARNING: Block quote ends without a blank line; unexpected unindent. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/performance.py:docstring of giskard.testing.tests.performance.test_precision:17: ERROR: Unexpected indentation. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/performance.py:docstring of giskard.testing.tests.performance.test_precision:18: WARNING: Block quote ends without a blank line; unexpected unindent. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/performance.py:docstring of giskard.testing.tests.performance.test_f1:17: ERROR: Unexpected indentation. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/performance.py:docstring of giskard.testing.tests.performance.test_f1:18: WARNING: Block quote ends without a blank line; unexpected unindent. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/performance.py:docstring of giskard.testing.tests.performance.test_r2:19: ERROR: Unexpected indentation. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/performance.py:docstring of giskard.testing.tests.performance.test_r2:20: WARNING: Block quote ends without a blank line; unexpected unindent. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/performance.py:docstring of giskard.testing.tests.performance.test_diff_recall:22: ERROR: Unexpected indentation. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/performance.py:docstring of giskard.testing.tests.performance.test_diff_recall:23: WARNING: Block quote ends without a blank line; unexpected unindent. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/performance.py:docstring of giskard.testing.tests.performance.test_diff_accuracy:22: ERROR: Unexpected indentation. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/performance.py:docstring of giskard.testing.tests.performance.test_diff_accuracy:23: WARNING: Block quote ends without a blank line; unexpected unindent. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/performance.py:docstring of giskard.testing.tests.performance.test_diff_precision:22: ERROR: Unexpected indentation. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/performance.py:docstring of giskard.testing.tests.performance.test_diff_precision:23: WARNING: Block quote ends without a blank line; unexpected unindent. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/performance.py:docstring of giskard.testing.tests.performance.test_diff_rmse:24: ERROR: Unexpected indentation. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/performance.py:docstring of giskard.testing.tests.performance.test_diff_rmse:25: WARNING: Block quote ends without a blank line; unexpected unindent. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/performance.py:docstring of giskard.testing.tests.performance.test_diff_f1:22: ERROR: Unexpected indentation. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/performance.py:docstring of giskard.testing.tests.performance.test_diff_f1:23: WARNING: Block quote ends without a blank line; unexpected unindent. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/statistic.py:docstring of giskard.testing.tests.statistic.test_right_label:23: ERROR: Unexpected indentation. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/statistic.py:docstring of giskard.testing.tests.statistic.test_right_label:24: WARNING: Block quote ends without a blank line; unexpected unindent. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/statistic.py:docstring of giskard.testing.tests.statistic.test_output_in_range:35: ERROR: Unexpected indentation. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/statistic.py:docstring of giskard.testing.tests.statistic.test_output_in_range:36: WARNING: Block quote ends without a blank line; unexpected unindent. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/statistic.py:docstring of giskard.testing.tests.statistic.test_disparate_impact:40: ERROR: Unexpected indentation. looking for now-outdated files... none found pickling environment... done checking consistency... /Users/rak/Documents/giskard_repos/giskard_main/docs/getting_started/quickstart/_quickstart_llm.ipynb: WARNING: document isn't included in any toctree done preparing documents... done copying assets... copying static files... done copying extra files... done done writing output... [100%] tutorials/tabular_tutorials/index /Users/rak/Documents/giskard_repos/giskard_main/docs/reference/notebooks/LLM_Description_Product.ipynb:: WARNING: Lexing literal_block '!pip install "giskard[llm]" --upgrade' as "python" resulted in an error at token: '!'. Retrying in relaxed mode. /Users/rak/Documents/giskard_repos/giskard_main/docs/reference/notebooks/LLM_QA_IPCC.ipynb:: WARNING: Lexing literal_block '!pip install "giskard[llm]" --upgrade' as "python" resulted in an error at token: '!'. Retrying in relaxed mode. /Users/rak/Documents/giskard_repos/giskard_main/docs/reference/notebooks/LLM_QA_IPCC.ipynb:: WARNING: Lexing literal_block '!pip install "langchain<=0.0.301" "pypdf<=3.17.0" "faiss-cpu<=1.7.4" "openai<=0.28.1" "tiktoken<=0.5.1"' as "python" resulted in an error at token: '!'. Retrying in relaxed mode. /Users/rak/Documents/giskard_repos/giskard_main/giskard/testing/tests/llm/hallucination.py:docstring of giskard.testing.tests.llm.hallucination.test_llm_output_coherency:11: WARNING: undefined label: 'coherencyevaluator'

I'll help with this when I get the time.

@Hartorn
Copy link
Member Author

Hartorn commented Nov 22, 2023

Good to see it's working:
image

@rabah-khalek
Copy link
Contributor

but it's not failing still on warnings right @Hartorn ?

@Hartorn
Copy link
Member Author

Hartorn commented Nov 22, 2023

but it's not failing still on warnings right @Hartorn ?

No I removed it here 684c7e3

@Hartorn Hartorn marked this pull request as ready for review November 22, 2023 18:00
@Hartorn Hartorn requested a review from a team November 22, 2023 18:00
Copy link
Contributor

@rabah-khalek rabah-khalek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Phenomenal job @Hartorn! Sorry for the late review, we should have this merged asap.

Just left you two comments, if you have time to look at, if not, we can fix them later.

## Building the docs locally (continuously)
```
cd giskard
pdm run watch-doc
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why remove it? let's just correct it pdm watch-doc

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's inside the doc folder source, and so it's watched by sphinx

Comment on lines -117 to -121
:::::{grid-item}
:child-align:
::::{grid} 1 1 1 1
:child-align:
:gutter: 1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one of these child-align was responsible for better alignment of cells.

our public doc:
Screenshot 2023-11-24 at 11 54 55

doc in this pr:
Screenshot 2023-11-24 at 11 56 51

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one of these child-align was responsible for better alignment of cells.

our public doc: Screenshot 2023-11-24 at 11 54 55

doc in this pr: Screenshot 2023-11-24 at 11 56 51

there were mistakes in it. Plus we should not nest grid. If it's only the order, I can change it back or find some stuff

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why we should not nest grid? I used the following doc: https://sphinx-design.readthedocs.io/en/furo-theme/grids.html#nesting-grids
Yes I think the order is important here, can we please revert this back?

@rabah-khalek rabah-khalek changed the title Feature/gsk 2172 improve documentation ci Feature/gsk 2172 improve documentation ci [GSK-2186] Nov 27, 2023
@linear
Copy link

linear bot commented Nov 27, 2023

@sonarqubecloud
Copy link

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

100.0% 100.0% Coverage
0.0% 0.0% Duplication

@Hartorn Hartorn merged commit ef802a2 into main Nov 27, 2023
@Hartorn Hartorn deleted the feature/gsk-2172-improve-documentation-ci branch November 27, 2023 14:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

4 participants