docs: merge os and hub documentation #68
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Updated .gitignore to include new-docs/oss/sdk/reference and commented out .idea directory. - Updated poetry.lock to reflect changes in Poetry version and removed unnecessary groups from dependencies. - Added sphinx-tabs to pyproject.toml for enhanced documentation features. - Deleted obsolete script-docs files including configuration, index, and various guides related to documentation. - Removed unused static assets and images from script-docs. - Cleaned up the documentation structure to streamline the user experience.
- Updated Poetry version from 2.1.3 to 1.8.5. - Removed unnecessary groups from several dependencies. - Added new package 'greenlet' version 3.2.4 and 'sqlalchemy' version 2.0.43. - Adjusted extras for several packages to streamline dependency requirements.
- Updated 'anyio' to version 4.10.0 with enhanced description. - Upgraded 'argon2-cffi-bindings' to version 25.1.0, adjusting Python version compatibility and dependencies. - Updated 'astroid' to version 3.3.11. - Upgraded 'certifi' to version 2025.8.3. - Added new package 'cffi' version 2.0.0b1. - Updated 'starlette' to version 0.47.2. - Upgraded 'tornado' to version 6.5.2. - Updated 'types-python-dateutil' to version 2.9.0.20250809.
- Added a new card for Deeplearning.AI's red teaming course in the index. - Renamed sections in the notebooks and SDK documentation for clarity. - Updated links and descriptions for existing resources. - Removed outdated documentation on legacy features. - Improved formatting and consistency across various sections.
…ocess - Added a link to the Giskard GitHub repository in the index. - Enhanced the business failures section with tips and notes for clarity. - Improved the structure of the RAGET documentation, including clearer instructions for generating test sets and evaluating RAG systems. - Added examples for loading and saving test sets and reports. - Included troubleshooting tips for common issues in business failure testing.
- Added new packages: 'aiohappyeyeballs' (2.6.1), 'aiosignal' (1.4.0), 'annotated-types' (0.7.0), and 'zstandard' (0.24.0). - Updated existing dependencies in 'pyproject.toml' for 'giskard', 'pyarrow', and 'ragas'. - Increased the max depth of the Open Source SDK section in the documentation for better navigation. - Improved the import structure in the business documentation to include additional question generators for test set generation.
- Introduced ThemeOptions in the Sphinx configuration for improved theme customization. - Updated references from "Open Source SDK" to "Open Source Library" for consistency. - Clarified the evaluation process in the documentation, emphasizing the importance of systematic evaluation runs. - Expanded the "Manage Projects" section to include model management, detailing creation, retrieval, updating, and deletion of models. - Removed outdated notes on synthetic data generation processes for business and security datasets to streamline content.
- Enhanced Sphinx configuration by adding `html_permalinks_icon` and resolving Dataset cross-reference ambiguity with `autodoc_type_aliases`. - Improved formatting and clarity in various notebooks, including consistent references to the "Open Source Library". - Updated links in documentation to point to the correct SDK resources. - Refactored code in notebooks for better readability and organization, including breaking long lines and improving indentation. - Minor adjustments to documentation structure for better navigation and consistency.
- Introduced demo.py to create a LangChain LLMChain with Giskard integration for generating test suites. - Expanded documentation on dataset import and generation processes, including detailed examples for creating knowledge bases and importing datasets from various formats. - Improved clarity in the adversarial tests generation section and added code snippets for better understanding. - Updated the structure of the datasets documentation for consistency and ease of navigation.
- Corrected the spelling of "denial" in multiple sections to enhance readability. - Updated code examples to ensure consistency and clarity in the usage of the HubClient and dataset generation methods. - Improved phrasing in the documentation to provide clearer instructions and descriptions for users.
- Deleted the entire old-docs directory, including configuration files, index files, and various guides. - Removed obsolete static assets and images to streamline the documentation structure. - Cleaned up the repository by eliminating outdated content, ensuring a more organized and user-friendly experience.
- Introduced new documentation files including configuration, index, and various guides for the Giskard Hub. - Added a .gitignore file to exclude build artifacts and README.md from version control. - Created a launch script for Sphinx documentation server to facilitate local development. - Included CSS and image assets to enhance the visual presentation of the documentation. - Expanded SDK documentation with sections on managing checks, evaluations, projects, and datasets, providing comprehensive guidance for users.
Member
|
hey @davidberenstein1957, I've seen there a lot of warnings when running |
Member
|
I also see plenty of extra arrows everywhere |
- Updated README, SDK, and various documentation files to reflect the new terminology, replacing "models" with "agents" for consistency. - Enhanced code examples and explanations to guide users in transitioning to the new API structure. - Improved clarity in sections related to project and agent management, evaluation processes, and synthetic test generation. - Added deprecation warnings in the SDK to inform users about the transition from "models" to "agents".
- Updated the access rights documentation to clarify user and group-level permissions, including new tabs for better organization. - Renamed sections for consistency, changing "Run Evaluations" to "Run and Schedule Evaluations" across relevant files. - Added new images to illustrate access settings for groups and users. - Introduced a new section for comparing evaluations to help users identify regressions in model performance. - Removed outdated content related to manual tests for consistency across models.
…arifications - Added notes emphasizing the importance of ongoing testing for business failures and security vulnerabilities. - Updated references to Giskard Hub guides for more comprehensive testing and monitoring capabilities. - Improved clarity in the business documentation regarding the need for regular updates to test suites. - Adjusted phrasing in the security documentation to reinforce the continuous nature of security testing.
- Upgraded the `cffi` package to version 2.0.0b1 and added new dependencies for `ipywidgets` version 8.1.7, `jupyterlab-widgets` version 3.0.15, and `widgetsnbextension` version 4.0.14 in the `pyproject.toml` and `poetry.lock` files. - Improved documentation clarity by correcting phrasing and enhancing links in various sections, including the Giskard Hub and security documentation. - Standardized formatting in documentation files for better readability and consistency.
- Improved the section on running evaluations by consolidating information and clarifying the distinction between remote and local evaluations. - Added a new section for scheduling evaluations, indicating ongoing work in this area. - Standardized phrasing and formatting for better readability and consistency throughout the document.
- Corrected typos and improved phrasing in multiple documentation files, enhancing readability and precision. - Updated descriptions to ensure consistent terminology, particularly regarding the Giskard Hub and its features. - Added tests for the HubClient to verify the consistency and behavior of agents and models, ensuring accurate functionality in the SDK.
- Upgraded cffi to version 2.0.0b1, removing the previous version 1.17.1 and its associated files. - Added new packages: pandoc (version 2.4) and its dependencies (plumbum and ply) to the project. - Updated the pyproject.toml to include pandoc as a dependency, ensuring compatibility with the latest version.
Member
Author
I agree, but this is related to warnings and not functional errors. RST not playing together nicely with the internal rendered reports in the jupyter notebooks in the OSS. There are no functional errors so I don't think we can avoid trying to resolve this as it will likely take a lot of time for the legacy OSS docs. |
- Changed the hub_url in the HubClient instantiation from HTTP to HTTPS in test cases to ensure secure connections. - Updated test cases to reflect the new secure URLs for consistency across client instances.
- Included a link to the Giskard Hub GitHub repository in the index.rst file to enhance accessibility and provide users with direct access to the source code.
- Renamed references in the documentation from "Enterprise UI," "Enterprise SDK," and "Open Source Library" to "Giskard Hub UI," "Giskard Hub SDK," and "Giskard Open Source," respectively, to ensure consistent branding and clarity throughout the documentation files.
- Revised the link text in the index.rst file to specify "Open Collective funding page for Phare," enhancing clarity and directing users to the correct resource for supporting research.
- Removed the extraneous arrow symbol from the RealPerformance grid item card in the index.rst file to improve formatting and consistency within the documentation.
- Included a step to install Pandoc in the GitHub Actions workflow for documentation updates, ensuring compatibility with NBSphinx requirements and enhancing the documentation generation process.
- Changed the installation command from "!pip" to "%pip" in LLM_QA_Google.ipynb and LLM_QA_Winter_Olympics.ipynb to ensure compatibility with Jupyter notebook environments.
- Included a step to install Pandoc in the CI workflow to meet NBSphinx requirements for Jupyter notebook processing, ensuring proper documentation generation.
alexcombessie
approved these changes
Aug 23, 2025
Member
alexcombessie
left a comment
alexcombessie
left a comment
There was a problem hiding this comment.
LGTM on the business copy! Just two small comments, but nothing blocking. Good job!
- Enhanced Open Graph metadata across multiple documentation files to improve clarity and consistency, including updated titles and descriptions for various sections related to LLM agent testing, evaluations, and dataset management. - Ensured that descriptions accurately reflect the capabilities and features of the Giskard Hub and Open Source SDK, enhancing user understanding and navigation.
- Changed all references from "docs-hub.giskard.ai" to "docs.giskard.ai" in the Makefile, pyproject.toml, and README.md to reflect the new documentation URL, ensuring consistency across the project.
henchaves
reviewed
Aug 26, 2025
henchaves
reviewed
Aug 26, 2025
henchaves
requested changes
Aug 26, 2025
Member
There was a problem hiding this comment.
Hey @davidberenstein1957, I have a few remarks:
-
It's missing "Metadata" check in some sections, and the new "Semantic Similarity" check in all pages about checks/annotation. Since it will be a lot of changes, it's up to you if we should do it in that branch or in a separate PR.
-
There is a problem with text color (in dark mode) when tabs are used (e.g. Set Access Rights page)
- The
AgentsResourceshould not be added to this PR. Let's rollback tohub.modelsand so on in the SDK part. Normally, only technical people will interact with the SDK, so we can just add a note saying that model nomenclature is the same as agent. Even though we have tests, it has introduced some inconsistencies in the code, for example:
- Do we really want to add these sections in SDK part that we use
hub.datasets.generate? I would prefer to comment them here and uncomment on #64. Same for Knowledge Base sections in SDK
… structure - Updated references in the documentation from "models" to "agents" to reflect the new terminology, ensuring consistency across all related files. - Enhanced various documentation sections, including README.md, SDK references, and user guides, to improve clarity and user understanding of the Giskard Hub and Open Source SDK functionalities. - Removed deprecated code related to the "models" attribute in the HubClient class, streamlining the client interface. - Added new CSS styles for tabbed content in documentation to improve user experience and navigation.
- Added detailed descriptions and examples for the new semantic similarity check across multiple documentation files, including README.md, checks.rst, and various UI sections. - Updated existing checks documentation to include metadata and semantic similarity, ensuring comprehensive coverage of evaluation criteria. - Improved clarity and consistency in the documentation regarding the parameters and usage of checks, enhancing user understanding of the Giskard Hub's capabilities.
Member
Author
|
@henchaves thank you for the review. I've processed all your comments. Before merging, let's merge https://github.com/Giskard-AI/giskard-hub/pull/64/files, #71 and https://github.com/Giskard-AI/llm-hub/pull/908, update those features and then explore a second iteration with updates on Lidar + CRT when they are fully ready. |
…te (#71) * Init restyling with the new brand guidelines * Enhance styling in custom.css - WIP#1 * Enhance styling in custom.css + shorten header item's names WIP#2 * Enhance styling in custom.css + add proper open-graph image * Enhance styling in custom.css - WIP#3 * Set sidebar auto-scroll at navigation * Update sidebar functionality and enhance styling * Fix toc current item highlight * Clean up custom.css * Improve search and text highlight render --------- Co-authored-by: lansolo99 <[email protected]>
- Clarified that the Giskard Hub SDK currently does not support the generation of synthetic test cases for business failures and security vulnerabilities. - Updated relevant sections in `business.rst` and `security.rst` to direct users to the Giskard Hub UI for generating these test cases instead.
henchaves
approved these changes
Aug 27, 2025
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.