🐛 Handle editable pip installs#2731
Merged
laurentsimon merged 12 commits intoossf:mainfrom Mar 13, 2023
Merged
Conversation
Codecov Report
Additional details and impacted files@@ Coverage Diff @@
## main #2731 +/- ##
==========================================
+ Coverage 49.78% 49.91% +0.13%
==========================================
Files 156 156
Lines 11638 11669 +31
==========================================
+ Hits 5794 5825 +31
Misses 5490 5490
Partials 354 354 |
laurentsimon
reviewed
Mar 9, 2023
laurentsimon
approved these changes
Mar 10, 2023
Contributor
|
Please rebase and we'll be able to merge. |
Editable pip installs (-e) should be considered secure if the package is installed from a local source or a remote source (VCS install) but pinned by commit hash. To keep the behaviour we have for normal pip installs, we need to guarantee the package dependencies are pinned by hash too. For normal pip installs, we verify that by using --require-hashes flag. Unfortunately, --require-hashes flag is not compatible with editable installs, so we use --no-deps flag to verify the dependencies are not installed since we can't verify if they are pinned. Signed-off-by: Gabriela Gutierrez <[email protected]>
Signed-off-by: Gabriela Gutierrez <[email protected]>
Signed-off-by: Gabriela Gutierrez <[email protected]>
Signed-off-by: Gabriela Gutierrez <[email protected]>
Signed-off-by: Gabriela Gutierrez <[email protected]>
Signed-off-by: Gabriela Gutierrez <[email protected]>
Signed-off-by: Gabriela Gutierrez <[email protected]>
Signed-off-by: Gabriela Gutierrez <[email protected]>
Signed-off-by: Gabriela Gutierrez <[email protected]>
Signed-off-by: Gabriela Gutierrez <[email protected]>
Signed-off-by: Gabriela Gutierrez <[email protected]>
auto-merge was automatically disabled
March 13, 2023 13:21
Head branch was pushed to by a user without write access
6eb976d to
6a2c051
Compare
Contributor
Author
|
rebased |
azeemshaikh38
added a commit
to azeemshaikh38/scorecard
that referenced
this pull request
Mar 14, 2023
commit 00da7a8be965c09d044f978d6b9eafee1350bd30 Author: Azeem Shaikh <[email protected]> Date: Tue Mar 14 23:07:19 2023 +0000 Pr comments commit 1127dd9 Merge: 274448f 23bd295 Author: Azeem Shaikh <[email protected]> Date: Wed Mar 15 04:23:32 2023 +0530 Merge branch 'main' into go-git commit 274448f Author: Azeem Shaikh <[email protected]> Date: Tue Mar 14 22:52:30 2023 +0000 Initial implementation of go-git client Signed-off-by: Azeem Shaikh <[email protected]> commit 23bd295 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue Mar 14 20:28:41 2023 +0000 :seedling: Bump github/codeql-action from 2.2.4 to 2.2.6 (ossf#2741) commit fc026ef Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue Mar 14 17:04:31 2023 +0000 :seedling: Bump github.com/google/ko from 0.12.0 to 0.13.0 in /tools (ossf#2742) commit 2e04214 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue Mar 14 14:02:34 2023 +0000 :seedling: Bump tj-actions/changed-files from 35.6.2 to 35.7.0 Bumps [tj-actions/changed-files](https://github.com/tj-actions/changed-files) from 35.6.2 to 35.7.0. - [Release notes](https://github.com/tj-actions/changed-files/releases) - [Changelog](https://github.com/tj-actions/changed-files/blob/main/HISTORY.md) - [Commits](tj-actions/changed-files@5ce975c...bd376fb) --- updated-dependencies: - dependency-name: tj-actions/changed-files dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> commit e36b590 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue Mar 14 08:59:20 2023 -0500 :seedling: Bump actions/cache from 3.3.0 to 3.3.1 (ossf#2740) Bumps [actions/cache](https://github.com/actions/cache) from 3.3.0 to 3.3.1. - [Release notes](https://github.com/actions/cache/releases) - [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md) - [Commits](actions/cache@940f3d7...88522ab) --- updated-dependencies: - dependency-name: actions/cache dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 6ff94eb Author: Gabriela Gutierrez <[email protected]> Date: Mon Mar 13 19:42:37 2023 +0000 :bug: Handle editable pip installs (ossf#2731) * fix: Handle editable pip install Editable pip installs (-e) should be considered secure if the package is installed from a local source or a remote source (VCS install) but pinned by commit hash. To keep the behaviour we have for normal pip installs, we need to guarantee the package dependencies are pinned by hash too. For normal pip installs, we verify that by using --require-hashes flag. Unfortunately, --require-hashes flag is not compatible with editable installs, so we use --no-deps flag to verify the dependencies are not installed since we can't verify if they are pinned. Signed-off-by: Gabriela Gutierrez <[email protected]> * test: Editable pip install in GHA Signed-off-by: Gabriela Gutierrez <[email protected]> * test: Editable pip install in Dockerfile Signed-off-by: Gabriela Gutierrez <[email protected]> * test: Editable pip install in shell script Signed-off-by: Gabriela Gutierrez <[email protected]> * fix: Code complexity increase Signed-off-by: Gabriela Gutierrez <[email protected]> * fix: Simplify boolean return Signed-off-by: Gabriela Gutierrez <[email protected]> * docs: Add pip editable install references in comments Signed-off-by: Gabriela Gutierrez <[email protected]> * fix: Handle multiple packages in editable pip install Signed-off-by: Gabriela Gutierrez <[email protected]> * test: Multi editable pip install in GHA Signed-off-by: Gabriela Gutierrez <[email protected]> * test: Multi editable pip install in Dockerfile Signed-off-by: Gabriela Gutierrez <[email protected]> * test: Multi editable pip install in shell script Signed-off-by: Gabriela Gutierrez <[email protected]> --------- Signed-off-by: Gabriela Gutierrez <[email protected]> Co-authored-by: laurentsimon <[email protected]> commit 110e352 Author: raghavkaul <[email protected]> Date: Mon Mar 13 11:13:50 2023 -0400 ✨ Gitlab support: RepoClient (ossf#2655) * Add make targets and E2E test target for GitLab only Signed-off-by: Raghav Kaul <[email protected]> * Add GitLab support to RepoClient Signed-off-by: Raghav Kaul <[email protected]> * Build * Make target for e2e-gitlab-token * Only run Gitlab tests in CI that don't require a token Signed-off-by: Raghav Kaul <[email protected]> * Add tests Signed-off-by: Raghav Kaul <[email protected]> * Remove spurious printf Signed-off-by: Raghav Kaul <[email protected]> * 🐛 Check OSS Fuzz build file for Fuzzing check (ossf#2719) * Check OSS-Fuzz using project list Signed-off-by: Spencer Schrock <[email protected]> * Use clients.RepoClient interface to perform the new OSS Fuzz check Signed-off-by: Spencer Schrock <[email protected]> * wip: add eager client for better repeated lookup of projects Signed-off-by: Spencer Schrock <[email protected]> * Split lazy and eager behavior into different implementations. Signed-off-by: Spencer Schrock <[email protected]> * Add tests and benchmarks Signed-off-by: Spencer Schrock <[email protected]> * Switch to always parsing JSON to determine if a project is present. The other approach of looking for a substring match would lead to false positives. Signed-off-by: Spencer Schrock <[email protected]> * Add eager constructor to surface status file errors sooner. Signed-off-by: Spencer Schrock <[email protected]> * Switch existing users to new OSS Fuzz client Signed-off-by: Spencer Schrock <[email protected]> * Mark old method as deprecated in the godoc Signed-off-by: Spencer Schrock <[email protected]> * remove unused comment. Signed-off-by: Spencer Schrock <[email protected]> * Use new OSS Fuzz client in e2e test. Signed-off-by: Spencer Schrock <[email protected]> * fix typo. Signed-off-by: Spencer Schrock <[email protected]> * Fix potential path bug with test server. Signed-off-by: Spencer Schrock <[email protected]> * Force include the two JSON files which were being ignored by .gitignore Signed-off-by: Spencer Schrock <[email protected]> * trim the status json file Signed-off-by: Spencer Schrock <[email protected]> --------- Signed-off-by: Spencer Schrock <[email protected]> --------- Signed-off-by: Raghav Kaul <[email protected]> Signed-off-by: Spencer Schrock <[email protected]> Co-authored-by: Spencer Schrock <[email protected]> commit 5625dda Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Sat Mar 11 17:14:42 2023 +0000 :seedling: Bump github.com/onsi/ginkgo/v2 from 2.8.3 to 2.9.0 in /tools Bumps [github.com/onsi/ginkgo/v2](https://github.com/onsi/ginkgo) from 2.8.3 to 2.9.0. - [Release notes](https://github.com/onsi/ginkgo/releases) - [Changelog](https://github.com/onsi/ginkgo/blob/master/CHANGELOG.md) - [Commits](onsi/ginkgo@v2.8.3...v2.9.0) --- updated-dependencies: - dependency-name: github.com/onsi/ginkgo/v2 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> commit d591e38 Author: Spencer Schrock <[email protected]> Date: Fri Mar 10 16:02:05 2023 -0800 🌱 Add RepoClient re-use E2E tests. (ossf#2625) * Add e2e test for re-used repoclient. Signed-off-by: Spencer Schrock <[email protected]> * Improve diff logging Signed-off-by: Spencer Schrock <[email protected]> * Skip scorecard e2e test during unit tests. Signed-off-by: Spencer Schrock <[email protected]> * Fix linter. Signed-off-by: Spencer Schrock <[email protected]> --------- Signed-off-by: Spencer Schrock <[email protected]> commit a7e81bb Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Fri Mar 10 08:20:28 2023 -0600 :seedling: Bump actions/cache from 3.2.6 to 3.3.0 (ossf#2738) Bumps [actions/cache](https://github.com/actions/cache) from 3.2.6 to 3.3.0. - [Release notes](https://github.com/actions/cache/releases) - [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md) - [Commits](actions/cache@69d9d44...940f3d7) --- updated-dependencies: - dependency-name: actions/cache dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What kind of change does this PR introduce?
Handle editable pip installs in Pinned-Dependencies check.
Description
When performing
pip install -e <pkg>, the install can be considered secure if the package and it's dependencies are hash-pinned. The package can come from a local or remote source (VCS install). If the package is local, it is considered secure. If the package is remote, it needs to pinned by commit hash to be secure. We have no way to identify if the dependencies installed from requirements are hash-pinned. The flag--require-hashesdoes not work with-eflag. To guarantee we are not installing malicious dependencies, we can use--no-depsflag to not install the dependencies. That way the install can be considered secure.Examples
References
(Is it a bug fix, feature, docs update, something else?)
What is the current behavior?
Currently, we don't handle editable pip installs. We suggest using
--require-hashesas remediation but it's not possible.What is the new behavior (if this is a feature change)?**
Which issue(s) this PR fixes
Closes #2228
Special notes for your reviewer
Does this PR introduce a user-facing change?
For user-facing changes, please add a concise, human-readable release note to
the
release-note(In particular, describe what changes users might need to make in their
application as a result of this pull request.)