Skip to content

add helper script to download the needed files to mirror for UV_PYTHON_INSTALL_MIRROR#8548

Merged
charliermarsh merged 5 commits intoastral-sh:mainfrom
MeitarR:add-mirror-script-for-python
Dec 4, 2024
Merged

add helper script to download the needed files to mirror for UV_PYTHON_INSTALL_MIRROR#8548
charliermarsh merged 5 commits intoastral-sh:mainfrom
MeitarR:add-mirror-script-for-python

Conversation

@MeitarR
Copy link
Contributor

@MeitarR MeitarR commented Oct 24, 2024

Summary

I added crates/uv-python/create-mirror.py to make it easy to download all the needed files to create a mirror for Python distributions in an offline environment.

the script also has an option to iterate over the git history of the download-metadata.json to make sure we have all the files needed for all the uv versions

Test Plan

uv run create-mirror.py --from-all-history --os linux --arch x86_64 --name cpython
2024-10-25 01:31:12,973 - INFO - Starting download of 466 files.
Downloading: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 466/466 [06:11<00:00,  1.26file/s]
Successfully downloaded: 466
now you can run UV_PYTHON_INSTALL_MIRROR='file:///home/meitar/dev/uv/crates/uv-python/mirror' uv python install

then checked (the unshare command make sure that the process don't have any netwok)

UV_PYTHON_INSTALL_MIRROR=file:///home/meitar/dev/uv/crates/uv-python/mirror sudo -E unshare -n /home/meitar/.local/bin/uv python install 3.13
Searching for Python versions matching: Python 3.13
Installed Python 3.13.0 in 2.91s
 + cpython-3.13.0-linux-x86_64-gnu

@zanieb
Copy link
Member

zanieb commented Oct 25, 2024

Cool! This may be better hosted outside this repository since it's just a Python script. I'll take a closer look though.

Related #8062

@zanieb zanieb self-assigned this Oct 25, 2024
@MeitarR
Copy link
Contributor Author

MeitarR commented Oct 25, 2024

Cool! This may be better hosted outside this repository since it's just a Python script. I'll take a closer look though.

Related #8062

Yes, I thought about it, but decided that it would be better to keep it close to the JSON for a couple of reasons:

  1. Simplicity – It removes the need to specify the path to the JSON file and the repo containing the full history.
  2. Coupling with JSON format – If the format of the JSON changes, having the script nearby ensures we can quickly update it to stay compatible.
  3. Discoverability – When someone looks into how to create a mirror, they’ll naturally find this script since it contains relevant strings like UV_PYTHON_INSTALL_MIRROR that they'll likely search for.

@MeitarR
Copy link
Contributor Author

MeitarR commented Nov 13, 2024

So, What do you think @zanieb ?

@MeitarR
Copy link
Contributor Author

MeitarR commented Nov 26, 2024

@charliermarsh

@zanieb
Copy link
Member

zanieb commented Nov 26, 2024

After looking more closely, I feel the same way — I don't want to maintain this and think it makes more sense as an external script. I think @charliermarsh is partial to merging it. I won't stand in the way of that, but since we're not testing this I wouldn't be surprised if it becomes outdated.

@MeitarR
Copy link
Contributor Author

MeitarR commented Nov 26, 2024

So we are waiting for @charliermarsh decision?

@charliermarsh charliermarsh force-pushed the add-mirror-script-for-python branch from 4bb9aaf to c271230 Compare December 4, 2024 01:24
@charliermarsh charliermarsh added the internal A refactor or improvement that is not user-facing label Dec 4, 2024
@charliermarsh charliermarsh enabled auto-merge (squash) December 4, 2024 01:32
@charliermarsh charliermarsh merged commit d2693da into astral-sh:main Dec 4, 2024
@MeitarR
Copy link
Contributor Author

MeitarR commented Dec 4, 2024

Thank you for merging ♥️

@jeffcarrico
Copy link

Thank you for contributing this script, I was about to set out to try to automate exactly this so it saved me a lot of time and worked perfectly.

@MeitarR
Copy link
Contributor Author

MeitarR commented Jan 4, 2025

@jeffcarrico
this issue might also interest you #10203

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

internal A refactor or improvement that is not user-facing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants