How to install Github packages from Dockerfile

[evanerwee]

Select Topic Area

Question

Body

After struggling for a day (Gitlab took minutes), I realized there is an undocumented secret getting releases downloaded (Private) and installed into a Docker container.

What is the command?

It is not: curl -L -H "Authorization: Bearer " https://github.com/RW-Lab/eXRag/releases/download/some_name.whl

None of these commads worked. I created a PAT TOKEN with all permissions and still does not work.

I can see all the releases when I submit to my repo, they are there. Just cant be found when I try try to download.

[evanerwee]

I did that, but had a significant problem with authentication. I end up doing this:

================================================================

Base image (ECR Public mirror of Docker Official Images)

================================================================

ARG BASE_REGISTRY=public.ecr.aws/docker
ARG BASE_IMAGE=library/python:3.11-slim
FROM ${BASE_REGISTRY}/${BASE_IMAGE} AS base

ENV PIP_NO_CACHE_DIR=1
PIP_DISABLE_PIP_VERSION_CHECK=1
PYTHONDONTWRITEBYTECODE=1
PYTHONUNBUFFERED=1
DEBIAN_FRONTEND=noninteractive

SHELL ["/bin/bash", "-euxo", "pipefail", "-c"]
WORKDIR /app

Minimal OS deps (jq for JSON, coreutils for sha256sum)

RUN apt-get update && apt-get install -y --no-install-recommends
curl ca-certificates jq coreutils
&& rm -rf /var/lib/apt/lists/*

------------------------------------------------

Public Python dependencies

------------------------------------------------

COPY requirements.txt .
RUN python -m pip install --upgrade pip setuptools wheel &&
python -m pip install -r requirements.txt &&
python -m pip install pytz

================================================================

Private wheels via GitHub Releases (secure with BuildKit secret)

Provide:

GH_OWNER, GH_REPO

GH_TAG_1..GH_TAG_10 (release tag names)

ASSET_1..ASSET_10 (asset filenames in each release)

Optional:

ASSET_1_SHA256..ASSET_10_SHA256 (checksum verification)

Build example (locally):

DOCKER_BUILDKIT=1 docker build \

--secret id=GH_TOKEN,env=GH_TOKEN \

--build-arg GH_OWNER=RW-Lab \

--build-arg GH_REPO=eXRag \

--build-arg GH_TAG_1=v1.2.19-ucbl-logger \

--build-arg ASSET_1=ucbl_logger-1.0.1-py3-none-any.whl \

-t yourimage:latest .

================================================================

ARG GH_OWNER
ARG GH_REPO
ARG GH_API_BASE=https://api.github.com

Up to 10 tag/asset pairs

ARG GH_TAG_1
ARG GH_TAG_2
ARG GH_TAG_3
ARG GH_TAG_4
ARG GH_TAG_5
ARG GH_TAG_6
ARG GH_TAG_7
ARG GH_TAG_8
ARG GH_TAG_9
ARG GH_TAG_10

ARG ASSET_1
ARG ASSET_2
ARG ASSET_3
ARG ASSET_4
ARG ASSET_5
ARG ASSET_6
ARG ASSET_7
ARG ASSET_8
ARG ASSET_9
ARG ASSET_10

Optional checksums

ARG ASSET_1_SHA256
ARG ASSET_2_SHA256
ARG ASSET_3_SHA256
ARG ASSET_4_SHA256
ARG ASSET_5_SHA256
ARG ASSET_6_SHA256
ARG ASSET_7_SHA256
ARG ASSET_8_SHA256
ARG ASSET_9_SHA256
ARG ASSET_10_SHA256

Use BuildKit secret (no token in ARG/ENV); follows GitHub's 302 to S3 (-L)

RUN --mount=type=secret,id=GH_TOKEN
set -euo pipefail;
api_base="${GH_API_BASE%/}"; \

Count provided pairs

count=0;
for i in $(seq 1 10); do
eval "tag=${GH_TAG_${i}:-}";
eval "name=${ASSET_${i}:-}";
if [ -n "${tag:-}" ] && [ -n "${name:-}" ]; then count=$((count+1)); fi;
done;
if [ "$count" -eq 0 ]; then
echo "No GH_TAG_n/ASSET_n pairs provided; skipping private wheel installs.";
else
if [ -z "${GH_OWNER:-}" ] || [ -z "${GH_REPO:-}" ]; then
echo "GH_OWNER and GH_REPO are required when using GH_TAG_n/ASSET_n."; exit 1;
fi;
token="$(cat /run/secrets/GH_TOKEN 2>/dev/null || true)";
[ -n "$token" ] || echo " GH_TOKEN is empty; private assets will 401/404.";
ua="curl/$(curl --version | head -1 | awk '{print $2}')";
for i in $(seq 1 10); do
eval "tag=${GH_TAG_${i}:-}";
eval "name=${ASSET_${i}:-}";
eval "sha=${ASSET_${i}_SHA256:-}";
[ -n "${tag:-}" ] && [ -n "${name:-}" ] || continue;
echo "→ Resolving release '${tag}' for asset '${name}'";
rel_url="${api_base}/repos/${GH_OWNER}/${GH_REPO}/releases/tags/${tag}";
code="$(curl -sS -w '%{http_code}' -o /tmp/release.json
-H 'Accept: application/vnd.github+json'
${token:+-H "Authorization: Bearer ${token}"}
"$rel_url")";
if [ "$code" -ge 300 ]; then
echo "Release lookup failed ($code) for tag=${tag}. Body:";
sed -n '1,200p' /tmp/release.json; exit 1;
fi;
asset_id="$(jq -r --arg n "$name" '.assets[] | select(.name==$n) | .id // empty' /tmp/release.json | head -1)";
if [ -z "$asset_id" ]; then
echo "Asset '${name}' not found in release ${tag}. Available assets:";
jq -r '.assets[].name' /tmp/release.json || true;
exit 1;
fi;
echo "→ Downloading asset id=${asset_id} (${name})";
dl_url="${api_base}/repos/${GH_OWNER}/${GH_REPO}/releases/assets/${asset_id}";
code="$(curl -sS -L -w '%{http_code}'
--retry 6 --retry-delay 2 --retry-all-errors
-H 'Accept: application/octet-stream'
-H "User-Agent: $ua"
${token:+-H "Authorization: Bearer ${token}"}
-o "/tmp/${name}" "$dl_url")";
if [ "$code" -lt 200 ] || [ "$code" -ge 300 ]; then
echo "Asset download failed ($code) for ${name}. Body (truncated):";
head -200 "/tmp/${name}" || true;
exit 1;
fi;
if [ -n "${sha:-}" ]; then
echo "${sha} /tmp/${name}" | sha256sum -c -;
else
echo "No SHA256 provided for ${name}; skipping integrity check.";
fi;
python -m pip install "/tmp/${name}";
rm -f "/tmp/${name}" /tmp/release.json;
done;
fi

------------------------------------------------

App code

------------------------------------------------

COPY app/ ./app/

------------------------------------------------

Runtime config

------------------------------------------------

ENV PYTHONPATH=/app
PORT=8040
ROOT_PATH=/
UPLOAD_API_ENDPOINT=https://upload.ai4triage.com/upload
DEV_MODE=true

EXPOSE 8040
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8040", "--workers", "1"]