Skip to content

Conversation

@shivasurya
Copy link
Owner

🎯 Overview

This PR adds automated Python stdlib registry uploads to Cloudflare R2 and bumps the version to 1.0.0.

✨ What's New

πŸ“¦ Version Bump

  • Version: 0.0.34 β†’ 1.0.0
  • Updated sourcecode-parser/VERSION and package.json

☁️ Cloudflare R2 Integration

  • New GitHub Action: .github/workflows/stdlib-r2-upload.yml
    • Runs on release publish
    • Supports manual workflow dispatch
  • Upload script: sourcecode-parser/tools/upload_to_r2.sh
    • Generates stdlib for Python 3.11, 3.12, 3.14
    • Uploads to R2 using AWS S3 sync
    • Uses --delete flag to replace existing registries
  • Testing script: sourcecode-parser/tools/test_generation_local.sh
    • Validates generation without uploading

πŸ”— URL Migration

Old: https://codepathfinder.dev/assets/registries/...
New: https://assets.codepathfinder.dev/registries/...

Updated files:

  • sourcecode-parser/tools/generate_stdlib_registry.py (manifest base_url)
  • sourcecode-parser/graph/callgraph/builder/builder.go (Go client)
  • sourcecode-parser/graph/callgraph/registry/stdlib_remote.go (comments)

🧹 Cleanup

  • Removed .github/workflows/deploy.yml (Cloudflare Pages - now using Netlify)

πŸ“š Documentation

  • Added R2_SETUP.md with complete setup instructions
  • Documents required GitHub secrets and Cloudflare configuration

πŸ§ͺ Testing

βœ… Local testing completed:

  • Generated 190 modules for Python 3.14
  • Total size: 8.2MB (well within 10GB free tier)
  • Validated JSON structure
  • Confirmed manifest base_url is correct
  • Failed Windows-only modules (expected on macOS): msvcrt, nt, winreg, winsound

πŸ”‘ Required Secrets (Already Added)

The following GitHub secrets are configured:

  • βœ… R2_ACCOUNT_ID
  • βœ… R2_ACCESS_KEY_ID
  • βœ… R2_SECRET_ACCESS_KEY

πŸ“‹ Cloudflare R2 Setup Checklist

See R2_SETUP.md for detailed instructions.

  • R2 bucket code-pathfinder-assets created
  • Custom domain assets.codepathfinder.dev configured (DNS CNAME)
  • Public access enabled on bucket
  • GitHub secrets configured
  • Test workflow dispatch after merge

πŸš€ How It Works

  1. On Release: When you publish a new release (e.g., v1.0.0):

    • GitHub Action triggers automatically
    • Generates stdlib for Python 3.11, 3.12, 3.14
    • Uploads to R2 (replaces existing files)
    • Verifies public URLs are accessible
  2. Manual Trigger: Go to Actions β†’ "Upload Stdlib Registries to R2" β†’ Run workflow

  3. Lazy Loading: Go client downloads modules on-demand from:

    • https://assets.codepathfinder.dev/registries/python3.11/stdlib/v1/manifest.json
    • https://assets.codepathfinder.dev/registries/python3.12/stdlib/v1/manifest.json
    • https://assets.codepathfinder.dev/registries/python3.14/stdlib/v1/manifest.json

πŸ’° Cost

Storage: 25-30 MB / 10 GB free tier = $0/month
Operations: ~100/month / 1M free = $0/month
Egress: Unlimited free = $0/month

Total: $0/month βœ…

⚠️ Breaking Changes

  • Stdlib registry URL changed (backward compatibility intentionally removed)
  • Go client already updated to use new URL

πŸ“ Next Steps

After merging:

  1. Complete R2 custom domain setup (if not done)
  2. Test workflow dispatch to verify R2 upload works
  3. Create v1.0.0 release to trigger automatic upload
  4. Verify registries are publicly accessible

πŸ”— Related

  • Closes #[issue-number] (if applicable)
  • Part of the stdlib registry infrastructure improvements

πŸ€– Generated with Claude Code

- Bump version from 0.0.34 to 1.0.0
- Add automated R2 upload via GitHub Actions
- Update stdlib registry URLs to assets.codepathfinder.dev
- Add upload and testing scripts
- Remove outdated Cloudflare Pages deploy workflow
- Add R2_SETUP.md documentation
@safedep
Copy link

safedep bot commented Nov 28, 2025

SafeDep Report Summary

Green Malicious Packages Badge Green Vulnerable Packages Badge Green Risky License Badge

No dependency changes detected. Nothing to scan.

This report is generated by SafeDep Github App

@shivasurya shivasurya self-assigned this Nov 28, 2025
@codecov
Copy link

codecov bot commented Nov 28, 2025

Codecov Report

βœ… All modified and coverable lines are covered by tests.
βœ… Project coverage is 80.27%. Comparing base (11f7448) to head (989072a).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #407      +/-   ##
==========================================
- Coverage   80.47%   80.27%   -0.20%     
==========================================
  Files          77       77              
  Lines        7763     7763              
==========================================
- Hits         6247     6232      -15     
- Misses       1269     1289      +20     
+ Partials      247      242       -5     

β˜” View full report in Codecov by Sentry.
πŸ“’ Have feedback on the report? Share it here.

πŸš€ New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • πŸ“¦ JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@shivasurya shivasurya merged commit 56aeabe into main Nov 28, 2025
4 of 5 checks passed
@shivasurya shivasurya deleted the feat/stdlib-r2-upload-v1.0.0 branch November 28, 2025 20:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants