Skip to content

Feature/updateinfo v2#20

Merged
rockythorn merged 4 commits intofeature/config-and-processing-refactorfrom
feature/updateinfo-v2
Nov 15, 2025
Merged

Feature/updateinfo v2#20
rockythorn merged 4 commits intofeature/config-and-processing-refactorfrom
feature/updateinfo-v2

Conversation

@rockythorn
Copy link
Owner

Summary

Implements v2 updateinfo.xml API endpoint with FK-based filtering and major version aggregation. Fixes critical module package bug causing 230+ missing advisories through ORM-level property pattern.

Problems Solved

Module Package Source RPM Mapping Bug

  • ~8,908 packages had "module." prefix in package_name field (e.g., "module.postgresql")
  • NEVRA parsing extracts "postgresql" but database has "module.postgresql" → mismatch
  • Impact: 230+ module advisories missing from Rocky 8 AppStream updateinfo.xml

V1 Endpoint Limitations

  • Uses denormalized product_name strings instead of FK relationships
  • No major version aggregation (separate XML per minor version)
  • Requires URL encoding for product names with spaces

Changes

1. ORM-Level Module Prefix Fix (Trinity Quirk)

Adds Python property pattern to AdvisoryPackage model to transparently strip "module." prefix:

class AdvisoryPackage(Model):
    _package_name = fields.TextField(source_field='package_name')

    @property
    def package_name(self):
        return self._clean_package_name(self._package_name)
  • No database migration required
  • Works for existing data (reads) and new data (writes)
  • Transparent to all business logic

2. Simplify Business Logic

Removed manual .removeprefix("module.") calls since ORM handles it automatically.

3. V2 Updateinfo API Endpoint

URL: /api/v3/updateinfo/{product_slug}/{major_version}/{repo}/updateinfo.xml?arch={arch}

Example: /api/v3/updateinfo/rocky-linux/8/BaseOS/updateinfo.xml?arch=x86_64

Key improvements:

  • FK-based filtering using supported_product_id instead of string matching
  • Major version aggregation (8 → all 8.x minor versions)
  • Required architecture parameter (validated against Architecture enum)
  • Product slug mapping (rocky-linux → Rocky Linux)
  • Data integrity validation via FK relationships

4. Architecture Validation

Uses centralized Architecture enum from validation module instead of hardcoded lists.

5. Helper Functions

Extracted reusable functions:

  • resolve_product_slug() - Convert URL slug to product name
  • get_source_package_name() - Extract source package identifier
  • build_source_rpm_mapping() - Map binary packages to source RPMs

Also fixed bug where binaries were mapped to source RPMs from different minor versions.

API Comparison

Feature V1 V2
URL /Rocky%20Linux%208%20x86_64/BaseOS/updateinfo.xml /rocky-linux/8/BaseOS/updateinfo.xml?arch=x86_64
Filtering String-based product_name FK-based supported_product_id
Version aggregation None (one product = one minor version) Major version aggregation
Architecture Optional query param Required query param
URL encoding Required (spaces in names) Not needed (slugs)

Testing

  • Created 14 unit tests for new helper functions
  • Integration testing shows 230 module advisories now appear
  • V1 endpoint produces identical output (backward compatible)
  • All tests pass

Deployment

  • Backward compatible: V1 endpoint unchanged
  • No schema changes: ORM uses property pattern
  • No migration required: Works with existing data
  • Module advisories appear automatically after deployment

Files Changed

apollo/db/__init__.py                     |  30 +++
apollo/server/routes/api_updateinfo.py    | 251 +++
apollo/tests/BUILD.bazel                  |   8 +
apollo/tests/test_api_updateinfo.py       | 182 +++
.github/workflows/test.yaml               |   1 +

rockythorn and others added 4 commits November 14, 2025 17:04
Implements new v2 endpoint that uses normalized database relationships
(supported_product_id, major_version) instead of denormalized product_name
strings to enable aggregation across minor versions within a major release.

Key improvements:
- FK-based package filtering prevents cross-product contamination
- Fixes module prefix bug by stripping "module." consistently
- Enables major version aggregation (e.g., all Rocky 8.x advisories)
- Maintains v1 backward compatibility with no functional changes
- Fixes source RPM mapping bug where binaries were incorrectly mapped to
  source RPMs from different minor versions (e.g., el8_7 binary mapped to
  el8_6 source). The refactored build_source_rpm_mapping() now correctly
  matches each binary package to its exact source RPM version.

New endpoint: /{product}/{major_version}/{repo}/updateinfo.xml?arch={arch}
Example: /rocky-linux/8/BaseOS/updateinfo.xml?arch=x86_64
Implements transparent stripping of 'module.' prefix from package_name field
at the ORM layer using Python property pattern.

The package_name field contains 'module.' prefix for module packages
(e.g., 'module.postgresql' instead of 'postgresql'), causing source RPM
mapping failures.

No database migration required. The ORM property pattern handles the data
quality issue transparently.

Co-authored-by: Trinity Quirk <[email protected]>
Removes manual module. prefix stripping from business logic now that
the AdvisoryPackage ORM model handles it transparently via property getter.

Changes:
- get_source_package_name(): Removed .removeprefix(module.) call
- build_source_rpm_mapping(): Removed manual prefix stripping before comparison
- Updated function docstrings to remove references to prefix handling
- Updated tests to use clean package names (ORM already stripped prefix)

The ORM property pattern ensures pkg.package_name always returns a clean
value without the module. prefix, eliminating the need for defensive
prefix stripping throughout the codebase.

Benefits:
- Simpler, more maintainable code
- Single source of truth for prefix handling (ORM layer)
- No risk of forgetting to strip prefix in future code
- Business logic focuses on domain concerns, not data quality issues
Replaces hardcoded architecture list with the Architecture enum from
apollo.server.validation module to eliminate duplication and ensure
consistency across the codebase.

The centralized enum supports all architectures Rocky Linux uses,
including riscv64 which was missing from the hardcoded list but exists
in the database.

Changes:
- Import Architecture enum from validation module
- Replace hardcoded list check with enum validation
- Remove duplicate architecture test (already tested in test_validation.py)

Benefits:
- Single source of truth for valid architectures
- Supports full set of architectures (x86_64, aarch64, i386, i686, ppc64, ppc64le, s390x, riscv64, noarch)
- Consistent validation behavior across all Apollo endpoints
- No risk of architecture lists diverging between modules
@rockythorn rockythorn merged commit f550104 into feature/config-and-processing-refactor Nov 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants