poe-affix-builder

Toolchain to build result/affixes, result/uniques, and result/economy from:

• poe2db snapshot data (config/poe2db_snapshot.json),
• PoE2 source data (data/poe2/data/mods.json, auto-fetched on demand), and
• PoE2Scout economy catalog data for currency coverage.

The generated affix JSON files are the canonical affix output. Unique items are emitted separately under result/uniques/{class}.json. Validation warnings are expected coverage advisories, not evidence that the generated affix files are wrong.

Licensing

• The repository's source code and original project-authored content are licensed under CC0 1.0; see LICENSE and LICENSE-CC0.txt.
• This project fetches and may bundle or derive data from PoE2DB (https://poe2db.tw), including the modifiers index used by refresh-poe2db. It also fetches economy catalog data from PoE2Scout (https://poe2scout.com) via refresh-economy.
• PoE2DB states: Wikis Content is available under CC BY-NC-SA 3.0 unless otherwise noted.
• As a result, bundled or generated PoE2DB-derived data in this repository should be treated as CC BY-NC-SA 3.0 unless otherwise noted, not as CC0.
• That includes the fetched snapshot and the generated affix/unique/mapping/report artifacts, including config/poe2db_snapshot.json, config/unmapped_uniques.json, config/item_mapping.json, result/affixes/*.json, result/uniques/*.json, result/poe2db_refresh_report.json, result/build_report.json, result/rebuild_mapping_report.json, and result/mapping_validation.json. Economy artifacts under result/economy/*.json and tier artifacts under result/tiers/* are PoE2Scout-derived.
• If you redistribute PoE2DB-derived artifacts from this repo, preserve attribution to PoE2DB and comply with the applicable non-commercial and share-alike terms.
• Attribution and scope details are documented in THIRD_PARTY_NOTICES.md.

Requirements

• Python 3.14+
• uv

Setup

uv sync

Local Checks

Run the fast hygiene and regression gates before logic changes are merged:

uv run ruff check src tests
uv run python -m unittest discover -s tests
uv run poe-affix-build validate-unique-overrides

Run uv run poe-affix-build build when generated affix, unique, or report parity needs to be verified locally.

Commands

Show the installed CLI version:

uv run poe-affix-build --version

Refresh poe2db snapshot (explicit/manual, not part of build):

uv run poe-affix-build refresh-poe2db \
  --snapshot config/poe2db_snapshot.json \
  --report result/poe2db_refresh_report.json \
  --unmapped-uniques config/unmapped_uniques.json

refresh-poe2db prints live progress ([n/total]) and retries transient upstream failures (including Cloudflare 52x responses) with exponential backoff. It first fetches the canonical PoE2DB Unique_item index and visits each unique detail page, adds explicit extra unique detail pages for known records missing from the index, then uses itemtype pages only for bases and modifier sections.

Refresh the standalone economy currency catalog:

uv run poe-affix-build refresh-economy \
  --out result/economy/currency.json \
  --report result/economy_currency_report.json

refresh-economy fetches PoE2Scout economy categories for the Fate of the Vaal league and stores static category/item identifiers only. It includes all Scout currency categories, including Uncut Gems, because PoE2DB economy pages are currently incomplete for currency coverage. It does not map to pickit identifiers and does not store market prices, Divine/Exalted conversions, or tiers.

Rebuild mapping from snapshot + mods source:

uv run poe-affix-build rebuild-mapping \
  --source-dir data/poe2/data \
  --snapshot config/poe2db_snapshot.json \
  --out config/item_mapping.json \
  --report result/rebuild_mapping_report.json

For contributor debugging, rebuild-mapping can also write a categorized unresolved diagnostic report:

uv run poe-affix-build rebuild-mapping \
  --diagnostics result/rebuild_mapping_diagnostics.json

Build affix output from source data:

uv run poe-affix-build build \
  --source-dir data/poe2/data \
  --manifest config/item_mapping.json \
  --out-dir result/affixes \
  --uniques-dir result/uniques \
  --report result/build_report.json \
  --unresolved-tiers result/unresolved_tiers.json \
  --unique-stat-missing result/unique_stat_mapping_missing.json \
  --mods-list data/eb2/ModsList.html

For contributor debugging, build can also write the same categorized unresolved diagnostics:

uv run poe-affix-build build \
  --diagnostics result/build_diagnostics.json

Validate mapping coverage against source data:

uv run poe-affix-build validate-mapping \
  --source-dir data/poe2/data \
  --manifest config/item_mapping.json \
  --out result/mapping_validation.json

Validate unique override/dropdown config against EB2 affix ids:

uv run poe-affix-build validate-unique-overrides \
  --config config/unique_stat_affix_overrides.json \
  --mods-list data/eb2/ModsList.html

Write a local ModsList review copy with known undefined text cells filled from local mappings:

uv run poe-affix-build annotate-mods-list \
  --mods-list data/eb2/ModsList.html \
  --out result/ModsList.annotated.html

Notes

• refresh-poe2db fails atomically: snapshot/report are written only after full successful fetch/parse.
• Unique items are detail-page records from https://poe2db.tw/us/Unique_item where possible. Known records missing from that index, such as The Remembered Tales, are fetched through explicit extra detail URLs instead of scanning itemtype unique tabs.
• Unique detail records that cannot be assigned to a known affix itemtype are written to config/unmapped_uniques.json and summarized in result/poe2db_refresh_report.json. The snapshot stores unmapped_uniques_path instead of embedding the full review list. During build, those records are still included in result/uniques/{class}.json when PoE2DB class metadata is available.
• result/affixes/*.json is affix-only. Unique output is grouped by PoE2DB class, for example result/uniques/Boots.json, result/uniques/Body_Armours.json, and result/uniques/Jewels.json.
• During build, unique stat templates are matched back to generated affixes where possible. If a stat is not present in generated affixes, the builder falls back to raw mods.json entries and then exact single-id matches from data/eb2/ModsList.html. Unique stat rows also include affix_ids, derived from stat_ids by replacing spaces with underscores. When a unique stat can roll one of several affixes, the row includes affix_options so configurator UIs can show dropdown text while storing the selected affix id. When affix_ids is present, no affix_mapping block is emitted. Grants Skill and (Hidden) rows are ignored for affix mapping because they are handled separately; unmatched or ambiguous non-skill visible stats are written to result/unique_stat_mapping_missing.json for review.
• Unique-only affix id overrides and dropdown descriptions are maintained in config/unique_stat_affix_overrides.json. Run validate-unique-overrides after edits; it checks shape rules and confirms override affix ids exist in data/eb2/ModsList.html.
• result/economy/currency.json is the standalone PoE2Scout economy catalog. It contains category names, item names, Scout API identifiers, item ids, and optional icon URLs only.
• result/unresolved_tiers.json is a build-time analysis artifact with all unresolved tiers in a flat list plus actionability categories. It is easier to inspect than the embedded unresolved_tiers array in result/build_report.json. Combined rows can include text_parts and affix_components, so configurators can see each component's display text, numeric values, and resolved affix ids. Component ids are resolved from generated/mods.json references first, then from exact and curated ModsList.html references for EB2-only rows. Socketable-only unresolved rows are ignored because they are outside the item-drop filter pool.
• For build, validate-mapping, and rebuild-mapping, if mods.json is missing under --source-dir and the source dir follows the default .../poe2/data layout, the CLI performs a shallow sparse checkout of https://github.com/repoe-fork/poe2.git (master) into data/poe2, materializing only data/mods.json.
• validate-mapping reports coverage advisories. These are useful for mapping maintenance, but the current affix output files can still be correct even when advisories exist.

Unresolved Groups

The rebuild and build reports can still contain unresolved modifier matches, mostly from non-normal rune/socket/soul-core style sections. The detailed interpretation lives in docs/unresolved-groups.md.

Architecture

• src/poe_affix_builder/domain: typed core models split by source, manifest, snapshot, output, unique, and report concerns; domain.models remains the compatibility facade
• src/poe_affix_builder/contracts: serializers and round-trip-safe JSON contracts
• src/poe_affix_builder/adapters: filesystem, sparse checkout, poe2db, EB2 ModsList, and Scout HTTP integration boundaries
• src/poe_affix_builder/services: build, validation, rebuild, unique-output, snapshot-assignment/reporting, economy catalog/reporting, and shared matching logic
• src/poe_affix_builder/policies: domain/spawn-tag rules

More detail is in docs/architecture.md.

Contributor Policy

• Preserve parity for result/affixes, result/uniques, result/economy, config/item_mapping.json, and the committed reports unless a change explicitly intends to alter logic.
• When changing matching behavior, update or justify any artifact diffs in tests and review notes.
• Run uv run ruff check src tests, uv run python -m unittest discover -s tests, and uv run poe-affix-build validate-unique-overrides before merging logic changes.