[OSDEV-2304] Upload MIT Living Wage data to the platform

.env.sample

@@ -7,3 +7,12 @@ REACT_APP_DROMO_SCHEMA_ID=
 HUBSPOT_API_KEY=
 HUBSPOT_SUBSCRIPTION_ID=
 EXTERNAL_DOMAIN=
+AWS_S3_ENDPOINT_URL=
+AWS_REGION=
+AWS_STORAGE_BUCKET_NAME=
+AWS_ACCESS_KEY_ID=
+AWS_SECRET_ACCESS_KEY=
+STRIPE_SECRET_KEY=
+STRIPE_PRICE_ID=
+STRIPE_WEBHOOK_SECRET=
+GOOGLE_SERVICE_ACCOUNT_CREDS_BASE64=

README.md

@@ -92,6 +92,31 @@ docker compose exec -T database pg_restore --verbose --clean --if-exists --no-ac
 docker compose up logstash
 ```
 
+### US County Tigerline Data Setup
+
+The platform uses US County Tigerline geometry data for MIT Living Wage integration. This data is loaded via Django migrations and can be sourced from either a CSV file in S3/MinIO or fixture data for local development.
+
+#### Local Development
+
+For local development, there are two options:
+
+1. **Using CSV file (recommended for full dataset)**:
+   - Download the CSV file from `s3://opensupplyhub-development-files-eu-west-1/data/us_county_tigerline_2025.csv`
+   - Place `us_county_tigerline_2025.csv` in the `src/django/` directory
+   - The migration will download it from MinIO and populate the database
+
+1. **Using fixture data (default for quick setup)**:
+   - If the CSV file is not found, the migration will skip data population gracefully
+   - Fixture data is automatically loaded via the `load_fixtures` management command
+   - The fixture file `src/django/api/fixtures/us_county_tigerline.json` contains sample county data for testing
+
+The fixture data is included in the `load_fixtures` command and will be loaded when running:
+```
+./scripts/manage load_fixtures
+```
+
+This is automatically executed as part of `./scripts/start_local_dev`.
+
 ### Creation of Superusers
 
 For local development we could create a superuser by Django Shell:
@@ -224,4 +249,3 @@ To run the Playwright tests, use the following command:
 ```
 docker compose -f docker-compose.tests.yml run --rm --build --entrypoint "npx playwright test -c playwright.config.ts" playwright-test
 ```
-

doc/release/RELEASE-NOTES.md

@@ -16,13 +16,17 @@ This project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html
 * 0191_create_wage_indicator_partner_field.py - This data migration creates the `wage_indicator` system partner field with type `object` and a comprehensive JSON schema defining six properties for wage indicator reference links. The field is marked as `system_field=True` and `active=True`, and includes source attribution.
 * 0192_create_wage_indicator_models.py - This migration creates two new models: `WageIndicatorCountryData` to store wage indicator URLs (living wage and minimum wage links in national and English languages) for each country indexed by ISO 3166-1 alpha-2 country code, and `WageIndicatorLinkTextConfig` to store customizable display text for each link type. The models support system-generated partner field data that will be automatically displayed on production location profiles based on the location's country.
 * 0193_populate_wage_indicator_data.py - This data migration populates the `WageIndicatorCountryData` table with wage indicator reference links for 171 countries. Each country entry includes URLs for living wage benchmarks and minimum wage information in both national languages and English where available. The migration also populates the `WageIndicatorLinkTextConfig` table with default display text for all three link types.
+* 0194_create_and_populate_us_county_tigerline.py - This migration creates the `USCountyTigerline` model to store US county boundary data with geometry in EPSG:5070 (Albers Equal Area projection). The model includes fields for geoid (primary key), county name, and MultiPolygon geometry. The migration attempts to populate the table with data from a CSV file stored in S3 (or MinIO for local development). If the CSV file is not available, the migration completes successfully with an empty table, allowing data to be populated later. The migration supports downloading the CSV file from S3 buckets with MinIO endpoint configuration for local development environments.
+* 0195_add_mit_livingwage_partner_field.py - This data migration creates the `mit_living_wage` system partner field with type `object` and a JSON schema defining a single property for county ID with uri-reference format. The field is marked as `system_field=True` and `active=True`, and includes a base URL pointing to the MIT Living Wage website and display text for the link. This partner field enables automatic display of MIT Living Wage data links on US production location profiles based on the facility's geographic location.
 
 #### Schema changes
 * [OSDEV-2305](https://opensupplyhub.atlassian.net/browse/OSDEV-2305) - Added wage indicator system partner field infrastructure: The `PartnerField` model has been updated with two new boolean fields (`active` and `system_field`) to support system-managed partner fields. Two new models were introduced: `WageIndicatorCountryData` (stores wage indicator URLs for 171 countries) and `WageIndicatorLinkTextConfig` (stores customizable display text for wage indicator links). A new `wage_indicator` system partner field was created to display country-specific living wage and minimum wage reference links on production location profiles. The implementation includes a custom manager for filtering active partner fields, a provider registry pattern for system-generated fields, and admin panel protections to prevent deletion or unauthorized modification of system fields.
+* [OSDEV-2304](https://opensupplyhub.atlassian.net/browse/OSDEV-2304) - Added infrastructure for MIT Living Wage data: The `USCountyTigerline` model was added to store US county boundary geometries for geographic lookups, and a `mit_living_wage` system partner field was created to display MIT Living Wage data links for US production locations based on their county location. The migration supports downloading CSV files from S3 buckets (with MinIO support for local development) and gracefully handles missing files by skipping data population without failing the migration.
 
 ### Code/API changes
 * [Follow-up][OSDEV-2114](https://opensupplyhub.atlassian.net/browse/OSDEV-2114) - Removed the `reindex_locations_with_environmental_data` Django management command from the parent `post_deployment` command so it no longer runs, as it was only needed for the `2.17.0` release.
 * [OSDEV-2305](https://opensupplyhub.atlassian.net/browse/OSDEV-2305) - Enhanced the `GET /api/facilities/{os_id}` endpoint to support system-generated partner fields. The `partner_fields` object in the response now includes the `wage_indicator` field when a contributor is assigned to it, automatically providing country-specific wage indicator reference links (living wage and minimum wage URLs in both national language and English) based on the production location's country code. System partner fields are populated dynamically through a provider registry pattern and follow the same structure as user-contributed partner fields, appearing alongside them in the API response.
+* [OSDEV-2304](https://opensupplyhub.atlassian.net/browse/OSDEV-2304) - Enhanced the `GET /api/facilities/{os_id}` endpoint to support MIT Living Wage system-generated partner fields. For US production locations, the `mit_living_wage` field is included when a contributor is assigned, automatically providing links to MIT Living Wage data based on the facility's geographic location (county). The field is populated dynamically through a provider registry pattern and follows the same structure as user-contributed partner fields.
 
 ### Architecture/Environment changes
 * [OSDEV-2047](https://opensupplyhub.atlassian.net/browse/OSDEV-2047) - Removed all Terraform configurations and ECS service definitions related to the deprecated standalone ContriCleaner service. Cleaned up the repository by deleting unused code and references, as ContriCleaner now operates exclusively as an internal Django library.
@@ -33,6 +37,7 @@ This project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html
 
 ### What's new
 * [OSDEV-2305](https://opensupplyhub.atlassian.net/browse/OSDEV-2305) - Introduced automatic wage indicator reference links on production location profiles for 171 countries. Each production location now displays country-specific links to authoritative living wage and minimum wage information and its regional partner sites. The wage indicator data is presented in both the national language and English, providing users with easy access to benchmarking information for fair wage assessments. The links appear automatically based on the production location's country and are managed as a system-generated partner field that cannot be manually edited or deleted, ensuring data consistency and reliability across the platform.
+* [OSDEV-2304](https://opensupplyhub.atlassian.net/browse/OSDEV-2304) - Introduced automatic MIT Living Wage data links on US production location profiles. Each US production location now displays county-specific links to MIT Living Wage Calculator data based on the facility's geographic location. The links provide access to county-specific living wage calculations and data, helping users assess fair wage benchmarks for US locations. The links appear automatically based on the facility's county location and are managed as a system-generated partner field that cannot be manually edited or deleted.
 * [OSDEV-2221](https://opensupplyhub.atlassian.net/browse/OSDEV-2221) - Updated post claim page `claimed/{claim-id}/`. This update includes:
     * Added emission data input fields with checkboxes. UI is consitent with other input fields in the `ClaimedFacilitiesDetails` component.
     * Applied Yup validation schema for `facility_website`, `point_of_contact_email`, `facility_workers_count`.

docker-compose.yml

@@ -40,17 +40,15 @@ services:
       - DJANGO_ENV=Local
       - DJANGO_SECRET_KEY=secret
       - DJANGO_LOG_LEVEL=INFO
-      - AWS_PROFILE=${AWS_PROFILE:-default}
       - OAR_CLIENT_KEY=
       - [email protected]
       - KAFKA_BOOTSTRAP_SERVERS=kafka:9092
       - KAFKA_TOPIC_DEDUPE_BASIC_NAME=dedupe.hub.topic
-      # NOTE the AWS_S3* variables are only used for testing s3 in development.
-      # The deployed application uses a policy attached to the ECS task instead
-      # of using keys.
-      - AWS_S3_ACCESS_KEY_ID
-      - AWS_S3_SECRET_ACCESS_KEY
-      - AWS_STORAGE_BUCKET_NAME
+      - AWS_STORAGE_BUCKET_NAME=os-hub-dev
+      - AWS_S3_ENDPOINT_URL=http://minio:9000
+      - AWS_ACCESS_KEY_ID=minioadmin
+      - AWS_SECRET_ACCESS_KEY=minioadmin
+      - AWS_REGION=us-east-1
       - OPENSEARCH_HOST=opensearch-single-node
       - OPENSEARCH_PORT=9200
       - OPENSEARCH_SSL=False
@@ -74,7 +72,6 @@ services:
         condition: service_started
       logstash:
         condition: service_started
-
     command:
       - "-b :8081"
       - "--reload"
@@ -90,6 +87,23 @@ services:
     networks:
       - proxynet
 
+  minio:
+    image: minio/minio:latest
+    environment:
+      - MINIO_ROOT_USER=minioadmin
+      - MINIO_ROOT_PASSWORD=minioadmin
+    command: server /data --console-address ":9001"
+    ports:
+      - "9000:9000"
+      - "9001:9001"
+    healthcheck:
+      test: ["CMD", "mc", "ready", "local"]
+      interval: 5s
+      timeout: 5s
+      retries: 5
+    networks:
+      - proxynet
+
   react:
     build:
       context: ./src/react

scripts/start_local_dev

@@ -10,6 +10,42 @@ function usage() {
     "data in the database."
 }
 
+function setup_minio() {
+  MINIO_ALIAS="dockerminio"
+  MINIO_USER="minioadmin"
+  MINIO_PASSWORD="minioadmin"
+  echo "Setting up MinIO alias $MINIO_ALIAS..."
+  docker compose exec minio bash \
+          -c "mc alias set $MINIO_ALIAS http://minio:9000 $MINIO_USER $MINIO_PASSWORD"
+
+  MINIO_BUCKET="os-hub-dev"
+  echo "Creating MinIO bucket $MINIO_BUCKET..."
+  docker compose exec minio bash -c "mc mb --ignore-existing $MINIO_ALIAS/$MINIO_BUCKET"
+
+
+  echo "Copying fixtures to MinIO bucket $MINIO_BUCKET..."
+  docker compose exec minio bash \
+          -c "mkdir -p /fixtures"
+  docker compose cp ./src/django/api/fixtures minio:/fixtures
+  docker compose exec minio bash \
+          -c "mc cp --recursive /fixtures/* $MINIO_ALIAS/$MINIO_BUCKET/api/"
+  docker compose exec minio bash \
+          -c "rm -rf /fixtures"
+
+  TIGERLINE_CSV="us_county_tigerline_2025.csv"
+
+  if [ -f "./src/django/$TIGERLINE_CSV" ]; then
+    echo "Copying $TIGERLINE_CSV to MinIO bucket $MINIO_BUCKET..."
+    docker compose exec minio bash \
+          -c "mkdir -p /temp"
+    docker compose cp ./src/django/$TIGERLINE_CSV minio:/temp/$TIGERLINE_CSV
+    docker compose exec minio bash \
+        -c "mc cp --recursive /temp/* $MINIO_ALIAS/$MINIO_BUCKET/data/"
+    docker compose exec minio bash \
+        -c "rm -rf /temp"
+  fi
+}
+
 if [ "${BASH_SOURCE[0]}" = "${0}" ]; then
     if [ "${1:-}" = "--help" ]; then
         usage
@@ -31,6 +67,9 @@ if [ "${BASH_SOURCE[0]}" = "${0}" ]; then
             django \
             manage.py collectstatic --noinput
 
+        # Setup MinIO backend for local development
+        setup_minio
+
         # Populate database with seeded facility lists.
         echo "Populating database with seeded facility lists..."
         ./scripts/reset_database
@@ -56,7 +95,7 @@ if [ "${BASH_SOURCE[0]}" = "${0}" ]; then
         # Launch the rest of the services.
         echo "Launching the rest of the services..."
         docker compose up -d
-        
+
         echo "Process completed successfully!"
     fi
 fi

scripts/update

@@ -22,7 +22,7 @@ if [ "${BASH_SOURCE[0]}" = "${0}" ]; then
 
         # Bring up PostgreSQL and Django in a way that respects
         # configured service health checks.
-        docker compose up -d database django
+        docker compose up -d database django minio
 
         # Apply any outstanding Django migrations
         docker compose \

src/django/api/admin.py

@@ -17,6 +17,7 @@
 from api.models.wage_indicator_link_text_config import (
     WageIndicatorLinkTextConfig
 )
+from api.models.us_county_tigerline import USCountyTigerline
 from allauth.account.models import EmailAddress
 from simple_history.admin import SimpleHistoryAdmin
 from waffle.models import Flag, Sample, Switch
@@ -398,6 +399,12 @@ class WageIndicatorLinkTextConfigAdmin(admin.ModelAdmin):
     search_fields = ('link_type', 'display_text')
 
 
+class USCountyTigerlineAdmin(admin.ModelAdmin):
+    list_display = ('geoid', 'name')
+    search_fields = ('geoid', 'name')
+    readonly_fields = ('geoid', 'name', 'geometry', 'created_at', 'updated_at')
+
+
 admin_site.register(models.Version)
 admin_site.register(models.User, OarUserAdmin)
 admin_site.register(models.Contributor, ContributorAdmin)
@@ -426,3 +433,4 @@ class WageIndicatorLinkTextConfigAdmin(admin.ModelAdmin):
 admin_site.register(
     WageIndicatorLinkTextConfig, WageIndicatorLinkTextConfigAdmin
 )
+admin_site.register(USCountyTigerline, USCountyTigerlineAdmin)

src/django/api/fixtures/us_county_tigerline.json

@@ -0,0 +1,52 @@
+[
+    {
+        "model": "api.uscountytigerline",
+        "pk": "40075",
+        "fields": {
+            "name": "Oklahoma County",
+            "geometry": "SRID=5070;MULTIPOLYGON (((-1000000 1000000, -950000 1000000, -950000 1050000, -1000000 1050000, -1000000 1000000)))",
+            "created_at": "2025-01-01T00:00:00+00:00",
+            "updated_at": "2025-01-01T00:00:00+00:00"
+        }
+    },
+    {
+        "model": "api.uscountytigerline",
+        "pk": "06037",
+        "fields": {
+            "name": "Los Angeles County",
+            "geometry": "SRID=5070;MULTIPOLYGON (((-2000000 500000, -1950000 500000, -1950000 550000, -2000000 550000, -2000000 500000)))",
+            "created_at": "2025-01-01T00:00:00+00:00",
+            "updated_at": "2025-01-01T00:00:00+00:00"
+        }
+    },
+    {
+        "model": "api.uscountytigerline",
+        "pk": "36061",
+        "fields": {
+            "name": "New York County",
+            "geometry": "SRID=5070;MULTIPOLYGON (((-1800000 600000, -1750000 600000, -1750000 650000, -1800000 650000, -1800000 600000)))",
+            "created_at": "2025-01-01T00:00:00+00:00",
+            "updated_at": "2025-01-01T00:00:00+00:00"
+        }
+    },
+    {
+        "model": "api.uscountytigerline",
+        "pk": "48201",
+        "fields": {
+            "name": "Harris County",
+            "geometry": "SRID=5070;MULTIPOLYGON (((-1200000 800000, -1150000 800000, -1150000 850000, -1200000 850000, -1200000 800000)))",
+            "created_at": "2025-01-01T00:00:00+00:00",
+            "updated_at": "2025-01-01T00:00:00+00:00"
+        }
+    },
+    {
+        "model": "api.uscountytigerline",
+        "pk": "04013",
+        "fields": {
+            "name": "Maricopa County",
+            "geometry": "SRID=5070;MULTIPOLYGON (((-1500000 700000, -1450000 700000, -1450000 750000, -1500000 750000, -1500000 700000)))",
+            "created_at": "2025-01-01T00:00:00+00:00",
+            "updated_at": "2025-01-01T00:00:00+00:00"
+        }
+    }
+]

src/django/api/management/commands/load_fixtures.py

@@ -17,6 +17,7 @@ def handle(self, *args, **options):
                          'sources.json',
                          'facility_lists.json',
                          'sectors.json',
-                         'moderation_events.json')
+                         'moderation_events.json',
+                         'us_county_tigerline.json')
         except CommandError as e:
             self.stderr.write("Error loading fixture data: {}".format(e))