Skip to content
Merged
Show file tree
Hide file tree
Changes from 24 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
e6bf7b7
implement draft logic to support json schema validation for partner f…
roman-stolar Nov 17, 2025
244dfb0
added json formatting and validation for json schema field in admin p…
roman-stolar Nov 18, 2025
751fb5e
fix issue with format validation
roman-stolar Nov 18, 2025
70f3974
added django json editor
roman-stolar Nov 18, 2025
3f312e6
small fix
roman-stolar Nov 18, 2025
15bf5d9
updated cdn access to cloud front
roman-stolar Nov 18, 2025
363b2cb
fix linter
roman-stolar Nov 18, 2025
20727be
refactored
roman-stolar Nov 18, 2025
88b4074
Merge commit 'b33cfd6b9226a103a79c14744de961202f596967' into OSDEV-22…
roman-stolar Nov 18, 2025
dab9528
fix linter
roman-stolar Nov 18, 2025
122c35a
possible fix tests
roman-stolar Nov 18, 2025
23aef81
small UI fixes
roman-stolar Nov 18, 2025
36efc2d
fix json format for json_schema value
roman-stolar Nov 18, 2025
97f11b6
Merge branch 'main' into OSDEV-2266-add-json-schema-support
roman-stolar Nov 18, 2025
bfcc208
properly save to db
roman-stolar Nov 18, 2025
97d14ec
Merge branch 'main' into OSDEV-2266-add-json-schema-support
roman-stolar Nov 18, 2025
8a029c1
fix linter
roman-stolar Nov 18, 2025
32e55b1
Merge commit '97d14eca63d2eec766451e2b3a9263962016651f' into OSDEV-22…
roman-stolar Nov 19, 2025
b7e860d
added unit tests
roman-stolar Nov 19, 2025
5ced5c7
possible fix
roman-stolar Nov 19, 2025
0480492
fix tests
roman-stolar Nov 19, 2025
64794f0
updated release notes
roman-stolar Nov 19, 2025
30e0fad
remove head issue
roman-stolar Nov 19, 2025
640c807
addressed Vadim comments
roman-stolar Nov 20, 2025
e1a581e
small fix
roman-stolar Nov 20, 2025
4b11136
Merge branch 'main' into OSDEV-2266-add-json-schema-support
roman-stolar Nov 20, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions deployment/terraform/cdn.tf
Original file line number Diff line number Diff line change
Expand Up @@ -674,6 +674,28 @@ resource "aws_cloudfront_distribution" "cdn" {
max_ttl = 300
}

ordered_cache_behavior {
path_pattern = "/static/*jsoneditor/*"
allowed_methods = ["DELETE", "GET", "HEAD", "OPTIONS", "PATCH", "POST", "PUT"]
cached_methods = ["GET", "HEAD", "OPTIONS"]
target_origin_id = "originAlb"

forwarded_values {
query_string = true
headers = ["*"]

cookies {
forward = "all"
}
}

compress = true
viewer_protocol_policy = "redirect-to-https"
min_ttl = 0
default_ttl = 0
max_ttl = 300
}

logging_config {
include_cookies = false
bucket = aws_s3_bucket.logs.bucket_domain_name
Expand Down
12 changes: 9 additions & 3 deletions doc/release/RELEASE-NOTES.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,18 @@ This project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html
* Product name: Open Supply Hub
* Release date: November 29, 2025

### Database changes

#### Migrations
* 0186_add_json_schema_to_partner_field.py - This migration adds a `json_schema` JSONField to the `PartnerField` model, allowing administrators to define JSON schemas for validating object type partner fields. The field is optional and is used to validate the structure and content of contributed data when the partner field type is `object`.

#### Schema changes
* [OSDEV-2266](https://opensupplyhub.atlassian.net/browse/OSDEV-2266) - The `PartnerField` model has been updated. Field `json_schema` added with a type JSONField.

### What's new
* [OSDEV-2112](https://opensupplyhub.atlassian.net/browse/OSDEV-2112) - Moved "Recruitment Agency" (previously classified as a location type) under the "Office / HQ" location type as a processing type. Also introduced a new processing type, "Union Headquarters/Office", under the "Office / HQ" location type. This update affects both search and newly contributed data: from now on, "Union Headquarters/Office" and "Recruitment Agency" will appear under the "Office / HQ" location type when displayed in search dropdowns or shown on location profiles for **newly** added locations.
* [OSDEV-2244](https://opensupplyhub.atlassian.net/browse/OSDEV-2244) - Implemented frontend formatting for the ISIC 4 extended field to display Section, Division, Group, and Class as separate labeled entries, building the full ISIC 4 hierarchy on production location profile pages.
* [OSDEV-2266](https://opensupplyhub.atlassian.net/browse/OSDEV-2266) - Added JSON schema validation support for object type partner fields. Partner fields with object type can now have an associated JSON schema that validates the structure and content of contributed data, providing detailed error messages for validation failures.

### Release instructions
* Ensure that the following commands are included in the `post_deployment` command:
Expand Down Expand Up @@ -61,11 +70,8 @@ This project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html
* [OSDEV-2259](https://opensupplyhub.atlassian.net/browse/OSDEV-2259) - Fixed an issue where the Company Phone field was being saved to the incorrect `office_phone_number` column instead of the `facility_phone_number` when submitting a claim. The Company Phone field now properly stores the value in the correct `facility_phone_number` column in `api_facilityclaim` table.
* [OSDEV-2262](https://opensupplyhub.atlassian.net/browse/OSDEV-2262) - Prevented unintended submission of the last-step claim form when pressing Enter in an input while the Submit button is not focused. Updated `src/react/src/components/InitialClaimFlow/ClaimForm/ClaimForm.jsx` to remove implicit form submission and trigger Formik submission explicitly via the Submit button, aligning with Material UI semantics.
* [OSDEV-2231](https://opensupplyhub.atlassian.net/browse/OSDEV-2231) - Fixed Django admin panel not displaying location type values for claims when they didn't match the predefined taxonomy. Removed the restrictive `choices` constraint to allow all location types to be visible.
<<<<<<< HEAD
* [OSDEV-2260](https://opensupplyhub.atlassian.net/browse/OSDEV-2260) - Removed the unnecessary `facility_type` checks in the `facilities_view_set.py` to support proper saving of the data. The function assumed that the value is an array and that caused errors in the creation of extended fields. On front-end data format has been changed to send array values as a pipe-separated (`|`) string instead of a comma-separated one. The `extended_fields.py` file has been updated to correctly parse this new pipe-delimited format, ensuring data is handled correctly.
=======
* [OSDEV-2264](https://opensupplyhub.atlassian.net/browse/OSDEV-2264) - Fixed an issue where the processing type field was being saved to the database (`api_extendedfield` table) as an empty list. Now it's not saved if the list is empty.
>>>>>>> d009243c ([OSDEV-2264] Prevent saving empty lists for facility production types (#805))

### What's new
* [OSDEV-2200](https://opensupplyhub.atlassian.net/browse/OSDEV-2200) - Implements a new claim introduction page for the new facility claiming process, accessible via `/claim/:osId`, which can be enabled or activated through a feature flag.
Expand Down
21 changes: 20 additions & 1 deletion src/django/api/admin.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
from waffle.models import Flag, Sample, Switch
from waffle.admin import FlagAdmin, SampleAdmin, SwitchAdmin
from ckeditor.widgets import CKEditorWidget
from jsoneditor.forms import JSONEditor

from api import models

Expand Down Expand Up @@ -249,10 +250,22 @@ class PartnerFieldAdminForm(forms.ModelForm):
required=False,
widget=CKEditorWidget()
)
json_schema = forms.JSONField(
required=False,
widget=JSONEditor(
init_options={"mode": "code", "modes": ["code", "tree"]},
attrs={
'style': 'width: 800px; height: 400px;'
}
)
)

class Meta:
model = PartnerField
fields = ['name', 'type', 'unit', 'label', 'source_by']
fields = ['name', 'type', 'unit', 'label', 'source_by', 'json_schema']

def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)


class PartnerFieldAdmin(admin.ModelAdmin):
Expand All @@ -261,6 +274,12 @@ class PartnerFieldAdmin(admin.ModelAdmin):
search_fields = ('name', 'type', 'label', 'unit', 'source_by')
readonly_fields = ('uuid', 'created_at', 'updated_at')

class Media:
js = (
'admin/js/jquery.init.js',
'admin/js/partner_field_admin.js',
)


class EmailAddressAdmin(admin.ModelAdmin):
list_display = ('email', 'user', 'primary', 'verified')
Expand Down
19 changes: 19 additions & 0 deletions src/django/api/migrations/0186_add_json_schema_to_partner_field.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Generated by Django 3.2.17 on 2025-01-18 12:00

from django.db import migrations, models


class Migration(migrations.Migration):

dependencies = [
('api', '0185_add_source_by_to_partner_field'),
]

operations = [
migrations.AddField(
model_name='partnerfield',
name='json_schema',
field=models.JSONField(blank=True, null=True, help_text='JSON Schema for validating object type partner fields. Used when type is "object".'),
),
]

9 changes: 9 additions & 0 deletions src/django/api/models/partner_field.py
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,15 @@ class Meta:
)
)

json_schema = models.JSONField(
blank=True,
null=True,
help_text=(
'JSON Schema for validating object type partner fields. '
'Used when type is "object".'
)
)

created_at = models.DateTimeField(auto_now_add=True)
updated_at = models.DateTimeField(auto_now=True)

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@
import ProductionLocationDataProcessor
from api.moderation_event_actions.creation.location_contribution \
.processors.geocoding_processor import GeocodingProcessor
from api.moderation_event_actions.creation.location_contribution.processors \
.partner_field_type_processor import PartnerFieldTypeProcessor
from api.moderation_event_actions.creation.location_contribution \
.processors.permission_processor import PermissionProcessor
from api.moderation_event_actions.creation.dtos.create_moderation_event_dto \
Expand Down Expand Up @@ -37,6 +39,7 @@ def serialize(
def __setup_location_data_processors() -> ContributionProcessor:
location_data_processors = (
PermissionProcessor(),
PartnerFieldTypeProcessor(),
SourceProcessor(),
ProductionLocationDataProcessor(),
GeocodingProcessor()
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,218 @@
import json
from typing import Dict, List, Mapping, Optional, Tuple

import jsonschema
from jsonschema.exceptions import ValidationError as JsonSchemaValidationError
from jsonschema.validators import Draft202012Validator
from rest_framework import status

from api.moderation_event_actions.creation.location_contribution \
.processors.contribution_processor import ContributionProcessor
from api.moderation_event_actions.creation.dtos.create_moderation_event_dto \
import CreateModerationEventDTO
from api.models.partner_field import PartnerField
from api.constants import APIV1CommonErrorMessages


class PartnerFieldTypeProcessor(ContributionProcessor):

TYPE_VALIDATORS = {
'int': lambda value: isinstance(value, int)
and not isinstance(value, bool),
'float': lambda value: isinstance(value, float)
and not isinstance(value, bool),
'string': lambda value: isinstance(value, str),
'object': lambda value: isinstance(
value, (dict, list)
),
}

FORMAT_CHECKER = jsonschema.FormatChecker()

def process(
self,
event_dto: CreateModerationEventDTO
) -> CreateModerationEventDTO:

raw = event_dto.raw_data or {}
if not raw:
return super().process(event_dto)

incoming_keys = set(raw.keys())

partner_fields_qs = PartnerField.objects \
.filter(name__in=incoming_keys) \
.values("name", "type", "json_schema")
partner_fields_data: Dict[str, Dict] = {
field["name"]: {
"type": field["type"],
"json_schema": (
PartnerFieldTypeProcessor.__parse_json_schema(
field["json_schema"]
)
)
}
for field in partner_fields_qs
}

if not partner_fields_data:
return super().process(event_dto)

validation_errors = self.__validate_partner_fields(
raw,
partner_fields_data
)

if validation_errors:
event_dto.errors = self.__transform_validation_errors(
validation_errors
)
event_dto.status_code = status.HTTP_422_UNPROCESSABLE_ENTITY
return event_dto

return super().process(event_dto)

@staticmethod
def __validate_partner_fields(
raw: Mapping[str, object],
partner_fields_data: Dict[str, Dict]
) -> List[Tuple[str, str]]:

validation_errors: List[Tuple[str, str]] = []

for field_name, field_info in partner_fields_data.items():
value = raw.get(field_name)
if value is None:
continue

field_type = field_info.get("type")
json_schema = field_info.get("json_schema")

error = PartnerFieldTypeProcessor.__validate_single_field(
field_name, value, field_type, json_schema
)
if error:
validation_errors.append(error)

return validation_errors

@staticmethod
def __validate_single_field(
field_name: str,
value: object,
field_type: str,
json_schema: object
) -> Optional[Tuple[str, str]]:

use_json_schema = (
field_type == PartnerField.OBJECT and json_schema
)

if use_json_schema:
return PartnerFieldTypeProcessor.__validate_with_json_schema(
field_name, value, json_schema
)

return PartnerFieldTypeProcessor.__validate_with_type_validator(
field_name, value, field_type
)

@staticmethod
def __validate_with_json_schema(
field_name: str,
value: object,
json_schema: object
) -> Optional[Tuple[str, str]]:

try:
validator = Draft202012Validator(
schema=json_schema,
format_checker=PartnerFieldTypeProcessor.FORMAT_CHECKER
)
validator.validate(instance=value)
return None
except JsonSchemaValidationError as e:
error_message = PartnerFieldTypeProcessor \
.__format_json_schema_error(
e
)
return (field_name, error_message)
except Exception as e:
return (field_name, f"Schema validation error: {str(e)}")

@staticmethod
def __format_json_schema_error(
error: JsonSchemaValidationError
) -> str:

error_message = error.message
error_path = PartnerFieldTypeProcessor.__extract_error_path(error)

if error_path:
return f"{error_path}: {error_message}"

return error_message

@staticmethod
def __extract_error_path(
error: JsonSchemaValidationError
) -> Optional[str]:

if hasattr(error, 'absolute_path') and error.absolute_path:
return ".".join(str(p) for p in error.absolute_path)

if hasattr(error, 'path') and error.path:
return ".".join(str(p) for p in error.path)

return None

@staticmethod
def __validate_with_type_validator(
field_name: str,
value: object,
field_type: str
) -> Optional[Tuple[str, str]]:

if field_type not in PartnerFieldTypeProcessor.TYPE_VALIDATORS:
return None

validator = PartnerFieldTypeProcessor.TYPE_VALIDATORS[field_type]
if validator(value):
return None

return (
field_name,
f'Field {field_name} must be {field_type}, '
f'not {type(value).__name__}.'
)

@staticmethod
def __parse_json_schema(json_schema: object) -> Optional[dict]:
if json_schema is None:
return None

if isinstance(json_schema, dict):
return json_schema

if isinstance(json_schema, str):
try:
return json.loads(json_schema)
except (json.JSONDecodeError, TypeError):
return None

return None

@staticmethod
def __transform_validation_errors(
validation_errors: List[Tuple[str, str]]
) -> Dict:
return {
'detail': APIV1CommonErrorMessages.COMMON_REQ_BODY_ERROR,
'errors': [
{
'field': field_name,
'detail': error_message,
}
for field_name, error_message in validation_errors
],
}
Loading