feat(minio): Add MinIO compatibility for lifecycle, object lock, and versioning#531
feat(minio): Add MinIO compatibility for lifecycle, object lock, and versioning#531weisd wants to merge 4 commits intos3s-project:mainfrom
Conversation
…ioning - Add minio-patches.json: DelMarkerExpiration, expiry_updated_at, del_marker_expiration, expired_object_all_versions - Codegen: BucketLifecycleConfiguration accepts both root element names (LifecycleConfiguration, BucketLifecycleConfiguration) - Codegen: skip unknown elements in BucketLifecycleConfiguration - Codegen: PutBucketVersioning uses take_versioning_configuration - Codegen: PutObjectLockConfiguration uses take_opt_object_lock_configuration - http: add take_versioning_configuration, take_opt_object_lock_configuration for literal " Enabled " (MinIO legacy format) - xml: add named_element_any for alternate root element names All changes are gated by #[cfg(feature = "minio")] or codegen patch.
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
There was a problem hiding this comment.
Pull request overview
This PR extends the minio feature in s3s to better ingest/migrate MinIO bucket metadata by tolerating MinIO-specific XML/body format differences for lifecycle, object lock, and versioning configurations.
Changes:
- Adds MinIO-specific Smithy patches to introduce/extend lifecycle-related shapes/fields (e.g.,
ExpiryUpdatedAt,DelMarkerExpiration,ExpiredObjectAllVersions). - Updates XML deserialization/codegen to accept alternate lifecycle root element names and (intended) root-level forward-compat handling.
- Adds HTTP body deserialization compatibility for MinIO’s legacy literal
"Enabled"bodies for object lock and versioning; adds a MinIO lifecycle root-name test.
Reviewed changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| data/minio-patches.json | Defines MinIO-specific model patches for lifecycle extensions. |
| crates/s3s/tests/xml.rs | Adjusts a lifecycle test and adds a MinIO test covering both lifecycle root element names. |
| crates/s3s/src/xml/de.rs | Adds named_element_any support in the XML deserializer for alternate root names. |
| crates/s3s/src/http/de.rs | Adds MinIO-only HTTP body parsing for legacy literal "Enabled" object lock/versioning configs. |
| crates/s3s/src/header/generated.rs | Regenerated header constants formatting/output. |
| crates/s3s/src/header/generated_minio.rs | Regenerated MinIO header constants formatting/output. |
| crates/s3s/src/access/generated.rs | Regenerated access trait output (currently not rustfmt-stable). |
| crates/s3s/src/access/generated_minio.rs | Regenerated MinIO access trait output (currently not rustfmt-stable). |
| codegen/src/v1/xml.rs | Codegen support for alternate lifecycle root names + intended unknown-root-element tolerance. |
| codegen/src/v1/ops.rs | Codegen support for MinIO legacy literal bodies for object lock/versioning operations. |
| codegen/src/v1/mod.rs | Makes Patch public and threads patch selection through codegen entrypoints. |
| } | ||
|
|
||
| g!("Ok(())"); | ||
| g!("}}"); | ||
| } | ||
| g!("_ => Err(DeError::UnexpectedTagName)"); | ||
| // MinIO compatibility: skip unknown elements for BucketLifecycleConfiguration | ||
| if ty.name == "BucketLifecycleConfiguration" && matches!(patch, Some(Patch::Minio)) { |
There was a problem hiding this comment.
The MinIO-only unknown-element handling for BucketLifecycleConfiguration returns Ok(()) without consuming the element’s content. With the current Deserializer::for_each_element implementation, this only works for empty elements; any unknown element that contains nested tags will cause expect_end(...) to see a nested Start event and fail. Consider adding a Deserializer helper to skip an element subtree (consume until the matching end tag, tracking nesting) and have the generated fallback arm call that skip routine instead of returning immediately.
PR Description
Summary
This PR adds MinIO compatibility to the s3s
miniofeature so that MinIO bucket metadata can be migrated to s3s-based S3 implementations (e.g., RustFS). MinIO stores bucket metadata in.metadata.bin(msgpack) with XML/JSON config blobs; migrated data can fail to parse due to format differences. These changes address those differences.Changes
1. BucketLifecycleConfiguration
LifecycleConfigurationandBucketLifecycleConfigurationas root elements (MinIO uses both).expiry_updated_at: Option<Timestamp>for MinIO’s extension.2. LifecycleRule
del_marker_expiration: Option<DelMarkerExpiration>for MinIO’s separateDelMarkerExpirationelement with<Days>.3. LifecycleExpiration
expired_object_all_versions: Option<bool>for MinIO’s extension.4. ObjectLockConfiguration & VersioningConfiguration
" Enabled ": Support MinIO’s legacy literal format (trimmed"Enabled") instead of full XML for both configs.Implementation
#[cfg(feature = "minio")].data/minio-patches.json.Testing
cargo test -p s3s(without minio) passes.cargo test -p s3s --features miniopasses.minio_bucket_lifecycle_configuration_roottest for both root element names.References