Edpub 1735 consolidate upload endpoints

[ckd002]

Description

Consolidates the 4 upload endpoints (getPostUrl, getGroupUploadUrl, getAttachmentUploadUrl, & getUploadStepUrl) to a single getURL endpoint.

Linked JIRA Task or Github Issue

JIRA Task: EDPUB-1735

Types of changes

What types of changes does your code introduce to Earthdata Pub (EDPub)?
Put an x in the boxes that apply

• Bugfix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation Update (if adding or updating the existing documentation resources)
• Other (if none of the other choices apply)

Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code.

• I have read the Contributing Guide
• I have updated the CHANGELOG
• Lint and unit tests pass locally with my changes
• I have added tests that prove my fix is effective or that my feature works
• I have added the necessary documentation (if appropriate)
• Any dependent changes have been merged and published in downstream modules

Validation Steps

This will help us get a jump start on validating your PR by describing the steps to replicate
and validate the expected behavior. (For an example of good validation instructions, check out Bryan's Bouncy Ball PR.)

1. Make sure all merge request checks have passed (CI/CD).
2. Pull related branches to SIT.
3. Set up authentication for curl access

Test the /api/data/upload/form/getUrl endpoint

1. Copy a submission ID from an existing or new submission
2. Hit the endpoint with the expected parameters - replacing SUBMISSION_ID with the value copied from step 1.

curl -H "Authorization: $TOKEN" -XPOST "https://pub.sit.earthdata.nasa.gov/api/data/upload/form/getUrl" -d '{ "submission_id": "SUBMISSION_ID", "file_name": "FileName", "file_type": "FileType", "checksum_value": "ChecksumString", "file_category": "sample"}' | jq .

3. Verify that the key returned matches the format ${submissionId}/${fileCategory}/${user}/${fileName}

Test the /api/data/upload/group/getUrl endpoint

1. Copy a group ID from any of the DAACS
2. Hit the endpoint with the expected parameters - replacing GROUP_ID with the value copied from step 1.

curl  -H "Authorization: $TOKEN" -XPOST "https://pub.sit.earthdata.nasa.gov/api/data/upload/group/getUrl" -d '{"file_name": "FileName",  "file_type": "FileType", "checksum_value": "ChecksumString", "prefix": "abc", "group_id": "GROUP_ID"}

3. Verify that the key returned matches the format group/${groupShortName}/${prefix}/${fileName}

Test the /api/data/upload/attachment/getUrl endpoint

1. Copy a conversation ID from an existing request
2. Hit the endpoint with the expected parameters - replacing CONVERSATION_ID with the value copied from step 1.

curl  -H "Authorization: $TOKEN" -XPOST "https://pub.sit.earthdata.nasa.gov/api/data/upload/attachment/getUrl" -d '{  "file_name": "FileName", "file_type": "FileType", "checksum_value": "ChecksumString","conversation_id": "CONVERSATION_ID"}' | jq .

3. Verify that the key returned matches the format drafts/${conversationId}/${user}/${fileName}

Test the /api/data/upload/step/getUrl endpoint

1. Copy a submission ID from an existing or new submission
2. Hit the endpoint with the expected parameters - replacing SUBMISSION_ID with the value copied from step 1.

curl -H "Authorization: $TOKEN" -XPOST "https://pub.sit.earthdata.nasa.gov/api/data/upload/step/getUrl" -d '{ "submission_id": "SUBMISSION_ID", "file_name": "FileName", "file_type": "FileType", "checksum_value": "ChecksumString","file_category": "documentation","destination": "StepDestination"}' | jq .

3. Verify that the key returned matches the format ${uploadDestination}/${submissionId}/${fileCategory}/${user}/${fileName}

Note on local testing instead

1. In your local docker-compose file add USE_CUE_UPLOAD=false as an environment variable (CUE upload setting doesn't work locally)
2. Start EDPUB locally
3. Navigate to the swagger and authenticate
4. Click the "Try it out" option on the /api/data/upload/{type}/getUrl endpoint
5. Try each type with the expected parameters verifying that the keys match the expectations mentioned above

Further comments

The goal of this ticket was code de-duplication. My initial hope was to get all the backend code into a single function. In practice that seemed less maintainable as each type of upload has different parameters and needs to perform different permission checks. I went with a structure where we have a singe endpoint and a main function that relies on the various key generation methods (which were the heart of the previous endpoints) to do the validation and produce the required key.

I also went ahead and added some parameter validation steps that were missing from the original endpoints that were identified in the API verification work.