Skip to content

[OSDEV-1886] Fix script for delete Lamda@Edge functions#597

Merged
roninzp merged 17 commits intomainfrom
OSDEV-1886/test-env-destruction
Apr 23, 2025
Merged

[OSDEV-1886] Fix script for delete Lamda@Edge functions#597
roninzp merged 17 commits intomainfrom
OSDEV-1886/test-env-destruction

Conversation

@roninzp
Copy link
Contributor

@roninzp roninzp commented Apr 18, 2025

Fixed the script to run within the Destroy Environment GitHub workflow to delete the Lambda@Edge functions before destroying the infrastructure.

@roninzp roninzp temporarily deployed to Quality Environment April 18, 2025 15:27 — with GitHub Actions Inactive
@roninzp roninzp temporarily deployed to Quality Environment April 18, 2025 15:27 — with GitHub Actions Inactive
@roninzp roninzp temporarily deployed to Quality Environment April 18, 2025 15:27 — with GitHub Actions Inactive
@roninzp roninzp temporarily deployed to Quality Environment April 18, 2025 15:27 — with GitHub Actions Inactive
@roninzp roninzp temporarily deployed to Quality Environment April 18, 2025 15:27 — with GitHub Actions Inactive
@roninzp roninzp temporarily deployed to Quality Environment April 18, 2025 15:27 — with GitHub Actions Inactive
@roninzp roninzp temporarily deployed to Quality Environment April 18, 2025 15:27 — with GitHub Actions Inactive
@roninzp roninzp temporarily deployed to Quality Environment April 18, 2025 15:27 — with GitHub Actions Inactive
@roninzp roninzp temporarily deployed to Quality Environment April 18, 2025 15:27 — with GitHub Actions Inactive
@roninzp roninzp temporarily deployed to Quality Environment April 18, 2025 15:27 — with GitHub Actions Inactive
@roninzp roninzp temporarily deployed to Quality Environment April 18, 2025 15:27 — with GitHub Actions Inactive
@roninzp roninzp temporarily deployed to Quality Environment April 18, 2025 15:27 — with GitHub Actions Inactive
@roninzp roninzp temporarily deployed to Quality Environment April 18, 2025 15:27 — with GitHub Actions Inactive
@roninzp roninzp temporarily deployed to Quality Environment April 18, 2025 15:27 — with GitHub Actions Inactive
@roninzp roninzp temporarily deployed to Quality Environment April 18, 2025 15:27 — with GitHub Actions Inactive
@roninzp roninzp temporarily deployed to Quality Environment April 18, 2025 15:27 — with GitHub Actions Inactive
@roninzp roninzp temporarily deployed to Quality Environment April 18, 2025 15:27 — with GitHub Actions Inactive
@roninzp roninzp temporarily deployed to Quality Environment April 18, 2025 15:27 — with GitHub Actions Inactive
@coderabbitai
Copy link

coderabbitai bot commented Apr 18, 2025

Warning

Rate limit exceeded

@roninzp has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 4 minutes and 23 seconds before requesting another review.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

📥 Commits

Reviewing files that changed from the base of the PR and between 18d0f82 and a921a24.

📒 Files selected for processing (1)
  • .gitignore (1 hunks)
📝 Walkthrough

Walkthrough

This update modifies several components related to Lambda@Edge function deletion and workflow management. The .gitignore file is extended to ignore backup and temporary files. The Lambda deletion script is updated to improve the removal process of Lambda@Edge associations in CloudFront distributions, introduce a retry mechanism for Lambda version deletions, and exclude the "$LATEST" version. The GitHub Actions workflow is corrected to reference the accurate dependency job name. Additionally, the release notes are updated to document a bugfix addressing the correct order of Lambda@Edge function deletion in the destroy workflow.

Changes

File(s) Change Summary
.gitignore Added patterns to ignore files ending with ~ and .bak.
deployment/delete_lambda Improved Lambda version deletion logic: excludes "$LATEST", adds retry on replication errors, refines CloudFront config update, and removes polling for function deletion.
.github/workflows/destroy.yml Corrected the needs attribute to match the actual job name destroy_lambda_edge_functions.
doc/release/RELEASE-NOTES.md Added a bugfix entry for OSDEV-1886, documenting the fix to ensure Lambda@Edge function deletion before infrastructure destruction.

Sequence Diagram(s)

sequenceDiagram
    participant Workflow
    participant DeleteLambdaScript
    participant AWS Lambda
    participant AWS CloudFront

    Workflow->>DeleteLambdaScript: Trigger destroy workflow
    DeleteLambdaScript->>AWS CloudFront: Get distribution config
    DeleteLambdaScript->>DeleteLambdaScript: Remove Lambda@Edge associations
    DeleteLambdaScript->>AWS CloudFront: Update distribution config
    loop For each Lambda version (excluding "$LATEST")
        DeleteLambdaScript->>AWS Lambda: Attempt to delete version
        alt Replication error
            DeleteLambdaScript->>DeleteLambdaScript: Wait 5 minutes, retry
        else Success
            DeleteLambdaScript->>Workflow: Exit on success
        else Other error
            DeleteLambdaScript->>Workflow: Print error, exit with failure
        end
    end
Loading

Possibly related PRs

  • #567: Introduces and manages Lambda@Edge deletion logic, directly preceding and informing the current changes.
  • #579: Alters the same deployment/delete_lambda script, focusing on CloudFront distribution variable handling.
  • #582: Adds a new Lambda@Edge function and modifies workflows for deployment and deletion, related through Lambda@Edge function management.

Suggested reviewers

  • VadimKovalenkoSNF
  • vladsha-dev

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@roninzp roninzp temporarily deployed to Quality Environment April 18, 2025 15:30 — with GitHub Actions Inactive
@barecheck
Copy link

barecheck bot commented Apr 18, 2025

React App | Jest test suite - Code coverage report

Total: 33.99%

Your code coverage diff: 0.00% ▴

✅ All code changes are covered

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
deployment/environments/terraform-preprod.tfvars (1)

31-32: Reduced CLI compute resources

The CLI task CPU and memory have been significantly reduced. The memory allocation of 124MB seems particularly low and might be a typo (common values are 128, 256, 512, etc.).

- cli_fargate_memory = "124"
+ cli_fargate_memory = "128"
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between bdfb700 and fabc6e3.

📒 Files selected for processing (3)
  • .gitignore (1 hunks)
  • deployment/delete_lambda (3 hunks)
  • deployment/environments/terraform-preprod.tfvars (2 hunks)
⏰ Context from checks skipped due to timeout of 90000ms (10)
  • GitHub Check: run-flake8-linter
  • GitHub Check: run-integration-test-code-quality
  • GitHub Check: run-dd-code-quality
  • GitHub Check: run-countries-code-quality
  • GitHub Check: run-contricleaner-code-quality
  • GitHub Check: get-base-branch-dd-cov
  • GitHub Check: get-base-branch-countries-cov
  • GitHub Check: get-base-branch-contricleaner-cov
  • GitHub Check: run-django-code-quality
  • GitHub Check: get-base-branch-django-cov
🔇 Additional comments (6)
.gitignore (1)

97-98: LGTM: Standard ignore patterns for backup files added

These patterns will prevent backup and temporary files created by various text editors from being committed to the repository, which is a good housekeeping practice.

deployment/delete_lambda (2)

1-1: Added debugging flags to shell execution

The -xv flags will enable verbose logging and command tracing, which is helpful for troubleshooting script execution issues. This is appropriate for a maintenance script dealing with infrastructure changes.


22-22: Improved version selection logic

The updated JQ expression now properly filters out the literal "$LATEST" version before selecting the maximum numeric version. This is more robust than the previous approach, especially if versions aren't returned in numeric order.

deployment/environments/terraform-preprod.tfvars (3)

19-19: Significant downgrade of RDS instance type

Changed from a higher tier instance to db.t3.micro, which is a burstable instance with limited CPU and memory. Ensure this is sufficient for your preprod database workload.


25-25: Reduced app ECS resources

App container count reduced to 1 and compute resources scaled down. This is appropriate for cost optimization in preprod, but verify that it meets your testing requirements.

Also applies to: 28-29


54-55: Adjusted resource allocations for additional services

Resources for data deduplication, OpenSearch, and Logstash components have been adjusted for the preprod environment. These changes align with the overall strategy to optimize resource usage in the preprod environment.

Also applies to: 58-58, 61-61

@barecheck
Copy link

barecheck bot commented Apr 18, 2025

Dedupe Hub App | Unittest test suite - Code coverage report

Total: 56.14%

Your code coverage diff: 0.00% ▴

✅ All code changes are covered

@roninzp roninzp temporarily deployed to Quality Environment April 18, 2025 15:34 — with GitHub Actions Inactive
@roninzp roninzp temporarily deployed to Quality Environment April 18, 2025 15:35 — with GitHub Actions Inactive
@roninzp roninzp temporarily deployed to Quality Environment April 18, 2025 15:35 — with GitHub Actions Inactive
@barecheck
Copy link

barecheck bot commented Apr 18, 2025

Countries App | Unittest test suite - Code coverage report

Total: 100%

Your code coverage diff: 0.00% ▴

✅ All code changes are covered

@barecheck
Copy link

barecheck bot commented Apr 18, 2025

Contricleaner App | Unittest test suite - Code coverage report

Total: 98.91%

Your code coverage diff: 0.00% ▴

✅ All code changes are covered

@roninzp roninzp temporarily deployed to Quality Environment April 22, 2025 10:55 — with GitHub Actions Inactive
@roninzp roninzp temporarily deployed to Quality Environment April 22, 2025 10:55 — with GitHub Actions Inactive
@roninzp roninzp temporarily deployed to Quality Environment April 22, 2025 10:55 — with GitHub Actions Inactive
@roninzp roninzp temporarily deployed to Quality Environment April 22, 2025 10:55 — with GitHub Actions Inactive
@roninzp roninzp temporarily deployed to Quality Environment April 22, 2025 10:55 — with GitHub Actions Inactive
@roninzp roninzp had a problem deploying to Quality Environment April 22, 2025 10:55 — with GitHub Actions Failure
@roninzp roninzp temporarily deployed to Quality Environment April 22, 2025 10:55 — with GitHub Actions Inactive
@roninzp roninzp temporarily deployed to Quality Environment April 22, 2025 10:55 — with GitHub Actions Inactive
@roninzp roninzp temporarily deployed to Quality Environment April 22, 2025 10:58 — with GitHub Actions Inactive
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (1)
deployment/delete_lambda (1)

42-49: Use temporary files and automatic cleanup for distribution config artefacts

Persisting full-config.json, dist-config.json, and new-config.json in the working directory may leak sensitive data and pollute the repo when the script is run locally.
Leverage mktemp and a trap to remove the files on exit.

-aws cloudfront get-distribution-config --id $DISTRIBUTION_ID > full-config.json
-ETAG=$(cat full-config.json | jq -r '.ETag')
-...
-cat full-config.json | jq '.DistributionConfig' > dist-config.json
+FULL_CFG="$(mktemp)"
+DIST_CFG="$(mktemp)"
+NEW_CFG="$(mktemp)"
+trap 'rm -f "$FULL_CFG" "$DIST_CFG" "$NEW_CFG"' EXIT
+
+aws cloudfront get-distribution-config --id "$DISTRIBUTION_ID" > "$FULL_CFG"
+ETAG=$(jq -r '.ETag' < "$FULL_CFG")
+jq '.DistributionConfig' "$FULL_CFG" > "$DIST_CFG"

Remember to replace later occurrences of dist-config.json/new-config.json with the new variables.

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between c4f89f6 and bc3325d.

📒 Files selected for processing (1)
  • deployment/delete_lambda (3 hunks)
⏰ Context from checks skipped due to timeout of 90000ms (13)
  • GitHub Check: run-integration-test-code-quality
  • GitHub Check: run-flake8-linter
  • GitHub Check: run-fe-code-quality
  • GitHub Check: run-eslint-linter-and-prettier-formatter
  • GitHub Check: run-django-code-quality
  • GitHub Check: run-dd-code-quality
  • GitHub Check: run-countries-code-quality
  • GitHub Check: run-contricleaner-code-quality
  • GitHub Check: get-base-branch-fe-cov
  • GitHub Check: get-base-branch-dd-cov
  • GitHub Check: get-base-branch-contricleaner-cov
  • GitHub Check: get-base-branch-countries-cov
  • GitHub Check: get-base-branch-django-cov

Copy link
Contributor

@vladsha-dev vladsha-dev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. I have a question: why did you test the deletion of only the AddSecurityHeaders function, and not both functions(AddSecurityHeaders and RedirectToS3origin), is it enough in this case? https://github.com/opensupplyhub/open-supply-hub/actions/runs/14594317960/job/40940180830
  2. I have left a few comments.

@roninzp
Copy link
Contributor Author

roninzp commented Apr 22, 2025

I have a question: why did you test the deletion of only the AddSecurityHeaders function, and not both functions(AddSecurityHeaders and RedirectToS3origin), is it enough in this case? https://github.com/opensupplyhub/open-supply-hub/actions/runs/14594317960/job/40940180830

@vladsha-dev

The first function was removed by the same script during local tests. Since the script removes only one function at a time, I decided that testing the removal of just one function in the pipeline would be sufficient, without redeploying the environment.

Copy link
Contributor

@vladsha-dev vladsha-dev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@sonarqubecloud
Copy link

Copy link
Contributor

@mazursasha1990 mazursasha1990 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants