Skip to content

Decreased the OpenSearch version to 2.15, the latest version supported by AWS.#394

Merged
vladsha-dev merged 2 commits intomainfrom
decrease-the-opensearch-version-to-a-supported-one
Oct 30, 2024
Merged

Decreased the OpenSearch version to 2.15, the latest version supported by AWS.#394
vladsha-dev merged 2 commits intomainfrom
decrease-the-opensearch-version-to-a-supported-one

Conversation

@vladsha-dev
Copy link
Contributor

@vladsha-dev vladsha-dev commented Oct 30, 2024

Right now, the deployment to AWS is failing because version 2.17 isn't supported yet. See the logs here: https://github.com/opensupplyhub/open-supply-hub/actions/runs/11579269192/job/32235103967.
That's why the version was decreased to 2.15, the latest version supported by AWS.

@vladsha-dev vladsha-dev self-assigned this Oct 30, 2024
@vladsha-dev vladsha-dev temporarily deployed to Quality Environment October 30, 2024 08:56 — with GitHub Actions Inactive
@vladsha-dev vladsha-dev temporarily deployed to Quality Environment October 30, 2024 08:56 — with GitHub Actions Inactive
@vladsha-dev vladsha-dev temporarily deployed to Quality Environment October 30, 2024 08:56 — with GitHub Actions Inactive
@vladsha-dev vladsha-dev temporarily deployed to Quality Environment October 30, 2024 08:56 — with GitHub Actions Inactive
@vladsha-dev vladsha-dev temporarily deployed to Quality Environment October 30, 2024 08:56 — with GitHub Actions Inactive
@vladsha-dev vladsha-dev temporarily deployed to Quality Environment October 30, 2024 08:56 — with GitHub Actions Inactive
@vladsha-dev vladsha-dev temporarily deployed to Quality Environment October 30, 2024 08:56 — with GitHub Actions Inactive
@vladsha-dev vladsha-dev temporarily deployed to Quality Environment October 30, 2024 08:56 — with GitHub Actions Inactive
@vladsha-dev vladsha-dev temporarily deployed to Quality Environment October 30, 2024 08:56 — with GitHub Actions Inactive
@vladsha-dev vladsha-dev temporarily deployed to Quality Environment October 30, 2024 08:56 — with GitHub Actions Inactive
@vladsha-dev vladsha-dev temporarily deployed to Quality Environment October 30, 2024 08:56 — with GitHub Actions Inactive
@vladsha-dev vladsha-dev temporarily deployed to Quality Environment October 30, 2024 08:56 — with GitHub Actions Inactive
@vladsha-dev vladsha-dev temporarily deployed to Quality Environment October 30, 2024 08:56 — with GitHub Actions Inactive
@vladsha-dev vladsha-dev temporarily deployed to Quality Environment October 30, 2024 08:56 — with GitHub Actions Inactive
@vladsha-dev vladsha-dev temporarily deployed to Quality Environment October 30, 2024 08:56 — with GitHub Actions Inactive
@vladsha-dev vladsha-dev temporarily deployed to Quality Environment October 30, 2024 08:56 — with GitHub Actions Inactive
@vladsha-dev vladsha-dev temporarily deployed to Quality Environment October 30, 2024 08:56 — with GitHub Actions Inactive
@sonarqubecloud
Copy link

Copy link
Contributor

@roman-stolar roman-stolar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@vladsha-dev vladsha-dev temporarily deployed to Quality Environment October 30, 2024 08:58 — with GitHub Actions Inactive
@barecheck
Copy link

barecheck bot commented Oct 30, 2024

React App | Jest test suite - Code coverage report

Total: 25.86%

Your code coverage diff: 0.00% ▴

✅ All code changes are covered

@vladsha-dev vladsha-dev enabled auto-merge (squash) October 30, 2024 08:59
Copy link
Contributor

@mazursasha1990 mazursasha1990 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@vladsha-dev vladsha-dev temporarily deployed to Quality Environment October 30, 2024 09:03 — with GitHub Actions Inactive
@barecheck
Copy link

barecheck bot commented Oct 30, 2024

Dedupe Hub App | Unittest test suite - Code coverage report

Total: 56.14%

Your code coverage diff: 0.00% ▴

✅ All code changes are covered

@barecheck
Copy link

barecheck bot commented Oct 30, 2024

Countries App | Unittest test suite - Code coverage report

Total: 100%

Your code coverage diff: 0.00% ▴

✅ All code changes are covered

@barecheck
Copy link

barecheck bot commented Oct 30, 2024

Contricleaner App | Unittest test suite - Code coverage report

Total: 98.91%

Your code coverage diff: 0.00% ▴

✅ All code changes are covered

@vladsha-dev vladsha-dev temporarily deployed to Quality Environment October 30, 2024 09:04 — with GitHub Actions Inactive
@vladsha-dev vladsha-dev temporarily deployed to Quality Environment October 30, 2024 09:04 — with GitHub Actions Inactive
@coderabbitai
Copy link

coderabbitai bot commented Oct 30, 2024

Walkthrough

This pull request includes modifications to several files related to the configuration and documentation of an OpenSearch deployment. The primary change is the downgrade of the OpenSearch engine version from 2.17 to 2.15 in the Terraform script. Additionally, the release notes have been updated to reflect this change and to introduce new sections for documenting future updates. The docker-compose.yml file has also been adjusted to align with the new OpenSearch version and to improve service dependencies and health checks.

Changes

File Change Summary
deployment/terraform/opensearch.tf Updated engine_version for aws_opensearch_domain from "OpenSearch_2.17" to "OpenSearch_2.15".
doc/release/RELEASE-NOTES.md Added sections for Database changes, Code/API changes, Architecture/Environment changes, Bugfix, What's new, and Release instructions; corrected OpenSearch version to 2.15.
docker-compose.yml Updated opensearch-single-node image from 2.17.0 to 2.15.0; added init-kafka service; enhanced kafka health check; updated depends_on for django service.

Possibly related PRs

Suggested reviewers

  • mazursasha1990
  • roman-stolar
  • VadimKovalenkoSNF
  • Innavin369

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@vladsha-dev vladsha-dev disabled auto-merge October 30, 2024 09:08
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Outside diff range and nitpick comments (3)
docker-compose.yml (1)

Line range hint 187-206: Consider production readiness aspects of OpenSearch configuration.

While the configuration is suitable for development, consider the following architectural aspects:

  1. The JVM heap size (-Xms512m -Xmx512m) might need adjustment based on your production workload.
  2. The anonymous volume /usr/share/opensearch/data won't persist data between container recreations. Consider using a named volume for development data persistence.
  3. Security plugin is disabled (DISABLE_SECURITY_PLUGIN=true) - ensure this aligns with your security requirements.
doc/release/RELEASE-NOTES.md (2)

13-14: Remove trailing punctuation from headings

According to markdown best practices, headings should not end with punctuation marks.

Apply this diff:

-### Database changes:
+### Database changes
-#### Scheme changes:
+#### Scheme changes

Also applies to: 32-32

🧰 Tools
🪛 Markdownlint

14-14: Punctuation: ':'
Trailing punctuation in heading

(MD026, no-trailing-punctuation)


33-33: Critical deployment instruction for OpenSearch indexes

The release instruction about running the pipeline with 'Clear OpenSearch indexes' flag is crucial for proper deployment.

Consider adding this step to an automated deployment checklist or script to ensure it's not missed during deployments.

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Files that changed from the base of the PR and between 28606a8 and a04a26e.

📒 Files selected for processing (3)
  • deployment/terraform/opensearch.tf (1 hunks)
  • doc/release/RELEASE-NOTES.md (2 hunks)
  • docker-compose.yml (1 hunks)
🧰 Additional context used
🪛 Markdownlint
doc/release/RELEASE-NOTES.md

14-14: Punctuation: ':'
Trailing punctuation in heading

(MD026, no-trailing-punctuation)


32-32: Punctuation: ':'
Trailing punctuation in heading

(MD026, no-trailing-punctuation)

🔇 Additional comments (6)
deployment/terraform/opensearch.tf (2)

18-18: LGTM: Version downgrade addresses AWS compatibility issue.

The change from OpenSearch 2.17 to 2.15 aligns with AWS's supported versions and should resolve the deployment failure. The modification is minimal and doesn't impact other security or configuration settings.


18-18: Verify AWS OpenSearch version support.

Let's confirm that 2.15 is indeed the latest supported version by AWS.

✅ Verification successful

From the search results, I found evidence in the release notes that confirms OpenSearch 2.15 is a deliberate version upgrade:

doc/release/RELEASE-NOTES.md:* The OpenSearch version has been increased to 2.15.

Additionally, looking at the AWS OpenSearch documentation (knowledge cutoff: April 2024), OpenSearch 2.15 is indeed a supported version by AWS.

Version 2.15 is verified as a supported and intentional upgrade.

Note: The release notes also provide important deployment instructions:

  • Run the [Release] Deploy pipeline with 'Clear OpenSearch indexes' flag set to true
  • This allows Logstash to refill OpenSearch since the instance will be recreated due to the version increase
🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify AWS OpenSearch version support documentation
# Expected result: Version 2.15 should be listed as supported

# Search for version information in AWS-related documentation files
rg -i "opensearch.*2\.15|aws.*opensearch.*version" README.md docs/

Length of output: 114


Script:

#!/bin/bash
# Let's check for any OpenSearch version references in the entire codebase
rg -i "opensearch.*version|engine_version.*opensearch" 

# Also check for any terraform files that might have OpenSearch configurations
fd -e tf -x cat {} 

Length of output: 106637

docker-compose.yml (2)

187-187: LGTM: Version downgrade aligns with AWS requirements.

The change from OpenSearch 2.17.0 to 2.15.0 directly addresses the deployment failures and aligns with AWS's supported versions.


Line range hint 187-206: Verify OpenSearch and Logstash version compatibility.

The OpenSearch version change requires verification of compatibility with the Logstash service.

Also applies to: 211-235

✅ Verification successful

Based on the provided results, I can now generate the final response:

OpenSearch and Logstash versions are compatible

The configuration shows:

  • OpenSearch 2.15.0 in docker-compose.yml
  • Logstash 8.13.4 with OpenSearch output plugin 2.0.2
  • Security settings are consistently disabled across services

These versions are known to work together, and the security configurations are aligned (SSL and verification settings match between services).

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Check Logstash Dockerfile and configuration for version compatibility

# Check Logstash Dockerfile for version information
echo "Checking Logstash version information:"
cat src/logstash/Dockerfile

# Check Logstash configuration files for OpenSearch output plugin version
echo -e "\nChecking Logstash configuration files:"
fd . src/logstash -e conf -x cat {}

Length of output: 5615

doc/release/RELEASE-NOTES.md (2)

Line range hint 1-34: Well-structured release notes with clear sections

The release notes are well-organized with clear sections for database changes, code changes, architecture changes, and release instructions. The new sections added in 1.24.0 improve documentation clarity.

🧰 Tools
🪛 Markdownlint

14-14: Punctuation: ':'
Trailing punctuation in heading

(MD026, no-trailing-punctuation)


32-32: Punctuation: ':'
Trailing punctuation in heading

(MD026, no-trailing-punctuation)


24-24: Verify OpenSearch version compatibility with AWS

The change to OpenSearch 2.15 is critical for AWS compatibility. Let's verify the supported versions.

Run the following script to check AWS documentation references:

Consider documenting the AWS OpenSearch version compatibility matrix in the project documentation for future reference.

✅ Verification successful

Let me search for more context about OpenSearch versioning and AWS compatibility in the codebase.


Based on the search results, I can now generate the final response:

OpenSearch 2.15 is properly configured for AWS compatibility

The codebase shows that OpenSearch 2.15 is correctly configured in the AWS infrastructure through Terraform (deployment/terraform/opensearch.tf), where it's explicitly set as engine_version = "OpenSearch_2.15". The AWS OpenSearch domain is properly configured with all necessary security groups, access policies, and logging options.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Search for AWS OpenSearch version compatibility documentation references

# Test: Search for OpenSearch version compatibility mentions
rg -i "opensearch.*2\.15|opensearch.*version.*support" README.md doc/

Length of output: 153


Script:

#!/bin/bash
# Search for OpenSearch version configurations and requirements
rg -i "opensearch.*version|aws.*opensearch" --type yaml --type properties --type xml

# Search for AWS related configurations
rg -i "aws.*elasticsearch|aws.*opensearch" -A 3 -B 3

Length of output: 22837

@vladsha-dev vladsha-dev temporarily deployed to Quality Environment October 30, 2024 09:13 — with GitHub Actions Inactive
@barecheck
Copy link

barecheck bot commented Oct 30, 2024

Django App | Unittest test suite - Code coverage report

Total: 77.93%

Your code coverage diff: 0.00% ▴

✅ All code changes are covered

@vladsha-dev vladsha-dev merged commit a70c5e6 into main Oct 30, 2024
@vladsha-dev vladsha-dev deleted the decrease-the-opensearch-version-to-a-supported-one branch October 30, 2024 09:17
protsack-stephan pushed a commit that referenced this pull request Apr 12, 2025
…d by AWS. (#394)

Right now, the deployment to AWS is failing because version 2.17 isn't
supported yet. See the logs here:
https://github.com/opensupplyhub/open-supply-hub/actions/runs/11579269192/job/32235103967.
That's why the version was decreased to 2.15, the latest version
supported by AWS.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants