Skip to content

Fix github api rate limit issue when test script try to fetch github issues state#19116

Merged
wangxin merged 5 commits intosonic-net:masterfrom
lerry-lee:chunangli/fix_github_api_rate_limit
Jul 1, 2025
Merged

Fix github api rate limit issue when test script try to fetch github issues state#19116
wangxin merged 5 commits intosonic-net:masterfrom
lerry-lee:chunangli/fix_github_api_rate_limit

Conversation

@lerry-lee
Copy link
Copy Markdown
Contributor

Description of PR

Fix github api rate limit issue by introduce a proxy.
During peak periods of PR testing, some test scripts encounter GitHub API rate limit issues when attempting to fetch GitHub issues. This is due to unauthenticated requests being subject to a strict limit of 60 requests per hour per IP address.

This PR introduces a workaround by routing these requests through a proxy service that performs authenticated GitHub API calls (with a higher limit of 5000 requests per hour per GitHub App) and supports caching.

If the proxy is unavailable or not configured, the system falls back to the original unauthenticated behavior, ensuring it does not worsen the existing situation.

For details, refer to GitHub API rate limits documentation:
https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api?apiVersion=2022-11-28#primary-rate-limit-for-unauthenticated-users

Type of change

  • Bug fix
  • Testbed and Framework(new/improvement)
  • New Test case
    • Skipped for non-supported platforms
  • Test case improvement

Back port request

  • 202205
  • 202305
  • 202311
  • 202405
  • 202411
  • 202505

Approach

What is the motivation for this PR?

Fix github api rate limit issue by introduce a proxy.

How did you do it?

When fetch github issues, try proxy first, if fails, fallback to original unauthenticated behavior.
And to support setting proxy environment vars, add args for setup-container.sh

How did you verify/test it?

When no proxy, the pr tests are good as before.
When sets proxy, the pr tests are good as well. (check log can see it via proxy to fetch github issues)

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

@mssonicbld
Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines
Copy link
Copy Markdown

Azure Pipelines successfully started running 1 pipeline(s).

@lerry-lee lerry-lee changed the title Fix github api rate limit issue by introduce a proxy Fix github api rate limit issue when test script try to fetch github issues state Jun 20, 2025
@mssonicbld
Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines
Copy link
Copy Markdown

Azure Pipelines successfully started running 1 pipeline(s).

@lerry-lee lerry-lee requested a review from wangxin June 30, 2025 02:28
@lerry-lee
Copy link
Copy Markdown
Contributor Author

/azpw run

@mssonicbld
Copy link
Copy Markdown
Collaborator

/AzurePipelines run

@azure-pipelines
Copy link
Copy Markdown

Azure Pipelines successfully started running 1 pipeline(s).

@mssonicbld
Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines
Copy link
Copy Markdown

Azure Pipelines successfully started running 1 pipeline(s).

@mssonicbld
Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines
Copy link
Copy Markdown

Azure Pipelines successfully started running 1 pipeline(s).

@wangxin wangxin merged commit 8bc9867 into sonic-net:master Jul 1, 2025
18 checks passed
@yejianquan
Copy link
Copy Markdown
Collaborator

@mssonicbld
Copy link
Copy Markdown
Collaborator

@lerry-lee PR conflicts with 202505 branch

yejianquan pushed a commit that referenced this pull request Jul 2, 2025
…116/files (#19321)

Description of PR
[202405]manual cherry pick PR 19116 to Fix github api rate limit issue
#19116
wangxin pushed a commit that referenced this pull request Jul 9, 2025
…ix github api rate limit issue (#19478)

The PR #19116 adds -e args for setup-container.sh for all PR test pipelines.

This PR is to cherry pick the change of adding -e for setup-container to fix github api rate limit issue.
wangxin pushed a commit that referenced this pull request Jul 9, 2025
…ix github api rate limit issue (#19477)

The PR #19116 adds -e args for setup-container.sh for all PR test pipelines.

This PR is to cherry pick the change of adding -e for setup-container to fix github api rate limit issue.
sdszhang pushed a commit to sdszhang/sonic-mgmt that referenced this pull request Aug 2, 2025
nissampa pushed a commit to nissampa/sonic-mgmt_dpu_test that referenced this pull request Aug 7, 2025
…issues state (sonic-net#19116)

Fix github api rate limit issue by introduce a proxy.
During peak periods of PR testing, some test scripts encounter GitHub API rate limit issues when attempting to fetch GitHub issues. This is due to unauthenticated requests being subject to a strict limit of 60 requests per hour per IP address.

This PR introduces a workaround by routing these requests through a proxy service that performs authenticated GitHub API calls (with a higher limit of 5000 requests per hour per GitHub App) and supports caching.

If the proxy is unavailable or not configured, the system falls back to the original unauthenticated behavior, ensuring it does not worsen the existing situation.

For details, refer to GitHub API rate limits documentation:
https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api?apiVersion=2022-11-28#primary-rate-limit-for-unauthenticated-users

What is the motivation for this PR?
Fix github api rate limit issue by introduce a proxy.

How did you do it?
When fetch github issues, try proxy first, if fails, fallback to original unauthenticated behavior.
And to support setting proxy environment vars, add args for setup-container.sh

How did you verify/test it?
When no proxy, the pr tests are good as before.
When sets proxy, the pr tests are good as well. (check log can see it via proxy to fetch github issues)
ashutosh-agrawal pushed a commit to ashutosh-agrawal/sonic-mgmt that referenced this pull request Aug 14, 2025
…issues state (sonic-net#19116)

Fix github api rate limit issue by introduce a proxy.
During peak periods of PR testing, some test scripts encounter GitHub API rate limit issues when attempting to fetch GitHub issues. This is due to unauthenticated requests being subject to a strict limit of 60 requests per hour per IP address.

This PR introduces a workaround by routing these requests through a proxy service that performs authenticated GitHub API calls (with a higher limit of 5000 requests per hour per GitHub App) and supports caching.

If the proxy is unavailable or not configured, the system falls back to the original unauthenticated behavior, ensuring it does not worsen the existing situation.

For details, refer to GitHub API rate limits documentation:
https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api?apiVersion=2022-11-28#primary-rate-limit-for-unauthenticated-users

What is the motivation for this PR?
Fix github api rate limit issue by introduce a proxy.

How did you do it?
When fetch github issues, try proxy first, if fails, fallback to original unauthenticated behavior.
And to support setting proxy environment vars, add args for setup-container.sh

How did you verify/test it?
When no proxy, the pr tests are good as before.
When sets proxy, the pr tests are good as well. (check log can see it via proxy to fetch github issues)
bingwang-ms added a commit to Azure/sonic-mgmt.msft that referenced this pull request Sep 24, 2025
vidyac86 pushed a commit to vidyac86/sonic-mgmt that referenced this pull request Oct 23, 2025
…issues state (sonic-net#19116)

Fix github api rate limit issue by introduce a proxy.
During peak periods of PR testing, some test scripts encounter GitHub API rate limit issues when attempting to fetch GitHub issues. This is due to unauthenticated requests being subject to a strict limit of 60 requests per hour per IP address.

This PR introduces a workaround by routing these requests through a proxy service that performs authenticated GitHub API calls (with a higher limit of 5000 requests per hour per GitHub App) and supports caching.

If the proxy is unavailable or not configured, the system falls back to the original unauthenticated behavior, ensuring it does not worsen the existing situation.

For details, refer to GitHub API rate limits documentation:
https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api?apiVersion=2022-11-28#primary-rate-limit-for-unauthenticated-users

What is the motivation for this PR?
Fix github api rate limit issue by introduce a proxy.

How did you do it?
When fetch github issues, try proxy first, if fails, fallback to original unauthenticated behavior.
And to support setting proxy environment vars, add args for setup-container.sh

How did you verify/test it?
When no proxy, the pr tests are good as before.
When sets proxy, the pr tests are good as well. (check log can see it via proxy to fetch github issues)
opcoder0 pushed a commit to opcoder0/sonic-mgmt that referenced this pull request Dec 8, 2025
…issues state (sonic-net#19116)

Fix github api rate limit issue by introduce a proxy.
During peak periods of PR testing, some test scripts encounter GitHub API rate limit issues when attempting to fetch GitHub issues. This is due to unauthenticated requests being subject to a strict limit of 60 requests per hour per IP address.

This PR introduces a workaround by routing these requests through a proxy service that performs authenticated GitHub API calls (with a higher limit of 5000 requests per hour per GitHub App) and supports caching.

If the proxy is unavailable or not configured, the system falls back to the original unauthenticated behavior, ensuring it does not worsen the existing situation.

For details, refer to GitHub API rate limits documentation:
https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api?apiVersion=2022-11-28#primary-rate-limit-for-unauthenticated-users

What is the motivation for this PR?
Fix github api rate limit issue by introduce a proxy.

How did you do it?
When fetch github issues, try proxy first, if fails, fallback to original unauthenticated behavior.
And to support setting proxy environment vars, add args for setup-container.sh

How did you verify/test it?
When no proxy, the pr tests are good as before.
When sets proxy, the pr tests are good as well. (check log can see it via proxy to fetch github issues)

Signed-off-by: opcoder0 <[email protected]>
gshemesh2 pushed a commit to gshemesh2/sonic-mgmt that referenced this pull request Dec 16, 2025
…issues state (sonic-net#19116)

Fix github api rate limit issue by introduce a proxy.
During peak periods of PR testing, some test scripts encounter GitHub API rate limit issues when attempting to fetch GitHub issues. This is due to unauthenticated requests being subject to a strict limit of 60 requests per hour per IP address.

This PR introduces a workaround by routing these requests through a proxy service that performs authenticated GitHub API calls (with a higher limit of 5000 requests per hour per GitHub App) and supports caching.

If the proxy is unavailable or not configured, the system falls back to the original unauthenticated behavior, ensuring it does not worsen the existing situation.

For details, refer to GitHub API rate limits documentation:
https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api?apiVersion=2022-11-28#primary-rate-limit-for-unauthenticated-users

What is the motivation for this PR?
Fix github api rate limit issue by introduce a proxy.

How did you do it?
When fetch github issues, try proxy first, if fails, fallback to original unauthenticated behavior.
And to support setting proxy environment vars, add args for setup-container.sh

How did you verify/test it?
When no proxy, the pr tests are good as before.
When sets proxy, the pr tests are good as well. (check log can see it via proxy to fetch github issues)

Signed-off-by: Guy Shemesh <[email protected]>
AharonMalkin pushed a commit to AharonMalkin/sonic-mgmt that referenced this pull request Dec 16, 2025
…issues state (sonic-net#19116)

Fix github api rate limit issue by introduce a proxy.
During peak periods of PR testing, some test scripts encounter GitHub API rate limit issues when attempting to fetch GitHub issues. This is due to unauthenticated requests being subject to a strict limit of 60 requests per hour per IP address.

This PR introduces a workaround by routing these requests through a proxy service that performs authenticated GitHub API calls (with a higher limit of 5000 requests per hour per GitHub App) and supports caching.

If the proxy is unavailable or not configured, the system falls back to the original unauthenticated behavior, ensuring it does not worsen the existing situation.

For details, refer to GitHub API rate limits documentation:
https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api?apiVersion=2022-11-28#primary-rate-limit-for-unauthenticated-users

What is the motivation for this PR?
Fix github api rate limit issue by introduce a proxy.

How did you do it?
When fetch github issues, try proxy first, if fails, fallback to original unauthenticated behavior.
And to support setting proxy environment vars, add args for setup-container.sh

How did you verify/test it?
When no proxy, the pr tests are good as before.
When sets proxy, the pr tests are good as well. (check log can see it via proxy to fetch github issues)

Signed-off-by: Aharon Malkin <[email protected]>
gshemesh2 pushed a commit to gshemesh2/sonic-mgmt that referenced this pull request Dec 21, 2025
…issues state (sonic-net#19116)

Fix github api rate limit issue by introduce a proxy.
During peak periods of PR testing, some test scripts encounter GitHub API rate limit issues when attempting to fetch GitHub issues. This is due to unauthenticated requests being subject to a strict limit of 60 requests per hour per IP address.

This PR introduces a workaround by routing these requests through a proxy service that performs authenticated GitHub API calls (with a higher limit of 5000 requests per hour per GitHub App) and supports caching.

If the proxy is unavailable or not configured, the system falls back to the original unauthenticated behavior, ensuring it does not worsen the existing situation.

For details, refer to GitHub API rate limits documentation:
https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api?apiVersion=2022-11-28#primary-rate-limit-for-unauthenticated-users

What is the motivation for this PR?
Fix github api rate limit issue by introduce a proxy.

How did you do it?
When fetch github issues, try proxy first, if fails, fallback to original unauthenticated behavior.
And to support setting proxy environment vars, add args for setup-container.sh

How did you verify/test it?
When no proxy, the pr tests are good as before.
When sets proxy, the pr tests are good as well. (check log can see it via proxy to fetch github issues)

Signed-off-by: Guy Shemesh <[email protected]>
venu-nexthop pushed a commit to venu-nexthop/sonic-mgmt that referenced this pull request Jan 13, 2026
…issues state (sonic-net#19116)

Fix github api rate limit issue by introduce a proxy.
During peak periods of PR testing, some test scripts encounter GitHub API rate limit issues when attempting to fetch GitHub issues. This is due to unauthenticated requests being subject to a strict limit of 60 requests per hour per IP address.

This PR introduces a workaround by routing these requests through a proxy service that performs authenticated GitHub API calls (with a higher limit of 5000 requests per hour per GitHub App) and supports caching.

If the proxy is unavailable or not configured, the system falls back to the original unauthenticated behavior, ensuring it does not worsen the existing situation.

For details, refer to GitHub API rate limits documentation:
https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api?apiVersion=2022-11-28#primary-rate-limit-for-unauthenticated-users

What is the motivation for this PR?
Fix github api rate limit issue by introduce a proxy.

How did you do it?
When fetch github issues, try proxy first, if fails, fallback to original unauthenticated behavior.
And to support setting proxy environment vars, add args for setup-container.sh

How did you verify/test it?
When no proxy, the pr tests are good as before.
When sets proxy, the pr tests are good as well. (check log can see it via proxy to fetch github issues)
gshemesh2 pushed a commit to gshemesh2/sonic-mgmt that referenced this pull request Jan 26, 2026
…issues state (sonic-net#19116)

Fix github api rate limit issue by introduce a proxy.
During peak periods of PR testing, some test scripts encounter GitHub API rate limit issues when attempting to fetch GitHub issues. This is due to unauthenticated requests being subject to a strict limit of 60 requests per hour per IP address.

This PR introduces a workaround by routing these requests through a proxy service that performs authenticated GitHub API calls (with a higher limit of 5000 requests per hour per GitHub App) and supports caching.

If the proxy is unavailable or not configured, the system falls back to the original unauthenticated behavior, ensuring it does not worsen the existing situation.

For details, refer to GitHub API rate limits documentation:
https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api?apiVersion=2022-11-28#primary-rate-limit-for-unauthenticated-users

What is the motivation for this PR?
Fix github api rate limit issue by introduce a proxy.

How did you do it?
When fetch github issues, try proxy first, if fails, fallback to original unauthenticated behavior.
And to support setting proxy environment vars, add args for setup-container.sh

How did you verify/test it?
When no proxy, the pr tests are good as before.
When sets proxy, the pr tests are good as well. (check log can see it via proxy to fetch github issues)

Signed-off-by: Guy Shemesh <[email protected]>
ytzur1 pushed a commit to ytzur1/sonic-mgmt that referenced this pull request Feb 2, 2026
…issues state (sonic-net#19116)

Fix github api rate limit issue by introduce a proxy.
During peak periods of PR testing, some test scripts encounter GitHub API rate limit issues when attempting to fetch GitHub issues. This is due to unauthenticated requests being subject to a strict limit of 60 requests per hour per IP address.

This PR introduces a workaround by routing these requests through a proxy service that performs authenticated GitHub API calls (with a higher limit of 5000 requests per hour per GitHub App) and supports caching.

If the proxy is unavailable or not configured, the system falls back to the original unauthenticated behavior, ensuring it does not worsen the existing situation.

For details, refer to GitHub API rate limits documentation:
https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api?apiVersion=2022-11-28#primary-rate-limit-for-unauthenticated-users

What is the motivation for this PR?
Fix github api rate limit issue by introduce a proxy.

How did you do it?
When fetch github issues, try proxy first, if fails, fallback to original unauthenticated behavior.
And to support setting proxy environment vars, add args for setup-container.sh

How did you verify/test it?
When no proxy, the pr tests are good as before.
When sets proxy, the pr tests are good as well. (check log can see it via proxy to fetch github issues)

Signed-off-by: Yael Tzur <[email protected]>
auspham pushed a commit to auspham/sonic-mgmt that referenced this pull request Feb 3, 2026
…issues state (sonic-net#19116)

Fix github api rate limit issue by introduce a proxy.
During peak periods of PR testing, some test scripts encounter GitHub API rate limit issues when attempting to fetch GitHub issues. This is due to unauthenticated requests being subject to a strict limit of 60 requests per hour per IP address.

This PR introduces a workaround by routing these requests through a proxy service that performs authenticated GitHub API calls (with a higher limit of 5000 requests per hour per GitHub App) and supports caching.

If the proxy is unavailable or not configured, the system falls back to the original unauthenticated behavior, ensuring it does not worsen the existing situation.

For details, refer to GitHub API rate limits documentation:
https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api?apiVersion=2022-11-28#primary-rate-limit-for-unauthenticated-users

What is the motivation for this PR?
Fix github api rate limit issue by introduce a proxy.

How did you do it?
When fetch github issues, try proxy first, if fails, fallback to original unauthenticated behavior.
And to support setting proxy environment vars, add args for setup-container.sh

How did you verify/test it?
When no proxy, the pr tests are good as before.
When sets proxy, the pr tests are good as well. (check log can see it via proxy to fetch github issues)
Pterosaur pushed a commit to Pterosaur/sonic-mgmt that referenced this pull request Mar 26, 2026
…e for setup-container to fix github api rate limit issue

The PR sonic-net#19116 adds -e args for setup-container.sh for all PR test pipelines.

This PR is to cherry pick the change of adding -e for setup-container to fix github api rate limit issue

Signed-off-by: Chun'ang Li <[email protected]>

----
#### AI description  (iteration 1)
#### PR Classification
New feature: enhancement to allow passing environment variables during container setup.

#### PR Summary
This pull request introduces the `-e` flag to the container setup script, enabling users to pass custom environment variables into the container.
- `setup-container.sh`: Added `-e` to the getopts flag list and implemented a case to append environment variables.
- `setup-container.sh`: Updated the docker run command to include the environment variables.
- `setup-container.sh`: Initialized the `ENV_VARS` variable and updated the help message to document the new flag.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->
venu-nexthop pushed a commit to venu-nexthop/sonic-mgmt that referenced this pull request Mar 27, 2026
…issues state (sonic-net#19116)

Fix github api rate limit issue by introduce a proxy.
During peak periods of PR testing, some test scripts encounter GitHub API rate limit issues when attempting to fetch GitHub issues. This is due to unauthenticated requests being subject to a strict limit of 60 requests per hour per IP address.

This PR introduces a workaround by routing these requests through a proxy service that performs authenticated GitHub API calls (with a higher limit of 5000 requests per hour per GitHub App) and supports caching.

If the proxy is unavailable or not configured, the system falls back to the original unauthenticated behavior, ensuring it does not worsen the existing situation.

For details, refer to GitHub API rate limits documentation:
https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api?apiVersion=2022-11-28#primary-rate-limit-for-unauthenticated-users

What is the motivation for this PR?
Fix github api rate limit issue by introduce a proxy.

How did you do it?
When fetch github issues, try proxy first, if fails, fallback to original unauthenticated behavior.
And to support setting proxy environment vars, add args for setup-container.sh

How did you verify/test it?
When no proxy, the pr tests are good as before.
When sets proxy, the pr tests are good as well. (check log can see it via proxy to fetch github issues)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment