Skip to content

Commit a4384d9

Browse files
raulcdkou
andauthored
GH-36634: [Dev] Ensure merge script goes over all pages when requesting info from GitHub (#36637)
### Rationale for this change We currently were missing maintenance branches due to pagination on GH API. ### What changes are included in this PR? Check whether the API is returning a paginated view and extend the list returned. ### Are these changes tested? I have tested locally: ``` (Pdb) pr.maintenance_branches ['maint-0.11.x', 'maint-0.12.x', 'maint-0.14.x', 'maint-0.15.x', 'maint-0.17.x', 'maint-1.0.x', 'maint-3.0.x', 'maint-4.0.x', 'maint-6.0.x', 'maint-7.0.x', 'maint-7.0.1', 'maint-8.0.x', 'maint-9.0.0', 'maint-10.0.x', 'maint-10.0.0', 'maint-10.0.1', 'maint-11.0.0', 'maint-12.0.x', 'maint-12.0.0', 'maint-12.0.1', 'maint-13.0.0'] (Pdb) c Enter fix version [14.0.0]: ``` ### Are there any user-facing changes? No * Closes: #36634 Lead-authored-by: Raúl Cumplido <raulcumplido@gmail.com> Co-authored-by: Sutou Kouhei <kou@cozmixng.org> Signed-off-by: Sutou Kouhei <kou@clear-code.com>
1 parent 8245b21 commit a4384d9

1 file changed

Lines changed: 18 additions & 1 deletion

File tree

dev/merge_arrow_pr.py

Lines changed: 18 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -78,7 +78,24 @@ def get_json(url, headers=None):
7878
response = requests.get(url, headers=headers)
7979
if response.status_code != 200:
8080
raise ValueError(response.json())
81-
return response.json()
81+
# GitHub returns a link header with the next, previous, last
82+
# page if there is pagination on the response. See:
83+
# https://docs.github.com/en/rest/guides/using-pagination-in-the-rest-api#using-link-headers
84+
next_responses = None
85+
if "link" in response.headers:
86+
links = response.headers['link'].split(', ')
87+
for link in links:
88+
if 'rel="next"' in link:
89+
# Format: '<url>; rel="next"'
90+
next_url = link.split(";")[0][1:-1]
91+
next_responses = get_json(next_url, headers)
92+
responses = response.json()
93+
if next_responses:
94+
if isinstance(responses, list):
95+
responses.extend(next_responses)
96+
else:
97+
raise ValueError('GitHub response was paginated and is not a list')
98+
return responses
8299

83100

84101
def run_cmd(cmd):

0 commit comments

Comments
 (0)