Error on search method with short term fields

[nsiriaporn]

• Operating System: Windows10 Home 19041.1052
• Node version: 14.16.0
• app-play-scraper version: 16.3

Description:

Describe your issue here

When performing a one-term search (i.e. "travel", "diet", "commute") and also "active travel", using the method 'search', the promise return an error. However, some two-word search works. I've used "active commute" which works but "active travel" does not.

Example code:

// Put code to reproduce the issue here
scraper = require('app-store-scraper');
scraper.search({
    term:  "diet",
    country: "us",
    num: 250,
    fullDetail: true
  }).then(console.log, console.log);

Error message:

The error is the json in attached file.

error.txt

[jonaskuske]

From my testing, this has to do with the amount of results, not the term itself. The /search endpoint can respond with many results, but the /lookup endpoint that is used afterwards to get result data is limited to 200 IDs per request.

When you choose a term with a lot of results (like most short terms) and the num option is > 200, more than 200 IDs are passed to the lookup request and we receive the 502 error linked in the issue description. Changing the term to something that has less than 200 results makes the error go away, so does removing the num option or setting it to a number < 200.

I've submitted a PR that makes sure we only send 200 IDs to the /lookup endpoint, and opts for multiple lookup requests if there are more than 200 IDs: #166