Skip to content

v0.2.7

Choose a tag to compare

View all tags
@D4Vinci D4Vinci released this 26 Nov 21:11
· 949 commits to main since this release
v0.2.7
26aebba
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

What's changed

New features

  • Now if you used the wait_selector argument with StealthyFetcher and PlayWrightFetcher classes, Scrapling will wait again for the JS to fully load and execute like normal. If you used the network_idle argument, Scrapling will wait for it again too after waiting for all of that. If the states are all fulfilled then no waiting happens, of course.
  • Now you can enable and disable ads on StealthyFetcher with the disable_ads argument. This is enabled by default and it installs the ublock origin addon.
  • Now you can set the locale used by PlayWrightFetcher with the locale argument. The default value is still en-US.
  • Now the basic requests done through Fetcher can accept proxies in this format http://username:password@localhost:8030.
  • The stealth mode improved a bit for PlayWrightFetcher.

Bugs Squashed/Improvements

  1. Now enabling proxies on the PlayWrightFetcher class is not tied to the stealth mode being on or off (Thanks to @AbdullahY36 for pointing that out)
  2. Now the ResponseEncoding tests if the encoding returned from the response can be used with the page or not. If the returned encoding triggered an error, Scrapling defaults to utf-8

Note

A friendly reminder that maintaining and improving Scrapling takes a lot of time and effort which I have been happily doing for months even though it's becoming harder. So, if you like Scrapling and want it to keep improving, you can help by supporting me through the Sponsor button.

Assets 2
Loading