scrapy-proxy-rotator

Scrapy downloader middleware that rotates proxies for requests.

Stars

8

Forks

4

Language

Python

Last Updated

Apr 05, 2021

Similar Repos
Repo Language Stars Description Updated At
Python 8 Proxymesh downloader middleware for Scrapy Aug 21, 2022
Python 2 Downloader middleware for Scrapy that uses Ntlm authentication. Nov 04, 2020
Python 5 Test Downloader Middleware of Scrapy Jul 23, 2020
Python 6 Scrapy Downloader Middleware for Crawlera Fetch API Mar 11, 2022
Python 17 Scrapy downloader middleware that stores response HTMLs to disk. Aug 18, 2022
Python 3 Elite Proxies (http://elite.proxies.online) middleware for scrapy http://rev.proxies.online May 02, 2019
Python 75 Downloader Middleware to support Playwright in Scrapy & Gerapy Aug 24, 2022
Python 124 Downloader Middleware to support Pyppeteer in Scrapy & Gerapy Aug 30, 2022
Python 24 Downloader Middleware to support Selenium in Scrapy & Gerapy Mar 19, 2023
Python 2 Rotates through a group of proxies Mar 07, 2023
Python 19 A downloader middleware to change user-agent of scrapy Dec 26, 2021
Python 6 Scrapy🕷 middleware for limiting requests based on a counter Mar 08, 2023
Python 20 Scrapy middleware to handle javascript pages using requests-html Aug 31, 2022
Python 4 Rotates through a group of proxies and user agents Mar 25, 2023
Python 96 PhantomJS Downloader for Scrapy, Yeah! Jan 28, 2022
Python 598 use multiple proxies with Scrapy Sep 03, 2022
Python 10 Delayed requests for Scrapy May 19, 2022
Python 4 Random proxy middleware for Scrapy Jan 10, 2020
Python 5 Scrapy middleware for the autologin Jan 05, 2022
Python 7 Statsd integration middleware for scrapy Jun 18, 2022
Python 1565 Random proxy middleware for Scrapy Sep 02, 2022
Python 36 Scrapy middleware for the autologin Dec 03, 2021
Python 2 Downloader for scrapy over socks proxy Jul 17, 2022
Python 5 Scrapy middleware that reads proxy config from settings Nov 21, 2020
JavaScript 2 Node js app that proxies requests to graphhopper. Jan 29, 2023
Python 2 Scrapy Spider Middleware Demo Jul 27, 2022
JavaScript 166 Proxies Puppeteer Page requests. Aug 07, 2022
Python 2 Scrapy🕷 middleware for link filtering Mar 08, 2023
Go 4 Middleware for signed requests Dec 29, 2020
JavaScript 5 NodeJS Express middleware that detects malicious requests Sep 14, 2021
Ruby 2 A Rack Middleware that blocks Slackbot requests Jun 27, 2022
Python 248 Scrapy spider middleware to ignore requests to pages containing items seen in previous crawls Aug 11, 2022
Python 21 WSGI middleware that requires Heroku OAuth for all requests. Jan 26, 2022
Python 19 A middleware of cookies persistence for Scrapy Sep 30, 2021
Python 4 Scrapy Middleware for proxy authentication with Smartproxy Oct 13, 2022
Nix 7 A HTTP nix store that proxies requests to Google Storage Aug 07, 2021
TypeScript 24 MyProxy is an application that proxies requests to other servers Apr 24, 2023
Python 43 A middleware layer for Scrapy that detects CAPTCHA tests and solves them Jan 07, 2022
TypeScript 2 A transport for pino that rotates logs Mar 06, 2024
Python 2 Scrape Amazon product listings utilising scrapy & residential proxies Feb 16, 2024
Python 376 a class that uses scraped proxies to make http GET/POST requests (Python requests) Aug 09, 2022
JavaScript 2 DEPRECATED: A simple utility that routes and proxies requests for 18F guides Feb 24, 2019
PHP 7 Middleware for data map requests Sep 03, 2020
Go 20 OpenTelemetry trace middleware for Fiber that adds traces to requests. Jan 28, 2023
Python 6 Yet another tumblr downloader based on Scrapy Mar 20, 2022
Go 2 Replicate any requests over configured proxies May 13, 2022
Go 3 Proxies requests and adds CORS headers. Sep 18, 2022
JavaScript 2 A Cloudflare Worker that proxies toastpack requests to the npm registry Apr 18, 2022
Go 33 Go library for a file that rotates daily Jan 28, 2023
JavaScript 3 NodeJS Express middleware that rejects requests listed on blacklist Aug 17, 2020