D4Vinci/Scrapling

↗ GitHub

🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl!

34,154

Stars

2,754

Forks

144

Watchers

Open Issues

Python·BSD 3-Clause "New" or "Revised" License·Last commit Apr 1, 2026·by @D4Vinci·Published April 1, 2026·Analyzed 6d ago

Safety Rating A

No hardcoded secrets, malicious code patterns, suspicious dependency usages, or prompt injection attempts were detected. The repository is a well-structured, openly licensed (BSD-3-Clause) Python framework with a clear legitimate purpose, active CI/CD pipelines, and a standard dependency set (Playwright, lxml, httpx, etc.). The disclaimer in the README appropriately notes that users must comply with applicable laws. No red flags were identified.

ℹAI-assisted review, not a professional security audit.

AI Analysis

Scrapling is an adaptive Python web scraping framework that handles everything from single HTTP requests to full-scale concurrent crawls. It features a Scrapy-like spider API, multiple fetcher classes (standard HTTP, stealthy headless browser, and dynamic Playwright-based), anti-bot bypass capabilities (including Cloudflare Turnstile), adaptive element tracking that survives website redesigns, proxy rotation, session management, an interactive shell, a CLI tool, and a built-in MCP server for AI-assisted scraping. It achieves high performance through optimized parsing and supports async patterns throughout.

Use Cases

Scraping websites protected by anti-bot systems such as Cloudflare Turnstile
Building full-scale concurrent web crawlers with pause/resume functionality
Adaptive scraping that automatically relocates elements after website layout changes
AI-assisted web data extraction via built-in MCP server integration
CLI-based content extraction without writing code
Multi-session crawling combining fast HTTP and stealthy browser requests in a single spider