Back to Agent Skills
CrewAI Web Scraping Tool
CrewAI FrameworkSkill Description
Scrapes entire web pages and converts them to markdown or text. Handles parsing HTML tags and extracting main article text.
Code / Definition File
skill_manifest.yaml / config.json
from crewai_tools import ScrapeWebsiteTool
# Initialize the tool with a specific website or leave it blank for dynamic use
scrape_tool = ScrapeWebsiteTool()
# Usage in a CrewAI Agent:
# agent = Agent(
# role='Research Analyst',
# goal='Analyze the content of the target website',
# backstory='Expert at extracting insight from raw HTML',
# tools=[scrape_tool]
# )
Tags
scraper html-parsing web
Author: CrewAI
Added on: 6/7/2026