The Fire Crawl Tool Node is a versatile tool designed for web scraping, mapping, extracting, crawling, and searching websites. It features a blue connection dot, allowing you to easily integrate it with the prompt node in the blue connection tool section.
Functionality
1. Scrape
- Description: Scrapes a specific URL and retrieves its content in markdown format.
- When to Use: Use this function when you need to extract content from a single web page for analysis or integration into applications.
2. Crawl
- Description: Crawls a given URL and all its accessible subpages, returning the content in a specified format.
- When to Use: This is ideal for gathering data from an entire website without needing a sitemap, making it useful for comprehensive data collection.
- Max Crawl Limit: Limit is 10.
3. Map
- Description: Inputs a website and retrieves all the URLs associated with it quickly.
- When to Use: Use this function when you need to understand the structure of a website or gather links for further scraping or crawling.
4. Extract
- Description: Extracts structured data from a single page, multiple pages, or entire websites using AI capabilities.
- When to Use: This function is beneficial when you need specific data points from web pages, such as product details, contact information, or other structured content.
5. Search
- Description: Performs web searches and retrieves full content from the results, optionally scraping the search results in one operation.
- When to Use: Use this function when you want to find information across the web and need to extract relevant content from the search results.
- Max Search Limit: Limit is 10 page.
- Additional Functionalities: You can also control the location, time and language to search.
With a plug and use node, there's no need for you to obtain an API key; we manage the API and cover the usage costs. There is + 1 credit requered to use this tool.