The Tavily Map Node maps a website's internal URL structure starting from a base URL. It discovers and lists all internal endpoints and pages, providing an index of a website's hierarchy. This is incredibly useful for planning crawl operations or building a site index.
Parameters
Required Parameters
- url (string): The base/root URL to begin mapping.
Optional Parameters
- limit (integer): The maximum number of links the mapper will process before stopping (Default:
100). - max_depth (integer): Maximum tree depth from the root URL.
- max_breadth (integer): Maximum links to follow per level.
- allow_external (boolean): Include external outbound links.
- instructions (string): Natural language guidance to direct mapping priorities.
- select_domains / select_paths (array of regexes): Restrict mapping to matching domains (e.g.,
^docs\\.) or paths (e.g.,/api/v1/.*).
Practical Use Cases
- Sitemap Generation: Automatically discover all live URLs on a newly launched site or documentation center.
- Pre-Crawl Planning: Run mapping first to identify specific paths (e.g.,
/help/) and filter out irrelevant URLs prior to running heavy extraction. - Broken Link Audit: Retrieve all internal links to map page boundaries and construct site navigation charts.