A Python script to crawl a website up to a specified depth and generate a summary using the OpenAI API.
- Install dependencies:
pip install -r requirements.txt
- Set API Key:
Open
linksage.pyand replace"YOUR_API_KEY"with your actual OpenAI API key.
python linksage.py <URL> [--depth N]<URL>: The starting URL to crawl (e.g.,https://example.com).--depth N: (Optional) The maximum crawl depth. Defaults to 2.
Example:
python linksage.py https://example.com --depth 2