Server-side Request Forgery In Recursive URL Loader
langchain-ai/langchain
Affected versions
| Product | Affected | Fixed in |
|---|---|---|
| langchain PyPI |
|
|
| langchain-exa PyPI |
|
|
Original description
With the following crawler configuration:
langchain-ai/langchain
| Product | Affected | Fixed in |
|---|---|---|
| langchain PyPI |
|
|
| langchain-exa PyPI |
|
|
With the following crawler configuration:
```python from bs4 import BeautifulSoup as Soup
url = "https://example.com" loader = RecursiveUrlLoader( url=url, max_depth=2, extractor=lambda x: Soup(x, "html.parser").text ) docs = loader.load() ```
An attacker in control of the contents of `https://example.com` could place a malicious HTML file in there with links like "https://example.completely.different/my_file.html" and the crawler would proceed to download that file as well even though `prevent_outside=True`.
https://github.com/langchain-ai/langchain/blob/bf0b3cc0b5ade1fb95a5b1b6fa260e99064c2e22/libs/community/langchain_community/document_loaders/recursive_url_loader.py#L51-L51
Resolved in https://github.com/langchain-ai/langchain/pull/15559
From the CVE record · @huntr_ai
RecursiveUrlLoader can follow links outside the configured domain despite `prevent_outside=True`. An attacker who controls the contents of a page the crawler loads can add a crafted link that causes the crawler to download a file from another domain. This requires the crawler to load the attacker-controlled page; the input does not specify an authentication requirement.
Check deployments using PyPI `langchain` or `langchain-exa` versions earlier than 0.1.0, particularly those using RecursiveUrlLoader.