Tim Asp 3ca1a387c2 Web Loader: Add proxy support (#6792) ...

Proxies are helpful, especially when you start querying against more
anti-bot websites.

[Proxy
services](https://developers.oxylabs.io/advanced-proxy-solutions/web-unblocker/making-requests)
(of which there are many) and `requests` make it easy to rotate IPs to
prevent banning by just passing along a simple dict to `requests`.

CC @rlancemartin, @eyurtsev

2023-06-27 22:27:49 -07:00

..

blob_loaders

YoutubeAudioLoader and updates to OpenAIWhisperParser (#5772)

2023-06-06 15:15:08 -07:00

parsers

feat (documents): add a source code loader based on AST manipulation (#6486)

2023-06-27 15:58:47 -07:00

__init__.py

feat: Add UnstructuredOrgModeLoader (#6842)

2023-06-27 16:34:17 -07:00

acreom.py

Feature/add acreom loader (#5780)

2023-06-15 11:53:00 -07:00

airbyte_json.py

Dev2049/add modern treasury (#3924)

2023-05-01 20:28:02 -07:00

airtable.py

Loader for OpenCityData and minor cleanups to Pandas, Airtable loaders (#6301)

2023-06-22 22:20:42 -07:00

apify_dataset.py

changed ValueError to ImportError (#5103)

2023-05-22 15:24:45 -07:00

arxiv.py

Arxiv document loader (#3627)

2023-04-26 21:04:56 -07:00

azlyrics.py

clean up loaders (#1178)

2023-02-20 08:20:48 -08:00

azure_blob_storage_container.py

Minor: Remove duplicated word in error message (#2706)

2023-04-11 13:10:33 -07:00

azure_blob_storage_file.py

Minor: Remove duplicated word in error message (#2706)

2023-04-11 13:10:33 -07:00

base.py

docs retriever improvements (#4430)

2023-05-17 15:29:22 -07:00

bibtex.py

Bibtex integration for document loader and retriever (#5137)

2023-05-25 00:21:31 -07:00

bigquery.py

Ability to specify credentials wihen using Google BigQuery as a data loader (#5466)

2023-05-30 16:25:22 -07:00

bilibili.py

Fix bilibili (#4860)

2023-05-18 09:56:51 -04:00

blackboard.py

hotfix (#1742)

2023-03-17 09:05:08 -07:00

blockchain.py

added docstrings where they missed (#6626)

2023-06-23 15:49:44 -07:00

chatgpt.py

added docstrings where they missed (#6626)

2023-06-23 15:49:44 -07:00

college_confidential.py

clean up loaders (#1178)

2023-02-20 08:20:48 -08:00

confluence.py

added docstrings where they missed (#6626)

2023-06-23 15:49:44 -07:00

conllu.py

add CoNLL-U document loader (#1297)

2023-02-26 17:27:00 -08:00

csv_loader.py

feat: Add UnstructuredCSVLoader for CSV files (#5844)

2023-06-07 19:18:01 -07:00

dataframe.py

Loader for OpenCityData and minor cleanups to Pandas, Airtable loaders (#6301)

2023-06-22 22:20:42 -07:00

diffbot.py

consistently use getLogger(__name__), no root logger (#2989)

2023-04-16 12:49:35 -07:00

directory.py

Add path validation to DirectoryLoader (#5327)

2023-05-28 15:31:23 -04:00

discord.py

Harrison/discord loader (#3200)

2023-04-19 21:04:12 -07:00

docugami.py

Change Data Loader Namespace (#6568)

2023-06-21 19:24:04 -07:00

duckdb_loader.py

changed ValueError to ImportError (#5103)

2023-05-22 15:24:45 -07:00

email.py

fix: pass unstructured kwargs down in all unstructured loaders (#2506)

2023-04-06 12:29:52 -07:00

embaas.py

added docstrings where they missed (#6626)

2023-06-23 15:49:44 -07:00

epub.py

fix: pass unstructured kwargs down in all unstructured loaders (#2506)

2023-04-06 12:29:52 -07:00

evernote.py

feature/4493 Improve Evernote Document Loader (#4577)

2023-05-19 14:28:17 -07:00

excel.py

feat: add UnstructuredExcelLoader for .xlsx and .xls files (#5617)

2023-06-03 12:44:12 -07:00

facebook_chat.py

Refactor TelegramChatLoader and FacebookChatLoader classes and add tests (#3863)

2023-05-03 15:59:19 -07:00

fauna.py

added docstrings where they missed (#6626)

2023-06-23 15:49:44 -07:00

figma.py

Dev2049/add modern treasury (#3924)

2023-05-01 20:28:02 -07:00

gcs_directory.py

Support GCS Objects with / in GCS Loaders (#3356)

2023-04-24 22:05:44 -07:00

gcs_file.py

Support GCS Objects with / in GCS Loaders (#3356)

2023-04-24 22:05:44 -07:00

generic.py

Add a generic document loader (#4875)

2023-05-17 22:38:55 -04:00

git.py

Add source field to metadata (#4462)

2023-05-15 14:30:12 -07:00

gitbook.py

Gitbook enhancements (#2279)

2023-04-06 22:55:07 -07:00

github.py

DocumentLoader for GitHub (#5408)

2023-05-29 20:11:21 -07:00

googledrive.py

Iterate through filtered file types instead of all listed files (#6258)

2023-06-18 17:47:58 -07:00

gutenberg.py

gutenberg books (#946)

2023-02-08 12:00:47 -08:00

helpers.py

feat #4479: TextLoader auto detect encoding and improved exceptions (#4927)

2023-05-18 09:55:14 -04:00

hn.py

Refactor some loops into list comprehensions (#1185)

2023-02-20 16:38:43 -08:00

html_bs.py

Add get_text_separator parameter to BSHTMLLoader (#3551)

2023-04-26 16:10:16 -07:00

html.py

feat: allow the unstructured kwargs to be passed in to Unstructured document loaders (#1667)

2023-03-14 18:15:28 -07:00

hugging_face_dataset.py

Hugging Face Loader: Add lazy load (#4799)

2023-05-17 12:04:23 -04:00

ifixit.py

Harrison/ifixit (#1680)

2023-03-14 21:17:50 -07:00

image_captions.py

changed ValueError to ImportError (#5103)

2023-05-22 15:24:45 -07:00

image.py

feat: allow the unstructured kwargs to be passed in to Unstructured document loaders (#1667)

2023-03-14 18:15:28 -07:00

imsdb.py

clean up loaders (#1178)

2023-02-20 08:20:48 -08:00

iugu.py

added docstrings where they missed (#6626)

2023-06-23 15:49:44 -07:00

joplin.py

Fixed regression in JoplinLoader's get note url (#5265)

2023-05-25 13:10:10 -07:00

json_loader.py

changed ValueError to ImportError (#5103)

2023-05-22 15:24:45 -07:00

markdown.py

fix: pass unstructured kwargs down in all unstructured loaders (#2506)

2023-04-06 12:29:52 -07:00

mastodon.py

Add Mastodon toots loader (#5036)

2023-05-22 16:43:07 -07:00

max_compute.py

add maxcompute (#5533)

2023-06-01 00:54:42 -07:00

mediawikidump.py

Harrison/media wiki xml (#4072)

2023-05-03 20:45:33 -07:00

merge.py

Create merge loader that combines documents from a set of loaders (#6659)

2023-06-23 13:02:48 -07:00

mhtml.py

Added a MHTML document loader (#6311)

2023-06-25 13:12:08 -07:00

modern_treasury.py

added docstrings where they missed (#6626)

2023-06-23 15:49:44 -07:00

notebook.py

changed ValueError to ImportError (#5103)

2023-05-22 15:24:45 -07:00

notion.py

Harrison/unstructured support (#903)

2023-02-05 23:02:07 -08:00

notiondb.py

Improve performance when retrieving Notion DB pages (#6710)

2023-06-26 05:46:09 -07:00

obsidian.py

Dev2049/obsidian patch (#4204)

2023-05-05 16:49:19 -07:00

odt.py

feat: add loader for open office odt files (#4405)

2023-05-10 01:37:17 -07:00

onedrive_file.py

Harrison/one drive loader (#4081)

2023-05-03 22:55:34 -07:00

onedrive.py

changed ValueError to ImportError (#5103)

2023-05-22 15:24:45 -07:00

open_city_data.py

Loader for OpenCityData and minor cleanups to Pandas, Airtable loaders (#6301)

2023-06-22 22:20:42 -07:00

org_mode.py

feat: Add UnstructuredOrgModeLoader (#6842)

2023-06-27 16:34:17 -07:00

pdf.py

Fixed PermissionError on windows (#6170)

2023-06-18 16:39:57 -07:00

powerpoint.py

feat: batch multiple files in a single Unstructured API request (#4525)

2023-05-21 20:48:20 -07:00

psychic.py

Update to the latest Psychic python library version (#6804)

2023-06-27 22:26:38 -07:00

pyspark_dataframe.py

fix (#5457)

2023-05-30 10:42:20 -07:00

python.py

Add PythonLoader which auto-detects encoding of Python files (#3311)

2023-04-21 10:47:57 -07:00

readthedocs.py

Allow readthedoc loader to pass custom html tag (#5175)

2023-05-24 10:40:27 -07:00

recursive_url_loader.py

RecusiveUrlLoader to RecursiveUrlLoader (#6787)

2023-06-26 23:12:14 -07:00

reddit.py

Langchain with reddit (#3661) (#3768)

2023-04-28 20:59:56 -07:00

roam.py

Harrison/add roam loader (#939)

2023-02-08 00:35:33 -08:00

rst.py

feat: Add UnstructuredRSTLoader (#6594)

2023-06-25 12:41:57 -07:00

rtf.py

feat: add loader for rich text files (#3227)

2023-04-20 15:51:49 -07:00

s3_directory.py

changed ValueError to ImportError (#5103)

2023-05-22 15:24:45 -07:00

s3_file.py

changed ValueError to ImportError (#5103)

2023-05-22 15:24:45 -07:00

sitemap.py

Strips whitespace and \n from loc before filtering urls from sitemap (#5728)

2023-06-05 16:33:55 -07:00

slack_directory.py

Add Slack Directory Loader (#2841)

2023-04-13 21:31:59 -07:00

snowflake_loader.py

Fix: SnowflakeLoader returning empty documents (#5967)

2023-06-10 13:03:50 -07:00

spreedly.py

added docstrings where they missed (#6626)

2023-06-23 15:49:44 -07:00

srt.py

changed ValueError to ImportError (#5103)

2023-05-22 15:24:45 -07:00

stripe.py

added docstrings where they missed (#6626)

2023-06-23 15:49:44 -07:00

telegram.py

changed ValueError to ImportError (#5103)

2023-05-22 15:24:45 -07:00

text.py

feat #4479: TextLoader auto detect encoding and improved exceptions (#4927)

2023-05-18 09:55:14 -04:00

tomarkdown.py

2markdown loader (#4796)

2023-05-16 23:42:53 -07:00

toml.py

Add PDF parser implementations (#4356)

2023-05-09 10:24:17 -04:00

trello.py

New Trello document loader (#4767)

2023-05-29 19:47:56 -07:00

twitter.py

changed ValueError to ImportError (#5103)

2023-05-22 15:24:45 -07:00

unstructured.py

Harrison/unstructured page number (#6464)

2023-06-19 22:31:43 -07:00

url_playwright.py

changed ValueError to ImportError (#5103)

2023-05-22 15:24:45 -07:00

url_selenium.py

changed ValueError to ImportError (#5103)

2023-05-22 15:24:45 -07:00

url.py

feat: added tqdm progress bar to UnstructuredURLLoader (#6600)

2023-06-25 12:41:25 -07:00

weather.py

Adding Weather Loader (#5056)

2023-05-23 15:57:33 -07:00

web_base.py

Web Loader: Add proxy support (#6792)

2023-06-27 22:27:49 -07:00

whatsapp_chat.py

Enhancement : Ignore deleted messages and media in WhatsAppChatLoader (#6839)

2023-06-27 16:36:55 -07:00

wikipedia.py

Wiki loader lint (#6670)

2023-06-23 16:05:42 -07:00

word_document.py

feat: batch multiple files in a single Unstructured API request (#4525)

2023-05-21 20:48:20 -07:00

xml.py

feat: Add UnstructuredXMLLoader for .xml files (#5955)

2023-06-10 16:24:42 -07:00

youtube.py

Harrison/youtube multi language (#5758)

2023-06-05 16:38:07 -07:00