re
https://github.com/hwchase17/langchain/issues/439#issuecomment-1510442791
I think it's not polite for a library to use the root logger
both of these forms are also used:
```
logger = logging.getLogger(__name__)
logger = logging.getLogger(__file__)
```
I am not sure if there is any reason behind one vs the other? (...I am
guessing maybe just contributed by different people)
it seems to me it'd be better to consistently use
`logging.getLogger(__name__)`
this makes it easier for consumers of the library to set up log
handlers, e.g. for everything with `langchain.` prefix
Solves #2247. Noted that the only test I added checks for the
BeautifulSoup behaviour change. Happy to add a test for
`DirectoryLoader` if deemed necessary.
Ran into a broken build if bs4 wasn't installed in the project.
Minor tweak to follow the other doc loaders optional package-loading
conventions.
Also updated html docs to include reference to this new html loader.
side note: Should there be 2 different html-to-text document loaders?
This new one only handles local files, while the existing unstructured
html loader handles HTML from local and remote. So it seems like the
improvement was adding the title to the metadata, which is useful but
could also be added to `html.py`
This `BSHTMLLoader` document_loader loads an HTML document, extracts
text and adds the page title to the returned Document's metadata. The
loader uses the already installed bs4 package to extract both text
content and the page title.
Included in this PR is an example HTML file and an integration test that
tests against this file.
---------
Co-authored-by: Daniel Chalef <daniel.chalef@private.org>
Daniel Chalef