Compare commits

..

9 Commits

Author SHA1 Message Date
Jerry Liu 45832e3ef9 cr 2024-06-11 09:55:16 -07:00
Jerry Liu b60b2e1f59 cr 2024-06-09 17:11:06 -07:00
Pierre-Loic Doulcet 58257d546b Pierre/new options (#216)
* Add spreadsheet extensions

* linting

* fix tests

* Add new parameters to the parser

* add option to API call

* lint

* lint remove trailing space

---------

Co-authored-by: Logan Markewich <logan.markewich@live.com>
2024-06-07 18:17:54 -06:00
Adam Reichert 5e99d810fd Fix Typo in description of invalidate_cache argument of LlamaParse constructor (#221)
Fix Typo
2024-06-07 17:32:57 -06:00
Jerry Liu a2dc717d85 add simple excel notebook (with dcf) (#215) 2024-06-06 08:48:33 -07:00
Logan 87062e6ca8 add link to docs 2024-06-05 16:10:28 -06:00
Jerry Liu ae3a21c5ff add KG agent (#211) 2024-06-05 12:58:07 -07:00
Ravi Theja ccebb8a2fa Add Excel spreadsheet example with llamaparse (#204) 2024-05-30 09:48:33 -06:00
Pierre-Loic Doulcet 2d21d6e688 Add spreadsheet extensions (#203)
* Add spreadsheet extensions

* linting

* fix tests

---------

Co-authored-by: Logan Markewich <logan.markewich@live.com>
2024-05-29 20:11:28 -06:00
11 changed files with 4978 additions and 3117 deletions
+1 -1
View File
@@ -72,7 +72,7 @@ repos:
args:
[
"--ignore-words-list",
"astroid,gallary,momento,narl,ot,rouge,nin,gere,te,inh",
"astroid,gallary,momento,narl,ot,rouge,nin,gere,te,inh,vor",
]
- repo: https://github.com/srstevenson/nb-clean
rev: 3.1.0
+6
View File
@@ -6,6 +6,8 @@ LlamaParse directly integrates with [LlamaIndex](https://github.com/run-llama/ll
Free plan is up to 1000 pages a day. Paid plan is free 7k pages per week + 0.3c per additional page.
Read below for some quickstart information, or see the [full documentation](https://docs.cloud.llamaindex.ai/).
## Getting Started
First, login and get an api-key from [**https://cloud.llamaindex.ai ↗**](https://cloud.llamaindex.ai).
@@ -87,6 +89,10 @@ Several end-to-end indexing examples can be found in the examples folder
- [Advanced RAG Example](examples/demo_advanced.ipynb)
- [Raw API Usage](examples/demo_api.ipynb)
## Documentation
[https://docs.cloud.llamaindex.ai/](https://docs.cloud.llamaindex.ai/)
## Terms of Service
See the [Terms of Service Here](./TOS.pdf).
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
+310
View File
@@ -0,0 +1,310 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "0db58db5-d4ee-4631-af5b-4fc53eb05170",
"metadata": {},
"source": [
"# RAG with Excel Spreadsheet using LlamaPrase\n",
"\n",
"<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/demo_excel.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
"\n",
"This notebook constructs a RAG pipeline over a simple DCF template [here](https://eqvista.com/app/uploads/2020/09/Eqvista_DCF-Excel-Template.xlsx).\n",
"\n"
]
},
{
"cell_type": "markdown",
"id": "5f7d99ad-6ebd-47d0-92a7-566630b0c22a",
"metadata": {},
"source": [
"## Setup\n",
"\n",
"We first setup and load the data. If you haven't already, [download the template](https://eqvista.com/app/uploads/2020/09/Eqvista_DCF-Excel-Template.xlsx) and name it `dcf_template.xlxs` locally."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d867d1a6-cfcf-4f53-952a-f4a6ff2fa205",
"metadata": {},
"outputs": [],
"source": [
"%pip install llama-index\n",
"%pip install llama-parse"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "103c7983-56d3-45be-b763-d1828d07c43e",
"metadata": {},
"outputs": [],
"source": [
"import nest_asyncio\n",
"\n",
"nest_asyncio.apply()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7b694b56-e04b-4d87-aa37-f0725d6b3adb",
"metadata": {},
"outputs": [],
"source": [
"from llama_parse import LlamaParse\n",
"\n",
"# api_key = \"llx-\" # get from cloud.llamaindex.ai"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9c4693c7-c1c8-47b4-8a8c-25d7e9ef9d2c",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Started parsing the file under job_id 9332a14a-78d2-4392-bbe8-a271d7572cd1\n"
]
}
],
"source": [
"parser = LlamaParse(\n",
" # api_key=api_key, # can also be set in your env as LLAMA_CLOUD_API_KEY\n",
" result_type=\"markdown\",\n",
")\n",
"\n",
"documents = parser.load_data(\"./dcf_template.xlsx\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7302f1c8-e405-4cda-8ff7-1d55185816f7",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"# DCF Model\n",
"\n",
"|Discounted Cash Flow Excel Template | | | | | | | | | | | |\n",
"|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------|-----------|-----------|-----------------------|-----------|-----------------------|--------------|-----------|-----------|-----------|--------------|\n",
"| | | | | | | | | | | | |\n",
"|Here is a simple discounted cash flow excel template for estimating your company value based on this income valuation approach | | | | | | | | | | | |\n",
"| | | | | | | | | | | | |\n",
"|Instructions: | | | | | | | | | | | |\n",
"|1) Fill out the two assumptions in yellow highlight | | | | | | | | | | | |\n",
"|2) Fill in either the 5 year or 3 year weighted average figures in yellow highlight | | | | | | | | | | | |\n",
"| | | | | | | | | | | | |\n",
"|Assumptions | | | | | | | | | | | |\n",
"|Tax Rate |20% | | | | | | | | | | |\n",
"|Discount Rate |15% | | | | | | | | | | |\n",
"| | | | | | | | | | | | |\n",
"|5 Year Weighted Moving Average | | | | | | | | | | | |\n",
"|Indication of Company Value |$242,995.43 | | | | | | | | | | |\n",
"| | | | | | | | | | | | |\n",
"|3 Year Weighted Moving Average | | | | | | | | | | | |\n",
"|Indication of Company Value |$158,651.07 | | | | | | | | | | |\n",
"| | | | | | | | | | | | |\n",
"| |5 Year Weighted Moving Average| | | | | | | | | | |\n",
"| |Past Years | | | | |Forecasted Future Years| | | | | |\n",
"| |Year 1 |Year 2 |Year 3 |Year 4 |Year 5 |Year 6 |Year 7 |Year 8 |Year 9 |Year 10 |Terminal Value|\n",
"|Pre-tax income |50,000.00 |55,000.00 |45,000.00 |52,000.00 |60,000.00 | | | | | | |\n",
"|Income Taxes |10,000.00 |11,000.00 |9,000.00 |10,400.00 |12,000.00 | | | | | | |\n",
"|Net Income |40,000.00 |44,000.00 |36,000.00 |41,600.00 |48,000.00 | | | | | | |\n",
"|Depreciation Expense |5,000.00 |4,000.00 |3,000.00 |2,000.00 |1,000.00 | | | | | | |\n",
"|Capital Expenditures |10,000.00 |8,000.00 |5,000.00 |5,000.00 |7,000.00 | | | | | | |\n",
"|Debt Repayments |5,000.00 |5,000.00 |5,000.00 |5,000.00 |5,000.00 | | | | | | |\n",
"|Net Cash Flow |20,000.00 |27,000.00 |23,000.00 |29,600.00 |35,000.00 |29,093.33 |29,817.78 |30,177.48 |30,469.23 |30,379.74 |287,188.00 |\n",
"|Discounting Factor | | | | | |0.8696 |0.7561 |0.6575 |0.5718 |0.4972 |0.4972 |\n",
"|Present Value of Future Cash Flow | | | | | |25,298.55 |22,546.52 |19,842.18 |17,420.88 |15,104.10 |142,783.19 |\n",
"| | | | | | | | | | | | |\n",
"| |3 Year Weighted Moving Average| | | | | | | | | | |\n",
"| |Past Years | | |Forecasted Future Years| | | | | | | |\n",
"| |Year 1 |Year 2 |Year 3 |Year 4 |Year 5 |Year 6 |Terminal Value| | | | |\n",
"|Pre-tax income |50,000.00 |55,000.00 |45,000.00 | | | | | | | | |\n",
"|Income Taxes |10,000.00 |11,000.00 |9,000.00 | | | | | | | | |\n",
"|Net Income |40,000.00 |44,000.00 |36,000.00 | | | | | | | | |\n",
"|Depreciation Expense |5,000.00 |4,000.00 |3,000.00 | | | | | | | | |\n",
"|Capital Expenditures |10,000.00 |8,000.00 |5,000.00 | | | | | | | | |\n",
"|Debt Repayments |5,000.00 |5,000.00 |5,000.00 | | | | | | | | |\n",
"|Net Cash Flow |20,000.00 |27,000.00 |23,000.00 |23,833.33 |24,083.33 |23,819.44 |158,253.59 | | | | |\n",
"|Discounting Factor | | | |0.8696 |0.7561 |0.6575 |0.6575 | | | | |\n",
"|Present Value of Future Cash Flow | | | |20,724.64 |18,210.46 |15,661.67 |104,054.30 | | | | |\n",
"| | | | | | | | | | | | |\n",
"|Notes: | | | | | | | | | | | |\n",
"|-We based this simple discounted cash flow excel model based on the weighted moving averages (5 year or 3 year) for simplicity, in case a constant growth rate cannot be easily determined.| | | | | | | | | | | |\n",
"|-The factors such as Depreciation Expense, Capital Expense and Debt Repayments remain constant, so consider this when looking at the forecasted figures. | | | | | | | | | | | |\n",
"|-For the terminal value constant growth rate, we make the assumption of the growth from the last forecasted year compared to the first forecasted year. Adjust in the formula as needed. | | | | | | | | | | | |\n",
"\n"
]
}
],
"source": [
"print(documents[1].get_content())"
]
},
{
"cell_type": "markdown",
"id": "1aedd4bb-7939-4fbc-8f07-d362e24d9772",
"metadata": {},
"source": [
"## Ask Questions over this Data\n",
"\n",
"Now let's try asking some questions over this data!"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f7c056a8-d098-4ebe-9341-d9f07081067c",
"metadata": {},
"outputs": [],
"source": [
"from llama_index.llms.openai import OpenAI\n",
"from llama_index.core import Settings\n",
"\n",
"llm = OpenAI(model=\"gpt-4-turbo-preview\")\n",
"Settings.llm = llm"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c0fa2630-ee1b-4ce7-91e9-f9ffff8347f9",
"metadata": {},
"outputs": [],
"source": [
"from llama_index.core import SummaryIndex\n",
"\n",
"index = SummaryIndex.from_documents([documents[1]])\n",
"\n",
"query_engine = index.as_query_engine()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a875a20e-a6b6-46b7-80d4-614546215ffc",
"metadata": {},
"outputs": [],
"source": [
"response = query_engine.query(\n",
" \"Tell me about the income taxes in the past years (year 3-5) for the 5 year WMA table\"\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "06b0b072-f159-47c4-9cad-9f0cc0d56b28",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"The income taxes in the past years (year 3 to year 5) for the 5-year Weighted Moving Average table are as follows: Year 3 - $9,000.00, Year 4 - $10,400.00, and Year 5 - $12,000.00.\n"
]
}
],
"source": [
"print(str(response))"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8bd0998f-4f7f-46f9-9b51-cfb510f384ee",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"# DCF Model\n",
"\n",
"|Discounted Cash Flow Excel Template | | | | | | | | | | | |\n",
"|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------|-----------|-----------|-----------------------|-----------|-----------------------|--------------|-----------|-----------|-----------|--------------|\n",
"| | | | | | | | | | | | |\n",
"|Here is a simple discounted cash flow excel template for estimating your company value based on this income valuation approach | | | | | | | | | | | |\n",
"| | | | | | | | | | | | |\n",
"|Instructions: | | | | | | | | | | | |\n",
"|1) Fill out the two assumptions in yellow highlight | | | | | | | | | | | |\n",
"|2) Fill in either the 5 year or 3 year weighted average figures in yellow highlight | | | | | | | | | | | |\n",
"| | | | | | | | | | | | |\n",
"|Assumptions | | | | | | | | | | | |\n",
"|Tax Rate |20% | | | | | | | | | | |\n",
"|Discount Rate |15% | | | | | | | | | | |\n",
"| | | | | | | | | | | | |\n",
"|5 Year Weighted Moving Average | | | | | | | | | | | |\n",
"|Indication of Company Value |$242,995.43 | | | | | | | | | | |\n",
"| | | | | | | | | | | | |\n",
"|3 Year Weighted Moving Average | | | | | | | | | | | |\n",
"|Indication of Company Value |$158,651.07 | | | | | | | | | | |\n",
"| | | | | | | | | | | | |\n",
"| |5 Year Weighted Moving Average| | | | | | | | | | |\n",
"| |Past Years | | | | |Forecasted Future Years| | | | | |\n",
"| |Year 1 |Year 2 |Year 3 |Year 4 |Year 5 |Year 6 |Year 7 |Year 8 |Year 9 |Year 10 |Terminal Value|\n",
"|Pre-tax income |50,000.00 |55,000.00 |45,000.00 |52,000.00 |60,000.00 | | | | | | |\n",
"|Income Taxes |10,000.00 |11,000.00 |9,000.00 |10,400.00 |12,000.00 | | | | | | |\n",
"|Net Income |40,000.00 |44,000.00 |36,000.00 |41,600.00 |48,000.00 | | | | | | |\n",
"|Depreciation Expense |5,000.00 |4,000.00 |3,000.00 |2,000.00 |1,000.00 | | | | | | |\n",
"|Capital Expenditures |10,000.00 |8,000.00 |5,000.00 |5,000.00 |7,000.00 | | | | | | |\n",
"|Debt Repayments |5,000.00 |5,000.00 |5,000.\n"
]
}
],
"source": [
"print(response.source_nodes[0].get_content())"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "fa699117-9867-4bb5-a290-30d35c59624f",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"The discounting factors for years 4-6 in the 3-year weighted moving average scenario are 0.8696, 0.7561, and 0.6575, respectively.\n"
]
}
],
"source": [
"response = query_engine.query(\n",
" \"Tell me about the discounting factors in years 4-6 for the 3 year WMA\"\n",
")\n",
"print(str(response))"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "llama_parse",
"language": "python",
"name": "llama_parse"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
File diff suppressed because it is too large Load Diff
Binary file not shown.

After

Width:  |  Height:  |  Size: 334 KiB

+21 -6
View File
@@ -63,7 +63,23 @@ class LlamaParse(BasePydanticReader):
)
invalidate_cache: Optional[bool] = Field(
default=False,
description="If set to true, the cache will be ignored and the document re-processes. All document are kept in cache for 48hours after the job was completed to avoid processing 2 time the same document.",
description="If set to true, the cache will be ignored and the document re-processes. All document are kept in cache for 48hours after the job was completed to avoid processing the same document twice.",
)
do_not_cache: Optional[bool] = Field(
default=False,
description="If set to true, the document will not be cached. This mean that you will be re-charged it you reprocess them as they will not be cached.",
)
fast_mode: Optional[bool] = Field(
default=False,
description="Note: Non compatible with gpt-4o. If set to true, the parser will use a faster mode to extract text from documents. This mode will skip OCR of images, and table/heading reconstruction.",
)
do_not_unroll_columns: Optional[bool] = Field(
default=False,
description="If set to true, the parser will keep column in the text according to document layout. Reduce reconstruction accuracy, and LLM's/embedings performances in most case.",
)
page_separator: Optional[str] = Field(
default=None,
description="The page separator to use to split the text. Default is None, which means the parser will use the default separator '\\n---\\n'.",
)
gpt4o_mode: bool = Field(
default=False,
@@ -73,10 +89,6 @@ class LlamaParse(BasePydanticReader):
default=None,
description="The API key for the GPT-4o API. Lowers the cost of parsing.",
)
fast_mode: bool = Field(
default=False,
description="Super fast mode for parsing. Only output Text, do not extract tables, images, etc.",
)
ignore_errors: bool = Field(
default=True,
description="Whether or not to ignore and skip errors raised during parsing.",
@@ -133,9 +145,12 @@ class LlamaParse(BasePydanticReader):
data={
"language": self.language.value,
"parsing_instruction": self.parsing_instruction,
"fast_mode": self.fast_mode,
"invalidate_cache": self.invalidate_cache,
"skip_diagonal_text": self.skip_diagonal_text,
"do_not_cache": self.do_not_cache,
"fast_mode": self.fast_mode,
"do_not_unroll_columns": self.do_not_unroll_columns,
"page_separator": self.page_separator,
"gpt4o_mode": self.gpt4o_mode,
"gpt4o_api_key": self.gpt4o_api_key,
},
+72 -38
View File
@@ -101,59 +101,93 @@ class Language(str, Enum):
SUPPORTED_FILE_TYPES = [
".pdf",
# Microsoft word - all versions
# document and presentations
".602",
".abw",
".cgm",
".cwk",
".doc",
".docx",
".docm",
".dot",
".dotx",
".dotm",
# Rich text format
".hwp",
".key",
".lwp",
".mw",
".mcw",
".pages",
".pbd",
".ppt",
".pptm",
".pptx",
".pot",
".potm",
".potx",
".rtf",
# Microsoft Works
".wps",
# Word Perfect
".wpd",
# Open Office
".sda",
".sdd",
".sdp",
".sdw",
".sgl",
".sti",
".sxi",
".sxw",
".stw",
".sxg",
# Apple
".pages",
# Mac Write
".mw",
".mcw",
# Unified Office Format text
".uot",
".txt",
".uof",
".uos",
".uop",
# Microsoft powerpoints
".ppt",
".pptx",
".pot",
".pptm",
".potx",
".potm",
# Apple keynote
".key",
# Open Office Presentations
".odp",
".odg",
".otp",
".fopd",
".sxi",
".sti",
# ebook
".uot",
".vor",
".wpd",
".wps",
".xml",
".zabw",
".epub",
# html
".html",
".htm",
# media,
# images
".jpg",
".jpeg",
".png",
".tiff",
".gif",
".bmp",
".svg",
".tiff",
".webp",
# web
".htm",
".html",
# spreadsheets
".xlsx",
".xls",
".xlsm",
".xlsb",
".xlw",
".csv",
".dif",
".sylk",
".slk",
".prn",
".numbers",
".et",
".ods",
".fods",
".uos1",
".uos2",
".dbf",
".wk1",
".wk2",
".wk3",
".wk4",
".wks",
".123",
".wq1",
".wq2",
".wb1",
".wb2",
".wb3",
".qpw",
".xlr",
".eth",
".tsv",
]