mirror of
https://github.com/run-llama/llama_cloud_services.git
synced 2026-07-19 16:43:32 -04:00
Compare commits
11 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 Jerry Liu
|
9df44ea15f | ||
|
Jerry Liu
|
f3c7a2e4e8 | ||
|
Jerry Liu
|
75e83f55a3 | ||
|
Jerry Liu
|
fc23db7d90 | ||
|
Jerry Liu
|
a9c4c6e8b4 | ||
|
Pierre-Loic Doulcet
|
58257d546b | ||
|
Adam Reichert
|
5e99d810fd | ||
|
Jerry Liu
|
a2dc717d85 | ||
|
Logan
|
87062e6ca8 | ||
|
Jerry Liu
|
ae3a21c5ff | ||
|
Ravi Theja
|
ccebb8a2fa |
@@ -6,6 +6,8 @@ LlamaParse directly integrates with [LlamaIndex](https://github.com/run-llama/ll
|
||||
|
||||
Free plan is up to 1000 pages a day. Paid plan is free 7k pages per week + 0.3c per additional page.
|
||||
|
||||
Read below for some quickstart information, or see the [full documentation](https://docs.cloud.llamaindex.ai/).
|
||||
|
||||
## Getting Started
|
||||
|
||||
First, login and get an api-key from [**https://cloud.llamaindex.ai ↗**](https://cloud.llamaindex.ai).
|
||||
@@ -87,6 +89,10 @@ Several end-to-end indexing examples can be found in the examples folder
|
||||
- [Advanced RAG Example](examples/demo_advanced.ipynb)
|
||||
- [Raw API Usage](examples/demo_api.ipynb)
|
||||
|
||||
## Documentation
|
||||
|
||||
[https://docs.cloud.llamaindex.ai/](https://docs.cloud.llamaindex.ai/)
|
||||
|
||||
## Terms of Service
|
||||
|
||||
See the [Terms of Service Here](./TOS.pdf).
|
||||
|
||||
Binary file not shown.
File diff suppressed because one or more lines are too long
@@ -0,0 +1,310 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "0db58db5-d4ee-4631-af5b-4fc53eb05170",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# RAG with Excel Spreadsheet using LlamaPrase\n",
|
||||
"\n",
|
||||
"<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/demo_excel.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
|
||||
"\n",
|
||||
"This notebook constructs a RAG pipeline over a simple DCF template [here](https://eqvista.com/app/uploads/2020/09/Eqvista_DCF-Excel-Template.xlsx).\n",
|
||||
"\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "5f7d99ad-6ebd-47d0-92a7-566630b0c22a",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"We first setup and load the data. If you haven't already, [download the template](https://eqvista.com/app/uploads/2020/09/Eqvista_DCF-Excel-Template.xlsx) and name it `dcf_template.xlxs` locally."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "d867d1a6-cfcf-4f53-952a-f4a6ff2fa205",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"%pip install llama-index\n",
|
||||
"%pip install llama-parse"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "103c7983-56d3-45be-b763-d1828d07c43e",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import nest_asyncio\n",
|
||||
"\n",
|
||||
"nest_asyncio.apply()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "7b694b56-e04b-4d87-aa37-f0725d6b3adb",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from llama_parse import LlamaParse\n",
|
||||
"\n",
|
||||
"# api_key = \"llx-\" # get from cloud.llamaindex.ai"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "9c4693c7-c1c8-47b4-8a8c-25d7e9ef9d2c",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Started parsing the file under job_id 9332a14a-78d2-4392-bbe8-a271d7572cd1\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"parser = LlamaParse(\n",
|
||||
" # api_key=api_key, # can also be set in your env as LLAMA_CLOUD_API_KEY\n",
|
||||
" result_type=\"markdown\",\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"documents = parser.load_data(\"./dcf_template.xlsx\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "7302f1c8-e405-4cda-8ff7-1d55185816f7",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"# DCF Model\n",
|
||||
"\n",
|
||||
"|Discounted Cash Flow Excel Template | | | | | | | | | | | |\n",
|
||||
"|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------|-----------|-----------|-----------------------|-----------|-----------------------|--------------|-----------|-----------|-----------|--------------|\n",
|
||||
"| | | | | | | | | | | | |\n",
|
||||
"|Here is a simple discounted cash flow excel template for estimating your company value based on this income valuation approach | | | | | | | | | | | |\n",
|
||||
"| | | | | | | | | | | | |\n",
|
||||
"|Instructions: | | | | | | | | | | | |\n",
|
||||
"|1) Fill out the two assumptions in yellow highlight | | | | | | | | | | | |\n",
|
||||
"|2) Fill in either the 5 year or 3 year weighted average figures in yellow highlight | | | | | | | | | | | |\n",
|
||||
"| | | | | | | | | | | | |\n",
|
||||
"|Assumptions | | | | | | | | | | | |\n",
|
||||
"|Tax Rate |20% | | | | | | | | | | |\n",
|
||||
"|Discount Rate |15% | | | | | | | | | | |\n",
|
||||
"| | | | | | | | | | | | |\n",
|
||||
"|5 Year Weighted Moving Average | | | | | | | | | | | |\n",
|
||||
"|Indication of Company Value |$242,995.43 | | | | | | | | | | |\n",
|
||||
"| | | | | | | | | | | | |\n",
|
||||
"|3 Year Weighted Moving Average | | | | | | | | | | | |\n",
|
||||
"|Indication of Company Value |$158,651.07 | | | | | | | | | | |\n",
|
||||
"| | | | | | | | | | | | |\n",
|
||||
"| |5 Year Weighted Moving Average| | | | | | | | | | |\n",
|
||||
"| |Past Years | | | | |Forecasted Future Years| | | | | |\n",
|
||||
"| |Year 1 |Year 2 |Year 3 |Year 4 |Year 5 |Year 6 |Year 7 |Year 8 |Year 9 |Year 10 |Terminal Value|\n",
|
||||
"|Pre-tax income |50,000.00 |55,000.00 |45,000.00 |52,000.00 |60,000.00 | | | | | | |\n",
|
||||
"|Income Taxes |10,000.00 |11,000.00 |9,000.00 |10,400.00 |12,000.00 | | | | | | |\n",
|
||||
"|Net Income |40,000.00 |44,000.00 |36,000.00 |41,600.00 |48,000.00 | | | | | | |\n",
|
||||
"|Depreciation Expense |5,000.00 |4,000.00 |3,000.00 |2,000.00 |1,000.00 | | | | | | |\n",
|
||||
"|Capital Expenditures |10,000.00 |8,000.00 |5,000.00 |5,000.00 |7,000.00 | | | | | | |\n",
|
||||
"|Debt Repayments |5,000.00 |5,000.00 |5,000.00 |5,000.00 |5,000.00 | | | | | | |\n",
|
||||
"|Net Cash Flow |20,000.00 |27,000.00 |23,000.00 |29,600.00 |35,000.00 |29,093.33 |29,817.78 |30,177.48 |30,469.23 |30,379.74 |287,188.00 |\n",
|
||||
"|Discounting Factor | | | | | |0.8696 |0.7561 |0.6575 |0.5718 |0.4972 |0.4972 |\n",
|
||||
"|Present Value of Future Cash Flow | | | | | |25,298.55 |22,546.52 |19,842.18 |17,420.88 |15,104.10 |142,783.19 |\n",
|
||||
"| | | | | | | | | | | | |\n",
|
||||
"| |3 Year Weighted Moving Average| | | | | | | | | | |\n",
|
||||
"| |Past Years | | |Forecasted Future Years| | | | | | | |\n",
|
||||
"| |Year 1 |Year 2 |Year 3 |Year 4 |Year 5 |Year 6 |Terminal Value| | | | |\n",
|
||||
"|Pre-tax income |50,000.00 |55,000.00 |45,000.00 | | | | | | | | |\n",
|
||||
"|Income Taxes |10,000.00 |11,000.00 |9,000.00 | | | | | | | | |\n",
|
||||
"|Net Income |40,000.00 |44,000.00 |36,000.00 | | | | | | | | |\n",
|
||||
"|Depreciation Expense |5,000.00 |4,000.00 |3,000.00 | | | | | | | | |\n",
|
||||
"|Capital Expenditures |10,000.00 |8,000.00 |5,000.00 | | | | | | | | |\n",
|
||||
"|Debt Repayments |5,000.00 |5,000.00 |5,000.00 | | | | | | | | |\n",
|
||||
"|Net Cash Flow |20,000.00 |27,000.00 |23,000.00 |23,833.33 |24,083.33 |23,819.44 |158,253.59 | | | | |\n",
|
||||
"|Discounting Factor | | | |0.8696 |0.7561 |0.6575 |0.6575 | | | | |\n",
|
||||
"|Present Value of Future Cash Flow | | | |20,724.64 |18,210.46 |15,661.67 |104,054.30 | | | | |\n",
|
||||
"| | | | | | | | | | | | |\n",
|
||||
"|Notes: | | | | | | | | | | | |\n",
|
||||
"|-We based this simple discounted cash flow excel model based on the weighted moving averages (5 year or 3 year) for simplicity, in case a constant growth rate cannot be easily determined.| | | | | | | | | | | |\n",
|
||||
"|-The factors such as Depreciation Expense, Capital Expense and Debt Repayments remain constant, so consider this when looking at the forecasted figures. | | | | | | | | | | | |\n",
|
||||
"|-For the terminal value constant growth rate, we make the assumption of the growth from the last forecasted year compared to the first forecasted year. Adjust in the formula as needed. | | | | | | | | | | | |\n",
|
||||
"\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"print(documents[1].get_content())"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "1aedd4bb-7939-4fbc-8f07-d362e24d9772",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Ask Questions over this Data\n",
|
||||
"\n",
|
||||
"Now let's try asking some questions over this data!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "f7c056a8-d098-4ebe-9341-d9f07081067c",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from llama_index.llms.openai import OpenAI\n",
|
||||
"from llama_index.core import Settings\n",
|
||||
"\n",
|
||||
"llm = OpenAI(model=\"gpt-4-turbo-preview\")\n",
|
||||
"Settings.llm = llm"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "c0fa2630-ee1b-4ce7-91e9-f9ffff8347f9",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from llama_index.core import SummaryIndex\n",
|
||||
"\n",
|
||||
"index = SummaryIndex.from_documents([documents[1]])\n",
|
||||
"\n",
|
||||
"query_engine = index.as_query_engine()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "a875a20e-a6b6-46b7-80d4-614546215ffc",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"response = query_engine.query(\n",
|
||||
" \"Tell me about the income taxes in the past years (year 3-5) for the 5 year WMA table\"\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "06b0b072-f159-47c4-9cad-9f0cc0d56b28",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"The income taxes in the past years (year 3 to year 5) for the 5-year Weighted Moving Average table are as follows: Year 3 - $9,000.00, Year 4 - $10,400.00, and Year 5 - $12,000.00.\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"print(str(response))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "8bd0998f-4f7f-46f9-9b51-cfb510f384ee",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"# DCF Model\n",
|
||||
"\n",
|
||||
"|Discounted Cash Flow Excel Template | | | | | | | | | | | |\n",
|
||||
"|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------|-----------|-----------|-----------------------|-----------|-----------------------|--------------|-----------|-----------|-----------|--------------|\n",
|
||||
"| | | | | | | | | | | | |\n",
|
||||
"|Here is a simple discounted cash flow excel template for estimating your company value based on this income valuation approach | | | | | | | | | | | |\n",
|
||||
"| | | | | | | | | | | | |\n",
|
||||
"|Instructions: | | | | | | | | | | | |\n",
|
||||
"|1) Fill out the two assumptions in yellow highlight | | | | | | | | | | | |\n",
|
||||
"|2) Fill in either the 5 year or 3 year weighted average figures in yellow highlight | | | | | | | | | | | |\n",
|
||||
"| | | | | | | | | | | | |\n",
|
||||
"|Assumptions | | | | | | | | | | | |\n",
|
||||
"|Tax Rate |20% | | | | | | | | | | |\n",
|
||||
"|Discount Rate |15% | | | | | | | | | | |\n",
|
||||
"| | | | | | | | | | | | |\n",
|
||||
"|5 Year Weighted Moving Average | | | | | | | | | | | |\n",
|
||||
"|Indication of Company Value |$242,995.43 | | | | | | | | | | |\n",
|
||||
"| | | | | | | | | | | | |\n",
|
||||
"|3 Year Weighted Moving Average | | | | | | | | | | | |\n",
|
||||
"|Indication of Company Value |$158,651.07 | | | | | | | | | | |\n",
|
||||
"| | | | | | | | | | | | |\n",
|
||||
"| |5 Year Weighted Moving Average| | | | | | | | | | |\n",
|
||||
"| |Past Years | | | | |Forecasted Future Years| | | | | |\n",
|
||||
"| |Year 1 |Year 2 |Year 3 |Year 4 |Year 5 |Year 6 |Year 7 |Year 8 |Year 9 |Year 10 |Terminal Value|\n",
|
||||
"|Pre-tax income |50,000.00 |55,000.00 |45,000.00 |52,000.00 |60,000.00 | | | | | | |\n",
|
||||
"|Income Taxes |10,000.00 |11,000.00 |9,000.00 |10,400.00 |12,000.00 | | | | | | |\n",
|
||||
"|Net Income |40,000.00 |44,000.00 |36,000.00 |41,600.00 |48,000.00 | | | | | | |\n",
|
||||
"|Depreciation Expense |5,000.00 |4,000.00 |3,000.00 |2,000.00 |1,000.00 | | | | | | |\n",
|
||||
"|Capital Expenditures |10,000.00 |8,000.00 |5,000.00 |5,000.00 |7,000.00 | | | | | | |\n",
|
||||
"|Debt Repayments |5,000.00 |5,000.00 |5,000.\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"print(response.source_nodes[0].get_content())"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "fa699117-9867-4bb5-a290-30d35c59624f",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"The discounting factors for years 4-6 in the 3-year weighted moving average scenario are 0.8696, 0.7561, and 0.6575, respectively.\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"response = query_engine.query(\n",
|
||||
" \"Tell me about the discounting factors in years 4-6 for the 3 year WMA\"\n",
|
||||
")\n",
|
||||
"print(str(response))"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "llama_parse",
|
||||
"language": "python",
|
||||
"name": "llama_parse"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
Binary file not shown.
|
After Width: | Height: | Size: 334 KiB |
@@ -0,0 +1,938 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "93ae9bad-b8cc-43de-ba7d-387e0155674c",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Building a Natively Multimodal RAG Pipeline (over a Slide Deck)\n",
|
||||
"\n",
|
||||
"<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/multimodal/multimodal_rag_ppt.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
|
||||
"\n",
|
||||
"In this cookbook we show you how to build a multimodal RAG pipeline over a slide deck, with text, tables, images, diagrams, and complex layouts.\n",
|
||||
"\n",
|
||||
"A gap of text-based RAG is that they struggle with purely text-based representations of complex documents. For instance, if a page contains a lot of images and diagrams, a text parser would need to rely on raw OCR to extract out text. You can also use a multimodal model (e.g. gpt-4o and up) to do text extraction, but this is inherently a lossy conversion.\n",
|
||||
"\n",
|
||||
"Instead a **native multimodal pipeline** stores both a text and image representation of a document chunk. They are indexed via embeddings (text or image), and during synthesis both text and image are directly fed to the multimodal model for synthesis.\n",
|
||||
"\n",
|
||||
"This can have the following advantages:\n",
|
||||
"- **Robustness**: This solution is more robust than a pure text or even a pure image-based approach. In a pure text RAG approach, the parsing piece can be lossy. In a pure image-based approach, multimodal OCR is not perfect and may lose out against text parsing for text-heavy documents.\n",
|
||||
"- **Cost Optimization**: You may choose to dynamically include text-only, or text + image depending on the content of the page.\n",
|
||||
"\n",
|
||||
""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "54e8d9a7-5036-4d32-818f-00b2e888521f",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Setup"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "70ccdd53-e68a-4199-aacb-cfe71ad1ff0b",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import nest_asyncio\n",
|
||||
"\n",
|
||||
"nest_asyncio.apply()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "225c5556-a789-4386-a1ee-cce01dbeb6cf",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Setup Observability\n",
|
||||
"\n",
|
||||
"We setup an integration with LlamaTrace (integration with Arize).\n",
|
||||
"\n",
|
||||
"If you haven't already done so, make sure to create an account here: https://llamatrace.com/login. Then create an API key and put it in the `PHOENIX_API_KEY` variable below."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "0eabee1f-290a-4c85-b362-54f45c8559ae",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"!pip install -U llama-index-callbacks-arize-phoenix"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "aaeb245c-730b-4c34-ad68-708fdde0e6cb",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# setup Arize Phoenix for logging/observability\n",
|
||||
"import llama_index.core\n",
|
||||
"import os\n",
|
||||
"\n",
|
||||
"PHOENIX_API_KEY = \"<PHOENIX_API_KEY>\"\n",
|
||||
"os.environ[\"OTEL_EXPORTER_OTLP_HEADERS\"] = f\"api_key={PHOENIX_API_KEY}\"\n",
|
||||
"llama_index.core.set_global_handler(\n",
|
||||
" \"arize_phoenix\", endpoint=\"https://llamatrace.com/v1/traces\"\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "fbb362db-b1b1-4eea-be1a-b1f78b0779d7",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Load Data\n",
|
||||
"\n",
|
||||
"Here we load the [Conoco Phillips 2023 investor meeting slide deck](https://static.conocophillips.com/files/2023-conocophillips-aim-presentation.pdf)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "8bce3407-a7d2-47e8-9eaf-ab297a94750c",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"!mkdir data\n",
|
||||
"!mkdir data_images\n",
|
||||
"!wget \"https://static.conocophillips.com/files/2023-conocophillips-aim-presentation.pdf\" -O data/conocophillips.pdf"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "246ba6b0-51af-42f9-b1b2-8d3e721ef782",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Model Setup\n",
|
||||
"\n",
|
||||
"Setup models that will be used for downstream orchestration."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "16e2071d-bbc2-4707-8ae7-cb4e1fecafd3",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from llama_index.core import Settings\n",
|
||||
"from llama_index.llms.openai import OpenAI\n",
|
||||
"from llama_index.embeddings.openai import OpenAIEmbedding\n",
|
||||
"\n",
|
||||
"embed_model = OpenAIEmbedding(model=\"text-embedding-3-large\")\n",
|
||||
"llm = OpenAI(model=\"gpt-4o\")\n",
|
||||
"\n",
|
||||
"Settings.embed_model = embed_model\n",
|
||||
"Settings.llm = llm"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "e3f6416f-f580-4722-aaa9-7f3500408547",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Use LlamaParse to Parse Text and Images\n",
|
||||
"\n",
|
||||
"In this example, use LlamaParse to parse both the text and images from the document.\n",
|
||||
"\n",
|
||||
"We parse out the text in two ways: \n",
|
||||
"- in regular `text` mode using our default text layout algorithm\n",
|
||||
"- in `markdown` mode using GPT-4o (`gpt4o_mode=True`). This also allows us to capture page screenshots"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "570089e5-238a-4dcc-af65-96e7393c2b4d",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from llama_parse import LlamaParse\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"parser_text = LlamaParse(result_type=\"text\")\n",
|
||||
"parser_gpt4o = LlamaParse(result_type=\"markdown\", gpt4o_mode=True)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "ef82a985-4088-4bb7-9a21-0318e1b9207d",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(f\"Parsing text...\")\n",
|
||||
"docs_text = parser_text.load_data(\"data/conocophillips.pdf\")\n",
|
||||
"print(f\"Parsing PDF file...\")\n",
|
||||
"md_json_objs = parser_gpt4o.get_json_result(\"data/conocophillips.pdf\")\n",
|
||||
"md_json_list = md_json_objs[0][\"pages\"]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "7506b603-c01f-45de-b354-4a0728dde03c",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(docs_text[0].get_content())"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "5318fb7b-fe6a-4a8a-b82e-4ed7b4512c37",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(md_json_list[10][\"md\"])"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "7a46a73e-c6e2-4b0b-bd10-31b0d3e4b70f",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"dict_keys(['page', 'text', 'md', 'images', 'items'])\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"print(md_json_list[1].keys())"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "eeadb16c-97eb-4622-9551-b34d7f90d72f",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"image_dicts = parser_gpt4o.get_images(md_json_objs, download_path=\"data_images\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "fd3e098b-0606-4429-b48d-d4fe0140fc0e",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Build Multimodal Index\n",
|
||||
"\n",
|
||||
"In this section we build the multimodal index over the parsed deck. \n",
|
||||
"\n",
|
||||
"We do this by creating **text** nodes from the document that contain metadata referencing the original image path.\n",
|
||||
"\n",
|
||||
"In this example we're indexing the text node for retrieval. The text node has a reference to both the parsed text as well as the image screenshot."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "3aae2dee-9d85-4604-8a51-705d4db527f7",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"#### Get Text Nodes"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "18c24174-05ce-417f-8dd2-79c3f375db03",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from llama_index.core.schema import TextNode\n",
|
||||
"from typing import Optional"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "8e331dfe-a627-4e23-8c57-70ab1d9342e4",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# get pages loaded through llamaparse\n",
|
||||
"import re\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"def get_page_number(file_name):\n",
|
||||
" match = re.search(r\"-page-(\\d+)\\.jpg$\", str(file_name))\n",
|
||||
" if match:\n",
|
||||
" return int(match.group(1))\n",
|
||||
" return 0\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"def _get_sorted_image_files(image_dir):\n",
|
||||
" \"\"\"Get image files sorted by page.\"\"\"\n",
|
||||
" raw_files = [f for f in list(Path(image_dir).iterdir()) if f.is_file()]\n",
|
||||
" sorted_files = sorted(raw_files, key=get_page_number)\n",
|
||||
" return sorted_files"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "346fe5ef-171e-4a54-9084-7a7805103a13",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from copy import deepcopy\n",
|
||||
"from pathlib import Path\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"# attach image metadata to the text nodes\n",
|
||||
"def get_text_nodes(docs, image_dir=None, json_dicts=None):\n",
|
||||
" \"\"\"Split docs into nodes, by separator.\"\"\"\n",
|
||||
" nodes = []\n",
|
||||
"\n",
|
||||
" image_files = _get_sorted_image_files(image_dir) if image_dir is not None else None\n",
|
||||
" md_texts = [d[\"md\"] for d in json_dicts] if json_dicts is not None else None\n",
|
||||
"\n",
|
||||
" doc_chunks = docs[0].text.split(\"---\")\n",
|
||||
" for idx, doc_chunk in enumerate(doc_chunks):\n",
|
||||
" chunk_metadata = {\"page_num\": idx + 1}\n",
|
||||
" if image_files is not None:\n",
|
||||
" image_file = image_files[idx]\n",
|
||||
" chunk_metadata[\"image_path\"] = str(image_file)\n",
|
||||
" if md_texts is not None:\n",
|
||||
" chunk_metadata[\"parsed_text_markdown\"] = md_texts[idx]\n",
|
||||
" chunk_metadata[\"parsed_text\"] = doc_chunk\n",
|
||||
" node = TextNode(\n",
|
||||
" text=\"\",\n",
|
||||
" metadata=chunk_metadata,\n",
|
||||
" )\n",
|
||||
" nodes.append(node)\n",
|
||||
"\n",
|
||||
" return nodes"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "f591669c-5a8e-491d-9cef-0b754abbf26f",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# this will split into pages\n",
|
||||
"text_nodes = get_text_nodes(docs_text, image_dir=\"data_images\", json_dicts=md_json_list)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "32c13950-c1db-435f-b5b4-89d62b8b7744",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"page_num: 11\n",
|
||||
"image_path: data_images/d9137e19-3974-4b5d-998f-dac0cf29dd9d-page-10.jpg\n",
|
||||
"parsed_text_markdown: # Commitment to Disciplined Reinvestment Rate\n",
|
||||
"\n",
|
||||
"| Year | Reinvestment Rate | WTI Average Price | Reinvestment Rate at $60/BBL WTI | Reinvestment Rate at $80/BBL WTI |\n",
|
||||
"|------------|-------------------|-------------------|----------------------------------|----------------------------------|\n",
|
||||
"| 2012-2016 | >100% | ~$75/BBL | | |\n",
|
||||
"| 2017-2022 | <60% | ~$63/BBL | | |\n",
|
||||
"| 2023E | | | | at $80/BBL WTI |\n",
|
||||
"| 2024-2028 | | | at $60/BBL WTI | at $80/BBL WTI |\n",
|
||||
"| 2029-2032 | | | at $60/BBL WTI | at $80/BBL WTI |\n",
|
||||
"\n",
|
||||
"**Disciplined Reinvestment Rate is the Foundation for Superior Returns on and of Capital, while Driving Durable CFO Growth**\n",
|
||||
"\n",
|
||||
"- ~50% 10-Year Reinvestment Rate\n",
|
||||
"- ~6% CFO CAGR 2024-2032 at $60/BBL WTI Mid-Cycle Planning Price\n",
|
||||
"\n",
|
||||
"**Note:** Reinvestment rate and cash from operations (CFO) are non-GAAP measures. Definitions and reconciliations are included in the Appendix.\n",
|
||||
"parsed_text: \n",
|
||||
"Commitment to Disciplined Reinvestment Rate\n",
|
||||
" Industry ConocoPhillips\n",
|
||||
" Strategy Reset Disciplined Reinvestment Rate is the Foundation for Superior\n",
|
||||
" Growth Focus Returns on and of Capital, while Driving Durable CFO Growth\n",
|
||||
" 100% <60% 50% 6% at $60/BBL WTI\n",
|
||||
" Reinvestment Rate Reinvestment Rate Reinvestment Rate10-YearCFO CAGR Planning PriceMid-Cycle\n",
|
||||
" 2024-2032\n",
|
||||
" 2 100%\n",
|
||||
" 1 75%\n",
|
||||
" 1 50%\n",
|
||||
" 1 WTIat $80/BBL at S80/BBL\n",
|
||||
" 25% 'S75/BBL $63/BBL WTI\n",
|
||||
" WTI WTI at S80/BBL at S60/BBL at S60/BBL\n",
|
||||
" Average Average WTI WTI WTI\n",
|
||||
" 0%\n",
|
||||
" 2012-2016 2017-2022 2023E 2024-2028 2029-2032\n",
|
||||
" Historic Reinvestment Rate Reinvestment Rate at $60/BBL WTI Reinvestment Rate at $80/BBL WTI\n",
|
||||
" Reinvestment rate andcashfrom operations (CFO) are non-GAAP measures: Definitions and reconciliations are included in the Appendix ConocoPhillips\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"print(text_nodes[10].get_content(metadata_mode=\"all\"))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "4f404f56-db1e-4ed7-9ba1-ead763546348",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"#### Build Index\n",
|
||||
"\n",
|
||||
"Once the text nodes are ready, we feed into our vector store index abstraction, which will index these nodes into a simple in-memory vector store (of course, you should definitely check out our 40+ vector store integrations!)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "6ea53c31-0e38-421c-8d9b-0e3adaa1677e",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import os\n",
|
||||
"from llama_index.core import (\n",
|
||||
" StorageContext,\n",
|
||||
" VectorStoreIndex,\n",
|
||||
" load_index_from_storage,\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"if not os.path.exists(\"storage_nodes\"):\n",
|
||||
" index = VectorStoreIndex(text_nodes, embed_model=embed_model)\n",
|
||||
" # save index to disk\n",
|
||||
" index.set_index_id(\"vector_index\")\n",
|
||||
" index.storage_context.persist(\"./storage_nodes\")\n",
|
||||
"else:\n",
|
||||
" # rebuild storage context\n",
|
||||
" storage_context = StorageContext.from_defaults(persist_dir=\"storage_nodes\")\n",
|
||||
" # load index\n",
|
||||
" index = load_index_from_storage(storage_context, index_id=\"vector_index\")\n",
|
||||
"\n",
|
||||
"retriever = index.as_retriever()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "5f0e33a4-9422-498d-87ee-d917bdf74d80",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Build Multimodal Query Engine\n",
|
||||
"\n",
|
||||
"We now use LlamaIndex abstractions to build a **custom query engine**. In contrast to a standard RAG query engine that will retrieve the text node and only put that into the prompt (response synthesis module), this custom query engine will also load the image document, and put both the text and image document into the response synthesis module."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "35a94be2-e289-41a6-92e4-d3cb428fb0c8",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from llama_index.core.query_engine import CustomQueryEngine, SimpleMultiModalQueryEngine\n",
|
||||
"from llama_index.core.retrievers import BaseRetriever\n",
|
||||
"from llama_index.multi_modal_llms.openai import OpenAIMultiModal\n",
|
||||
"from llama_index.core.schema import ImageNode, NodeWithScore, MetadataMode\n",
|
||||
"from llama_index.core.prompts import PromptTemplate\n",
|
||||
"from llama_index.core.base.response.schema import Response\n",
|
||||
"from typing import Optional\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"gpt_4o = OpenAIMultiModal(model=\"gpt-4o\", max_new_tokens=4096)\n",
|
||||
"\n",
|
||||
"QA_PROMPT_TMPL = \"\"\"\\\n",
|
||||
"Below we give parsed text from slides in two different formats, as well as the image.\n",
|
||||
"\n",
|
||||
"We parse the text in both 'markdown' mode as well as 'raw text' mode. Markdown mode attempts \\\n",
|
||||
"to convert relevant diagrams into tables, whereas raw text tries to maintain the rough spatial \\\n",
|
||||
"layout of the text.\n",
|
||||
"\n",
|
||||
"Use the image information first and foremost. ONLY use the text/markdown information \n",
|
||||
"if you can't understand the image.\n",
|
||||
"\n",
|
||||
"---------------------\n",
|
||||
"{context_str}\n",
|
||||
"---------------------\n",
|
||||
"Given the context information and not prior knowledge, answer the query. Explain whether you got the answer\n",
|
||||
"from the parsed markdown or raw text or image, and if there's discrepancies, and your reasoning for the final answer.\n",
|
||||
"\n",
|
||||
"Query: {query_str}\n",
|
||||
"Answer: \"\"\"\n",
|
||||
"\n",
|
||||
"QA_PROMPT = PromptTemplate(QA_PROMPT_TMPL)\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"class MultimodalQueryEngine(CustomQueryEngine):\n",
|
||||
" \"\"\"Custom multimodal Query Engine.\n",
|
||||
"\n",
|
||||
" Takes in a retriever to retrieve a set of document nodes.\n",
|
||||
" Also takes in a prompt template and multimodal model.\n",
|
||||
"\n",
|
||||
" \"\"\"\n",
|
||||
"\n",
|
||||
" qa_prompt: PromptTemplate\n",
|
||||
" retriever: BaseRetriever\n",
|
||||
" multi_modal_llm: OpenAIMultiModal\n",
|
||||
"\n",
|
||||
" def __init__(self, qa_prompt: Optional[PromptTemplate] = None, **kwargs) -> None:\n",
|
||||
" \"\"\"Initialize.\"\"\"\n",
|
||||
" super().__init__(qa_prompt=qa_prompt or QA_PROMPT, **kwargs)\n",
|
||||
"\n",
|
||||
" def custom_query(self, query_str: str):\n",
|
||||
" # retrieve text nodes\n",
|
||||
" nodes = self.retriever.retrieve(query_str)\n",
|
||||
" # create ImageNode items from text nodes\n",
|
||||
" image_nodes = [\n",
|
||||
" NodeWithScore(node=ImageNode(image_path=n.metadata[\"image_path\"]))\n",
|
||||
" for n in nodes\n",
|
||||
" ]\n",
|
||||
"\n",
|
||||
" # create context string from text nodes, dump into the prompt\n",
|
||||
" context_str = \"\\n\\n\".join(\n",
|
||||
" [r.get_content(metadata_mode=MetadataMode.LLM) for r in nodes]\n",
|
||||
" )\n",
|
||||
" fmt_prompt = self.qa_prompt.format(context_str=context_str, query_str=query_str)\n",
|
||||
"\n",
|
||||
" # synthesize an answer from formatted text and images\n",
|
||||
" llm_response = self.multi_modal_llm.complete(\n",
|
||||
" prompt=fmt_prompt,\n",
|
||||
" image_documents=[image_node.node for image_node in image_nodes],\n",
|
||||
" )\n",
|
||||
" return Response(\n",
|
||||
" response=str(llm_response),\n",
|
||||
" source_nodes=nodes,\n",
|
||||
" metadata={\"text_nodes\": text_nodes, \"image_nodes\": image_nodes},\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
" return response"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "0890be59-fb12-4bb5-959b-b2d9600f7774",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"query_engine = MultimodalQueryEngine(\n",
|
||||
" retriever=index.as_retriever(similarity_top_k=9), multi_modal_llm=gpt_4o\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "a92aa4f1-7501-4711-b054-f02338e54e74",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Define Baseline\n",
|
||||
"\n",
|
||||
"In addition, we define a \"baseline\" where we rely only on text-based indexing. Here we define an index using only the nodes that are parsed in text-mode from LlamaParse. \n",
|
||||
"\n",
|
||||
"**NOTE**: We don't currently include the markdown-parsed text because that was parsed with GPT-4o, so already uses a multimodal model during the text extraction phase.\n",
|
||||
"\n",
|
||||
"It is of course a valid experiment to compare RAG where multimodal extraction only happens during indexing, vs. the current multimodal RAG implementation where images are fed during synthesis to the LLM. "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "c0b15a48-d177-4666-aec2-98ee90664642",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def get_nodes(docs):\n",
|
||||
" \"\"\"Split docs into nodes, by separator.\"\"\"\n",
|
||||
" nodes = []\n",
|
||||
" for doc in docs:\n",
|
||||
" doc_chunks = doc.text.split(\"\\n---\\n\")\n",
|
||||
" for doc_chunk in doc_chunks:\n",
|
||||
" node = TextNode(\n",
|
||||
" text=doc_chunk,\n",
|
||||
" metadata=deepcopy(doc.metadata),\n",
|
||||
" )\n",
|
||||
" nodes.append(node)\n",
|
||||
"\n",
|
||||
" return nodes"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "2065d2c6-d6ba-4ee3-8e9e-dbc83cbcec1b",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"base_nodes = get_nodes(docs_text)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "bcaea1a8-26c9-4385-8f62-32855aa898b6",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Our Differentiated Portfolio: Deep; Durable and Diverse\n",
|
||||
" 20 BBOE of Resource Diverse Production Base\n",
|
||||
" Under $40/BBL Cost of Supply 10-Year Plan Cumulative Production (BBOE)\n",
|
||||
" S50 S32/BBL Lower 48 Alaska\n",
|
||||
" Average Cost of Supply\n",
|
||||
" 3$40 GKA GWA\n",
|
||||
" GPA WNS\n",
|
||||
" $30 EMENA\n",
|
||||
" 3 Norway\n",
|
||||
" 8 $20\n",
|
||||
" E Qatar Libya\n",
|
||||
" Asia Pacific Canada\n",
|
||||
" $10 Permian\n",
|
||||
" APLNG Montney\n",
|
||||
" S0\n",
|
||||
" 10 15 20 Bakken\n",
|
||||
" Resource (BBOE) Eagle Ford Other MalaysiaChina Surmont\n",
|
||||
" Lower 48 Canada Alaska EMENA Asia Pacific\n",
|
||||
"Costs assumemid-cycle price environment of S60/BBL WTI:\n",
|
||||
" ConocoPhillips\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"print(base_nodes[13].get_content(metadata_mode=\"all\"))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "f6bcfbc6-4e9b-41ad-ad81-1c4245b95cd5",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"base_index = VectorStoreIndex(base_nodes, embed_model=embed_model)\n",
|
||||
"base_query_engine = base_index.as_query_engine(llm=llm, similarity_top_k=9)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "1f94ef26-0df5-4468-a156-903d686f02ce",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Build a Multimodal Agent\n",
|
||||
"\n",
|
||||
"Build an agent around the multimodal query engine. This gives you agent capabilities like query planning/decomposition and memory around a central QA interface."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "5b7a8c5f-39fc-4d04-8c56-3642f5718437",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from llama_index.core.tools import QueryEngineTool\n",
|
||||
"from llama_index.core.agent import FunctionCallingAgentWorker\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"vector_tool = QueryEngineTool.from_defaults(\n",
|
||||
" query_engine=query_engine,\n",
|
||||
" name=\"vector_tool\",\n",
|
||||
" description=(\n",
|
||||
" \"Useful for retrieving specific context from the data. Do NOT select if question asks for a summary of the data.\"\n",
|
||||
" ),\n",
|
||||
")\n",
|
||||
"agent = FunctionCallingAgentWorker.from_tools(\n",
|
||||
" [vector_tool], llm=llm, verbose=True\n",
|
||||
").as_agent()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "2b4f7eb1-d247-45fa-bb41-c02fc353a22a",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# define a similar agent for the baseline\n",
|
||||
"base_vector_tool = QueryEngineTool.from_defaults(\n",
|
||||
" query_engine=base_query_engine,\n",
|
||||
" name=\"vector_tool\",\n",
|
||||
" description=(\n",
|
||||
" \"Useful for retrieving specific context from the data. Do NOT select if question asks for a summary of the data.\"\n",
|
||||
" ),\n",
|
||||
")\n",
|
||||
"base_agent = FunctionCallingAgentWorker.from_tools(\n",
|
||||
" [base_vector_tool], llm=llm, verbose=True\n",
|
||||
").as_agent()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "2336f98b-c0a1-413a-849d-8a89bacb90b5",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Try out Queries\n",
|
||||
"\n",
|
||||
"Let's try out queries against these documents and compare against each other."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "1cdce5d8-6bb3-4cd3-929d-1cec249d9052",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Added user message to memory: How does the Conoco Phillips capex/EUR in the delaware basin compare against other competitors?\n",
|
||||
"=== Calling Function ===\n",
|
||||
"Calling function: vector_tool with args: {\"input\": \"Conoco Phillips capex/EUR in the Delaware Basin\"}\n",
|
||||
"=== Function Output ===\n",
|
||||
"The ConocoPhillips capex/EUR in the Delaware Basin is $10/BOE.\n",
|
||||
"\n",
|
||||
"I obtained this information from the image provided. The image clearly shows a bar chart under the section \"Delaware Basin Well Capex/EUR ($/BOE)\" where ConocoPhillips is listed with a capex/EUR of $10/BOE. This information is consistent with the parsed markdown text, which also lists ConocoPhillips' capex/EUR as $10/BOE in the Delaware Basin. There are no discrepancies between the image and the parsed markdown text in this case.\n",
|
||||
"=== Calling Function ===\n",
|
||||
"Calling function: vector_tool with args: {\"input\": \"competitors capex/EUR in the Delaware Basin\"}\n",
|
||||
"=== Function Output ===\n",
|
||||
"The competitors' Capex/EUR in the Delaware Basin can be found in the image on the slide titled \"Delaware: Vast Inventory with Proven Track Record of Performance.\" The relevant information is presented in a bar chart under the section \"Delaware Basin Well Capex/EUR ($/BOE)\".\n",
|
||||
"\n",
|
||||
"Here are the details:\n",
|
||||
"\n",
|
||||
"- ConocoPhillips: $10/BOE\n",
|
||||
"- Competitor 1: $15/BOE\n",
|
||||
"- Competitor 2: $20/BOE\n",
|
||||
"- Competitor 3: $25/BOE\n",
|
||||
"- Competitor 4: $30/BOE\n",
|
||||
"- Competitor 5: $35/BOE\n",
|
||||
"- Competitor 6: $40/BOE\n",
|
||||
"- Competitor 7: $45/BOE\n",
|
||||
"\n",
|
||||
"This information was obtained directly from the image, which provides a clear visual representation of the Capex/EUR values for ConocoPhillips and its competitors in the Delaware Basin. The parsed markdown text also confirms these values, ensuring consistency between the image and the text.\n",
|
||||
"=== LLM Response ===\n",
|
||||
"The capital expenditure per estimated ultimate recovery (capex/EUR) for ConocoPhillips in the Delaware Basin is $10 per barrel of oil equivalent (BOE). When compared to its competitors, ConocoPhillips has a significantly lower capex/EUR. Here are the capex/EUR values for ConocoPhillips and its competitors:\n",
|
||||
"\n",
|
||||
"- **ConocoPhillips**: $10/BOE\n",
|
||||
"- **Competitor 1**: $15/BOE\n",
|
||||
"- **Competitor 2**: $20/BOE\n",
|
||||
"- **Competitor 3**: $25/BOE\n",
|
||||
"- **Competitor 4**: $30/BOE\n",
|
||||
"- **Competitor 5**: $35/BOE\n",
|
||||
"- **Competitor 6**: $40/BOE\n",
|
||||
"- **Competitor 7**: $45/BOE\n",
|
||||
"\n",
|
||||
"This data indicates that ConocoPhillips has a more cost-efficient operation in the Delaware Basin compared to its competitors.\n",
|
||||
"The capital expenditure per estimated ultimate recovery (capex/EUR) for ConocoPhillips in the Delaware Basin is $10 per barrel of oil equivalent (BOE). When compared to its competitors, ConocoPhillips has a significantly lower capex/EUR. Here are the capex/EUR values for ConocoPhillips and its competitors:\n",
|
||||
"\n",
|
||||
"- **ConocoPhillips**: $10/BOE\n",
|
||||
"- **Competitor 1**: $15/BOE\n",
|
||||
"- **Competitor 2**: $20/BOE\n",
|
||||
"- **Competitor 3**: $25/BOE\n",
|
||||
"- **Competitor 4**: $30/BOE\n",
|
||||
"- **Competitor 5**: $35/BOE\n",
|
||||
"- **Competitor 6**: $40/BOE\n",
|
||||
"- **Competitor 7**: $45/BOE\n",
|
||||
"\n",
|
||||
"This data indicates that ConocoPhillips has a more cost-efficient operation in the Delaware Basin compared to its competitors.\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# response = agent.query(\"Tell me about the different regions and subregions where Conoco Phillips has a production base.\")\n",
|
||||
"response = agent.query(\n",
|
||||
" \"How does the Conoco Phillips capex/EUR in the delaware basin compare against other competitors?\"\n",
|
||||
")\n",
|
||||
"print(str(response))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "d584c560-8f49-4c10-a4db-2e0d3b7085d2",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"page_num: 38\n",
|
||||
"image_path: data_images/d9137e19-3974-4b5d-998f-dac0cf29dd9d-page-37.jpg\n",
|
||||
"parsed_text_markdown: # Delaware: Vast Inventory with Proven Track Record of Performance\n",
|
||||
"\n",
|
||||
"## Prolific Acreage Spanning Over ~659,000 Net Acres¹\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"### Total 10-Year Operated Permian Inventory\n",
|
||||
"\n",
|
||||
"- Delaware Basin: 65%\n",
|
||||
"- Midland Basin: 35%\n",
|
||||
"\n",
|
||||
"### High Single-Digit Production Growth\n",
|
||||
"\n",
|
||||
"## 12-Month Cumulative Production³ (BOE/FT)\n",
|
||||
"\n",
|
||||
"| Months | 2019 | 2020 | 2021 | 2022 |\n",
|
||||
"|--------|------|------|------|------|\n",
|
||||
"| 1 | 0 | 0 | 0 | 0 |\n",
|
||||
"| 2 | 5 | 6 | 7 | 8 |\n",
|
||||
"| 3 | 10 | 12 | 14 | 16 |\n",
|
||||
"| 4 | 15 | 18 | 21 | 24 |\n",
|
||||
"| 5 | 20 | 24 | 28 | 32 |\n",
|
||||
"| 6 | 25 | 30 | 35 | 40 |\n",
|
||||
"| 7 | 30 | 36 | 42 | 48 |\n",
|
||||
"| 8 | 35 | 42 | 49 | 56 |\n",
|
||||
"| 9 | 40 | 48 | 56 | 64 |\n",
|
||||
"| 10 | 45 | 54 | 63 | 72 |\n",
|
||||
"| 11 | 50 | 60 | 70 | 80 |\n",
|
||||
"| 12 | 55 | 66 | 77 | 88 |\n",
|
||||
"\n",
|
||||
"~30% Improved Performance from 2019 to 2022\n",
|
||||
"\n",
|
||||
"## Delaware Basin Well Capex/EUR⁴ ($/BOE)\n",
|
||||
"\n",
|
||||
"| Company | Capex/EUR |\n",
|
||||
"|------------------|-----------|\n",
|
||||
"| ConocoPhillips | 10 |\n",
|
||||
"| Competitor 1 | 15 |\n",
|
||||
"| Competitor 2 | 20 |\n",
|
||||
"| Competitor 3 | 25 |\n",
|
||||
"| Competitor 4 | 30 |\n",
|
||||
"| Competitor 5 | 35 |\n",
|
||||
"| Competitor 6 | 40 |\n",
|
||||
"| Competitor 7 | 45 |\n",
|
||||
"\n",
|
||||
"---\n",
|
||||
"\n",
|
||||
"¹ Unconventional acres. \n",
|
||||
"² Source: Enverus and ConocoPhillips (March 2023). \n",
|
||||
"³ Source: Enverus (March 2023) based on wells online year. \n",
|
||||
"⁴ Source: Enverus (March 2023). Average single well capex/EUR. Top eight public operators based on wells online in years 2021-2022, greater than 50% oil weight. COP based on COP well design. Competitors include: CVX, DVN, EOG, MTDR, OXY, PR and XOM.\n",
|
||||
"parsed_text: \n",
|
||||
"Delaware: Vast Inventory with Proven Track Record of Performance\n",
|
||||
" New Prolific Acreage Spanning Over 12-Month Cumulative Production? (BOE/FT)\n",
|
||||
" Mexico 659,000 Net Acres' 40\n",
|
||||
" Texas 3828\n",
|
||||
" 30 2019\n",
|
||||
" 20 30%\n",
|
||||
" 10 Improved Performancefrom 2019 to 2022\n",
|
||||
" Total\n",
|
||||
" Permian Inventory\n",
|
||||
" 10-Year Operated\n",
|
||||
" 2 10 11 12\n",
|
||||
" Months\n",
|
||||
" Delaware Basin Well Capex/EUR4 (S/BOE)\n",
|
||||
" 65% 25\n",
|
||||
" Delaware Basin 20\n",
|
||||
" Midland Basin 15\n",
|
||||
" Low HighCost of Supplyz 10 ConocoPhillips\n",
|
||||
" High Single-Digit Production Growth\n",
|
||||
" \"Unconventional acres. 2Source: Enverus and ConocoPhillips (March 2023). 3SourceEnverus (March 2023) based on wells online year: \"Source; Enverus (March 2023). Average single well capex/EUR Top eight public operators based on\n",
|
||||
"wells online in years 2021-2022, greater than 50% oil weight; COP based on COP well design: Competitors include; CVX DVN, EOG; MTDR, OXY, PR and XOM: ConocoPhillips\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"print(response.source_nodes[0].get_content(metadata_mode=\"all\"))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "d21d694b-6618-4d04-a6f6-8b0c2625f539",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Added user message to memory: How does the Conoco Phillips capex/EUR in the delaware basin compare against other competitors?\n",
|
||||
"=== Calling Function ===\n",
|
||||
"Calling function: vector_tool with args: {\"input\": \"Conoco Phillips capex/EUR in the Delaware Basin\"}\n",
|
||||
"=== Function Output ===\n",
|
||||
"ConocoPhillips' capex/EUR in the Delaware Basin is approximately $20/BOE.\n",
|
||||
"=== Calling Function ===\n",
|
||||
"Calling function: vector_tool with args: {\"input\": \"competitors capex/EUR in the Delaware Basin\"}\n",
|
||||
"=== Function Output ===\n",
|
||||
"The average single well capex/EUR for competitors in the Delaware Basin is between $10 and $25 per BOE.\n",
|
||||
"=== LLM Response ===\n",
|
||||
"ConocoPhillips' capex/EUR in the Delaware Basin is approximately $20 per BOE. In comparison, the average capex/EUR for competitors in the Delaware Basin ranges between $10 and $25 per BOE. This places ConocoPhillips' capex/EUR towards the higher end of the competitive range.\n",
|
||||
"ConocoPhillips' capex/EUR in the Delaware Basin is approximately $20 per BOE. In comparison, the average capex/EUR for competitors in the Delaware Basin ranges between $10 and $25 per BOE. This places ConocoPhillips' capex/EUR towards the higher end of the competitive range.\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# base_response = base_agent.query(\"Tell me about the different regions and subregions where Conoco Phillips has a production base.\")\n",
|
||||
"base_response = base_agent.query(\n",
|
||||
" \"How does the Conoco Phillips capex/EUR in the delaware basin compare against other competitors?\"\n",
|
||||
")\n",
|
||||
"print(str(base_response))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "d3afccae-ad8d-4c5d-9d93-810dba413a5d",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Deep, Durable and Diverse Portfolio with Significant Growth Runway\n",
|
||||
" 1,2002022 Lower 48 Unconventional Production' (MBOED S50 ~S32/BBL\n",
|
||||
" 000 ConocoPhillips Cost of SupplyAverage\n",
|
||||
" 00 S40\n",
|
||||
" 500 3\n",
|
||||
" 400 1 S30\n",
|
||||
" 200\n",
|
||||
" 5\n",
|
||||
" 15,000ConocoPhillipsNet Remaining Well Inventory? 1 S20\n",
|
||||
" 12,000 S10\n",
|
||||
" 000\n",
|
||||
" 0o0 SO\n",
|
||||
" 3,000 10\n",
|
||||
" Resource (BBOE)\n",
|
||||
" Delaware Basin Midland Basin Eagle Ford Bakken Other\n",
|
||||
" Largest Lower 48 Unconventional Producer; Growing into the Next Decade\n",
|
||||
" onshore operated inventory that achieves 15% IRR at $SO/BBL WTI, Competitors include CVX, DVN, EOG, FANG, MRO, OXY, PXD,and XOM:\n",
|
||||
" Source: Wood Mackenzie Lower 48 Unconventional Plays 2022 ProductionCompetitors include CVX, DVN; EOG, FANG, MRO, OXY, PXD and XOM; greaterthan50% liquids weight: ?Source: Wood Mackenzie (March 2023), Lower 48\n",
|
||||
" ConocoPhillips\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"print(base_response.source_nodes[0].get_content(metadata_mode=\"llm\"))"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "llama_index_v3",
|
||||
"language": "python",
|
||||
"name": "llama_index_v3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 271 KiB |
+21
-1
@@ -63,7 +63,23 @@ class LlamaParse(BasePydanticReader):
|
||||
)
|
||||
invalidate_cache: Optional[bool] = Field(
|
||||
default=False,
|
||||
description="If set to true, the cache will be ignored and the document re-processes. All document are kept in cache for 48hours after the job was completed to avoid processing 2 time the same document.",
|
||||
description="If set to true, the cache will be ignored and the document re-processes. All document are kept in cache for 48hours after the job was completed to avoid processing the same document twice.",
|
||||
)
|
||||
do_not_cache: Optional[bool] = Field(
|
||||
default=False,
|
||||
description="If set to true, the document will not be cached. This mean that you will be re-charged it you reprocess them as they will not be cached.",
|
||||
)
|
||||
fast_mode: Optional[bool] = Field(
|
||||
default=False,
|
||||
description="Note: Non compatible with gpt-4o. If set to true, the parser will use a faster mode to extract text from documents. This mode will skip OCR of images, and table/heading reconstruction.",
|
||||
)
|
||||
do_not_unroll_columns: Optional[bool] = Field(
|
||||
default=False,
|
||||
description="If set to true, the parser will keep column in the text according to document layout. Reduce reconstruction accuracy, and LLM's/embedings performances in most case.",
|
||||
)
|
||||
page_separator: Optional[str] = Field(
|
||||
default=None,
|
||||
description="The page separator to use to split the text. Default is None, which means the parser will use the default separator '\\n---\\n'.",
|
||||
)
|
||||
gpt4o_mode: bool = Field(
|
||||
default=False,
|
||||
@@ -131,6 +147,10 @@ class LlamaParse(BasePydanticReader):
|
||||
"parsing_instruction": self.parsing_instruction,
|
||||
"invalidate_cache": self.invalidate_cache,
|
||||
"skip_diagonal_text": self.skip_diagonal_text,
|
||||
"do_not_cache": self.do_not_cache,
|
||||
"fast_mode": self.fast_mode,
|
||||
"do_not_unroll_columns": self.do_not_unroll_columns,
|
||||
"page_separator": self.page_separator,
|
||||
"gpt4o_mode": self.gpt4o_mode,
|
||||
"gpt4o_api_key": self.gpt4o_api_key,
|
||||
},
|
||||
|
||||
Reference in New Issue
Block a user