Compare commits

...

34 Commits

Author SHA1 Message Date
Logan Markewich 3c46824afb fix publish flow 2025-02-11 17:53:50 -06:00
Logan c872617b4e add organization id and project id as args (#616) 2025-02-11 17:46:50 -06:00
Jen Person 47c8682761 fixing colab links (#611) 2025-02-10 11:29:02 -06:00
Jerry Liu 683400788b add gemini2 flash notebook (#606) 2025-02-07 14:48:40 -06:00
Logan 05065a8329 v0.6.0 (#603)
* v0.6.0

* nit release

* nit
2025-02-06 17:39:40 -06:00
Logan 1ae4d2bbc7 Refactor into llama-cloud-services (#597) 2025-02-06 16:15:57 -06:00
Sacha Bron ae38f406fd fix release pipeline no-dev issue (#592) 2025-01-24 15:17:01 +01:00
Pierre-Loic Doulcet 4897d01cb0 add new formatting instruction parameters (#582)
* add new formatting instruction parameters

* bump version

* wip

* s3 region

* update test
2025-01-22 15:56:57 +01:00
Logan bd7b563463 v0.5.19 (#569) 2024-12-27 13:07:01 -06:00
apostoli 530241dd0b Stoli/feat/connection handling (#568) 2024-12-27 11:34:00 -06:00
Pierre-Loic Doulcet 6338641107 Extract layout, audio files (#557) 2024-12-18 16:29:17 +01:00
Bharath Lakshman Kumar 6d62fb89c3 Fix docstring for aget_xlsx method (#551)
Updated docstring to describe xlsx download instead of image download
2024-12-13 20:35:18 +05:30
Ravi Theja 7d4df3b6e5 Add cookbook for parsing instructions (#550) 2024-12-13 06:44:17 -08:00
Ravi Theja bc28db5b92 Update cache parameter (#548) 2024-12-11 16:15:59 +01:00
Jerry Liu f78186c0f7 update auto-mode (#545) 2024-12-09 16:13:09 -06:00
Laurie Voss e3292f5566 Expanding auto mode notebook with strings and regex triggers (#544) 2024-12-09 12:03:09 -08:00
Jerry Liu 58f980f411 auto-mode notebook (#540)
Co-authored-by: Laurie Voss <github@seldo.com>
2024-12-09 08:59:21 -08:00
Ravi Theja 4740d0611d Add get charts function (#542)
* Add get charts function

* code refactoring

* solve linting

* Add cookbook
2024-12-09 21:28:48 +05:30
Laurie Voss 3651a10e80 JSON mode tour notebook (#531) 2024-12-06 14:21:15 -08:00
Pierre-Loic Doulcet 483b51c51c Add support for html_remove_navigation_elements. (#532) 2024-12-06 12:05:46 +01:00
Ravi Theja cdbddef86d Add demo videos notebooks (#529) 2024-12-05 08:38:34 -08:00
Pierre-Loic Doulcet 3690109abf Add more parameters (#525)
* add after revert

* 3.8 so numpy work

* change defaults

* change requested

* change requested
2024-12-04 15:39:00 +01:00
Pierre-Loic Doulcet 2e322b4fc8 Revert "Add more paramerters"
This reverts commit 735e5f3ddc.
2024-12-04 10:20:07 +01:00
Pierre-Loic Doulcet 735e5f3ddc Add more paramerters 2024-12-04 10:17:08 +01:00
Logan e4cb4c75e5 add test for downloading images (#506) 2024-11-21 13:08:29 -06:00
Jerry Liu 1693deff72 dynamic section retrieval nb (#484) 2024-11-13 13:29:30 +01:00
Jerry Liu 3270f1228d multimodal report generation image (#461)
* cr

* cr
2024-11-13 13:28:07 +01:00
Pierre-Loic Doulcet eeabf48d29 add input url and http_proxy (#475) 2024-11-12 12:56:58 -06:00
Pierre-Loic Doulcet 89348aa8e5 add xlsx support (#472) 2024-11-01 10:09:17 -06:00
Thiago Salvatore 3ab2ce27b5 Add PurePosixPath to list of allowed file-paths (#464) 2024-10-25 10:45:47 -06:00
Sacha Bron 265261862f Add continuous_mode (#460) 2024-10-22 19:45:46 +02:00
Sacha Bron 66cf052b8c Update issue templates (#457)
* Update issue templates

* Update issue templates
2024-10-21 19:51:46 +02:00
Jerry Liu 2ca2d81e58 fix RFP example (#455) 2024-10-21 09:13:24 -07:00
Sacha Bron 951ba4dfd8 Release is_formatting_instruction parameter (#446)
* Release is_formatting_instruction parameter

* Add annotate links
2024-10-17 12:29:05 +02:00
123 changed files with 14052 additions and 3363 deletions
+4 -10
View File
@@ -7,8 +7,6 @@ assignees: ''
---
_Note: we're aware of some missing content in the output and layout issues on tables. Please refrain from opening new issues on this topic unless if you think it's different from what has already been reported._
**Describe the bug**
Write a concise description of what the bug is.
@@ -19,19 +17,15 @@ If possible, please provide the PDF file causing the issue.
If you have it, please provide the ID of the job you ran.
You can find it here: https://cloud.llamaindex.ai/parse in the "History" tab.
**Screenshots**
Feel free to also provide screenshots if relevant.
**Client:**
Please remove untested options:
- Frontend (cloud.llamaindex.ai)
- Python Library
- API
- Frontend (cloud.llamaindex.ai)
- Typescript Library
- Notebook
- API
**Options**
What options did you use? Multimodal, fast mode, parsing instructions, etc.
**Additional context**
Add any additional context about the problem here.
What options did you use? Premium mode, multimodal, fast mode, parsing instructions, etc.
Screenshots, code snippets, etc.
+1 -1
View File
@@ -45,4 +45,4 @@ jobs:
- name: Test import
shell: bash
working-directory: ${{ vars.RUNNER_TEMP }}
run: python -c "import llama_parse"
run: python -c "import llama_cloud_services"
+15 -4
View File
@@ -14,7 +14,7 @@ env:
jobs:
build-n-publish:
name: Build and publish to PyPI
if: github.repository == 'run-llama/llama_parse'
if: github.repository == 'run-llama/llama_cloud_services'
runs-on: ubuntu-latest
steps:
@@ -23,18 +23,28 @@ jobs:
uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
- name: Install Poetry
uses: snok/install-poetry@v1
with:
version: ${{ env.POETRY_VERSION }}
- name: Install deps
shell: bash
run: pip install -e .
- name: Build and publish to pypi
uses: JRubics/poetry-publish@v1.17
- name: Build and publish llama-cloud-services
uses: JRubics/poetry-publish@v2.1
with:
pypi_token: ${{ secrets.LLAMA_PARSE_PYPI_TOKEN }}
ignore_dev_requirements: "yes"
poetry_install_options: "--without dev"
- name: Build and publish llama-parse
uses: JRubics/poetry-publish@v2.1
with:
working_directory: "llama_parse"
pypi_token: ${{ secrets.LLAMA_PARSE_PYPI_TOKEN }}
poetry_install_options: "--without dev"
- name: Create GitHub Release
id: create_release
@@ -52,6 +62,7 @@ jobs:
export PKG=$(ls dist/ | grep tar)
set -- $PKG
echo "name=$1" >> $GITHUB_ENV
- name: Upload Release Asset (sdist) to GitHub
id: upload-release-asset
uses: actions/upload-release-asset@v1
+1 -1
View File
@@ -17,7 +17,7 @@ jobs:
# You can use PyPy versions in python-version.
# For example, pypy-2.7 and pypy-3.8
matrix:
python-version: ["3.8", "3.10", "3.11"]
python-version: ["3.9", "3.10", "3.11", "3.12"]
steps:
- uses: actions/checkout@v3
with:
+2 -1
View File
@@ -33,6 +33,7 @@ repos:
rev: v1.0.1
hooks:
- id: mypy
exclude: ^tests/
additional_dependencies:
[
"types-requests",
@@ -46,7 +47,7 @@ repos:
[
--disallow-untyped-defs,
--ignore-missing-imports,
--python-version=3.8,
--python-version=3.10,
]
- repo: https://github.com/adamchainz/blacken-docs
rev: 1.16.0
+22 -137
View File
@@ -1,158 +1,45 @@
# LlamaParse
[![PyPI - Downloads](https://img.shields.io/pypi/dm/llama-parse)](https://pypi.org/project/llama-parse/)
[![GitHub contributors](https://img.shields.io/github/contributors/run-llama/llama_parse)](https://github.com/run-llama/llama_parse/graphs/contributors)
[![PyPI - Downloads](https://img.shields.io/pypi/dm/llama-cloud-services)](https://pypi.org/project/llama-cloud-services/)
[![GitHub contributors](https://img.shields.io/github/contributors/run-llama/llama_cloud_services)](https://github.com/run-llama/llama_cloud_services/graphs/contributors)
[![Discord](https://img.shields.io/discord/1059199217496772688)](https://discord.gg/dGcwcsnxhU)
LlamaParse is a **GenAI-native document parser** that can parse complex document data for any downstream LLM use case (RAG, agents).
# Llama Cloud Services
It is really good at the following:
This repository contains the code for hand-written SDKs and clients for interacting with LlamaCloud.
-**Broad file type support**: Parsing a variety of unstructured file types (.pdf, .pptx, .docx, .xlsx, .html) with text, tables, visual elements, weird layouts, and more.
-**Table recognition**: Parsing embedded tables accurately into text and semi-structured representations.
-**Multimodal parsing and chunking**: Extracting visual elements (images/diagrams) into structured formats and return image chunks using the latest multimodal models.
-**Custom parsing**: Input custom prompt instructions to customize the output the way you want it.
This includes:
LlamaParse directly integrates with [LlamaIndex](https://github.com/run-llama/llama_index).
The free plan is up to 1000 pages a day. Paid plan is free 7k pages per week + 0.3c per additional page by default. There is a sandbox available to test the API [**https://cloud.llamaindex.ai/parse ↗**](https://cloud.llamaindex.ai/parse).
Read below for some quickstart information, or see the [full documentation](https://docs.cloud.llamaindex.ai/).
If you're a company interested in enterprise RAG solutions, and/or high volume/on-prem usage of LlamaParse, come [talk to us](https://www.llamaindex.ai/contact).
- [LlamaParse](./parse.md) - A GenAI-native document parser that can parse complex document data for any downstream LLM use case (Agents, RAG, data processing, etc.).
- [LlamaReport (beta/invite-only)](./report.md) - A prebuilt agentic report builder that can be used to build reports from a variety of data sources.
- [LlamaExtract (coming soon!)]() - A prebuilt agentic data extractor that can be used to transform data into a structured JSON representation.
## Getting Started
First, login and get an api-key from [**https://cloud.llamaindex.ai/api-key ↗**](https://cloud.llamaindex.ai/api-key).
Then, make sure you have the latest LlamaIndex version installed.
**NOTE:** If you are upgrading from v0.9.X, we recommend following our [migration guide](https://pretty-sodium-5e0.notion.site/v0-10-0-Migration-Guide-6ede431dcb8841b09ea171e7f133bd77), as well as uninstalling your previous version first.
```
pip uninstall llama-index # run this if upgrading from v0.9.x or older
pip install -U llama-index --upgrade --no-cache-dir --force-reinstall
```
Lastly, install the package:
`pip install llama-parse`
Now you can parse your first PDF file using the command line interface. Use the command `llama-parse [file_paths]`. See the help text with `llama-parse --help`.
Install the package:
```bash
export LLAMA_CLOUD_API_KEY='llx-...'
# output as text
llama-parse my_file.pdf --result-type text --output-file output.txt
# output as markdown
llama-parse my_file.pdf --result-type markdown --output-file output.md
# output as raw json
llama-parse my_file.pdf --output-raw-json --output-file output.json
pip install llama-cloud-services
```
You can also create simple scripts:
Then, get your API key from [LlamaCloud](https://cloud.llamaindex.ai/).
Then, you can use the services in your code:
```python
import nest_asyncio
from llama_cloud_services import LlamaParse, LlamaReport
nest_asyncio.apply()
from llama_parse import LlamaParse
parser = LlamaParse(
api_key="llx-...", # can also be set in your env as LLAMA_CLOUD_API_KEY
result_type="markdown", # "markdown" and "text" are available
num_workers=4, # if multiple files passed, split in `num_workers` API calls
verbose=True,
language="en", # Optionally you can define a language, default=en
)
# sync
documents = parser.load_data("./my_file.pdf")
# sync batch
documents = parser.load_data(["./my_file1.pdf", "./my_file2.pdf"])
# async
documents = await parser.aload_data("./my_file.pdf")
# async batch
documents = await parser.aload_data(["./my_file1.pdf", "./my_file2.pdf"])
parser = LlamaParse(api_key="YOUR_API_KEY")
report = LlamaReport(api_key="YOUR_API_KEY")
```
## Using with file object
See the quickstart guides for each service for more information:
You can parse a file object directly:
```python
import nest_asyncio
nest_asyncio.apply()
from llama_parse import LlamaParse
parser = LlamaParse(
api_key="llx-...", # can also be set in your env as LLAMA_CLOUD_API_KEY
result_type="markdown", # "markdown" and "text" are available
num_workers=4, # if multiple files passed, split in `num_workers` API calls
verbose=True,
language="en", # Optionally you can define a language, default=en
)
file_name = "my_file1.pdf"
extra_info = {"file_name": file_name}
with open(f"./{file_name}", "rb") as f:
# must provide extra_info with file_name key with passing file object
documents = parser.load_data(f, extra_info=extra_info)
# you can also pass file bytes directly
with open(f"./{file_name}", "rb") as f:
file_bytes = f.read()
# must provide extra_info with file_name key with passing file bytes
documents = parser.load_data(file_bytes, extra_info=extra_info)
```
## Using with `SimpleDirectoryReader`
You can also integrate the parser as the default PDF loader in `SimpleDirectoryReader`:
```python
import nest_asyncio
nest_asyncio.apply()
from llama_parse import LlamaParse
from llama_index.core import SimpleDirectoryReader
parser = LlamaParse(
api_key="llx-...", # can also be set in your env as LLAMA_CLOUD_API_KEY
result_type="markdown", # "markdown" and "text" are available
verbose=True,
)
file_extractor = {".pdf": parser}
documents = SimpleDirectoryReader(
"./data", file_extractor=file_extractor
).load_data()
```
Full documentation for `SimpleDirectoryReader` can be found on the [LlamaIndex Documentation](https://docs.llamaindex.ai/en/stable/module_guides/loading/simpledirectoryreader.html).
## Examples
Several end-to-end indexing examples can be found in the examples folder
- [Getting Started](examples/demo_basic.ipynb)
- [Advanced RAG Example](examples/demo_advanced.ipynb)
- [Raw API Usage](examples/demo_api.ipynb)
- [LlamaParse](./parse.md)
- [LlamaReport (beta/invite-only)](./report.md)
- [LlamaExtract (coming soon!)]()
## Documentation
[https://docs.cloud.llamaindex.ai/](https://docs.cloud.llamaindex.ai/)
You can see complete SDK and API documentation for each service on [our official docs](https://docs.cloud.llamaindex.ai/).
## Terms of Service
@@ -160,6 +47,4 @@ See the [Terms of Service Here](./TOS.pdf).
## Get in Touch (LlamaCloud)
LlamaParse is part of LlamaCloud, our e2e enterprise RAG platform that provides out-of-the-box, production-ready connectors, indexing, and retrieval over your complex data sources. We offer SaaS and VPC options.
LlamaCloud is currently available via waitlist (join by [creating an account](https://cloud.llamaindex.ai/)). If you're interested in state-of-the-art quality and in centralizing your RAG efforts, come [get in touch with us](https://www.llamaindex.ai/contact).
You can get in touch with us by following our [contact link](https://www.llamaindex.ai/contact).
File diff suppressed because it is too large Load Diff
Binary file not shown.

After

Width:  |  Height:  |  Size: 6.9 MiB

@@ -22,7 +22,7 @@
"metadata": {},
"outputs": [],
"source": [
"!pip install llama-parse llama-index llama-index-postprocessor-sbert-rerank"
"!pip install llama-cloud-services llama-index llama-index-postprocessor-sbert-rerank"
]
},
{
@@ -82,7 +82,7 @@
"metadata": {},
"outputs": [],
"source": [
"from llama_parse import LlamaParse\n",
"from llama_cloud_services import LlamaParse\n",
"\n",
"parser = LlamaParse(\n",
" result_type=\"markdown\",\n",
@@ -7,7 +7,7 @@
"source": [
"# RAG over the Caltrain Weekend Schedule \n",
"\n",
"<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/caltrain/caltrain_text_mode.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
"<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/parse/caltrain/caltrain_text_mode.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
"\n",
"This example shows off LlamaParse parsing capabilities to build a functioning query pipeline over the Caltrain weekend schedule, a big timetable containing all trains northbound and southbound and their stops in various cities.\n",
"\n",
@@ -81,7 +81,7 @@
}
],
"source": [
"from llama_parse import LlamaParse\n",
"from llama_cloud_services import LlamaParse\n",
"\n",
"docs = LlamaParse(result_type=\"text\").load_data(\"./caltrain_schedule_weekend.pdf\")"
]
@@ -26,7 +26,7 @@
"!pip install llama-index-embeddings-openai\n",
"!pip install llama-index-postprocessor-flag-embedding-reranker\n",
"!pip install git+https://github.com/FlagOpen/FlagEmbedding.git\n",
"!pip install llama-parse"
"!pip install llama-cloud-services"
]
},
{
@@ -108,7 +108,7 @@
}
],
"source": [
"from llama_parse import LlamaParse\n",
"from llama_cloud_services import LlamaParse\n",
"\n",
"documents = LlamaParse(result_type=\"markdown\").load_data(\"./apple_2021_10k.pdf\")"
]
@@ -22,7 +22,7 @@
"%pip install llama-index-embeddings-openai\n",
"%pip install llama-index-postprocessor-flag-embedding-reranker\n",
"%pip install git+https://github.com/FlagOpen/FlagEmbedding.git\n",
"%pip install llama-parse\n",
"%pip install llama-cloud-services\n",
"%pip install llama-index-vector-stores-astra-db"
]
},
@@ -107,7 +107,7 @@
}
],
"source": [
"from llama_parse import LlamaParse\n",
"from llama_cloud_services import LlamaParse\n",
"\n",
"documents = LlamaParse(result_type=\"markdown\").load_data(\"./uber_10q_march_2022.pdf\")"
]
@@ -6,7 +6,7 @@
"source": [
"# Advanced RAG with LlamaParse + Weaviate\n",
"\n",
"<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/demo_advanced_weaviate.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\\\" alt=\"Open In Colab\"/></a>\n",
"<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/parse/demo_advanced_weaviate.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\\\" alt=\"Open In Colab\"/></a>\n",
"\n",
"This notebook shows you how to use `LlamaParse` for advancd RAG applications with `LlamaIndex` and [Weaviate](https://weaviate.io/).\n",
"\n",
@@ -176,7 +176,7 @@
}
],
"source": [
"from llama_parse import LlamaParse\n",
"from llama_cloud_services import LlamaParse\n",
"\n",
"documents = LlamaParse(result_type=\"markdown\").load_data(\"./uber_10q_march_2022.pdf\")"
]
@@ -130,7 +130,7 @@
}
],
"source": [
"from llama_parse import LlamaParse\n",
"from llama_cloud_services import LlamaParse\n",
"\n",
"documents = LlamaParse(result_type=\"text\").load_data(file_path)"
]
@@ -73,7 +73,7 @@
}
],
"source": [
"from llama_parse import LlamaParse\n",
"from llama_cloud_services import LlamaParse\n",
"\n",
"documents = LlamaParse(result_type=\"text\").load_data(\"./attention.pdf\")"
]
@@ -120,7 +120,7 @@
}
],
"source": [
"from llama_parse import LlamaParse\n",
"from llama_cloud_services import LlamaParse\n",
"\n",
"documents = LlamaParse(result_type=\"markdown\").load_data(\"./attention.pdf\")"
]
@@ -142,7 +142,7 @@
}
],
"source": [
"from llama_parse import LlamaParse\n",
"from llama_cloud_services import LlamaParse\n",
"\n",
"documents = LlamaParse(result_type=\"text\").load_data(file_path)"
]
@@ -6,7 +6,7 @@
"source": [
"# RAG with Excel Spreadsheet using LlamaPrase\n",
"\n",
"<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/demo_excel.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
"<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/demo_excel.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
"\n",
"This notebook shows you using LlamaParse with Excel Spreadsheet.\n",
"\n",
@@ -21,7 +21,7 @@
"outputs": [],
"source": [
"%pip install llama-index\n",
"%pip install llama-parse"
"%pip install llama-cloud-services"
]
},
{
@@ -41,7 +41,7 @@
"\n",
"nest_asyncio.apply()\n",
"\n",
"from llama_parse import LlamaParse\n",
"from llama_cloud_services import LlamaParse\n",
"\n",
"api_key = \"llx-\" # get from cloud.llamaindex.ai"
]
File diff suppressed because one or more lines are too long
@@ -6,7 +6,7 @@
"source": [
"# LlamaParse - Fast checking Insurance Contract for Coverage\n",
"\n",
"<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/demo_insurance.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
"<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/demo_insurance.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
"\n",
"In this notebook we will look at how LlamaParse can be used to extract structured coverage information from an insurance policy."
]
@@ -116,7 +116,7 @@
}
],
"source": [
"from llama_parse import LlamaParse\n",
"from llama_cloud_services import LlamaParse\n",
"\n",
"documents = LlamaParse(result_type=\"markdown\").load_data(\"./policy.pdf\")"
]
@@ -7,7 +7,7 @@
"source": [
"# LlamaParse JSON Mode + Multimodal RAG\n",
"\n",
"<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/demo_json.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
"<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/parse/demo_json.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
"\n",
"This notebook shows you how to use LlamaParse JSON mode with LlamaIndex to build a simple multimodal RAG pipeline.\n",
"\n",
@@ -35,7 +35,7 @@
"!pip install llama-index-core\n",
"!pip install llama-index-llms-anthropic llama-index-multi-modal-llms-anthropic\n",
"!pip install llama-index-embeddings-huggingface\n",
"!pip install llama-parse"
"!pip install llama-cloud-services"
]
},
{
@@ -129,7 +129,7 @@
}
],
"source": [
"from llama_parse import LlamaParse\n",
"from llama_cloud_services import LlamaParse\n",
"\n",
"parser = LlamaParse(verbose=True)\n",
"json_objs = parser.get_json_result(\"./uber_10q_march_2022.pdf\")\n",
@@ -7,7 +7,7 @@
"source": [
"# LlamaParse JSON Mode + Advanced RAG with `LlamaParseJsonNodeParser`\n",
"\n",
"<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/demo_json_parsing.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
"<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/demo_json_parsing.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
"\n",
"This notebook shows you how to use LlamaParse JSON mode with LlamaIndex to build a simple recursive retrieval RAG pipeline using `LlamaParseJsonNodeParser`.\n",
"\n",
@@ -37,7 +37,7 @@
"%pip install llama-index-core\n",
"%pip install llama-index-llms-anthropic llama-index-multi-modal-llms-anthropic\n",
"%pip install llama-index-embeddings-huggingface\n",
"%pip install llama-parse"
"%pip install llama-cloud-services"
]
},
{
@@ -110,7 +110,7 @@
}
],
"source": [
"from llama_parse import LlamaParse\n",
"from llama_cloud_services import LlamaParse\n",
"\n",
"parser = LlamaParse(verbose=True)\n",
"json_objs = parser.get_json_result(\"./uber_10q_march_2022.pdf\")\n",
File diff suppressed because it is too large Load Diff
@@ -9,7 +9,7 @@
"\n",
"LlamaParse supports users to specify a `language` parameter before uploading documents, giving users better OCR capabilities over non-English PDFs, parsing images into more accurate representations.\n",
"\n",
"You can specify 80+ different languages: see this file for a full list of supported languages: https://github.com/run-llama/llama_parse/blob/main/llama_parse/base.py.\n",
"You can specify 80+ different languages: see this file for a full list of supported languages: https://github.com/run-llama/llama_cloud_services/blob/main/llama_parse/base.py.\n",
"\n",
"This notebook shows a demo of this in action. "
]
@@ -77,7 +77,7 @@
}
],
"source": [
"from llama_parse import LlamaParse\n",
"from llama_cloud_services import LlamaParse\n",
"\n",
"parser = LlamaParse(result_type=\"text\", language=\"fr\")\n",
"documents = parser.load_data(\"./treasury_report.pdf\")"
@@ -250,7 +250,7 @@
}
],
"source": [
"from llama_parse import LlamaParse\n",
"from llama_cloud_services import LlamaParse\n",
"\n",
"parser = LlamaParse(result_type=\"text\", language=\"ch_sim\")\n",
"documents = parser.load_data(\"./chinese_pdf.pdf\")"
@@ -404,7 +404,7 @@
}
],
"source": [
"from llama_parse import LlamaParse\n",
"from llama_cloud_services import LlamaParse\n",
"\n",
"base_parser = LlamaParse(result_type=\"text\", language=\"en\")\n",
"base_documents = parser.load_data(\"./chinese_pdf2.pdf\")"
@@ -7,7 +7,7 @@
"source": [
"# LlamaParse With MongoDB\n",
"\n",
"<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/demo_mongodb.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
"<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/parse/demo_mongodb.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
"\n",
"In this notebook, we provide a straightforward example of using LlamaParse with MongoDB Atlas VectorSearch.\n",
"\n",
@@ -69,7 +69,7 @@
"import pymongo\n",
"\n",
"from llama_index.vector_stores.mongodb import MongoDBAtlasVectorSearch\n",
"from llama_parse import LlamaParse\n",
"from llama_cloud_services import LlamaParse\n",
"from llama_index.embeddings.openai import OpenAIEmbedding\n",
"from llama_index.core import VectorStoreIndex, StorageContext\n",
"from llama_index.core.node_parser import SimpleNodeParser"
@@ -114,7 +114,7 @@
}
],
"source": [
"%pip install llama-parse"
"%pip install llama-cloud-services"
]
},
{
@@ -169,7 +169,7 @@
"metadata": {},
"outputs": [],
"source": [
"from llama_parse import LlamaParse"
"from llama_cloud_services import LlamaParse"
]
},
{
@@ -0,0 +1,357 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "97c79c38-38a3-40f3-ba2e-250649347d63",
"metadata": {},
"source": [
"<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/parse/demo_starter_multimodal.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
]
},
{
"cell_type": "markdown",
"id": "4e081457",
"metadata": {},
"source": [
"# Multimodal Parsing using LlamaParse\n",
"\n",
"This cookbook shows you how to use LlamaParse to parse any document with the multimodal capabilities of Multi-Modal LLMs from Anthropic/ OpenAI.\n",
"\n",
"LlamaParse allows you to plug in external, multimodal model vendors for parsing - we handle the error correction, validation, and scalability/reliability for you.\n"
]
},
{
"cell_type": "markdown",
"id": "qOdqBxCS51Ow",
"metadata": {},
"source": [
"### Installation"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "H_Vqcylb50vm",
"metadata": {},
"outputs": [],
"source": [
"!pip install llama-cloud-services"
]
},
{
"cell_type": "markdown",
"id": "15e60ecf-519c-41fc-911b-765adaf8bad4",
"metadata": {},
"source": [
"### Setup\n",
"\n",
"Here we setup `LLAMA_CLOUD_API_KEY` for using `LlamaParse`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "91a9e532-1454-40e0-bbf0-fd442c350121",
"metadata": {},
"outputs": [],
"source": [
"import nest_asyncio\n",
"\n",
"nest_asyncio.apply()\n",
"\n",
"import os\n",
"\n",
"# API access to llama-cloud\n",
"os.environ[\"LLAMA_CLOUD_API_KEY\"] = \"<YOUR LLAMACLOUD API KEY>\""
]
},
{
"cell_type": "markdown",
"id": "LGwBNPNotZRQ",
"metadata": {},
"source": [
"## Download Data\n",
"\n",
"For this demonstration, we will use OpenAI's recent paper `Evaluation of OpenAI o1: Opportunities and Challenges of AGI`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "IjtKDQRLrylI",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"--2024-12-05 18:54:24-- https://arxiv.org/pdf/2409.18486\n",
"Resolving arxiv.org (arxiv.org)... 151.101.67.42, 151.101.131.42, 151.101.3.42, ...\n",
"Connecting to arxiv.org (arxiv.org)|151.101.67.42|:443... connected.\n",
"HTTP request sent, awaiting response... 200 OK\n",
"Length: 13986265 (13M) [application/pdf]\n",
"Saving to: o1.pdf\n",
"\n",
"o1.pdf 100%[===================>] 13.34M 11.8MB/s in 1.1s \n",
"\n",
"2024-12-05 18:54:26 (11.8 MB/s) - o1.pdf saved [13986265/13986265]\n",
"\n"
]
}
],
"source": [
"!wget \"https://arxiv.org/pdf/2409.18486\" -O \"o1.pdf\""
]
},
{
"cell_type": "markdown",
"id": "4e29a9d7-5bd9-4fb8-8ec1-4c128a748662",
"metadata": {},
"source": [
"## Initialize LlamaParse\n",
"\n",
"Initialize LlamaParse in multimodal mode, and specify the vendor.\n",
"\n",
"**NOTE**: optionally you can specify the Anthropic/ OpenAI API key. If you choose to do so LlamaParse will only charge you 1 credit (0.3c) per page. \n",
"\n",
"\n",
"Using your own API key may incur additional costs from your model provider and could result in failed pages or documents if you do not have sufficient usage limits."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "dc921729-3446-42ca-8e1b-a6fd26195ed9",
"metadata": {},
"outputs": [],
"source": [
"from llama_index.core.schema import TextNode\n",
"from typing import List\n",
"\n",
"\n",
"def get_text_nodes(json_list: List[dict]):\n",
" text_nodes = []\n",
" for idx, page in enumerate(json_list):\n",
" text_node = TextNode(text=page[\"md\"], metadata={\"page\": page[\"page\"]})\n",
" text_nodes.append(text_node)\n",
" return text_nodes"
]
},
{
"cell_type": "markdown",
"id": "1b5d6da6",
"metadata": {},
"source": [
"### With anthropic-sonnet-3.5"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f2e9d9cf-8189-4fcb-b34f-cde6cc0b59c8",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Started parsing the file under job_id dd9d5e0f-160e-486a-89a2-6005e5a1c2ac\n"
]
}
],
"source": [
"from llama_cloud_services import LlamaParse\n",
"\n",
"parser = LlamaParse(\n",
" result_type=\"markdown\",\n",
" use_vendor_multimodal_model=True,\n",
" vendor_multimodal_model_name=\"anthropic-sonnet-3.5\",\n",
" target_pages=\"24\"\n",
" # invalidate_cache=True\n",
")\n",
"json_objs = parser.get_json_result(\"o1.pdf\")\n",
"json_list = json_objs[0][\"pages\"]\n",
"docs = get_text_nodes(json_list)"
]
},
{
"cell_type": "markdown",
"id": "4f3c51b0-7878-48d7-9bc3-02b516500128",
"metadata": {},
"source": [
"### With GPT-4o\n",
"\n",
"For comparison, we will also parse the document using GPT-4o."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6fc3f258-50ae-4988-b904-c105463a498f",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Started parsing the file under job_id 6a4dea44-4f90-406b-b290-9e98620b1232\n"
]
}
],
"source": [
"from llama_cloud_services import LlamaParse\n",
"\n",
"parser_gpt4o = LlamaParse(\n",
" result_type=\"markdown\",\n",
" use_vendor_multimodal_model=True,\n",
" vendor_multimodal_model=\"openai-gpt4o\",\n",
" target_pages=\"24\",\n",
" # invalidate_cache=True\n",
")\n",
"json_objs_gpt4o = parser_gpt4o.get_json_result(\"o1.pdf\")\n",
"json_list_gpt4o = json_objs_gpt4o[0][\"pages\"]\n",
"docs_gpt4o = get_text_nodes(json_list_gpt4o)"
]
},
{
"cell_type": "markdown",
"id": "44c20f7a-2901-4dd0-b635-a4b33c5664c1",
"metadata": {},
"source": [
"### View Results\n",
"\n",
"Let's visualize the results along with the original document page."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "778698aa-da7e-4081-b3b5-0372f228536f",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"page: 25\n",
"\n",
"| Participant_ID | clinical Description Reference |\n",
"|-----------------|----------------------------------|\n",
"| Attribute | Value | Basic Personal Information: Subject 098_S_0896 is a 72.0-year-old Female who has completed 15 years of education. The ethnicity is Not Hisp/Latino and race is White. Marital status is Married. Initially diagnosed as AD, as of the date 2007-10-24, the final diagnosis was Dementia. |\n",
"| Age | 72.0 |\n",
"| Sex | Female |\n",
"| Education | 15 |\n",
"| Race | White | Biomarker Measurements: The subject's genetic profile includes an ApoE4 status of 0.0... |\n",
"| DX_bl | AD |\n",
"| DX | Dementia |\n",
"| ... | ... | Cognitive and Neurofunctional Assessments: The Mini-Mental State Examination score stands at 29.0. The Clinical Dementia Rating, sum of boxes, is 1.0. ADAS 11 and 13 scores are 4.67 and 4.67 respectively, with a score of 1.0 in delayed word recall... |\n",
"| APOE4 | 1.0 |\n",
"| TAU | 212.5 |\n",
"| ... | ... |\n",
"| MMSE | 29.0 | Volumetric Data: Under MRI conditions at a field strength of 1.5 Tesla MRI Tesla, using Cross Sectional FreeSurfer (FreeSurfer Version 4.3), the imaging data recorded includes ventricles volume at 54422.0, hippocampus volume at 6677.0, whole brain volume at 1147980.0, entorhinal cortex volume at 2782.0, fusiform gyrus volume at 19432.0, and middle temporal area volume at 24951.0. The intracranial volume measured is 1799580.0.... |\n",
"| CDRSB | 0.0 |\n",
"| ... | ... |\n",
"| FLDSTRENG | 1.5 Tesla MRI |\n",
"| Ventricles | 84599 |\n",
"| Hippocampus | 5319 |\n",
"| ... | ... |\n",
"\n",
"Figure 2: An example of a patient table and its corresponding clinical description.\n",
"\n",
"skills. Mathematics, as a highly structured and logic-driven discipline, provides an ideal testing ground for evaluating this reasoning ability. To investigate o1-preview's performance, we designed a series of tests covering various difficulty levels. We begin with high school-level math competition problems in this section, followed by college-level mathematics problems in the next section, allowing us to observe the model's logical reasoning across varying levels of complexity.\n",
"\n",
"In this section, we selected two primary areas of mathematics: algebra and counting and probability in this section. We chose these two topics because of their heavy reliance on problem-solving skills and their frequent use in assessing logical and abstract thinking [46]. The dataset used in testing is from the MATH dataset [46]. The problems in the dataset cover a wide range of subjects, including Prealgebra, Intermediate Algebra, Algebra, Geometry, Counting and Probability, Number Theory, and Precalculus. Each problem is categorized based on difficulty, ranked from level 1 to 5, according to the Art of Problem Solving (AoPS). The dataset mainly comprises problems from various high school math competitions, including the American Mathematics Competitions (AMC) 10 and 12, as well as the American Invitational Mathematics Examination (AIME), and other similar contests. Each problem comes with detailed reference solutions, allowing for a comprehensive comparison of o1-preview's solutions.\n",
"\n",
"In addition to evaluating the final answers produced by o1-preview, our analysis delves into the step-by-step reasoning process of the o1-preview's solutions. By comparing o1-preview's solutions with the dataset's solutions, we assess its ability to engage in logical reasoning, handle abstract problem-solving tasks, and apply structured approaches to reach correct answers. This deeper analysis offers insights into o1-preview's overall reasoning capabilities, using mathematics as a reliable indicator for logical and structured thought processes.\n"
]
}
],
"source": [
"# using Sonnet-3.5\n",
"print(docs[0].get_content(metadata_mode=\"all\"))"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "1511a30f-3efc-4142-9668-7dc056a24d0c",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"page: 25\n",
"\n",
"\n",
"| Participant_ID | clinical Description Reference |\n",
"|----------------|--------------------------------|\n",
"| **Attribute** | **Value** |\n",
"| Age | 72.0 |\n",
"| Sex | Female |\n",
"| Education | 15 |\n",
"| Race | White |\n",
"| DX_bl | AD |\n",
"| DX | Dementia |\n",
"| ... | ... |\n",
"| APOE4 | 1.0 |\n",
"| TAU | 212.5 |\n",
"| ... | ... |\n",
"| MMSE | 29.0 |\n",
"| CDRSB | 0.0 |\n",
"| ... | ... |\n",
"| FLDSTRENG | 1.5 Tesla MRI |\n",
"| Ventricles | 84599 |\n",
"| Hippocampus | 5319 |\n",
"| ... | ... |\n",
"\n",
"**Basic Personal Information:** Subject 098_S_0896 is a 72.0-year-old Female who has completed 15 years of education. The ethnicity is Not Hisp/Latino and race is White. Marital status is Married. Initially diagnosed as AD, as of the date 2007-10-24, the final diagnosis was Dementia.\n",
"\n",
"**Biomarker Measurements:** The subject's genetic profile includes an ApoE4 status of 0.0...\n",
"\n",
"**Cognitive and Neurofunctional Assessments:** The Mini-Mental State Examination score stands at 29.0. The Clinical Dementia Rating, sum of boxes, is 1.0. ADAS 11 and 13 scores are 4.67 and 4.67 respectively, with a score of 1.0 in delayed word recall...\n",
"\n",
"**Volumetric Data:** Under MRI conditions at a field strength of 1.5 Tesla MRI Tesla, using Cross-Sectional FreeSurfer (FreeSurfer Version 4.3), the imaging data recorded includes ventricles volume at 84422.0, hippocampus volume at 6677.0, whole brain volume at 1147980.0, entorhinal cortex volume at 27820.0, fusiform gyrus volume at 19432.0, and middle temporal area volume at 24951.0. The intracranial volume measured is 1799580.0...\n",
"\n",
"Figure 2: An example of a patient table and its corresponding clinical description.\n",
"\n",
"----\n",
"\n",
"Skills. Mathematics, as a highly structured and logic-driven discipline, provides an ideal testing ground for evaluating this reasoning ability. To investigate o1-previews performance, we designed a series of tests covering various difficulty levels. We begin with high school-level math competition problems in this section, followed by college-level mathematics problems in the next section, allowing us to observe the models logical reasoning across varying levels of complexity.\n",
"\n",
"In this section, we selected two primary areas of mathematics: algebra and counting and probability in this section. We chose these two topics because of their heavy reliance on problem-solving skills and their frequent use in assessing logical and abstract thinking [46]. The dataset used in testing is from the MATH dataset [46]. The problems in the dataset cover a wide range of subjects, including Prealgebra, Intermediate Algebra, Algebra, Geometry, Counting and Probability, Number Theory, and Precalculus. Each problem is categorized based on difficulty, ranked from level 1 to 5, according to the Art of Problem Solving (AoPS). The dataset mainly comprises problems from various high school math competitions, including the American Mathematics Competitions (AMC) 10 and 12, as well as the American Invitational Mathematics Examination (AIME), and other similar contests. Each problem comes with detailed reference solutions, allowing for a comprehensive comparison of o1-previews solutions.\n",
"\n",
"In addition to evaluating the final answers produced by o1-preview, our analysis delves into the step-by-step reasoning process of the o1-previews solutions. By comparing o1-previews solutions with the datasets solutions, we assess its ability to engage in logical reasoning, handle abstract problem-solving tasks, and apply structured approaches to reach correct answers. This deeper analysis offers insights into o1-previews overall reasoning capabilities, using mathematics as a reliable indicator for logical and structured thought processes.\n"
]
}
],
"source": [
"# using GPT-4o\n",
"print(docs_gpt4o[0].get_content(metadata_mode=\"all\"))"
]
}
],
"metadata": {
"colab": {
"provenance": []
},
"kernelspec": {
"display_name": "llamacloud",
"language": "python",
"name": "llamacloud"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
@@ -0,0 +1,170 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/parse/demo_starter_parse_selected_pages.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Parse Selected Pages \n",
"\n",
"In this notebook we will demonstrate how to parse selected pages in a document using LlamaParse."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Installation\n",
"\n",
"Here we install `llama-parse` used for parsing the document"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"!pip install llama-cloud-services"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Set API Key"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# llama-parse is async-first, running the async code in a notebook requires the use of nest_asyncio\n",
"import nest_asyncio\n",
"\n",
"nest_asyncio.apply()\n",
"\n",
"import os\n",
"\n",
"# API access to llama-cloud\n",
"os.environ[\"LLAMA_CLOUD_API_KEY\"] = \"<YOUR LLAMACLOUD API KEY>\""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Download Data\n",
"\n",
"Here we download Uber 2021 10K SEC filings data for the demonstration."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"--2024-12-05 11:40:59-- https://raw.githubusercontent.com/run-llama/llama_index/main/docs/docs/examples/data/10k/uber_2021.pdf\n",
"Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 2606:50c0:8000::154, 2606:50c0:8002::154, 2606:50c0:8003::154, ...\n",
"Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|2606:50c0:8000::154|:443... connected.\n",
"HTTP request sent, awaiting response... 200 OK\n",
"Length: 1880483 (1.8M) [application/octet-stream]\n",
"Saving to: ./uber_2021.pdf\n",
"\n",
"./uber_2021.pdf 100%[===================>] 1.79M --.-KB/s in 0.1s \n",
"\n",
"2024-12-05 11:40:59 (14.2 MB/s) - ./uber_2021.pdf saved [1880483/1880483]\n",
"\n"
]
}
],
"source": [
"!wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/docs/examples/data/10k/uber_2021.pdf' -O './uber_2021.pdf'"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Parse the PDF file in selected pages\n",
"\n",
"Here we will parse the PDF file in selected pages and get the text in `markdown` format."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Started parsing the file under job_id ad1087c1-b085-4dc7-9aa8-d13cdd440f2b\n"
]
}
],
"source": [
"from llama_cloud_services import LlamaParse\n",
"\n",
"parser = LlamaParse(target_pages=\"0,1,2\", result_type=\"markdown\")\n",
"\n",
"documents = parser.load_data(\"./uber_2021.pdf\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[Document(id_='d0b34f4a-27ef-48e2-a92a-386e5e265f4c', embedding=None, metadata={}, excluded_embed_metadata_keys=[], excluded_llm_metadata_keys=[], relationships={}, metadata_template='{key}: {value}', metadata_separator='\\n', text='# UNITED STATES SECURITIES AND EXCHANGE COMMISSION\\n\\n# Washington, D.C. 20549\\n\\n# FORM 10-K\\n\\n(Mark One)\\n\\n☒ ANNUAL REPORT PURSUANT TO SECTION 13 OR 15(d) OF THE SECURITIES EXCHANGE ACT OF 1934\\n\\nFor the fiscal year ended December 31, 2021\\n\\nOR\\n\\n☐ TRANSITION REPORT PURSUANT TO SECTION 13 OR 15(d) OF THE SECURITIES EXCHANGE ACT OF 1934\\n\\nFor the transition period from _____ to _____\\n\\nCommission File Number: 001-38902\\n\\n# UBER TECHNOLOGIES, INC.\\n\\n(Exact name of registrant as specified in its charter)\\n\\nDelaware\\n\\n45-2647441\\n\\n(State or other jurisdiction of incorporation or organization) (I.R.S. Employer Identification No.)\\n\\n1515 3rd Street\\n\\nSan Francisco, California 94158\\n\\n(Address of principal executive offices, including zip code)\\n\\n(415) 612-8582\\n\\n(Registrants telephone number, including area code)\\n\\n# Securities registered pursuant to Section 12(b) of the Act:\\n\\n|Title of each class|Trading Symbol(s)|Name of each exchange on which registered|\\n|---|---|---|\\n|Common Stock, par value $0.00001 per share|UBER|New York Stock Exchange|\\n\\nSecurities registered pursuant to Section 12(g) of the Act: None\\n\\nIndicate by check mark whether the registrant is a well-known seasoned issuer, as defined in Rule 405 of the Securities Act. Yes ☒ No ☐\\n\\nIndicate by check mark whether the registrant is not required to file reports pursuant to Section 13 or Section 15(d) of the Act. Yes ☐ No ☒\\n\\nIndicate by check mark whether the registrant (1) has filed all reports required to be filed by Section 13 or 15(d) of the Securities Exchange Act of 1934 during the preceding 12 months (or for such shorter period that the registrant was required to file such reports), and (2) has been subject to such filing requirements for the past 90 days. Yes ☒ No ☐\\n\\nIndicate by check mark whether the registrant has submitted electronically every Interactive Data File required to be submitted pursuant to Rule 405 of Regulation S-T (§232.405 of this chapter) during the preceding 12 months (or for such shorter period that the registrant was required to submit such files). Yes ☒ No ☐\\n\\nIndicate by check mark whether the registrant is a large accelerated filer, an accelerated filer, a non-accelerated filer, a smaller reporting company, or an emerging growth company. See the definitions of “large accelerated filer,” “accelerated filer,” “smaller reporting company,” and “emerging growth company” in Rule 12b-2 of the Exchange Act.', mimetype='text/plain', start_char_idx=None, end_char_idx=None, metadata_seperator='\\n', text_template='{metadata_str}\\n\\n{content}'),\n",
" Document(id_='253b1141-a260-466e-b164-b39df67ef799', embedding=None, metadata={}, excluded_embed_metadata_keys=[], excluded_llm_metadata_keys=[], relationships={}, metadata_template='{key}: {value}', metadata_separator='\\n', text=\"# Large accelerated filer\\n\\n☒\\n\\n# Accelerated filer\\n\\n☐\\n\\n# Non-accelerated filer\\n\\n☐\\n\\n# Smaller reporting company\\n\\n☐\\n\\n# Emerging growth company\\n\\n☐\\n\\nIf an emerging growth company, indicate by check mark if the registrant has elected not to use the extended transition period for complying with any new or revised financial accounting standards provided pursuant to Section 13(a) of the Exchange Act.\\n\\n☐\\n\\nIndicate by check mark whether the registrant has filed a report on and attestation to its managements assessment of the effectiveness of its internal control over financial reporting under Section 404(b) of the Sarbanes-Oxley Act (15 U.S.C. 7262(b)) by the registered public accounting firm that prepared or issued\\n\\n☒\\n\\nIndicate by check mark whether the registrant is a shell company (as defined in Rule 12b-2 of the Exchange Act). Yes\\n\\n☐\\n\\nNo\\n\\n☒\\n\\nThe aggregate market value of the voting and non-voting common equity held by non-affiliates of the registrant as of June 30, 2021, the last business day of the registrant's most recently completed second fiscal quarter, was approximately $90.5 billion based upon the closing price reported for such date on the New York Stock Exchange.\\n\\nThe number of shares of the registrant's common stock outstanding as of February 22, 2022 was 1,954,464,088.\\n\\n# DOCUMENTS INCORPORATED BY REFERENCE\\n\\nPortions of the registrants Definitive Proxy Statement relating to the Annual Meeting of Stockholders are incorporated by reference into Part III of this Annual Report on Form 10-K where indicated. Such Definitive Proxy Statement will be filed with the Securities and Exchange Commission within 120 days after the end of the registrants fiscal year ended December 31, 2021.\", mimetype='text/plain', start_char_idx=None, end_char_idx=None, metadata_seperator='\\n', text_template='{metadata_str}\\n\\n{content}'),\n",
" Document(id_='ad988239-3ab5-498d-85ba-a29241db24d4', embedding=None, metadata={}, excluded_embed_metadata_keys=[], excluded_llm_metadata_keys=[], relationships={}, metadata_template='{key}: {value}', metadata_separator='\\n', text='# UBER TECHNOLOGIES, INC.\\n\\n# TABLE OF CONTENTS\\n\\n|Special Note Regarding Forward-Looking Statements|2|\\n|---|---|\\n|PART I|PART I|\\n|Item 1. Business|4|\\n|Item 1A. Risk Factors|11|\\n|Item 1B. Unresolved Staff Comments|46|\\n|Item 2. Properties|46|\\n|Item 3. Legal Proceedings|46|\\n|Item 4. Mine Safety Disclosures|47|\\n|PART II|PART II|\\n|Item 5. Market for Registrants Common Equity, Related Stockholder Matters and Issuer Purchases of Equity Securities|47|\\n|Item 6. [Reserved]|48|\\n|Item 7. Managements Discussion and Analysis of Financial Condition and Results of Operations|48|\\n|Item 7A. Quantitative and Qualitative Disclosures About Market Risk|69|\\n|Item 8. Financial Statements and Supplementary Data|70|\\n|Item 9. Changes in and Disagreements with Accountants on Accounting and Financial Disclosure|146|\\n|Item 9A. Controls and Procedures|147|\\n|Item 9B. Other Information|147|\\n|Item 9C. Disclosure Regarding Foreign Jurisdictions that Prevent Inspections|147|\\n|PART III|PART III|\\n|Item 10. Directors, Executive Officers and Corporate Governance|147|\\n|Item 11. Executive Compensation|147|\\n|Item 12. Security Ownership of Certain Beneficial Owners and Management and Related Stockholder Matters|148|\\n|Item 13. Certain Relationships and Related Transactions, and Director Independence|148|\\n|Item 14. Principal Accounting Fees and Services|148|\\n|PART IV|PART IV|\\n|Item 15. Exhibits, Financial Statement Schedules|148|\\n|Item 16. Form 10-K Summary|148|\\n|Exhibit Index|149|\\n|Signatures|152|', mimetype='text/plain', start_char_idx=None, end_char_idx=None, metadata_seperator='\\n', text_template='{metadata_str}\\n\\n{content}')]"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"documents"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "llamacloud",
"language": "python",
"name": "llamacloud"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
@@ -6,7 +6,7 @@
"source": [
"# RAG for Table Comparisons with LlamaParse + LlamaIndex\n",
"\n",
"<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/demo_table_comparisons.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
"<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/parse/demo_table_comparisons.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
"\n",
"This notebook shows you how to do comparisons across both tabular and text data across multiple PDF documents.\n",
"\n",
@@ -34,7 +34,7 @@
"%pip install llama-index-question-gen-openai\n",
"%pip install llama-index-postprocessor-flag-embedding-reranker\n",
"%pip install git+https://github.com/FlagOpen/FlagEmbedding.git\n",
"%pip install llama-parse"
"%pip install llama-cloud-services"
]
},
{
@@ -109,7 +109,7 @@
"metadata": {},
"outputs": [],
"source": [
"from llama_parse import LlamaParse\n",
"from llama_cloud_services import LlamaParse\n",
"\n",
"docs_2021 = LlamaParse(result_type=\"markdown\").load_data(\"./apple_2021_10k.pdf\")\n",
"docs_2020 = LlamaParse(result_type=\"markdown\").load_data(\"./apple_2020_10k.pdf\")"
@@ -7,7 +7,7 @@
"source": [
"# RAG with Excel Spreadsheet using LlamaPrase\n",
"\n",
"<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/demo_excel.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
"<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/parse/excel/dcf_rag.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
"\n",
"This notebook constructs a RAG pipeline over a simple DCF template [here](https://eqvista.com/app/uploads/2020/09/Eqvista_DCF-Excel-Template.xlsx).\n",
"\n"
@@ -31,7 +31,7 @@
"outputs": [],
"source": [
"%pip install llama-index\n",
"%pip install llama-parse"
"%pip install llama-cloud-services"
]
},
{
@@ -53,7 +53,7 @@
"metadata": {},
"outputs": [],
"source": [
"from llama_parse import LlamaParse\n",
"from llama_cloud_services import LlamaParse\n",
"\n",
"# api_key = \"llx-\" # get from cloud.llamaindex.ai"
]
@@ -4,7 +4,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/excel/o1_excel_rag.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
"<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/excel/o1_excel_rag.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
]
},
{
@@ -37,7 +37,7 @@
"outputs": [],
"source": [
"# !pip install llama-index\n",
"# !pip install llama-parse"
"# !pip install llama-cloud-services"
]
},
{
@@ -59,7 +59,7 @@
"from llama_index.core import VectorStoreIndex\n",
"from IPython.display import Image, Markdown\n",
"\n",
"from llama_parse import LlamaParse\n",
"from llama_cloud_services import LlamaParse\n",
"\n",
"from llama_index.core.node_parser import MarkdownElementNodeParser"
]

Before

Width:  |  Height:  |  Size: 195 KiB

After

Width:  |  Height:  |  Size: 195 KiB

Before

Width:  |  Height:  |  Size: 363 KiB

After

Width:  |  Height:  |  Size: 363 KiB

Before

Width:  |  Height:  |  Size: 343 KiB

After

Width:  |  Height:  |  Size: 343 KiB

Before

Width:  |  Height:  |  Size: 185 KiB

After

Width:  |  Height:  |  Size: 185 KiB

Before

Width:  |  Height:  |  Size: 254 KiB

After

Width:  |  Height:  |  Size: 254 KiB

Before

Width:  |  Height:  |  Size: 650 KiB

After

Width:  |  Height:  |  Size: 650 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 72 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 173 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 72 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 88 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 200 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 115 KiB

@@ -7,7 +7,7 @@
"source": [
"# Knowledge Graph Agent with LlamaParse\n",
"\n",
"<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/knowledge_graphs/kg_agent.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
"<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/parse/knowledge_graphs/kg_agent.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
"\n",
"Here we build a knowledge graph agent over the SF 2023 Budget Proposal. We use LlamaIndex abstractions to construct a knowledge graph, and we store the property graph in neo4j. We then build an agent that can interact with the knowledge graph as a tool."
]
@@ -33,7 +33,7 @@
"!pip install llama-index-postprocessor-flag-embedding-reranker\n",
"!pip install git+https://github.com/FlagOpen/FlagEmbedding.git\n",
"!pip install llama-index-graph-stores-neo4j\n",
"!pip install llama-parse"
"!pip install llama-cloud-services"
]
},
{
@@ -125,7 +125,7 @@
}
],
"source": [
"from llama_parse import LlamaParse\n",
"from llama_cloud_services import LlamaParse\n",
"\n",
"docs = LlamaParse(result_type=\"text\").load_data(\"./data/budget_2023.pdf\")"
]

Before

Width:  |  Height:  |  Size: 334 KiB

After

Width:  |  Height:  |  Size: 334 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 202 KiB

@@ -7,7 +7,7 @@
"source": [
"# Multimodal Parsing using Anthropic Claude (Sonnet 3.5)\n",
"\n",
"<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/multimodal/claude_parse.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
"<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/parse/multimodal/claude_parse.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
"\n",
"This cookbook shows you how to use LlamaParse to parse any document with the multimodal capabilities of Sonnet 3.5. \n",
"\n",
@@ -141,7 +141,7 @@
}
],
"source": [
"from llama_parse import LlamaParse\n",
"from llama_cloud_services import LlamaParse\n",
"\n",
"parser = LlamaParse(\n",
" result_type=\"markdown\",\n",
@@ -205,7 +205,7 @@
}
],
"source": [
"from llama_parse import LlamaParse\n",
"from llama_cloud_services import LlamaParse\n",
"\n",
"parser_gpt4o = LlamaParse(\n",
" result_type=\"markdown\",\n",
@@ -0,0 +1,633 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "97c79c38-38a3-40f3-ba2e-250649347d63",
"metadata": {},
"source": [
"# Multimodal Parsing with Gemini 2.0 Flash\n",
"\n",
"<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/parse/multimodal/gemini2_flash.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
"\n",
"This cookbook shows you how to use LlamaParse to parse any document with the multimodal capabilities of Gemini 2.0 Flash.\n",
"\n",
"LlamaParse allows you to plug in external, multimodal model vendors for parsing - we handle the error correction, validation, and scalability/reliability for you.\n"
]
},
{
"cell_type": "markdown",
"id": "15e60ecf-519c-41fc-911b-765adaf8bad4",
"metadata": {},
"source": [
"## Setup\n",
"\n",
"Download the data - we'll use a technical datasheet for a programmable logic device (Xilinx's XC9500 In-System Programmable CPLD)."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "91a9e532-1454-40e0-bbf0-fd442c350121",
"metadata": {},
"outputs": [],
"source": [
"import nest_asyncio\n",
"\n",
"nest_asyncio.apply()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0d9fb0aa-74cd-476f-8161-efd9e04248bf",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"--2025-02-06 20:24:19-- https://media.digikey.com/pdf/Data%20Sheets/AMD/XC9500_CPLD_Family.pdf\n",
"Resolving media.digikey.com (media.digikey.com)... 23.37.18.160\n",
"Connecting to media.digikey.com (media.digikey.com)|23.37.18.160|:443... connected.\n",
"HTTP request sent, awaiting response... 200 OK\n",
"Length: 201899 (197K) [application/pdf]\n",
"Saving to: data/XC9500_CPLD_Family.pdf\n",
"\n",
"data/XC9500_CPLD_Fa 100%[===================>] 197.17K --.-KB/s in 0.03s \n",
"\n",
"2025-02-06 20:24:19 (7.67 MB/s) - data/XC9500_CPLD_Family.pdf saved [201899/201899]\n",
"\n"
]
}
],
"source": [
"!wget \"https://media.digikey.com/pdf/Data%20Sheets/AMD/XC9500_CPLD_Family.pdf\" -O data/XC9500_CPLD_Family.pdf"
]
},
{
"cell_type": "markdown",
"id": "4e29a9d7-5bd9-4fb8-8ec1-4c128a748662",
"metadata": {},
"source": [
"## Initialize LlamaParse\n",
"\n",
"Initialize LlamaParse in multimodal mode, and specify the vendor as `gemini-2.0-flash-001`.\n",
"\n",
"**NOTE**: Current pricing is 2 credits for a 1 page ($0.006 USD / page). This includes core model, infra, and algorithm costs to fully process the page. "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "dc921729-3446-42ca-8e1b-a6fd26195ed9",
"metadata": {},
"outputs": [],
"source": [
"from llama_index.core.schema import TextNode\n",
"from typing import List\n",
"import json\n",
"\n",
"\n",
"def get_text_nodes(json_list: List[dict]):\n",
" text_nodes = []\n",
" for idx, page in enumerate(json_list):\n",
" text_node = TextNode(text=page[\"md\"], metadata={\"page\": page[\"page\"]})\n",
" text_nodes.append(text_node)\n",
" return text_nodes\n",
"\n",
"\n",
"def save_jsonl(data_list, filename):\n",
" \"\"\"Save a list of dictionaries as JSON Lines.\"\"\"\n",
" with open(filename, \"w\") as file:\n",
" for item in data_list:\n",
" json.dump(item, file)\n",
" file.write(\"\\n\")\n",
"\n",
"\n",
"def load_jsonl(filename):\n",
" \"\"\"Load a list of dictionaries from JSON Lines.\"\"\"\n",
" data_list = []\n",
" with open(filename, \"r\") as file:\n",
" for line in file:\n",
" data_list.append(json.loads(line))\n",
" return data_list"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f2e9d9cf-8189-4fcb-b34f-cde6cc0b59c8",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Started parsing the file under job_id 51538aa0-13e6-4429-a458-a492ba7eec04\n"
]
}
],
"source": [
"from llama_parse import LlamaParse\n",
"\n",
"parsing_instruction = \"\"\"\n",
"You are given a technical datasheet of an electronic component.\n",
"For any graphs, try to create a 2D table of relevant values, along with a description of the graph.\n",
"For any schematic diagrams, MAKE SURE to describe a list of all components and their connections to each other.\n",
"Make sure that you always parse out the text with the correct reading order.\n",
"\"\"\"\n",
"\n",
"parser = LlamaParse(\n",
" result_type=\"markdown\",\n",
" use_vendor_multimodal_model=True,\n",
" vendor_multimodal_model_name=\"gemini-2.0-flash-001\",\n",
" invalidate_cache=True,\n",
" parsing_instruction=parsing_instruction,\n",
")\n",
"json_objs = parser.get_json_result(\"./data/XC9500_CPLD_Family.pdf\")\n",
"json_list = json_objs[0][\"pages\"]\n",
"docs = get_text_nodes(json_list)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "96a81df0-1026-4e30-a930-f677dc31e344",
"metadata": {},
"outputs": [],
"source": [
"# Optional: Save\n",
"save_jsonl([d.dict() for d in docs], \"docs_gemini_2.0_flash.jsonl\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ee2e6920-8893-4b39-ae12-94d13c651406",
"metadata": {},
"outputs": [],
"source": [
"# Optional: Load\n",
"from llama_index.core import Document\n",
"\n",
"docs_dicts = load_jsonl(\"docs_gemini_2.0_flash.jsonl\")\n",
"docs = [Document.parse_obj(d) for d in docs_dicts]"
]
},
{
"cell_type": "markdown",
"id": "4f3c51b0-7878-48d7-9bc3-02b516500128",
"metadata": {},
"source": [
"### Setup GPT-4o baseline\n",
"\n",
"For comparison, we will also parse the document using GPT-4o ($0.03 per page)."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6fc3f258-50ae-4988-b904-c105463a498f",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Started parsing the file under job_id 23c6627c-2e3d-46c9-88a0-7945d7e65d96\n"
]
}
],
"source": [
"from llama_parse import LlamaParse\n",
"\n",
"parser_gpt4o = LlamaParse(\n",
" result_type=\"markdown\",\n",
" use_vendor_multimodal_model=True,\n",
" vendor_multimodal_model=\"openai-gpt4o\",\n",
" invalidate_cache=True,\n",
" parsing_instruction=parsing_instruction,\n",
")\n",
"json_objs_gpt4o = parser_gpt4o.get_json_result(\"./data/XC9500_CPLD_Family.pdf\")\n",
"json_list_gpt4o = json_objs_gpt4o[0][\"pages\"]\n",
"docs_gpt4o = get_text_nodes(json_list_gpt4o)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6a47f04e-12e1-4c80-a71d-ef7721f96401",
"metadata": {},
"outputs": [],
"source": [
"# Optional: Save\n",
"save_jsonl([d.dict() for d in docs_gpt4o], \"docs_gpt4o.jsonl\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c38b5ca3-fa87-434b-b477-bf6a4962eb3d",
"metadata": {},
"outputs": [],
"source": [
"# Optional: Load\n",
"from llama_index.core import Document\n",
"\n",
"docs_gpt4o_dicts = load_jsonl(\"docs_gpt4o.jsonl\")\n",
"docs_gpt4o = [Document.parse_obj(d) for d in docs_gpt4o_dicts]"
]
},
{
"cell_type": "markdown",
"id": "44c20f7a-2901-4dd0-b635-a4b33c5664c1",
"metadata": {},
"source": [
"## View Results\n",
"\n",
"Let's visualize the results between GPT-4o and Gemini Flash 2.0 along with the original document page."
]
},
{
"cell_type": "markdown",
"id": "bf314141-9f6d-4453-beb9-0106cdf196bf",
"metadata": {},
"source": [
"Check out an example page 2 below."
]
},
{
"cell_type": "markdown",
"id": "c70d420d-1778-4b0d-81e2-db09276e90cf",
"metadata": {},
"source": [
"![xc9500_img](XC9500_CPLD_Family_p3.png)"
]
},
{
"cell_type": "markdown",
"id": "0950ecad-248c-4c3c-98b9-ab1a9dabd5b4",
"metadata": {},
"source": [
"We see that the parsed text is fairly similar between Gemini 2.0 Flash and GPT-4o. "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "778698aa-da7e-4081-b3b5-0372f228536f",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"page: 3\n",
"\n",
"The image shows the architecture of the XC9500 In-System Programmable CPLD Family, which is marked as obsolete. Here's a breakdown of the components and their connections:\n",
"\n",
"### Components and Connections:\n",
"\n",
"1. **JTAG Port:**\n",
" - Connects to the JTAG Controller.\n",
"\n",
"2. **JTAG Controller:**\n",
" - Interfaces with the In-System Programming Controller.\n",
" - Connects to the I/O Blocks.\n",
"\n",
"3. **In-System Programming Controller:**\n",
" - Interfaces with the JTAG Controller and the Fast CONNECT Switch Matrix.\n",
"\n",
"4. **I/O Blocks:**\n",
" - Multiple I/O lines connect to the Fast CONNECT Switch Matrix.\n",
" - Includes special I/O lines for GCK, GSR, and GTS.\n",
"\n",
"5. **Fast CONNECT Switch Matrix:**\n",
" - Connects to the I/O Blocks and Function Blocks.\n",
" - Provides 36 inputs and 18 outputs to each Function Block.\n",
"\n",
"6. **Function Blocks (FB):**\n",
" - Each block contains 18 macrocells.\n",
" - Outputs from the Function Blocks drive the I/O Blocks directly.\n",
" - Multiple Function Blocks (1 to N) are shown, each with 18 macrocells.\n",
"\n",
"### Function Block Details:\n",
"\n",
"- Each Function Block consists of 18 independent macrocells.\n",
"- Capable of implementing combinatorial or registered functions.\n",
"- Receives global clock, output enable, and set/reset signals.\n",
"- Generates 18 outputs for the Fast CONNECT switch matrix.\n",
"- Logic is implemented using a sum-of-products representation.\n",
"- 36 inputs provide 72 true and complement signals to form 90 product terms.\n",
"- Product terms can be allocated to each macrocell by the product term allocator.\n",
"- Supports local feedback paths for fast counters and state machines.\n",
"\n",
"This architecture is designed for flexibility in implementing complex logic functions within a programmable logic device.\n"
]
}
],
"source": [
"# using Gemini 2.0 Flash\n",
"print(docs[2].get_content(metadata_mode=\"all\"))"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "1511a30f-3efc-4142-9668-7dc056a24d0c",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"page: 3\n",
"\n",
"The diagram illustrates the architecture of the XC9500 In-System Programmable CPLD Family. Here's a breakdown of the components and their connections:\n",
"\n",
"1. **JTAG Port**: \n",
" - Connects to the JTAG Controller.\n",
"\n",
"2. **JTAG Controller**: \n",
" - Interfaces with the In-System Programming Controller.\n",
"\n",
"3. **In-System Programming Controller**: \n",
" - Manages programming of the device.\n",
"\n",
"4. **I/O Blocks**: \n",
" - Connect to external I/O pins.\n",
" - Interface with the Fast CONNECT Switch Matrix.\n",
"\n",
"5. **Fast CONNECT Switch Matrix**: \n",
" - Connects I/O Blocks to Function Blocks.\n",
" - Provides 36 inputs and 18 outputs to each Function Block.\n",
"\n",
"6. **Function Blocks (FB)**: \n",
" - Each block contains 18 macrocells.\n",
" - Capable of implementing combinatorial or registered functions.\n",
" - Receives global clock, output enable, and set/reset signals.\n",
" - Outputs drive the Fast CONNECT Switch Matrix.\n",
" - Supports local feedback paths for fast counters and state machines.\n",
"\n",
"7. **I/O/GCK, I/O/GSR, I/O/GTS**: \n",
" - Special I/O pins for global clock, set/reset, and output enable signals.\n",
"\n",
"The architecture is designed for flexibility and high-speed operation, with each Function Block capable of handling complex logic functions.\n"
]
}
],
"source": [
"# using GPT-4o\n",
"print(docs_gpt4o[2].get_content(metadata_mode=\"all\"))"
]
},
{
"cell_type": "markdown",
"id": "705f7729-fa0f-4ca0-8562-c42afeaa8532",
"metadata": {},
"source": [
"## Setup RAG Pipeline\n",
"\n",
"Let's setup a RAG pipeline over this data.\n",
"\n",
"(we also use gpt4o-mini for the actual text synthesis step)."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5a53ee5d-cc63-421b-8896-588c83edfcf0",
"metadata": {},
"outputs": [],
"source": [
"from llama_index.core import Settings\n",
"from llama_index.llms.openai import OpenAI\n",
"from llama_index.embeddings.openai import OpenAIEmbedding\n",
"\n",
"Settings.llm = OpenAI(model=\"o3-mini\")\n",
"Settings.embed_model = OpenAIEmbedding(model=\"text-embedding-3-large\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "60972d7a-7948-4ad7-89df-57004acee917",
"metadata": {},
"outputs": [],
"source": [
"# from llama_index.core import SummaryIndex\n",
"from llama_index.core import VectorStoreIndex\n",
"from llama_index.llms.openai import OpenAI\n",
"\n",
"index = VectorStoreIndex(docs)\n",
"query_engine = index.as_query_engine(similarity_top_k=5)\n",
"\n",
"index_gpt4o = VectorStoreIndex(docs_gpt4o)\n",
"query_engine_gpt4o = index_gpt4o.as_query_engine(similarity_top_k=5)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e7df7bcb-1df4-4a01-88fc-2d596b1cc74d",
"metadata": {},
"outputs": [],
"source": [
"query = \"Give me the full output slew-Rate curve for (a) Rising and (b) Falling Outputs\"\n",
"\n",
"response = query_engine.query(query)\n",
"response_gpt4o = query_engine_gpt4o.query(query)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b7070a31-3bb8-4134-8338-20bc2fd6f3d6",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"The full output slew-rate curve for (a) Rising and (b) Falling Outputs is represented in a graph where the output voltage starts at 1.5V and reaches the desired output level over a time period defined as T<sub>SLEW</sub>. The curve illustrates the gradual increase in voltage for rising outputs and the gradual decrease for falling outputs, effectively showing how the output edge rates can be controlled to reduce system noise.\n"
]
}
],
"source": [
"print(response)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7bee8167-f021-4c87-8d28-9f40a4f7b69d",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"# XC9500 In-System Programmable CPLD Family\n",
"\n",
"Each output has independent slew rate control. Output edge rates may be slowed down to reduce system noise (with an additional time delay of T<sub>SLEW</sub>) through programming. See Figure 11.\n",
"\n",
"Each IOB provides user programmable ground pin capability. This allows device I/O pins to be configured as additional ground pins. By tying strategically located programmable ground pins to the external ground connection, system noise generated from large numbers of simultaneous switching outputs may be reduced.\n",
"\n",
"A control pull-up resistor (typically 10K ohms) is attached to each device I/O pin to prevent them from floating when the device is not in normal user operation. This resistor is active during device programming mode and system power-up. It is also activated for an erased device. The resistor is deactivated during normal operation.\n",
"\n",
"The output driver is capable of supplying 24 mA output drive. All output drivers in the device may be configured for either 5V TTL levels or 3.3V levels by connecting the device output voltage supply (V<sub>CCIO</sub>) to a 5V or 3.3V voltage supply. Figure 12 shows how the XC9500 device can be used in 5V only and mixed 3.3V/5V systems.\n",
"\n",
"## Pin-Locking Capability\n",
"\n",
"The capability to lock the user defined pin assignments during design changes depends on the ability of the architecture to adapt to unexpected changes. The XC9500 devices have architectural features that enhance the ability to accept design changes while maintaining the same pinout.\n",
"\n",
"The XC9500 architecture provides maximum routing within the Fast CONNECT switch matrix, and incorporates a flexible Function Block that allows block-wide allocation of available product terms. This provides a high level of confidence of maintaining both input and output pin assignments for unexpected design changes.\n",
"\n",
"For extensive design changes requiring higher logic capacity than is available in the initially chosen device, the new design may be able to fit into a larger pin-compatible device using the same pin assignments. The same board may be used with a higher density device without the expense of board rework.\n",
"\n",
"!Output slew-Rate for (a) Rising and (b) Falling Outputs\n",
"\n",
"**Figure 11:** Output slew-Rate for (a) Rising and (b) Falling Outputs\n",
"\n",
"| Output Voltage | Time |\n",
"|----------------|------|\n",
"| 1.5V | 0 |\n",
"| T<sub>SLEW</sub> | |\n",
"\n",
"**Figure 12:** XC9500 Devices in (a) 5V Systems and (b) Mixed 5V/3.3V Systems\n",
"\n",
"| 5V CMOS or 5V TTL | 3.3V |\n",
"|-------------------|------|\n",
"| 5V | 0V |\n",
"| 3.6V | 0V |\n",
"| 3.3V | 0V |\n",
"\n",
"- **(a) 5V System:**\n",
" - V<sub>CCINT</sub> V<sub>CCIO</sub>\n",
" - XC9500 CPLD\n",
" - IN OUT\n",
" - GND\n",
"\n",
"- **(b) Mixed 5V/3.3V System:**\n",
" - V<sub>CCINT</sub> V<sub>CCIO</sub>\n",
" - XC9500 CPLD\n",
" - IN OUT\n",
" - GND\n",
"\n",
"www.xilinx.com\n",
"\n",
"DS063 (v6.0) May 17, 2013 \n",
"Product Specification\n"
]
}
],
"source": [
"print(response.source_nodes[0].get_content())"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5f9fef7f-510b-46a5-8716-f5616f542035",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"The output slew-rate curve for (a) Rising and (b) Falling Outputs is represented in a timing diagram where the output voltage transitions from a low state to a high state and vice versa. \n",
"\n",
"For the rising output, the curve starts at 1.5V and transitions to the desired output voltage level over a time period defined as T<sub>SLEW</sub>. \n",
"\n",
"For the falling output, the curve similarly begins at the high output voltage and decreases to a low state, also taking the time defined as T<sub>SLEW</sub> to complete the transition.\n",
"\n",
"The specific values and graphical representation would typically be illustrated in a figure, but the key takeaway is that the output slew rate can be controlled to manage system noise by programming the desired T<sub>SLEW</sub> time.\n"
]
}
],
"source": [
"print(response_gpt4o)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d40f9dd4-2dd4-4fa5-b636-1f901dc1601b",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"# XC9500 In-System Programmable CPLD Family\n",
"\n",
"Each output has independent slew rate control. Output edge rates may be slowed down to reduce system noise (with an additional time delay of T<sub>SLEW</sub>) through programming. See Figure 11.\n",
"\n",
"Each IOB provides user programmable ground pin capability. This allows device I/O pins to be configured as additional ground pins. By tying strategically located programmable ground pins to the external ground connection, system noise generated from large numbers of simultaneous switching outputs may be reduced.\n",
"\n",
"A control pull-up resistor (typically 10K ohms) is attached to each device I/O pin to prevent them from floating when the device is not in normal user operation. This resistor is active during device programming mode and system power-up. It is also activated for an erased device. The resistor is deactivated during normal operation.\n",
"\n",
"The output driver is capable of supplying 24 mA output drive. All output drivers in the device may be configured for either 5V TTL levels or 3.3V levels by connecting the device output voltage supply (V<sub>CCIO</sub>) to a 5V or 3.3V voltage supply. Figure 12 shows how the XC9500 device can be used in 5V only and mixed 3.3V/5V systems.\n",
"\n",
"## Pin-Locking Capability\n",
"\n",
"The capability to lock the user defined pin assignments during design changes depends on the ability of the architecture to adapt to unexpected changes. The XC9500 devices have architectural features that enhance the ability to accept design changes while maintaining the same pinout.\n",
"\n",
"The XC9500 architecture provides maximum routing within the Fast CONNECT switch matrix, and incorporates a flexible Function Block that allows block-wide allocation of available product terms. This provides a high level of confidence of maintaining both input and output pin assignments for unexpected design changes.\n",
"\n",
"For extensive design changes requiring higher logic capacity than is available in the initially chosen device, the new design may be able to fit into a larger pin-compatible device using the same pin assignments. The same board may be used with a higher density device without the expense of board rework.\n",
"\n",
"!Output slew-Rate for (a) Rising and (b) Falling Outputs\n",
"\n",
"**Figure 11:** Output slew-Rate for (a) Rising and (b) Falling Outputs\n",
"\n",
"| Output Voltage | Time |\n",
"|----------------|------|\n",
"| 1.5V | 0 |\n",
"| T<sub>SLEW</sub> | |\n",
"\n",
"**Figure 12:** XC9500 Devices in (a) 5V Systems and (b) Mixed 5V/3.3V Systems\n",
"\n",
"| 5V CMOS or 5V TTL | 3.3V |\n",
"|-------------------|------|\n",
"| 5V | 0V |\n",
"| 3.6V | 0V |\n",
"| 3.3V | 0V |\n",
"\n",
"- **XC9500 CPLD** \n",
" - **IN** \n",
" - **OUT** \n",
" - **GND** \n",
"\n",
"www.xilinx.com \n",
"DS063 (v6.0) May 17, 2013 \n",
"Product Specification\n"
]
}
],
"source": [
"print(response_gpt4o.source_nodes[0].get_content())"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "llama_parse",
"language": "python",
"name": "llama_parse"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
@@ -7,7 +7,7 @@
"source": [
"# Multimodal Parsing using GPT4o-mini\n",
"\n",
"<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/multimodal/gpt4o_mini.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
"<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/parse/multimodal/gpt4o_mini.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
"\n",
"This cookbook shows you how to use LlamaParse to parse any document with the multimodal capabilities of GPT4o-mini.\n",
"\n",
@@ -118,7 +118,7 @@
}
],
"source": [
"from llama_parse import LlamaParse\n",
"from llama_cloud_services import LlamaParse\n",
"\n",
"parser = LlamaParse(\n",
" result_type=\"markdown\",\n",
@@ -181,7 +181,7 @@
}
],
"source": [
"from llama_parse import LlamaParse\n",
"from llama_cloud_services import LlamaParse\n",
"\n",
"parser_gpt4o = LlamaParse(\n",
" result_type=\"markdown\",\n",
@@ -6,7 +6,7 @@
"source": [
"# Building a Multimodal RAG Pipeline over an Auto Insurance Claim\n",
"\n",
"<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/multimodal/insurance_rag.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
"<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/parse/multimodal/insurance_rag.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
]
},
{
@@ -99,7 +99,7 @@
"metadata": {},
"outputs": [],
"source": [
"from llama_parse import LlamaParse\n",
"from llama_cloud_services import LlamaParse\n",
"\n",
"parser = LlamaParse(\n",
" result_type=\"markdown\",\n",
@@ -6,7 +6,7 @@
"source": [
"# Building a RAG Pipeline over Legal Documents\n",
"\n",
"<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/multimodal/legal_rag.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
"<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/parse/multimodal/legal_rag.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
"\n",
"This example shows how LlamaParse and LlamaIndex can be used to parse various types of legal documents, which may contain complex tabular data. The advantage of this is being able to quickly retrieve a specific answer to a legal question with comprehensive context — knowledge of precedents, statutes, and cases presented in the given documents. A user can quickly find the answer to or find out more details about a specific legal question without having to read through the often long documents by using LLMs.\n",
"\n",
@@ -102,7 +102,7 @@
"metadata": {},
"outputs": [],
"source": [
"from llama_parse import LlamaParse\n",
"from llama_cloud_services import LlamaParse\n",
"\n",
"parser = LlamaParse(\n",
" result_type=\"markdown\",\n",

Before

Width:  |  Height:  |  Size: 1.2 MiB

After

Width:  |  Height:  |  Size: 1.2 MiB

Before

Width:  |  Height:  |  Size: 170 KiB

After

Width:  |  Height:  |  Size: 170 KiB

@@ -7,7 +7,7 @@
"source": [
"# Contextual Retrieval for Multimodal RAG\n",
"\n",
"<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/multimodal/multimodal_contextual_retrieval_rag.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
"<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/parse/multimodal/multimodal_contextual_retrieval_rag.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
"\n",
"In this cookbook we show you how to build a multimodal RAG pipeline with **contextual retrieval**.\n",
"\n",
@@ -169,7 +169,7 @@
"metadata": {},
"outputs": [],
"source": [
"from llama_parse import LlamaParse\n",
"from llama_cloud_services import LlamaParse\n",
"\n",
"\n",
"parser = LlamaParse(\n",

Before

Width:  |  Height:  |  Size: 580 KiB

After

Width:  |  Height:  |  Size: 580 KiB

@@ -7,7 +7,7 @@
"source": [
"# Building a Natively Multimodal RAG Pipeline (over a Slide Deck)\n",
"\n",
"<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/multimodal/multimodal_rag_slide_deck.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
"<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/parse/multimodal/multimodal_rag_slide_deck.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
"\n",
"In this cookbook we show you how to build a multimodal RAG pipeline over a slide deck, with text, tables, images, diagrams, and complex layouts.\n",
"\n",
@@ -153,7 +153,7 @@
"metadata": {},
"outputs": [],
"source": [
"from llama_parse import LlamaParse\n",
"from llama_cloud_services import LlamaParse\n",
"\n",
"\n",
"parser_text = LlamaParse(result_type=\"text\")\n",

Before

Width:  |  Height:  |  Size: 271 KiB

After

Width:  |  Height:  |  Size: 271 KiB

@@ -7,7 +7,7 @@
"source": [
"# Multimodal Report Generation (from a Slide Deck)\n",
"\n",
"<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/multimodal/multimodal_report_generation.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
"<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/parse/multimodal/multimodal_report_generation.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
"\n",
"In this cookbook we show you how to build a multimodal report generator. The pipeline parses a slide deck and stores both text and image chunks. It generates a detailed response that contains interleaving text and images.\n",
"\n",
@@ -143,7 +143,7 @@
"metadata": {},
"outputs": [],
"source": [
"from llama_parse import LlamaParse\n",
"from llama_cloud_services import LlamaParse\n",
"\n",
"parser = LlamaParse(\n",
" result_type=\"markdown\",\n",
@@ -7,10 +7,12 @@
"source": [
"# Multimodal Report Generation Agent \n",
"\n",
"<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/multimodal/multimodal_report_generation_agent.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
"<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/parse/multimodal/multimodal_report_generation_agent.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
"\n",
"In this cookbook we show you how to build a multimodal report generation agent from a bank of research reports. We use the a set of ICLR papers (which were also used as the dataset in our [DeepLearning.ai course](https://www.deeplearning.ai/short-courses/building-agentic-rag-with-llamaindex/?utm_campaign=llamaindexC2-launch&utm_medium=headband&utm_source=dlai-homepage).\n",
"\n",
"![](multimodal_report_generation_agent_img.png)\n",
"\n",
"We use our workflow abstraction to define an agentic system that contains two main phases: a research phase that pulls in relevant files through chunk-level or file-level retrieval, and then a blog generation phase that synthesizes the final report."
]
},
@@ -170,7 +172,7 @@
"metadata": {},
"outputs": [],
"source": [
"from llama_parse import LlamaParse\n",
"from llama_cloud_services import LlamaParse\n",
"\n",
"parser = LlamaParse(\n",
" result_type=\"markdown\",\n",
Binary file not shown.

After

Width:  |  Height:  |  Size: 1.5 MiB

@@ -6,7 +6,7 @@
"source": [
"# Building a RAG Pipeline over IKEA Product Instruction Manuals\n",
"\n",
"<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/multimodal/product_manual_rag.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
"<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/parse/multimodal/product_manual_rag.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
]
},
{
@@ -104,7 +104,7 @@
"metadata": {},
"outputs": [],
"source": [
"from llama_parse import LlamaParse\n",
"from llama_cloud_services import LlamaParse\n",
"\n",
"parser = LlamaParse(\n",
" result_type=\"markdown\",\n",
@@ -25,7 +25,7 @@
"\n",
"nest_asyncio.apply()\n",
"\n",
"from llama_parse import LlamaParse"
"from llama_cloud_services import LlamaParse"
]
},
{
@@ -27,7 +27,7 @@
"outputs": [],
"source": [
"%pip install llama-index\n",
"%pip install llama-parse\n",
"%pip install llama-cloud-services\n",
"%pip install torch transformers python-pptx Pillow"
]
},
@@ -85,7 +85,7 @@
"metadata": {},
"outputs": [],
"source": [
"from llama_parse import LlamaParse"
"from llama_cloud_services import LlamaParse"
]
},
{
Binary file not shown.

After

Width:  |  Height:  |  Size: 350 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 47 KiB

@@ -0,0 +1,602 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/parse/parsing_instructions.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
"\n",
"# Parsing documents with Instructions\n",
"\n",
"Parsing instructions allow you to guide our parsing model in the same way you would instruct an LLM.\n",
"\n",
"These instructions can be useful for improving the parser's performance on complex document layouts, extracting data in a specific format, or transforming the document in other ways.\n",
"\n",
"### Why This Matters:\n",
"Traditional document parsing can be rigid and error-prone, often missing crucial context and nuances in complex layouts. Our instruction-based parsing allows you to:\n",
"\n",
"1. Extract specific information with pinpoint accuracy\n",
"2. Handle complex document layouts with ease\n",
"3. Transform unstructured data into structured formats effortlessly\n",
"4. Save hours of manual data entry and verification\n",
"5. Reduce errors in document processing workflows\n",
"\n",
"In this demonstration, we showcase how parsing instructions can be used to extract specific information from unstructured documents. Below are the documents we use for testing:\n",
"\n",
"1. McDonald's Receipt - Extracting the price of each order and the final amount to be paid.\n",
"\n",
"2. Expense Report Document - Extracting employee name, employee ID, position, department, date ranges, individual expense items with dates, categories, and amounts.\n",
"\n",
"3. Purchase Order Document - Identifying the PO number, vendor details, shipping terms, and an itemized list of products with quantities and unit prices.\n",
"\n",
"Let's jump into these real-world examples and see how parsing instructions can help us extract specific information."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Installation"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"!pip install llama-cloud-services"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Setup API Key"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import nest_asyncio\n",
"\n",
"nest_asyncio.apply()\n",
"\n",
"import os\n",
"\n",
"os.environ[\"LLAMA_CLOUD_API_KEY\"] = \"llx-...\""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### McDonald's Receipt\n",
"\n",
"Here we extract the price of each order and the final amount to be paid."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<img src=\"mcdonalds_receipt.png\" alt=\"Alt Text\" width=\"500\">"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Started parsing the file under job_id 66643b81-e2f4-408b-890b-8e116472210b\n"
]
}
],
"source": [
"from llama_cloud_services import LlamaParse\n",
"\n",
"vanilaParsing = LlamaParse(result_type=\"markdown\").load_data(\"./mcdonalds_receipt.png\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"# Rate us HIGHLY SATISFIED\n",
"\n",
"Purchase any sandwich and receive a FREE ITEM\n",
"\n",
"Go to WWW.mcdvoice.com within 7 days of purchase of equal or lesser value and tell us about your visit.\n",
"\n",
"Validation Code: 31278-01121-21018-20481-00081-0\n",
"\n",
"Valid at participating US McDonald's\n",
"\n",
"Expires 30 days after receipt date\n",
"\n",
"# McDonald's Restaurant #312782378\n",
"\n",
"PINE RD NW\n",
"\n",
"RICE MN 56367-9740\n",
"\n",
"TEL# 320 393 4600\n",
"\n",
"KS# 12/08/2022 08:48 PM\n",
"\n",
"# Order\n",
"\n",
"|Happy Meal 6 Pc|$4.89|\n",
"|---|---|\n",
"|Creamy Ranch Cup| |\n",
"|Extra Kids Fry| |\n",
"|Wreck It Ralph 2 Snack| |\n",
"|Oreo McFlurry|$2.69|\n",
"\n",
"# Summary\n",
"\n",
"|Subtotal|$7.58|\n",
"|---|---|\n",
"|Tax|$0.52|\n",
"|Take-Out Total|$8.10|\n",
"|Cash Tendered|$10.00|\n",
"|Change|$1.90|\n",
"\n",
"### Not ACCEPTING APPLICATIONS *++ McDonald's Restaurant Rice\n",
"\n",
"Text to #36453 apply 31278\n"
]
}
],
"source": [
"print(vanilaParsing[0].text)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Started parsing the file under job_id 1a04fdbb-5415-4a36-a1bd-26bfb5d618fa\n"
]
}
],
"source": [
"parsingInstruction = \"\"\"The provided document is a McDonald's receipt.\n",
" Provide the price of each order and final amount to be paid.\"\"\"\n",
"withInstructionParsing = LlamaParse(\n",
" result_type=\"markdown\", parsing_instruction=parsingInstruction\n",
").load_data(\"./mcdonalds_receipt.png\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Here are the prices for each order from the McDonald's receipt:\n",
"\n",
"1. Happy Meal 6 Pc: $4.89\n",
"2. Snack Oreo McFlurry: $2.69\n",
"\n",
"**Subtotal:** $7.58\n",
"**Tax:** $0.52\n",
"**Total Amount to be Paid:** $8.10\n",
"\n",
"The cash tendered was $10.00, and the change given was $1.90.\n"
]
}
],
"source": [
"print(withInstructionParsing[0].text)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Expense Report Document\n",
"\n",
"Here we extract employee name, employee ID, position, department, date ranges, individual expense items with dates, categories, and amounts."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<img src=\"expense_report_document.png\" alt=\"Alt Text\" width=\"500\">"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Started parsing the file under job_id b6bcc6e1-7d30-4522-9abd-ace196781a70\n"
]
}
],
"source": [
"vanilaParsing = LlamaParse(result_type=\"markdown\").load_data(\n",
" \"./expense_report_document.pdf\"\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"# QUANTUM DYNAMICS CORPORATION\n",
"\n",
"# EMPLOYEE EXPENSE REPORT\n",
"\n",
"# FISCAL YEAR 2024\n",
"\n",
"# EMPLOYEE INFORMATION:\n",
"\n",
"Name: Dr. Alexandra Chen-Martinez, PhD\n",
"\n",
"Employee ID: QD-2022-1457\n",
"\n",
"Department: Advanced Research & Development\n",
"\n",
"Cost Center: CC-ARD-NA-003\n",
"\n",
"Project Codes: QD-QUANTUM-2024-01, QD-AI-2024-03\n",
"\n",
"Position: Principal Research Scientist\n",
"\n",
"Reporting Manager: Dr. James Thompson\n",
"\n",
"# TRIP/EXPENSE PERIOD:\n",
"\n",
"Start Date: November 15, 2024\n",
"\n",
"End Date: December 10, 2024\n",
"\n",
"Purpose: International Conference Attendance & Client Meetings\n",
"\n",
"Locations: Tokyo, Japan → Singapore → Sydney, Australia\n",
"\n",
"# CURRENCY CONVERSION RATES APPLIED:\n",
"\n",
"JPY (¥) → USD: 0.0068 (as of 11/15/2024)\n",
"\n",
"SGD (S$) → USD: 0.74 (as of 11/28/2024)\n",
"\n",
"AUD (A$) → USD: 0.65 (as of 12/03/2024)\n",
"\n",
"# ITEMIZED EXPENSES:\n",
"\n",
"|Date|Category|Description|Original|Currency|USD|\n",
"|---|---|---|---|---|---|\n",
"|11/15/2024|Transportation|JFK → NRT Business Class|4,250.00|USD|4,250.00|\n",
"|Booking Ref: QF78956 - Corporate Rate Applied|Booking Ref: QF78956 - Corporate Rate Applied|Booking Ref: QF78956 - Corporate Rate Applied|Booking Ref: QF78956 - Corporate Rate Applied|Booking Ref: QF78956 - Corporate Rate Applied|Booking Ref: QF78956 - Corporate Rate Applied|\n",
"|Project Code: QD-QUANTUM-2024-01|Project Code: QD-QUANTUM-2024-01|Project Code: QD-QUANTUM-2024-01|Project Code: QD-QUANTUM-2024-01|Project Code: QD-QUANTUM-2024-01|Project Code: QD-QUANTUM-2024-01|\n",
"|11/16/2024|Accommodation|Hilton Tokyo - 5 nights|225,000|JPY|1,530.00|\n",
"|Confirmation: HTK-2024-78956|Confirmation: HTK-2024-78956|Confirmation: HTK-2024-78956|Confirmation: HTK-2024-78956|Confirmation: HTK-2024-78956|Confirmation: HTK-2024-78956|\n"
]
}
],
"source": [
"print(vanilaParsing[0].text)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Started parsing the file under job_id 7b0d05bb-947b-4475-8d0f-f10386f7446e\n"
]
}
],
"source": [
"parsingInstruction = \"\"\"You are provided with an expense report. \n",
"Extract employee name, employee id, position, department, date ranges, individual expense items with dates, categories, and amounts.\"\"\"\n",
"\n",
"withInstructionParsing = LlamaParse(\n",
" result_type=\"markdown\", parsing_instruction=parsingInstruction\n",
").load_data(\"./expense_report_document.pdf\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"**Employee Information:**\n",
"- **Name:** Dr. Alexandra Chen-Martinez, PhD\n",
"- **Employee ID:** QD-2022-1457\n",
"- **Position:** Principal Research Scientist\n",
"- **Department:** Advanced Research & Development\n",
"\n",
"**Trip/Expense Period:**\n",
"- **Start Date:** November 15, 2024\n",
"- **End Date:** December 10, 2024\n",
"\n",
"**Expense Items:**\n",
"1. **Date:** 11/15/2024\n",
"- **Category:** Transportation\n",
"- **Description:** JFK → NRT Business Class\n",
"- **Original Amount:** $4,250.00\n",
"- **Currency:** USD\n",
"- **USD Amount:** $4,250.00\n",
"- **Booking Reference:** QF78956 - Corporate Rate Applied\n",
"- **Project Code:** QD-QUANTUM-2024-01\n",
"\n",
"2. **Date:** 11/16/2024\n",
"- **Category:** Accommodation\n",
"- **Description:** Hilton Tokyo - 5 nights\n",
"- **Original Amount:** ¥225,000\n",
"- **Currency:** JPY\n",
"- **USD Amount:** $1,530.00\n",
"- **Confirmation:** HTK-2024-78956\n",
"\n",
"**Locations:**\n",
"- Tokyo, Japan\n",
"- Singapore\n",
"- Sydney, Australia\n",
"\n",
"**Currency Conversion Rates Applied:**\n",
"- JPY (¥) → USD: 0.0068 (as of 11/15/2024)\n",
"- SGD (S$) → USD: 0.74 (as of 11/28/2024)\n",
"- AUD (A$) → USD: 0.65 (as of 12/03/2024)\n"
]
}
],
"source": [
"print(withInstructionParsing[0].text)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Purchase Order Document \n",
"\n",
"Here we identify the PO number, vendor details, shipping terms, and an itemized list of products with quantities and unit prices."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<img src=\"purchase_order_document.png\" alt=\"Alt Text\" width=\"500\">"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Started parsing the file under job_id b8cb11c3-7dce-4e6a-94bb-1a4e50e45e55\n"
]
}
],
"source": [
"vanilaParsing = LlamaParse(result_type=\"markdown\").load_data(\n",
" \"./purchase_order_document.pdf\"\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"# GLOBAL TECH SOLUTIONS, INC.\n",
"\n",
"# PURCHASE ORDER\n",
"\n",
"Document Reference: PO-2024-GT-9876/REV.2\n",
"\n",
"[Original: PO-2024-GT-9876]\n",
"\n",
"Amendment Date: 12/10/2024\n",
"\n",
"# VENDOR INFORMATION:\n",
"\n",
"Quantum Electronics Manufacturing\n",
"\n",
"DUNS: 78-456-7890\n",
"\n",
"Tax ID: EU8976543210\n",
"\n",
"Hoofdorp, Netherlands\n",
"\n",
"Vendor #: QEM-EU-2024-001\n",
"\n",
"# SHIP TO:\n",
"\n",
"Global Tech Solutions, Inc.\n",
"\n",
"Building 7A, Innovation Park\n",
"\n",
"2100 Technology Drive\n",
"\n",
"Austin, TX 78701\n",
"\n",
"USA\n",
"\n",
"Attn: Sarah Martinez, Receiving Manager\n",
"\n",
"Tel: +1 (512) 555-0123\n",
"\n",
"# PAYMENT TERMS:\n",
"\n",
"Net 45\n",
"\n",
"2% discount if paid within 15 days\n",
"\n",
"# SHIPPING TERMS:\n",
"\n",
"DDP (Delivered Duty Paid) - Incoterms 2020\n",
"\n",
"Insurance Required: Yes\n",
"\n",
"Preferred Carrier: DHL/FedEx\n",
"\n",
"Required Delivery Date: 01/15/2025\n",
"\n",
"# SPECIAL INSTRUCTIONS:\n",
"\n",
"1. All shipments must include Certificate of Conformance\n",
"2. ESD-sensitive items must be properly packaged\n",
"3. Temperature logging required for items marked with *\n",
"4. Partial shipments accepted with prior approval\n",
"5. Quote PO number on all correspondence\n",
"\n",
"# ITEM DETAILS:\n",
"\n",
"|Line|Part Number|Description|Qty|UOM|Unit Price|Total|\n",
"|---|---|---|---|---|---|---|\n",
"|1|QE-MCU-5590|Microcontroller Unit|500|EA|$12.50|$6,250.00|\n"
]
}
],
"source": [
"print(vanilaParsing[0].text)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Started parsing the file under job_id d2731305-984d-4633-8a52-0493748cf10b\n"
]
}
],
"source": [
"parsingInstruction = \"\"\"You are provided with a purchase order. \n",
"Identify the PO number, vendor details, shipping terms, and itemized list of products with quantities and unit prices.\"\"\"\n",
"\n",
"withInstructionParsing = LlamaParse(\n",
" result_type=\"markdown\", parsing_instruction=parsingInstruction\n",
").load_data(\"./purchase_order_document.pdf\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Here are the details extracted from the purchase order:\n",
"\n",
"**PO Number:** PO-2024-GT-9876/REV.2\n",
"\n",
"**Vendor Details:**\n",
"- **Vendor Name:** Quantum Electronics Manufacturing\n",
"- **DUNS:** 78-456-7890\n",
"- **Tax ID:** EU8976543210\n",
"- **Address:** Hoofdorp, Netherlands\n",
"- **Vendor Number:** QEM-EU-2024-001\n",
"- **Contact Person:** Sarah Martinez, Receiving Manager\n",
"- **Phone:** +1 (512) 555-0123\n",
"\n",
"**Shipping Terms:**\n",
"- **Terms:** DDP (Delivered Duty Paid) - Incoterms 2020\n",
"- **Insurance Required:** Yes\n",
"- **Preferred Carrier:** DHL/FedEx\n",
"- **Required Delivery Date:** 01/15/2025\n",
"\n",
"**Itemized List of Products:**\n",
"1. **Part Number:** QE-MCU-5590\n",
"- **Description:** Microcontroller Unit\n",
"- **Quantity:** 500 EA\n",
"- **Unit Price:** $12.50\n",
"- **Total:** $6,250.00\n",
"\n",
"**Payment Terms:**\n",
"- Net 45\n",
"- 2% discount if paid within 15 days\n",
"\n",
"**Special Instructions:**\n",
"1. All shipments must include Certificate of Conformance\n",
"2. ESD-sensitive items must be properly packaged\n",
"3. Temperature logging required for items marked with *\n",
"4. Partial shipments accepted with prior approval\n",
"5. Quote PO number on all correspondence\n"
]
}
],
"source": [
"print(withInstructionParsing[0].text)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "llamacloud",
"language": "python",
"name": "llamacloud"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
Binary file not shown.

After

Width:  |  Height:  |  Size: 344 KiB

File diff suppressed because it is too large Load Diff
Binary file not shown.

After

Width:  |  Height:  |  Size: 2.3 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 100 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 464 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 410 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 444 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 610 KiB

File diff suppressed because it is too large Load Diff

Before

Width:  |  Height:  |  Size: 986 KiB

After

Width:  |  Height:  |  Size: 986 KiB

@@ -8,7 +8,7 @@
"# LlamaParse with GPT-4o\n",
"\n",
"\n",
"<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/test_tesla_impact_report/test_gpt4o.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
"<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/parse/test_tesla_impact_report/test_gpt4o.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
"\n",
"GPT-4o is a [fully multimodal model by OpenAI](https://openai.com/index/hello-gpt-4o/) released in May 2024. It matches GPT-4 Turbo performance in text and code, and has significantly improved vision and audio capabilities.\n",
"\n",
@@ -107,7 +107,7 @@
"metadata": {},
"outputs": [],
"source": [
"from llama_parse import LlamaParse\n",
"from llama_cloud_services import LlamaParse\n",
"\n",
"parser_gpt4o = LlamaParse(\n",
" result_type=\"markdown\",\n",
+762
View File
@@ -0,0 +1,762 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Report Generation with LlamaReport\n",
"\n",
"In this notebook, we'll walk through the basic process of generating a report with LlamaReport, and highlight some of the key features of the library.\n",
"\n",
"TLDR:\n",
"1. Download source data to use as knowledge base for the report\n",
"2. Kick off report generation with a template\n",
"3. Get the plan and review/accept/reject suggestions\n",
"4. Get the final report\n",
"5. Review/accept/reject suggestions to edit the final report\n",
"6. Print the final report"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"%pip install llama-cloud-services"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 1. Download Source Data\n",
"\n",
"Here, we download the `Attention is All You Need` paper as a PDF.\n",
"\n",
"LlamaReport currently supports up to 5 files as input, and essentially any file type that can be parsed by LlamaParse.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"!wget \"https://arxiv.org/pdf/1706.03762.pdf\" -O \"./attention.pdf\""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 2. Kick off Report Generation\n",
"\n",
"Here, we kick off report generation with a template.\n",
"\n",
"The template can either be a string or a file path, but here we'll use a string.\n",
"\n",
"In our experiments, anything works as a template, but some general guidelines:\n",
"\n",
"- Use markdown formatting + instructions in each section to guide the report generation\n",
"- If using an existing file as a template, provide extra instructions to guide the report generation\n",
"\n",
"**NOTE:** Since we are in a notebook, we will use async functions and `await` throughout. Synchronous methods that work without `await` are available by just removing the `a` from the method name and removing the `await` keyword."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from llama_cloud_services import LlamaReport\n",
"\n",
"llama_report = LlamaReport(\n",
" api_key=\"llx-...\",\n",
")\n",
"\n",
"report_client = await llama_report.acreate_report(\n",
" name=\"my_cool_report_on_attention\",\n",
" # can pass in file paths or bytes\n",
" input_files=[\"./attention.pdf\"],\n",
" template_text=\"\"\"\\\n",
"# [Some title]\\n\\n\n",
"## TLDR\\n\n",
"A quick summary of the paper.\\n\\n\n",
"## Details\\n\n",
"More details about the paper, possibly more than one section here.\\n\n",
"\"\"\",\n",
" # optional additional instructions for the report generation\n",
" # template_instructions=None,\n",
" # optional file path to an existing template instead of template_text\n",
" # template_file=None,\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The returned `ReportClient` object is used to interact with the report generation process for this specific report."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Report(id=0a394b33-1a3e-463c-b5cb-7ff8ab827d0a, name=my_cool_report_on_attention)\n"
]
}
],
"source": [
"print(report_client)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 3. Get the plan\n",
"\n",
"The first phases of report generation involve ingesting the source data and generating a plan.\n",
"\n",
"The plan is a list of instructions for the report generation, and can be reviewed/accepted/rejected by the user.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"plan = await report_client.await_for_plan(\n",
" timeout=10000,\n",
" poll_interval=10,\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"# {title}\n",
"[ReportQuery(field='title', prompt='Generate a clear and concise title for this paper about the Transformer model and attention mechanisms', context='The paper discusses the Transformer architecture for sequence transduction using attention mechanisms, focusing on machine translation applications')]\n",
"==================\n",
"## TLDR\n",
"\n",
"{tldr_content}\n",
"[ReportQuery(field='tldr_content', prompt='Write a brief, clear summary of the key points about the Transformer model', context='Focus on the main innovations: attention mechanisms, efficiency improvements, and state-of-the-art results in machine translation')]\n",
"==================\n",
"## Details\n",
"\n",
"{details_content}\n",
"[ReportQuery(field='details_content', prompt='Provide detailed information about the Transformer model architecture and its applications', context='Include information about:\\n- The attention mechanism implementation\\n- Advantages over recurrent and convolutional models\\n- Performance in machine translation tasks\\n- Training efficiency improvements')]\n",
"==================\n"
]
}
],
"source": [
"for plan_block in plan.blocks:\n",
" print(plan_block.block.template)\n",
" print(plan_block.queries)\n",
" print(\"==================\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"With the plan, we can either use it to kick off generation of the final report, or we can edit the plan and adjust it as needed.\n",
"\n",
"While we could manually edit the objects here and use `await report_client.aupdate_plan(action=\"edit\", updated_plan=plan)`, we can also use `LlamaReport` to agentically edit the plan."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"suggestions = await report_client.asuggest_edits(\n",
" \"Can you split the details section into two sections?\"\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Justification for change: \n",
"I'll help you break down the details section into two distinct parts - one focusing on the architecture and another on the practical applications and performance. This will make the content more organized and easier to follow. The original block at index 2 will be replaced with these two new sections.\n",
"\n",
"Proposed changes:\n",
"\n",
"## Architecture Details\n",
"\n",
"{architecture_content}\n",
"\n",
"[ReportQuery(field='architecture_content', prompt='Describe the technical details of the Transformer model architecture', context='Focus on:\\n- Core components of the Transformer architecture\\n- Self-attention mechanism implementation\\n- Multi-head attention details\\n- Position encoding approach\\n- Feed-forward network structure')]\n",
"==================\n",
"\n",
"## Performance and Applications\n",
"\n",
"{applications_content}\n",
"\n",
"[ReportQuery(field='applications_content', prompt='Explain the practical applications and performance advantages of the Transformer model', context='Cover:\\n- Comparison with RNN and CNN models\\n- Machine translation results and benchmarks\\n- Training efficiency improvements\\n- Real-world applications and use cases\\n- Scalability benefits')]\n",
"==================\n"
]
}
],
"source": [
"for suggestion in suggestions:\n",
" print(\"Justification for change:\", suggestion.justification)\n",
" print(\"Proposed changes:\")\n",
" for plan_block in suggestion.blocks:\n",
" print(plan_block.block.template)\n",
" print(plan_block.queries)\n",
" print(\"==================\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This looks pretty good! We can also use the client to automatically accept and apply, or reject, these suggestions.\n",
"\n",
"This will (locally) keep track of the history of changes, so that future suggestions can be based on the previous changes."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"for suggestion in suggestions:\n",
" await report_client.aaccept_edit(suggestion)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"What effect did that have on the tracked local history? Let's see!"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[EditAction(block_idx=2, old_content='## Details\\n\\n{details_content}\\n\\nField: details_content, Prompt: Provide detailed information about the Transformer model architecture and its applications, Context: Include information about:\\n- The attention mechanism implementation\\n- Advantages over recurrent and convolutional models\\n- Performance in machine translation tasks\\n- Training efficiency improvements\\nDepends on: none', new_content='\\n## Architecture Details\\n\\n{architecture_content}\\n\\n\\nField: architecture_content, Prompt: Describe the technical details of the Transformer model architecture, Context: Focus on:\\n- Core components of the Transformer architecture\\n- Self-attention mechanism implementation\\n- Multi-head attention details\\n- Position encoding approach\\n- Feed-forward network structure\\nDepends on: none', action='approved', timestamp=datetime.datetime(2025, 2, 4, 20, 59, 55, 773558)),\n",
" EditAction(block_idx=3, old_content='[No old content]', new_content='\\n## Performance and Applications\\n\\n{applications_content}\\n\\n\\nField: applications_content, Prompt: Explain the practical applications and performance advantages of the Transformer model, Context: Cover:\\n- Comparison with RNN and CNN models\\n- Machine translation results and benchmarks\\n- Training efficiency improvements\\n- Real-world applications and use cases\\n- Scalability benefits\\nDepends on: previous', action='approved', timestamp=datetime.datetime(2025, 2, 4, 20, 59, 55, 773687))]"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"report_client.edit_history"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[Message(role=<MessageRole.USER: 'user'>, content='Can you split the details section into two sections?', timestamp=datetime.datetime(2025, 2, 4, 20, 59, 47, 754848)),\n",
" Message(role=<MessageRole.ASSISTANT: 'assistant'>, content=\"\\nI'll help you break down the details section into two distinct parts - one focusing on the architecture and another on the practical applications and performance. This will make the content more organized and easier to follow. The original block at index 2 will be replaced with these two new sections.\\n\", timestamp=datetime.datetime(2025, 2, 4, 20, 59, 55, 482070))]"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"report_client.chat_history"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"These two items are used to provide context for future suggestions! You can always clear this, or provide your own history."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# report_client.suggest_edits(\"....\", chat_history=[{\"role\": \"user\", \"content\": \"...\"}, ...])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 4. Get the final report\n",
"\n",
"Now that we have a plan, we can kick off generation of the final report."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# kicks off report generation\n",
"await report_client.aupdate_plan(action=\"approve\")\n",
"\n",
"# waits for report generation to complete\n",
"report = await report_client.await_completion(\n",
" timeout=10000,\n",
" poll_interval=10,\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"# Attention Is All You Need: A Pure Attention-Based Architecture for Neural Machine Translation\n",
"\n",
"## TLDR\n",
"\n",
"The Transformer introduced a revolutionary architecture that relies entirely on attention mechanisms, eliminating the need for recurrence or convolution in sequence processing. Its key innovations include multi-head self-attention for parallel processing of input sequences, scaled dot-product attention for efficient computation, and positional encodings for sequence order awareness. The model achieved breakthrough results in machine translation (28.4 BLEU on English-to-German, 41.8 BLEU on English-to-French) while requiring significantly less training time than previous approaches, training in 3.5 days on 8 GPUs. This architecture demonstrated that attention mechanisms alone are sufficient for state-of-the-art sequence modeling, setting a new direction for natural language processing.\n",
"\n",
"\n",
"## Architecture Details\n",
"\n",
"The Transformer architecture represents a groundbreaking approach to sequence processing, built entirely on attention mechanisms without recurrence or convolution. Here are its key technical details:\n",
"\n",
"Core Components:\n",
"- Encoder-decoder architecture with stacked self-attention and point-wise feed-forward layers\n",
"- Each layer contains two main sub-layers: multi-head self-attention mechanism and position-wise feed-forward network\n",
"- Layer normalization and residual connections between sub-layers\n",
"- No recurrent or convolutional elements, enabling parallel processing\n",
"\n",
"Self-Attention Mechanism:\n",
"- Processes relationships between all positions in a sequence simultaneously\n",
"- Computes attention weights using queries, keys, and values derived from input representations\n",
"- Implements scaled dot-product attention to prevent gradient issues with large input dimensions\n",
"- Allows direct modeling of dependencies regardless of positional distance\n",
"- Uses masking in decoder to prevent leftward information flow and maintain auto-regressive property\n",
"\n",
"Multi-Head Attention:\n",
"- Employs multiple attention heads operating in parallel\n",
"- Each head processes information in different representation subspaces\n",
"- Three types of attention applications:\n",
" 1. Encoder self-attention (all positions attend to each other)\n",
" 2. Decoder self-attention (each position attends to previous positions)\n",
" 3. Encoder-decoder attention (decoder queries attend to encoder outputs)\n",
"- Counteracts reduced resolution from attention averaging through parallel processing\n",
"\n",
"Position-wise Feed-Forward Network:\n",
"- Applied identically to each position separately\n",
"- Consists of two linear transformations with ReLU activation\n",
"- Structure: FFN(x) = max(0, xW1 + b1)W2 + b2\n",
"- Input and output dimensionality: dmodel = 512\n",
"- Inner-layer dimensionality: dff = 2048\n",
"- Parameters vary between layers but remain constant across positions\n",
"\n",
"Position Encoding:\n",
"- Adds positional information to input embeddings\n",
"- Enables the model to consider sequential order without recurrence\n",
"- Implements sinusoidal position encodings to allow model to attend to relative positions\n",
"- Maintains constant number of operations between any two positions, unlike convolutional approaches\n",
"- Allows effective modeling of both local and long-range dependencies\n",
"\n",
"\n",
"\n",
"## Performance and Applications\n",
"\n",
"The Transformer model demonstrates significant performance advantages and practical applications across multiple domains:\n",
"\n",
"Performance Advantages over RNN/CNN Models:\n",
"- Eliminates sequential computation constraints present in RNNs, enabling superior parallelization\n",
"- Reduces operations needed for relating distant positions to a constant number, compared to linear/logarithmic scaling in CNNs\n",
"- Processes all input and output positions simultaneously through self-attention mechanisms\n",
"- Achieves state-of-the-art results while requiring significantly less computational resources\n",
"\n",
"Machine Translation Benchmarks:\n",
"- WMT 2014 English-to-German: 28.4 BLEU score, exceeding previous best results by over 2 BLEU points\n",
"- WMT 2014 English-to-French: 41.8 BLEU score (single-model state-of-the-art)\n",
"- Surpasses performance of existing model ensembles in translation tasks\n",
"\n",
"Training Efficiency:\n",
"- Requires only 3.5 days of training on eight GPUs for state-of-the-art performance\n",
"- Achieves superior results at \"a small fraction of the training costs\" compared to previous models\n",
"- Enables significantly faster training through parallel processing of input/output sequences\n",
"- Can reach production-quality performance in as little as twelve hours on modern GPU hardware\n",
"\n",
"Real-world Applications:\n",
"- Machine translation systems\n",
"- Natural language understanding tasks\n",
"- Reading comprehension\n",
"- Abstractive summarization\n",
"- Text entailment analysis\n",
"- Constituency parsing (achieving 92.7 F1 score in semi-supervised settings)\n",
"- Adaptable to both large and limited training data scenarios\n",
"\n",
"Scalability Benefits:\n",
"- Highly parallelizable architecture enables efficient scaling across multiple GPUs\n",
"- Constant computational complexity for relating any input/output positions\n",
"- Effective handling of long-range dependencies in sequences\n",
"- Maintains performance quality while scaling to larger datasets and model sizes\n",
"- Generalizes well across different tasks and domains without architectural changes\n",
"- Supports efficient inference and deployment in production environments\n",
"\n"
]
}
],
"source": [
"report_text = \"\\n\\n\".join([block.template for block in report.blocks])\n",
"print(report_text)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 5. Edit the final report\n",
"\n",
"Now that we have a report, we can edit it.\n",
"\n",
"We can use the `asuggest_edits` method to get suggestions for edits, and then use the `aaccept_edit`/`areject_edit` methods to apply them.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Justification for change: \n",
"I'd suggest changing \"TLDR\" to \"Executive Summary\" which is more appropriate for a professional or academic report. This term is widely used in formal documents and better reflects the nature of this concise overview section while maintaining the same function of providing a quick summary of the key points.\n",
"\n",
"Proposed changes:\n",
"## Executive Summary\n",
"\n",
"The Transformer introduced a revolutionary architecture that relies entirely on attention mechanisms, eliminating the need for recurrence or convolution in sequence processing. Its key innovations include multi-head self-attention for parallel processing of input sequences, scaled dot-product attention for efficient computation, and positional encodings for sequence order awareness. The model achieved breakthrough results in machine translation (28.4 BLEU on English-to-German, 41.8 BLEU on English-to-French) while requiring significantly less training time than previous approaches, training in 3.5 days on 8 GPUs. This architecture demonstrated that attention mechanisms alone are sufficient for state-of-the-art sequence modeling, setting a new direction for natural language processing.\n",
"==================\n"
]
}
],
"source": [
"suggestions = await report_client.asuggest_edits(\n",
" \"Can you change the TLDR header to something more professional?\"\n",
")\n",
"for suggestion in suggestions:\n",
" print(\"Justification for change:\", suggestion.justification)\n",
" print(\"Proposed changes:\")\n",
" for block in suggestion.blocks:\n",
" print(block.template)\n",
" print(\"==================\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Changing to \"Executive Summary\" sounds reasonable, lets accept that!\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"for suggestion in suggestions:\n",
" await report_client.aaccept_edit(suggestion)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 7. Print the final report\n",
"\n",
"Now that we have a report, we can print it."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"# Attention Is All You Need: A Pure Attention-Based Architecture for Neural Machine Translation\n",
"\n",
"## Executive Summary\n",
"\n",
"The Transformer introduced a revolutionary architecture that relies entirely on attention mechanisms, eliminating the need for recurrence or convolution in sequence processing. Its key innovations include multi-head self-attention for parallel processing of input sequences, scaled dot-product attention for efficient computation, and positional encodings for sequence order awareness. The model achieved breakthrough results in machine translation (28.4 BLEU on English-to-German, 41.8 BLEU on English-to-French) while requiring significantly less training time than previous approaches, training in 3.5 days on 8 GPUs. This architecture demonstrated that attention mechanisms alone are sufficient for state-of-the-art sequence modeling, setting a new direction for natural language processing.\n",
"\n",
"\n",
"## Architecture Details\n",
"\n",
"The Transformer architecture represents a groundbreaking approach to sequence processing, built entirely on attention mechanisms without recurrence or convolution. Here are its key technical details:\n",
"\n",
"Core Components:\n",
"- Encoder-decoder architecture with stacked self-attention and point-wise feed-forward layers\n",
"- Each layer contains two main sub-layers: multi-head self-attention mechanism and position-wise feed-forward network\n",
"- Layer normalization and residual connections between sub-layers\n",
"- No recurrent or convolutional elements, enabling parallel processing\n",
"\n",
"Self-Attention Mechanism:\n",
"- Processes relationships between all positions in a sequence simultaneously\n",
"- Computes attention weights using queries, keys, and values derived from input representations\n",
"- Implements scaled dot-product attention to prevent gradient issues with large input dimensions\n",
"- Allows direct modeling of dependencies regardless of positional distance\n",
"- Uses masking in decoder to prevent leftward information flow and maintain auto-regressive property\n",
"\n",
"Multi-Head Attention:\n",
"- Employs multiple attention heads operating in parallel\n",
"- Each head processes information in different representation subspaces\n",
"- Three types of attention applications:\n",
" 1. Encoder self-attention (all positions attend to each other)\n",
" 2. Decoder self-attention (each position attends to previous positions)\n",
" 3. Encoder-decoder attention (decoder queries attend to encoder outputs)\n",
"- Counteracts reduced resolution from attention averaging through parallel processing\n",
"\n",
"Position-wise Feed-Forward Network:\n",
"- Applied identically to each position separately\n",
"- Consists of two linear transformations with ReLU activation\n",
"- Structure: FFN(x) = max(0, xW1 + b1)W2 + b2\n",
"- Input and output dimensionality: dmodel = 512\n",
"- Inner-layer dimensionality: dff = 2048\n",
"- Parameters vary between layers but remain constant across positions\n",
"\n",
"Position Encoding:\n",
"- Adds positional information to input embeddings\n",
"- Enables the model to consider sequential order without recurrence\n",
"- Implements sinusoidal position encodings to allow model to attend to relative positions\n",
"- Maintains constant number of operations between any two positions, unlike convolutional approaches\n",
"- Allows effective modeling of both local and long-range dependencies\n",
"\n",
"\n",
"\n",
"## Performance and Applications\n",
"\n",
"The Transformer model demonstrates significant performance advantages and practical applications across multiple domains:\n",
"\n",
"Performance Advantages over RNN/CNN Models:\n",
"- Eliminates sequential computation constraints present in RNNs, enabling superior parallelization\n",
"- Reduces operations needed for relating distant positions to a constant number, compared to linear/logarithmic scaling in CNNs\n",
"- Processes all input and output positions simultaneously through self-attention mechanisms\n",
"- Achieves state-of-the-art results while requiring significantly less computational resources\n",
"\n",
"Machine Translation Benchmarks:\n",
"- WMT 2014 English-to-German: 28.4 BLEU score, exceeding previous best results by over 2 BLEU points\n",
"- WMT 2014 English-to-French: 41.8 BLEU score (single-model state-of-the-art)\n",
"- Surpasses performance of existing model ensembles in translation tasks\n",
"\n",
"Training Efficiency:\n",
"- Requires only 3.5 days of training on eight GPUs for state-of-the-art performance\n",
"- Achieves superior results at \"a small fraction of the training costs\" compared to previous models\n",
"- Enables significantly faster training through parallel processing of input/output sequences\n",
"- Can reach production-quality performance in as little as twelve hours on modern GPU hardware\n",
"\n",
"Real-world Applications:\n",
"- Machine translation systems\n",
"- Natural language understanding tasks\n",
"- Reading comprehension\n",
"- Abstractive summarization\n",
"- Text entailment analysis\n",
"- Constituency parsing (achieving 92.7 F1 score in semi-supervised settings)\n",
"- Adaptable to both large and limited training data scenarios\n",
"\n",
"Scalability Benefits:\n",
"- Highly parallelizable architecture enables efficient scaling across multiple GPUs\n",
"- Constant computational complexity for relating any input/output positions\n",
"- Effective handling of long-range dependencies in sequences\n",
"- Maintains performance quality while scaling to larger datasets and model sizes\n",
"- Generalizes well across different tasks and domains without architectural changes\n",
"- Supports efficient inference and deployment in production environments\n",
"\n"
]
}
],
"source": [
"report_response = await report_client.aget()\n",
"report_text = \"\\n\\n\".join([block.template for block in report_response.report.blocks])\n",
"print(report_text)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We can also see the sources for each block!"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"0.99687636\n",
"# Abstract\n",
"\n",
"The dominant sequence transduction models are based on complex recurrent or convolutiona\n",
"==================\n",
"0.99591404\n",
"# 2 Background\n",
"\n",
"The goal of reducing sequential computation also forms the foundation of the Extende\n",
"==================\n",
"0.9951325\n",
"# 1 Introduction\n",
"\n",
"Recurrent neural networks, long short-term memory [13] and gated recurrent [7] neu\n",
"==================\n",
"0.99442345\n",
"# 7 Conclusion\n",
"\n",
"In this work, we presented the Transformer, the first sequence transduction model ba\n",
"==================\n",
"0.9967649\n",
"# 3.2.3 Applications of Attention in our Model\n",
"\n",
"The Transformer uses multi-head attention in three d\n",
"==================\n",
"0.99533635\n",
"# 2 Background\n",
"\n",
"The goal of reducing sequential computation also forms the foundation of the Extende\n",
"==================\n",
"0.9935868\n",
"# Abstract\n",
"\n",
"The dominant sequence transduction models are based on complex recurrent or convolutiona\n",
"==================\n",
"0.98780584\n",
"# Outputs\n",
"\n",
"(shifted right)\n",
"\n",
"Figure 1: The Transformer - model architecture.\n",
"\n",
"The Transformer follows\n",
"==================\n",
"0.9205043\n",
"# 3.3 Position-wise Feed-Forward Networks\n",
"\n",
"In addition to attention sub-layers, each of the layers i\n",
"==================\n",
"0.79581684\n",
"# 1 Introduction\n",
"\n",
"Recurrent neural networks, long short-term memory [13] and gated recurrent [7] neu\n",
"==================\n",
"0.9946774\n",
"# Abstract\n",
"\n",
"The dominant sequence transduction models are based on complex recurrent or convolutiona\n",
"==================\n",
"0.97079873\n",
"# 7 Conclusion\n",
"\n",
"In this work, we presented the Transformer, the first sequence transduction model ba\n",
"==================\n",
"0.9535353\n",
"# 6.3 English Constituency Parsing\n",
"\n",
"To evaluate if the Transformer can generalize to other tasks we \n",
"==================\n",
"0.9514138\n",
"# 2 Background\n",
"\n",
"The goal of reducing sequential computation also forms the foundation of the Extende\n",
"==================\n",
"0.9790758\n",
"# 1 Introduction\n",
"\n",
"Recurrent neural networks, long short-term memory [13] and gated recurrent [7] neu\n",
"==================\n",
"0.92262185\n",
"# Outputs\n",
"\n",
"(shifted right)\n",
"\n",
"Figure 1: The Transformer - model architecture.\n",
"\n",
"The Transformer follows\n",
"==================\n"
]
}
],
"source": [
"for block in report_response.report.blocks:\n",
" # Each block has a list of sources, which are the nodes that were used to generate the block\n",
" for source in block.sources:\n",
" print(source.score)\n",
" print(source.node.text[:100])\n",
" print(\"==================\")"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "llama-parse-aNC435Vv-py3.10",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
File diff suppressed because it is too large Load Diff
+8
View File
@@ -0,0 +1,8 @@
from llama_cloud_services.parse import LlamaParse
from llama_cloud_services.report import ReportClient, LlamaReport
__all__ = [
"LlamaParse",
"ReportClient",
"LlamaReport",
]
+3
View File
@@ -0,0 +1,3 @@
from llama_cloud_services.parse.base import LlamaParse, ResultType
__all__ = ["LlamaParse", "ResultType"]
File diff suppressed because it is too large Load Diff
@@ -5,7 +5,7 @@ from pathlib import Path
from pydantic.fields import FieldInfo
from typing import Any, Callable, List
from llama_parse.base import LlamaParse
from llama_cloud_services.parse.base import LlamaParse
def pydantic_field_to_click_option(name: str, field: FieldInfo) -> click.Option:
@@ -10,6 +10,8 @@ class ResultType(str, Enum):
TXT = "text"
MD = "markdown"
JSON = "json"
STRUCTURED = "structured"
class Language(str, Enum):
@@ -189,4 +191,11 @@ SUPPORTED_FILE_TYPES = [
".xlr",
".eth",
".tsv",
".mp3",
".mp4",
".mpeg",
".mpga",
".m4a",
".wav",
".webm",
]
+4
View File
@@ -0,0 +1,4 @@
from llama_cloud_services.report.report import ReportClient
from llama_cloud_services.report.base import LlamaReport
__all__ = ["ReportClient", "LlamaReport"]
+269
View File
@@ -0,0 +1,269 @@
import asyncio
import httpx
import os
import io
from concurrent.futures import ThreadPoolExecutor
from typing import Optional, List, Union, Any, Coroutine, TypeVar
from urllib.parse import urljoin
from llama_cloud.types import ReportMetadata
from llama_cloud_services.report.report import ReportClient
T = TypeVar("T")
class LlamaReport:
"""Client for managing reports and general report operations."""
def __init__(
self,
api_key: Optional[str] = None,
project_id: Optional[str] = None,
organization_id: Optional[str] = None,
base_url: Optional[str] = None,
timeout: Optional[int] = None,
async_httpx_client: Optional[httpx.AsyncClient] = None,
):
self.api_key = api_key or os.getenv("LLAMA_CLOUD_API_KEY", None)
if not self.api_key:
raise ValueError("No API key provided.")
self.base_url = base_url or os.getenv(
"LLAMA_CLOUD_BASE_URL", "https://api.cloud.llamaindex.ai"
)
self.timeout = timeout or 60
# Initialize HTTP clients
self._aclient = async_httpx_client or httpx.AsyncClient(timeout=self.timeout)
# Set auth headers
self.headers = {
"Authorization": f"Bearer {self.api_key}",
}
self.organization_id = organization_id
self.project_id = project_id
self._client_params = {
"timeout": self._aclient.timeout,
"headers": self._aclient.headers,
"base_url": self._aclient.base_url,
"auth": self._aclient.auth,
"event_hooks": self._aclient.event_hooks,
"cookies": self._aclient.cookies,
"max_redirects": self._aclient.max_redirects,
"params": self._aclient.params,
"trust_env": self._aclient.trust_env,
}
self._thread_pool = ThreadPoolExecutor(
max_workers=min(10, (os.cpu_count() or 1) + 4)
)
@property
def aclient(self) -> httpx.AsyncClient:
if self._aclient is None:
self._aclient = httpx.AsyncClient(**self._client_params)
return self._aclient
def _run_sync(self, coro: Coroutine[Any, Any, T]) -> T:
"""Run coroutine in a separate thread to avoid event loop issues"""
# force a new client for this thread/event loop
original_client = self._aclient
self._aclient = None
def run_coro() -> T:
async def wrapped_coro() -> T:
return await coro
return asyncio.run(wrapped_coro())
result = self._thread_pool.submit(run_coro).result()
# restore the original client
self._aclient = original_client
return result
async def _get_default_project(self) -> str:
response = await self.aclient.get(
urljoin(str(self.base_url), "/api/v1/projects"), headers=self.headers
)
response.raise_for_status()
projects = response.json()
default_project = [p for p in projects if p.get("is_default")]
return default_project[0]["id"]
async def _build_url(
self, endpoint: str, extra_params: Optional[List[str]] = None
) -> str:
"""Helper method to build URLs with common query parameters."""
url = urljoin(str(self.base_url), endpoint)
if not self.project_id:
self.project_id = await self._get_default_project()
query_params = []
if self.organization_id:
query_params.append(f"organization_id={self.organization_id}")
if self.project_id:
query_params.append(f"project_id={self.project_id}")
if extra_params:
query_params.extend([p for p in extra_params if p is not None])
if query_params:
url += "?" + "&".join(query_params)
return url
async def acreate_report(
self,
name: str,
template_instructions: Optional[str] = None,
template_text: Optional[str] = None,
template_file: Optional[Union[str, tuple[str, bytes]]] = None,
input_files: Optional[List[Union[str, tuple[str, bytes]]]] = None,
existing_retriever_id: Optional[str] = None,
) -> ReportClient:
"""Create a new report asynchronously."""
url = await self._build_url("/api/v1/reports/")
open_files: List[io.BufferedReader] = []
data = {"name": name}
if template_instructions:
data["template_instructions"] = template_instructions
if template_text:
data["template_text"] = template_text
if existing_retriever_id:
data["existing_retriever_id"] = str(existing_retriever_id)
files: List[tuple[str, io.BufferedReader | bytes]] = []
if template_file:
if isinstance(template_file, str):
open_files.append(open(template_file, "rb"))
files.append(("template_file", open_files[-1]))
else:
files.append(("template_file", template_file[1]))
if input_files:
for f in input_files:
if isinstance(f, str):
open_files.append(open(f, "rb"))
files.append(("files", open_files[-1]))
else:
files.append(("files", f[1]))
response = await self.aclient.post(
url, headers=self.headers, data=data, files=files
)
try:
response.raise_for_status()
report_id = response.json()["id"]
return ReportClient(report_id, name, self)
except httpx.HTTPStatusError as e:
raise ValueError(
f"Failed to create report: {e.response.text}\nError Code: {e.response.status_code}"
)
finally:
for open_file in open_files:
open_file.close()
def create_report(
self,
name: str,
template_instructions: Optional[str] = None,
template_text: Optional[str] = None,
template_file: Optional[Union[str, tuple[str, bytes]]] = None,
input_files: Optional[List[Union[str, tuple[str, bytes]]]] = None,
existing_retriever_id: Optional[str] = None,
) -> ReportClient:
"""Create a new report."""
return self._run_sync(
self.acreate_report(
name=name,
template_instructions=template_instructions,
template_text=template_text,
template_file=template_file,
input_files=input_files,
existing_retriever_id=existing_retriever_id,
)
)
async def alist_reports(
self, state: Optional[str] = None, limit: int = 100, offset: int = 0
) -> List[ReportClient]:
"""List all reports asynchronously."""
params = []
if state:
params.append(f"state={state}")
if limit:
params.append(f"limit={limit}")
if offset:
params.append(f"offset={offset}")
url = await self._build_url(
"/api/v1/reports/list",
extra_params=params,
)
response = await self.aclient.get(url, headers=self.headers)
response.raise_for_status()
data = response.json()
return [
ReportClient(r["report_id"], r["name"], self)
for r in data["report_responses"]
]
def list_reports(
self, state: Optional[str] = None, limit: int = 100, offset: int = 0
) -> List[ReportClient]:
"""Synchronous wrapper for listing reports."""
return self._run_sync(self.alist_reports(state, limit, offset))
async def aget_report(self, report_id: str) -> ReportClient:
"""Get a Report instance for working with a specific report."""
url = await self._build_url(f"/api/v1/reports/{report_id}")
response = await self.aclient.get(url, headers=self.headers)
response.raise_for_status()
data = response.json()
return ReportClient(data["report_id"], data["name"], self)
def get_report(self, report_id: str) -> ReportClient:
"""Synchronous wrapper for getting a report."""
return self._run_sync(self.aget_report(report_id))
async def aget_report_metadata(self, report_id: str) -> ReportMetadata:
"""Get metadata for a specific report asynchronously.
Returns:
dict containing:
- id: Report ID
- name: Report name
- state: Current report state
- report_metadata: Additional metadata
- template_file: Name of template file if used
- template_instructions: Template instructions if provided
- input_files: List of input file names
"""
url = await self._build_url(f"/api/v1/reports/{report_id}/metadata")
response = await self.aclient.get(url, headers=self.headers)
response.raise_for_status()
return ReportMetadata(**response.json())
def get_report_metadata(self, report_id: str) -> ReportMetadata:
"""Synchronous wrapper for getting report metadata."""
return self._run_sync(self.aget_report_metadata(report_id))
async def adelete_report(self, report_id: str) -> None:
"""Delete a specific report asynchronously."""
url = await self._build_url(f"/api/v1/reports/{report_id}")
response = await self.aclient.delete(url, headers=self.headers)
response.raise_for_status()
def delete_report(self, report_id: str) -> None:
"""Synchronous wrapper for deleting a report."""
return self._run_sync(self.adelete_report(report_id))
+527
View File
@@ -0,0 +1,527 @@
import asyncio
import httpx
import time
from typing import Optional, List, Literal, Union, TYPE_CHECKING
from dataclasses import dataclass
from datetime import datetime
from enum import Enum
from llama_cloud.types import (
ReportEventItemEventData_Progress,
ReportMetadata,
EditSuggestion,
ReportResponse,
ReportPlan,
ReportBlock,
ReportPlanBlock,
Report,
)
if TYPE_CHECKING:
from llama_cloud_services.report.base import LlamaReport
class MessageRole(str, Enum):
USER = "user"
ASSISTANT = "assistant"
@dataclass
class Message:
role: MessageRole
content: str
timestamp: datetime
@dataclass
class EditAction:
block_idx: int
old_content: str
new_content: Optional[str]
action: Literal["approved", "rejected"]
timestamp: datetime
DEFAULT_POLL_INTERVAL = 5
DEFAULT_TIMEOUT = 600
class ReportClient:
"""Client for operations on a specific report."""
def __init__(self, report_id: str, name: str, parent_client: "LlamaReport"):
self.report_id = report_id
self.name = name
self._client = parent_client
self._headers = parent_client.headers
self._run_sync = parent_client._run_sync
self._build_url = parent_client._build_url
self.chat_history: List[Message] = []
self.edit_history: List[EditAction] = []
@property
def aclient(self) -> httpx.AsyncClient:
return self._client.aclient
def __str__(self) -> str:
return f"Report(id={self.report_id}, name={self.name})"
def __repr__(self) -> str:
return f"Report(id={self.report_id}, name={self.name})"
def _get_block_content(self, block: Union[ReportBlock, ReportPlanBlock]) -> str:
if isinstance(block, ReportBlock):
return block.template
elif isinstance(block, ReportPlanBlock):
return block.block.template
else:
raise ValueError(f"Invalid block type: {type(block)}")
def _get_block_idx(self, block: Union[ReportBlock, ReportPlanBlock]) -> int:
if isinstance(block, ReportBlock):
return block.idx
elif isinstance(block, ReportPlanBlock):
return block.block.idx
else:
raise ValueError(f"Invalid block type: {type(block)}")
async def aget(self, version: Optional[int] = None) -> ReportResponse:
"""Get this report's details asynchronously."""
extra_params = []
if version is not None:
extra_params.append(f"version={version}")
url = await self._build_url(f"/api/v1/reports/{self.report_id}", extra_params)
response = await self.aclient.get(url, headers=self._headers)
response.raise_for_status()
return ReportResponse(**response.json())
def get(self, version: Optional[int] = None) -> ReportResponse:
"""Synchronous wrapper for getting this report's details."""
return self._run_sync(self.aget(version))
async def aupdate_report(self, updated_report: Report) -> ReportResponse:
"""Update this report's content asynchronously."""
url = await self._build_url(f"/api/v1/reports/{self.report_id}")
response = await self.aclient.patch(
url, headers=self._headers, json={"content": updated_report.dict()}
)
response.raise_for_status()
return ReportResponse(**response.json())
def update_report(self, updated_report: Report) -> ReportResponse:
"""Synchronous wrapper for updating this report's content."""
return self._run_sync(self.aupdate_report(updated_report))
async def aupdate_plan(
self,
action: Literal["approve", "reject", "edit"],
updated_plan: Optional[ReportPlan] = None,
) -> ReportResponse:
"""Update this report's plan asynchronously."""
if action == "edit" and not updated_plan:
raise ValueError("updated_plan is required when action is 'edit'")
url = await self._build_url(
f"/api/v1/reports/{self.report_id}/plan", [f"action={action}"]
)
data = None
if updated_plan is not None:
plan_dict = updated_plan.dict()
plan_dict.pop("generated_at", None)
data = plan_dict
if updated_plan is None and action == "edit":
raise ValueError("updated_plan is required when action is 'edit'")
response = await self.aclient.patch(url, headers=self._headers, json=data)
response.raise_for_status()
return ReportResponse(**response.json())
def update_plan(
self,
action: Literal["approve", "reject", "edit"],
updated_plan: Optional[ReportPlan] = None,
) -> ReportResponse:
"""Synchronous wrapper for updating this report's plan."""
return self._run_sync(self.aupdate_plan(action, updated_plan))
async def asuggest_edits(
self,
user_query: str,
auto_history: bool = True,
chat_history: Optional[List[dict]] = None,
) -> List[EditSuggestion]:
"""Get AI suggestions for edits to this report asynchronously.
Args:
user_query: The user's request/question about what to edit
auto_history: Whether to automatically add the user's message to the chat history
chat_history:
A list of chat messages to include in the chat history.
The format being a list of dictionaries with "role" and "content" keys.
"""
# Add user message to history
self.chat_history.append(
Message(role=MessageRole.USER, content=user_query, timestamp=datetime.now())
)
# Format chat history with edit summaries
chat_history_dicts = []
for msg in self.chat_history[:-1]: # Exclude current message
content = msg.content
if msg.role == MessageRole.USER:
# Add edit summary for user messages
edit_summary = self._get_edit_summary_after_message(msg.timestamp)
if edit_summary:
content = f"{content}\n\nActions taken:\n{edit_summary}"
chat_history_dicts.append({"role": msg.role.value, "content": content})
# decide whether to include chat history or not
if chat_history:
chat_history_dicts = chat_history
elif auto_history:
chat_history_dicts = chat_history_dicts
else:
chat_history_dicts = []
# Make the API call
url = await self._build_url(f"/api/v1/reports/{self.report_id}/suggest_edits")
data = {"user_query": user_query, "chat_history": chat_history_dicts}
response = await self.aclient.post(url, headers=self._headers, json=data)
response.raise_for_status()
suggestions = response.json()
suggestions = [EditSuggestion(**suggestion) for suggestion in suggestions]
# Add assistant response to history
if suggestions:
for suggestion in suggestions:
self.chat_history.append(
Message(
role=MessageRole.ASSISTANT,
content=suggestion.justification,
timestamp=datetime.now(),
)
)
return suggestions
def suggest_edits(
self,
user_query: str,
auto_history: bool = True,
chat_history: Optional[List[dict]] = None,
) -> List[EditSuggestion]:
"""Synchronous wrapper for getting edit suggestions."""
return self._run_sync(
self.asuggest_edits(user_query, auto_history, chat_history)
)
async def await_completion(
self, timeout: int = DEFAULT_TIMEOUT, poll_interval: int = DEFAULT_POLL_INTERVAL
) -> Report:
"""Wait for this report to complete processing."""
start_time = time.time()
while True:
report_response = await self.aget()
status = report_response.status
if status == "completed":
return report_response.report
elif status == "error":
events = await self.aget_events()
raise ValueError(f"Report entered error state: {events[-1].msg}")
elif time.time() - start_time > timeout:
raise TimeoutError(f"Report did not complete within {timeout} seconds")
await asyncio.sleep(poll_interval)
def wait_for_completion(
self, timeout: int = DEFAULT_TIMEOUT, poll_interval: int = DEFAULT_POLL_INTERVAL
) -> Report:
"""Synchronous wrapper for awaiting report completion."""
return self._run_sync(self.await_completion(timeout, poll_interval))
async def await_for_plan(
self, timeout: int = DEFAULT_TIMEOUT, poll_interval: int = DEFAULT_POLL_INTERVAL
) -> ReportPlan:
"""Wait for this report's plan to be ready for review."""
start_time = time.time()
while True:
report_metadata = await self.aget_metadata()
state = report_metadata.state
if state == "waiting_approval":
report_response = await self.aget()
return report_response.plan
elif state == "error":
events = await self.aget_events()
raise ValueError(f"Report entered error state: {events[-1].msg}")
elif time.time() - start_time > timeout:
raise TimeoutError(f"Plan was not ready within {timeout} seconds")
await asyncio.sleep(poll_interval)
def wait_for_plan(
self, timeout: int = DEFAULT_TIMEOUT, poll_interval: int = DEFAULT_POLL_INTERVAL
) -> ReportPlan:
"""Synchronous wrapper for awaiting plan readiness."""
return self._run_sync(self.await_for_plan(timeout, poll_interval))
async def aget_metadata(self) -> ReportMetadata:
"""Get this report's metadata asynchronously."""
return await self._client.aget_report_metadata(self.report_id)
def get_metadata(self) -> ReportMetadata:
"""Synchronous wrapper for getting this report's metadata."""
return self._run_sync(self.aget_metadata())
async def adelete(self) -> None:
"""Delete this report asynchronously."""
return await self._client.adelete_report(self.report_id)
def delete(self) -> None:
"""Synchronous wrapper for deleting this report."""
return self._run_sync(self.adelete())
async def aaccept_edit(self, suggestion: EditSuggestion) -> None:
"""Accept a suggested edit.
Args:
suggestion: The EditSuggestion to accept, typically from suggest_edits()
"""
if len(suggestion.blocks) == 0:
return
# Determine if we're editing a plan or report based on first block type
is_plan_edit = isinstance(suggestion.blocks[0], ReportPlanBlock)
# Get current content
report_response = await self.aget()
current_blocks = (
report_response.plan.blocks
if is_plan_edit
else report_response.report.blocks
)
# Track the edit
new_blocks = []
for edit_block in suggestion.blocks:
# Find matching block in current content
old_block = next(
(
b
for b in current_blocks
if self._get_block_idx(b) == self._get_block_idx(edit_block)
),
None,
)
old_content = (
self._get_block_content(old_block) if old_block else "[No old content]"
)
new_content = self._get_block_content(edit_block)
if is_plan_edit:
new_queries_str = "\n".join(
[
f"Field: {q.field}, Prompt: {q.prompt}, Context: {q.context}"
for q in edit_block.queries
]
)
new_dependency_str = (
f"Depends on: {edit_block.dependency}"
if edit_block.dependency
else ""
)
new_content += f"\n\n{new_queries_str}\n{new_dependency_str}"
if old_block:
old_queries_str = "\n".join(
[
f"Field: {q.field}, Prompt: {q.prompt}, Context: {q.context}"
for q in old_block.queries
]
)
old_dependency_str = (
f"Depends on: {old_block.dependency}"
if old_block.dependency
else ""
)
old_content += f"\n\n{old_queries_str}\n{old_dependency_str}"
self.edit_history.append(
EditAction(
block_idx=self._get_block_idx(edit_block),
old_content=old_content,
new_content=new_content,
action="approved",
timestamp=datetime.now(),
)
)
# Create updated block
if is_plan_edit:
new_blocks.append(
ReportPlanBlock(
block=ReportBlock(
idx=edit_block.block.idx,
template=self._get_block_content(edit_block),
sources=edit_block.block.sources,
),
queries=edit_block.queries,
dependency=edit_block.dependency,
)
)
else:
new_blocks.append(
ReportBlock(
idx=edit_block.idx,
template=self._get_block_content(edit_block),
sources=edit_block.sources,
)
)
if new_blocks:
if is_plan_edit:
# Update plan in place
plan = report_response.plan
# Replace edited blocks and add new ones
for new_block in new_blocks:
block_idx = self._get_block_idx(new_block)
existing_block_idx = next(
(
i
for i, b in enumerate(plan.blocks)
if b.block.idx == block_idx
),
None,
)
if existing_block_idx is not None:
# Replace existing block
plan.blocks[existing_block_idx] = new_block
else:
# Add new block to end
plan.blocks.append(new_block)
await self.aupdate_plan("edit", plan)
else:
# Update report in place
report = report_response.report
# Replace edited blocks and add new ones
for new_block in new_blocks:
block_idx = self._get_block_idx(new_block)
existing_block_idx = next(
(i for i, b in enumerate(report.blocks) if b.idx == block_idx),
None,
)
if existing_block_idx is not None:
# Replace existing block
report.blocks[existing_block_idx] = new_block
else:
# Add new block to end
report.blocks.append(new_block)
await self.aupdate_report(report)
def accept_edit(self, suggestion: EditSuggestion) -> None:
"""Synchronous wrapper for accepting an edit."""
return self._run_sync(self.aaccept_edit(suggestion))
async def areject_edit(self, suggestion: EditSuggestion) -> None:
"""Reject a suggested edit.
Args:
suggestion: The EditSuggestion to reject, typically from suggest_edits()
"""
# Track the rejections
for edit_block in suggestion.blocks:
self.edit_history.append(
EditAction(
block_idx=self._get_block_idx(edit_block),
old_content=self._get_block_content(edit_block),
new_content=None,
action="rejected",
timestamp=datetime.now(),
)
)
def reject_edit(self, suggestion: EditSuggestion) -> None:
"""Synchronous wrapper for rejecting an edit."""
return self._run_sync(self.areject_edit(suggestion))
def _get_edit_summary_after_message(
self, message_timestamp: datetime
) -> Optional[str]:
"""Get a summary of edits that occurred after a specific message."""
relevant_edits = [
edit for edit in self.edit_history if edit.timestamp > message_timestamp
]
if not relevant_edits:
return None
approved = [edit for edit in relevant_edits if edit.action == "approved"]
rejected = [edit for edit in relevant_edits if edit.action == "rejected"]
summary = []
if approved:
summary.append("Approved edits:")
for edit in approved:
summary.append(
f'Block {edit.block_idx}: "{edit.old_content}" -> "{edit.new_content}"'
)
if rejected:
if approved: # Add spacing if we had approved edits
summary.append("")
summary.append("Rejected edits:")
for edit in rejected:
summary.append(f'Block {edit.block_idx}: "{edit.old_content}"')
return "\n".join(summary)
async def aget_events(
self, last_sequence: Optional[int] = None
) -> List[ReportEventItemEventData_Progress]:
"""Get all events for this report asynchronously.
Args:
last_sequence: If provided, only get events after this sequence number
Returns:
List of ReportEvent objects
"""
extra_params = []
if last_sequence is not None:
extra_params.append(f"last_sequence={last_sequence}")
url = await self._build_url(
f"/api/v1/reports/{self.report_id}/events", extra_params
)
response = await self.aclient.get(url, headers=self._headers)
response.raise_for_status()
progress_events = []
for event in response.json():
if event["event_type"] == "progress":
progress_events.append(
ReportEventItemEventData_Progress(**event["event_data"])
)
return progress_events
def get_events(
self, last_sequence: Optional[int] = None
) -> List[ReportEventItemEventData_Progress]:
"""Synchronous wrapper for getting report events."""
return self._run_sync(self.aget_events(last_sequence))
+165
View File
@@ -0,0 +1,165 @@
# LlamaParse
[![PyPI - Downloads](https://img.shields.io/pypi/dm/llama-parse)](https://pypi.org/project/llama-parse/)
[![GitHub contributors](https://img.shields.io/github/contributors/run-llama/llama_parse)](https://github.com/run-llama/llama_parse/graphs/contributors)
[![Discord](https://img.shields.io/discord/1059199217496772688)](https://discord.gg/dGcwcsnxhU)
LlamaParse is a **GenAI-native document parser** that can parse complex document data for any downstream LLM use case (RAG, agents).
It is really good at the following:
-**Broad file type support**: Parsing a variety of unstructured file types (.pdf, .pptx, .docx, .xlsx, .html) with text, tables, visual elements, weird layouts, and more.
-**Table recognition**: Parsing embedded tables accurately into text and semi-structured representations.
-**Multimodal parsing and chunking**: Extracting visual elements (images/diagrams) into structured formats and return image chunks using the latest multimodal models.
-**Custom parsing**: Input custom prompt instructions to customize the output the way you want it.
LlamaParse directly integrates with [LlamaIndex](https://github.com/run-llama/llama_index).
The free plan is up to 1000 pages a day. Paid plan is free 7k pages per week + 0.3c per additional page by default. There is a sandbox available to test the API [**https://cloud.llamaindex.ai/parse ↗**](https://cloud.llamaindex.ai/parse).
Read below for some quickstart information, or see the [full documentation](https://docs.cloud.llamaindex.ai/).
If you're a company interested in enterprise RAG solutions, and/or high volume/on-prem usage of LlamaParse, come [talk to us](https://www.llamaindex.ai/contact).
## Getting Started
First, login and get an api-key from [**https://cloud.llamaindex.ai/api-key ↗**](https://cloud.llamaindex.ai/api-key).
Then, make sure you have the latest LlamaIndex version installed.
**NOTE:** If you are upgrading from v0.9.X, we recommend following our [migration guide](https://pretty-sodium-5e0.notion.site/v0-10-0-Migration-Guide-6ede431dcb8841b09ea171e7f133bd77), as well as uninstalling your previous version first.
```
pip uninstall llama-index # run this if upgrading from v0.9.x or older
pip install -U llama-index --upgrade --no-cache-dir --force-reinstall
```
Lastly, install the package:
`pip install llama-parse`
Now you can parse your first PDF file using the command line interface. Use the command `llama-parse [file_paths]`. See the help text with `llama-parse --help`.
```bash
export LLAMA_CLOUD_API_KEY='llx-...'
# output as text
llama-parse my_file.pdf --result-type text --output-file output.txt
# output as markdown
llama-parse my_file.pdf --result-type markdown --output-file output.md
# output as raw json
llama-parse my_file.pdf --output-raw-json --output-file output.json
```
You can also create simple scripts:
```python
import nest_asyncio
nest_asyncio.apply()
from llama_parse import LlamaParse
parser = LlamaParse(
api_key="llx-...", # can also be set in your env as LLAMA_CLOUD_API_KEY
result_type="markdown", # "markdown" and "text" are available
num_workers=4, # if multiple files passed, split in `num_workers` API calls
verbose=True,
language="en", # Optionally you can define a language, default=en
)
# sync
documents = parser.load_data("./my_file.pdf")
# sync batch
documents = parser.load_data(["./my_file1.pdf", "./my_file2.pdf"])
# async
documents = await parser.aload_data("./my_file.pdf")
# async batch
documents = await parser.aload_data(["./my_file1.pdf", "./my_file2.pdf"])
```
## Using with file object
You can parse a file object directly:
```python
import nest_asyncio
nest_asyncio.apply()
from llama_parse import LlamaParse
parser = LlamaParse(
api_key="llx-...", # can also be set in your env as LLAMA_CLOUD_API_KEY
result_type="markdown", # "markdown" and "text" are available
num_workers=4, # if multiple files passed, split in `num_workers` API calls
verbose=True,
language="en", # Optionally you can define a language, default=en
)
file_name = "my_file1.pdf"
extra_info = {"file_name": file_name}
with open(f"./{file_name}", "rb") as f:
# must provide extra_info with file_name key with passing file object
documents = parser.load_data(f, extra_info=extra_info)
# you can also pass file bytes directly
with open(f"./{file_name}", "rb") as f:
file_bytes = f.read()
# must provide extra_info with file_name key with passing file bytes
documents = parser.load_data(file_bytes, extra_info=extra_info)
```
## Using with `SimpleDirectoryReader`
You can also integrate the parser as the default PDF loader in `SimpleDirectoryReader`:
```python
import nest_asyncio
nest_asyncio.apply()
from llama_parse import LlamaParse
from llama_index.core import SimpleDirectoryReader
parser = LlamaParse(
api_key="llx-...", # can also be set in your env as LLAMA_CLOUD_API_KEY
result_type="markdown", # "markdown" and "text" are available
verbose=True,
)
file_extractor = {".pdf": parser}
documents = SimpleDirectoryReader(
"./data", file_extractor=file_extractor
).load_data()
```
Full documentation for `SimpleDirectoryReader` can be found on the [LlamaIndex Documentation](https://docs.llamaindex.ai/en/stable/module_guides/loading/simpledirectoryreader.html).
## Examples
Several end-to-end indexing examples can be found in the examples folder
- [Getting Started](/examples/parse/demo_basic.ipynb)
- [Advanced RAG Example](/examples/parse/demo_advanced.ipynb)
- [Raw API Usage](/examples/parse/demo_api.ipynb)
## Documentation
[https://docs.cloud.llamaindex.ai/](https://docs.cloud.llamaindex.ai/)
## Terms of Service
See the [Terms of Service Here](./TOS.pdf).
## Get in Touch (LlamaCloud)
LlamaParse is part of LlamaCloud, our e2e enterprise RAG platform that provides out-of-the-box, production-ready connectors, indexing, and retrieval over your complex data sources. We offer SaaS and VPC options.
LlamaCloud is currently available via waitlist (join by [creating an account](https://cloud.llamaindex.ai/)). If you're interested in state-of-the-art quality and in centralizing your RAG efforts, come [get in touch with us](https://www.llamaindex.ai/contact).
-3
View File
@@ -1,3 +0,0 @@
from llama_parse.base import LlamaParse, ResultType
__all__ = ["LlamaParse", "ResultType"]
-611
View File
@@ -1,611 +0,0 @@
import os
import asyncio
from io import TextIOWrapper
import httpx
import mimetypes
import time
from pathlib import Path, PurePath, PurePosixPath
from typing import AsyncGenerator, Any, Dict, List, Optional, Union
from contextlib import asynccontextmanager
from io import BufferedIOBase
from fsspec import AbstractFileSystem
from fsspec.spec import AbstractBufferedFile
from llama_index.core.async_utils import run_jobs
from llama_index.core.bridge.pydantic import Field, field_validator
from llama_index.core.constants import DEFAULT_BASE_URL
from llama_index.core.readers.base import BasePydanticReader
from llama_index.core.readers.file.base import get_default_fs
from llama_index.core.schema import Document
from llama_parse.utils import (
nest_asyncio_err,
nest_asyncio_msg,
ResultType,
Language,
SUPPORTED_FILE_TYPES,
)
from copy import deepcopy
# can put in a path to the file or the file bytes itself
# if passing as bytes or a buffer, must provide the file_name in extra_info
FileInput = Union[str, bytes, BufferedIOBase]
_DEFAULT_SEPARATOR = "\n---\n"
class LlamaParse(BasePydanticReader):
"""A smart-parser for files."""
api_key: str = Field(
default="",
description="The API key for the LlamaParse API.",
validate_default=True,
)
base_url: str = Field(
default=DEFAULT_BASE_URL,
description="The base URL of the Llama Parsing API.",
)
result_type: ResultType = Field(
default=ResultType.TXT, description="The result type for the parser."
)
num_workers: int = Field(
default=4,
gt=0,
lt=10,
description="The number of workers to use sending API requests for parsing.",
)
check_interval: int = Field(
default=1,
description="The interval in seconds to check if the parsing is done.",
)
max_timeout: int = Field(
default=2000,
description="The maximum timeout in seconds to wait for the parsing to finish.",
)
verbose: bool = Field(
default=True, description="Whether to print the progress of the parsing."
)
show_progress: bool = Field(
default=True, description="Show progress when parsing multiple files."
)
language: Language = Field(
default=Language.ENGLISH, description="The language of the text to parse."
)
parsing_instruction: Optional[str] = Field(
default="", description="The parsing instruction for the parser."
)
skip_diagonal_text: Optional[bool] = Field(
default=False,
description="If set to true, the parser will ignore diagonal text (when the text rotation in degrees modulo 90 is not 0).",
)
invalidate_cache: Optional[bool] = Field(
default=False,
description="If set to true, the cache will be ignored and the document re-processes. All document are kept in cache for 48hours after the job was completed to avoid processing the same document twice.",
)
do_not_cache: Optional[bool] = Field(
default=False,
description="If set to true, the document will not be cached. This mean that you will be re-charged it you reprocess them as they will not be cached.",
)
fast_mode: Optional[bool] = Field(
default=False,
description="Note: Non compatible with gpt-4o. If set to true, the parser will use a faster mode to extract text from documents. This mode will skip OCR of images, and table/heading reconstruction.",
)
premium_mode: bool = Field(
default=False,
description="Use our best parser mode if set to True.",
)
do_not_unroll_columns: Optional[bool] = Field(
default=False,
description="If set to true, the parser will keep column in the text according to document layout. Reduce reconstruction accuracy, and LLM's/embedings performances in most case.",
)
page_separator: Optional[str] = Field(
default=None,
description="A templated page separator to use to split the text. If it contain `{page_number}`,it will be replaced by the next page number. If not set will the default separator '\\n---\\n' will be used.",
)
page_prefix: Optional[str] = Field(
default=None,
description="A templated prefix to add to the beginning of each page. If it contain `{page_number}`, it will be replaced by the page number.",
)
page_suffix: Optional[str] = Field(
default=None,
description="A templated suffix to add to the beginning of each page. If it contain `{page_number}`, it will be replaced by the page number.",
)
gpt4o_mode: bool = Field(
default=False,
description="Whether to use gpt-4o extract text from documents.",
)
gpt4o_api_key: Optional[str] = Field(
default=None,
description="The API key for the GPT-4o API. Lowers the cost of parsing.",
)
bounding_box: Optional[str] = Field(
default=None,
description="The bounding box to use to extract text from documents describe as a string containing the bounding box margins",
)
target_pages: Optional[str] = Field(
default=None,
description="The target pages to extract text from documents. Describe as a comma separated list of page numbers. The first page of the document is page 0",
)
ignore_errors: bool = Field(
default=True,
description="Whether or not to ignore and skip errors raised during parsing.",
)
split_by_page: bool = Field(
default=True,
description="Whether to split by page using the page separator",
)
vendor_multimodal_api_key: Optional[str] = Field(
default=None,
description="The API key for the multimodal API.",
)
use_vendor_multimodal_model: bool = Field(
default=False,
description="Whether to use the vendor multimodal API.",
)
vendor_multimodal_model_name: Optional[str] = Field(
default=None,
description="The model name for the vendor multimodal API.",
)
take_screenshot: bool = Field(
default=False,
description="Whether to take screenshot of each page of the document.",
)
custom_client: Optional[httpx.AsyncClient] = Field(
default=None, description="A custom HTTPX client to use for sending requests."
)
disable_ocr: bool = Field(
default=False,
description="Disable the OCR on the document. LlamaParse will only extract the copyable text from the document.",
)
# Coming Soon
# annotate_links: bool = Field(
# default=False,
# description="Annotate links found in the document to extract their URL.",
# )
webhook_url: Optional[str] = Field(
default=None,
description="A URL that needs to be called at the end of the parsing job.",
)
azure_openai_deployment_name: Optional[str] = Field(
default=None, description="Azure Openai Deployment Name"
)
azure_openai_endpoint: Optional[str] = Field(
default=None, description="Azure Openai Endpoint"
)
azure_openai_api_version: Optional[str] = Field(
default=None, description="Azure Openai API Version"
)
azure_openai_key: Optional[str] = Field(
default=None, description="Azure Openai Key"
)
@field_validator("api_key", mode="before", check_fields=True)
@classmethod
def validate_api_key(cls, v: str) -> str:
"""Validate the API key."""
if not v:
import os
api_key = os.getenv("LLAMA_CLOUD_API_KEY", None)
if api_key is None:
raise ValueError("The API key is required.")
return api_key
return v
@field_validator("base_url", mode="before", check_fields=True)
@classmethod
def validate_base_url(cls, v: str) -> str:
"""Validate the base URL."""
url = os.getenv("LLAMA_CLOUD_BASE_URL", None)
return url or v or DEFAULT_BASE_URL
@asynccontextmanager
async def client_context(self) -> AsyncGenerator[httpx.AsyncClient, None]:
"""Create a context for the HTTPX client."""
if self.custom_client is not None:
yield self.custom_client
else:
async with httpx.AsyncClient(timeout=self.max_timeout) as client:
yield client
# upload a document and get back a job_id
async def _create_job(
self,
file_input: FileInput,
extra_info: Optional[dict] = None,
fs: Optional[AbstractFileSystem] = None,
) -> str:
headers = {"Authorization": f"Bearer {self.api_key}"}
url = f"{self.base_url}/api/parsing/upload"
files = None
file_handle = None
if isinstance(file_input, (bytes, BufferedIOBase)):
if not extra_info or "file_name" not in extra_info:
raise ValueError(
"file_name must be provided in extra_info when passing bytes"
)
file_name = extra_info["file_name"]
mime_type = mimetypes.guess_type(file_name)[0]
files = {"file": (file_name, file_input, mime_type)}
elif isinstance(file_input, (str, Path, PurePosixPath, PurePath)):
file_path = str(file_input)
file_ext = os.path.splitext(file_path)[1].lower()
if file_ext not in SUPPORTED_FILE_TYPES:
raise Exception(
f"Currently, only the following file types are supported: {SUPPORTED_FILE_TYPES}\n"
f"Current file type: {file_ext}"
)
mime_type = mimetypes.guess_type(file_path)[0]
# Open the file here for the duration of the async context
# load data, set the mime type
fs = fs or get_default_fs()
file_handle = fs.open(file_input, "rb")
files = {"file": (os.path.basename(file_path), file_handle, mime_type)}
else:
raise ValueError(
"file_input must be either a file path string, file bytes, or buffer object"
)
data = {
"language": self.language.value,
"parsing_instruction": self.parsing_instruction,
"invalidate_cache": self.invalidate_cache,
"skip_diagonal_text": self.skip_diagonal_text,
"do_not_cache": self.do_not_cache,
"fast_mode": self.fast_mode,
"premium_mode": self.premium_mode,
"do_not_unroll_columns": self.do_not_unroll_columns,
"gpt4o_mode": self.gpt4o_mode,
"gpt4o_api_key": self.gpt4o_api_key,
"vendor_multimodal_api_key": self.vendor_multimodal_api_key,
"use_vendor_multimodal_model": self.use_vendor_multimodal_model,
"vendor_multimodal_model_name": self.vendor_multimodal_model_name,
"take_screenshot": self.take_screenshot,
"disable_ocr": self.disable_ocr,
# "annotate_links": self.annotate_links,
}
# only send page separator to server if it is not None
# as if a null, "" string is sent the server will then ignore the page separator instead of using the default
if self.page_separator is not None:
data["page_separator"] = self.page_separator
if self.page_prefix is not None:
data["page_prefix"] = self.page_prefix
if self.page_suffix is not None:
data["page_suffix"] = self.page_suffix
if self.bounding_box is not None:
data["bounding_box"] = self.bounding_box
if self.target_pages is not None:
data["target_pages"] = self.target_pages
if self.webhook_url is not None:
data["webhook_url"] = self.webhook_url
# Azure OpenAI
if self.azure_openai_deployment_name is not None:
data["azure_openai_deployment_name"] = self.azure_openai_deployment_name
if self.azure_openai_endpoint is not None:
data["azure_openai_endpoint"] = self.azure_openai_endpoint
if self.azure_openai_api_version is not None:
data["azure_openai_api_version"] = self.azure_openai_api_version
if self.azure_openai_key is not None:
data["azure_openai_key"] = self.azure_openai_key
try:
async with self.client_context() as client:
response = await client.post(
url,
files=files,
headers=headers,
data=data,
)
if not response.is_success:
raise Exception(f"Failed to parse the file: {response.text}")
job_id = response.json()["id"]
return job_id
finally:
if file_handle is not None:
file_handle.close()
@staticmethod
def __get_filename(f: Union[TextIOWrapper, AbstractBufferedFile]) -> str:
if isinstance(f, TextIOWrapper):
return f.name
return f.full_name
async def _get_job_result(
self, job_id: str, result_type: str, verbose: bool = False
) -> Dict[str, Any]:
result_url = f"{self.base_url}/api/parsing/job/{job_id}/result/{result_type}"
status_url = f"{self.base_url}/api/parsing/job/{job_id}"
headers = {"Authorization": f"Bearer {self.api_key}"}
start = time.time()
tries = 0
while True:
await asyncio.sleep(self.check_interval)
async with self.client_context() as client:
tries += 1
result = await client.get(status_url, headers=headers)
if result.status_code != 200:
end = time.time()
if end - start > self.max_timeout:
raise Exception(f"Timeout while parsing the file: {job_id}")
if verbose and tries % 10 == 0:
print(".", end="", flush=True)
await asyncio.sleep(self.check_interval)
continue
# Allowed values "PENDING", "SUCCESS", "ERROR", "CANCELED"
result_json = result.json()
status = result_json["status"]
if status == "SUCCESS":
parsed_result = await client.get(result_url, headers=headers)
return parsed_result.json()
elif status == "PENDING":
end = time.time()
if end - start > self.max_timeout:
raise Exception(f"Timeout while parsing the file: {job_id}")
if verbose and tries % 10 == 0:
print(".", end="", flush=True)
await asyncio.sleep(self.check_interval)
else:
error_code = result_json.get("error_code", "No error code found")
error_message = result_json.get(
"error_message", "No error message found"
)
exception_str = f"Job ID: {job_id} failed with status: {status}, Error code: {error_code}, Error message: {error_message}"
raise Exception(exception_str)
async def _aload_data(
self,
file_path: FileInput,
extra_info: Optional[dict] = None,
fs: Optional[AbstractFileSystem] = None,
verbose: bool = False,
) -> List[Document]:
"""Load data from the input path."""
try:
job_id = await self._create_job(file_path, extra_info=extra_info, fs=fs)
if verbose:
print("Started parsing the file under job_id %s" % job_id)
result = await self._get_job_result(
job_id, self.result_type.value, verbose=verbose
)
docs = [
Document(
text=result[self.result_type.value],
metadata=extra_info or {},
)
]
if self.split_by_page:
return self._get_sub_docs(docs)
else:
return docs
except Exception as e:
file_repr = file_path if isinstance(file_path, str) else "<bytes/buffer>"
print(f"Error while parsing the file '{file_repr}':", e)
if self.ignore_errors:
return []
else:
raise e
async def aload_data(
self,
file_path: Union[List[FileInput], FileInput],
extra_info: Optional[dict] = None,
fs: Optional[AbstractFileSystem] = None,
) -> List[Document]:
"""Load data from the input path."""
if isinstance(file_path, (str, Path, bytes, BufferedIOBase)):
return await self._aload_data(
file_path, extra_info=extra_info, fs=fs, verbose=self.verbose
)
elif isinstance(file_path, list):
jobs = [
self._aload_data(
f,
extra_info=extra_info,
fs=fs,
verbose=self.verbose and not self.show_progress,
)
for f in file_path
]
try:
results = await run_jobs(
jobs,
workers=self.num_workers,
desc="Parsing files",
show_progress=self.show_progress,
)
# return flattened results
return [item for sublist in results for item in sublist]
except RuntimeError as e:
if nest_asyncio_err in str(e):
raise RuntimeError(nest_asyncio_msg)
else:
raise e
else:
raise ValueError(
"The input file_path must be a string or a list of strings."
)
def load_data(
self,
file_path: Union[List[FileInput], FileInput],
extra_info: Optional[dict] = None,
fs: Optional[AbstractFileSystem] = None,
) -> List[Document]:
"""Load data from the input path."""
try:
return asyncio.run(self.aload_data(file_path, extra_info, fs=fs))
except RuntimeError as e:
if nest_asyncio_err in str(e):
raise RuntimeError(nest_asyncio_msg)
else:
raise e
async def _aget_json(
self, file_path: FileInput, extra_info: Optional[dict] = None
) -> List[dict]:
"""Load data from the input path."""
try:
job_id = await self._create_job(file_path, extra_info=extra_info)
if self.verbose:
print("Started parsing the file under job_id %s" % job_id)
result = await self._get_job_result(job_id, "json")
result["job_id"] = job_id
if not isinstance(file_path, (bytes, BufferedIOBase)):
result["file_path"] = str(file_path)
return [result]
except Exception as e:
file_repr = file_path if isinstance(file_path, str) else "<bytes/buffer>"
print(f"Error while parsing the file '{file_repr}':", e)
if self.ignore_errors:
return []
else:
raise e
async def aget_json(
self,
file_path: Union[List[FileInput], FileInput],
extra_info: Optional[dict] = None,
) -> List[dict]:
"""Load data from the input path."""
if isinstance(file_path, (str, Path)):
return await self._aget_json(file_path, extra_info=extra_info)
elif isinstance(file_path, list):
jobs = [self._aget_json(f, extra_info=extra_info) for f in file_path]
try:
results = await run_jobs(
jobs,
workers=self.num_workers,
desc="Parsing files",
show_progress=self.show_progress,
)
# return flattened results
return [item for sublist in results for item in sublist]
except RuntimeError as e:
if nest_asyncio_err in str(e):
raise RuntimeError(nest_asyncio_msg)
else:
raise e
else:
raise ValueError(
"The input file_path must be a string or a list of strings."
)
def get_json_result(
self,
file_path: Union[List[FileInput], FileInput],
extra_info: Optional[dict] = None,
) -> List[dict]:
"""Parse the input path."""
try:
return asyncio.run(self.aget_json(file_path, extra_info))
except RuntimeError as e:
if nest_asyncio_err in str(e):
raise RuntimeError(nest_asyncio_msg)
else:
raise e
async def aget_images(
self, json_result: List[dict], download_path: str
) -> List[dict]:
"""Download images from the parsed result."""
headers = {"Authorization": f"Bearer {self.api_key}"}
# make the download path
if not os.path.exists(download_path):
os.makedirs(download_path)
try:
images = []
for result in json_result:
job_id = result["job_id"]
for page in result["pages"]:
if self.verbose:
print(f"> Image for page {page['page']}: {page['images']}")
for image in page["images"]:
image_name = image["name"]
# get the full path
image_path = os.path.join(
download_path, f"{job_id}-{image_name}"
)
# get a valid image path
if not image_path.endswith(".png"):
if not image_path.endswith(".jpg"):
image_path += ".png"
image["path"] = image_path
image["job_id"] = job_id
image["original_file_path"] = result.get("file_path", None)
image["page_number"] = page["page"]
with open(image_path, "wb") as f:
image_url = f"{self.base_url}/api/parsing/job/{job_id}/result/image/{image_name}"
async with self.client_context() as client:
res = await client.get(
image_url, headers=headers, timeout=self.max_timeout
)
res.raise_for_status()
f.write(res.content)
images.append(image)
return images
except Exception as e:
print("Error while downloading images from the parsed result:", e)
if self.ignore_errors:
return []
else:
raise e
def get_images(self, json_result: List[dict], download_path: str) -> List[dict]:
"""Download images from the parsed result."""
try:
return asyncio.run(self.aget_images(json_result, download_path))
except RuntimeError as e:
if nest_asyncio_err in str(e):
raise RuntimeError(nest_asyncio_msg)
else:
raise e
def _get_sub_docs(self, docs: List[Document]) -> List[Document]:
"""Split docs into pages, by separator."""
sub_docs = []
separator = self.page_separator or _DEFAULT_SEPARATOR
for doc in docs:
doc_chunks = doc.text.split(separator)
for doc_chunk in doc_chunks:
sub_doc = Document(
text=doc_chunk,
metadata=deepcopy(doc.metadata),
)
sub_docs.append(sub_doc)
return sub_docs
+3
View File
@@ -0,0 +1,3 @@
from llama_cloud_services.parse import LlamaParse, ResultType
__all__ = ["LlamaParse", "ResultType"]
+19
View File
@@ -0,0 +1,19 @@
from llama_cloud_services.parse.base import (
LlamaParse,
ResultType,
FileInput,
_DEFAULT_SEPARATOR,
JOB_RESULT_URL,
JOB_STATUS_ROUTE,
JOB_UPLOAD_ROUTE,
)
__all__ = [
"LlamaParse",
"ResultType",
"FileInput",
"_DEFAULT_SEPARATOR",
"JOB_RESULT_URL",
"JOB_STATUS_ROUTE",
"JOB_UPLOAD_ROUTE",
]

Some files were not shown because too many files have changed in this diff Show More