Add support for html_remove_navigation_elements.

.github/workflows/build_package.yml

@@ -45,4 +45,4 @@ jobs:
       - name: Test import
         shell: bash
         working-directory: ${{ vars.RUNNER_TEMP }}
-        run: python -c "import llama_cloud_services"
+        run: python -c "import llama_parse"

.github/workflows/publish_release.yml

@@ -23,33 +23,18 @@ jobs:
         uses: actions/setup-python@v4
         with:
           python-version: ${{ env.PYTHON_VERSION }}
-
       - name: Install Poetry
         uses: snok/install-poetry@v1
         with:
           version: ${{ env.POETRY_VERSION }}
-
       - name: Install deps
         shell: bash
         run: pip install -e .
-
-      - name: Build and publish llama-cloud-services
-        uses: JRubics/poetry-publish@v2.1
+      - name: Build and publish to pypi
+        uses: JRubics/poetry-publish@v1.17
         with:
-          poetry_version: ${{ env.POETRY_VERSION }}
-          python_version: ${{ env.PYTHON_VERSION }}
-          working_directory: "llama_cloud_services"
           pypi_token: ${{ secrets.LLAMA_PARSE_PYPI_TOKEN }}
-          poetry_install_options: "--without dev"
-
-      - name: Build and publish llama-parse
-        uses: JRubics/poetry-publish@v2.1
-        with:
-          poetry_version: ${{ env.POETRY_VERSION }}
-          python_version: ${{ env.PYTHON_VERSION }}
-          working_directory: "llama_parse"
-          pypi_token: ${{ secrets.LLAMA_PARSE_PYPI_TOKEN }}
-          poetry_install_options: "--without dev"
+          ignore_dev_requirements: "yes"
 
       - name: Create GitHub Release
         id: create_release
@@ -67,7 +52,6 @@ jobs:
           export PKG=$(ls dist/ | grep tar)
           set -- $PKG
           echo "name=$1" >> $GITHUB_ENV
-
       - name: Upload Release Asset (sdist) to GitHub
         id: upload-release-asset
         uses: actions/upload-release-asset@v1

.github/workflows/unit_test.yml

@@ -17,7 +17,7 @@ jobs:
       # You can use PyPy versions in python-version.
       # For example, pypy-2.7 and pypy-3.8
       matrix:
-        python-version: ["3.9", "3.10", "3.11", "3.12"]
+        python-version: ["3.8", "3.10", "3.11"]
     steps:
       - uses: actions/checkout@v3
         with:

.pre-commit-config.yaml

@@ -33,7 +33,6 @@ repos:
     rev: v1.0.1
     hooks:
       - id: mypy
-        exclude: ^tests/
         additional_dependencies:
           [
             "types-requests",
@@ -47,7 +46,7 @@ repos:
           [
             --disallow-untyped-defs,
             --ignore-missing-imports,
-            --python-version=3.10,
+            --python-version=3.8,
           ]
   - repo: https://github.com/adamchainz/blacken-docs
     rev: 1.16.0

README.md

@@ -1,45 +1,158 @@
-[![PyPI - Downloads](https://img.shields.io/pypi/dm/llama-cloud-services)](https://pypi.org/project/llama-cloud-services/)
-[![GitHub contributors](https://img.shields.io/github/contributors/run-llama/llama_cloud_services)](https://github.com/run-llama/llama_cloud_services/graphs/contributors)
+# LlamaParse
+
+[![PyPI - Downloads](https://img.shields.io/pypi/dm/llama-parse)](https://pypi.org/project/llama-parse/)
+[![GitHub contributors](https://img.shields.io/github/contributors/run-llama/llama_parse)](https://github.com/run-llama/llama_parse/graphs/contributors)
 [![Discord](https://img.shields.io/discord/1059199217496772688)](https://discord.gg/dGcwcsnxhU)
 
-# Llama Cloud Services
+LlamaParse is a **GenAI-native document parser** that can parse complex document data for any downstream LLM use case (RAG, agents).
 
-This repository contains the code for hand-written SDKs and clients for interacting with LlamaCloud.
+It is really good at the following:
 
-This includes:
+- ✅ **Broad file type support**: Parsing a variety of unstructured file types (.pdf, .pptx, .docx, .xlsx, .html) with text, tables, visual elements, weird layouts, and more.
+- ✅ **Table recognition**: Parsing embedded tables accurately into text and semi-structured representations.
+- ✅ **Multimodal parsing and chunking**: Extracting visual elements (images/diagrams) into structured formats and return image chunks using the latest multimodal models.
+- ✅ **Custom parsing**: Input custom prompt instructions to customize the output the way you want it.
 
-- [LlamaParse](./parse.md) - A GenAI-native document parser that can parse complex document data for any downstream LLM use case (Agents, RAG, data processing, etc.).
-- [LlamaReport (beta/invite-only)](./report.md) - A prebuilt agentic report builder that can be used to build reports from a variety of data sources.
-- [LlamaExtract (coming soon!)]() - A prebuilt agentic data extractor that can be used to transform data into a structured JSON representation.
+LlamaParse directly integrates with [LlamaIndex](https://github.com/run-llama/llama_index).
+
+The free plan is up to 1000 pages a day. Paid plan is free 7k pages per week + 0.3c per additional page by default. There is a sandbox available to test the API [**https://cloud.llamaindex.ai/parse ↗**](https://cloud.llamaindex.ai/parse).
+
+Read below for some quickstart information, or see the [full documentation](https://docs.cloud.llamaindex.ai/).
+
+If you're a company interested in enterprise RAG solutions, and/or high volume/on-prem usage of LlamaParse, come [talk to us](https://www.llamaindex.ai/contact).
 
 ## Getting Started
 
-Install the package:
+First, login and get an api-key from [**https://cloud.llamaindex.ai/api-key ↗**](https://cloud.llamaindex.ai/api-key).
+
+Then, make sure you have the latest LlamaIndex version installed.
+
+**NOTE:** If you are upgrading from v0.9.X, we recommend following our [migration guide](https://pretty-sodium-5e0.notion.site/v0-10-0-Migration-Guide-6ede431dcb8841b09ea171e7f133bd77), as well as uninstalling your previous version first.
+
+```
+pip uninstall llama-index  # run this if upgrading from v0.9.x or older
+pip install -U llama-index --upgrade --no-cache-dir --force-reinstall
+```
+
+Lastly, install the package:
+
+`pip install llama-parse`
+
+Now you can parse your first PDF file using the command line interface. Use the command `llama-parse [file_paths]`. See the help text with `llama-parse --help`.
 
 ```bash
-pip install llama-cloud-services
+export LLAMA_CLOUD_API_KEY='llx-...'
+
+# output as text
+llama-parse my_file.pdf --result-type text --output-file output.txt
+
+# output as markdown
+llama-parse my_file.pdf --result-type markdown --output-file output.md
+
+# output as raw json
+llama-parse my_file.pdf --output-raw-json --output-file output.json
 ```
 
-Then, get your API key from [LlamaCloud](https://cloud.llamaindex.ai/).
-
-Then, you can use the services in your code:
+You can also create simple scripts:
 
 ```python
-from llama_cloud_services import LlamaParse, LlamaReport
+import nest_asyncio
 
-parser = LlamaParse(api_key="YOUR_API_KEY")
-report = LlamaReport(api_key="YOUR_API_KEY")
+nest_asyncio.apply()
+
+from llama_parse import LlamaParse
+
+parser = LlamaParse(
+    api_key="llx-...",  # can also be set in your env as LLAMA_CLOUD_API_KEY
+    result_type="markdown",  # "markdown" and "text" are available
+    num_workers=4,  # if multiple files passed, split in `num_workers` API calls
+    verbose=True,
+    language="en",  # Optionally you can define a language, default=en
+)
+
+# sync
+documents = parser.load_data("./my_file.pdf")
+
+# sync batch
+documents = parser.load_data(["./my_file1.pdf", "./my_file2.pdf"])
+
+# async
+documents = await parser.aload_data("./my_file.pdf")
+
+# async batch
+documents = await parser.aload_data(["./my_file1.pdf", "./my_file2.pdf"])
 ```
 
-See the quickstart guides for each service for more information:
+## Using with file object
 
-- [LlamaParse](./parse.md)
-- [LlamaReport (beta/invite-only)](./report.md)
-- [LlamaExtract (coming soon!)]()
+You can parse a file object directly:
+
+```python
+import nest_asyncio
+
+nest_asyncio.apply()
+
+from llama_parse import LlamaParse
+
+parser = LlamaParse(
+    api_key="llx-...",  # can also be set in your env as LLAMA_CLOUD_API_KEY
+    result_type="markdown",  # "markdown" and "text" are available
+    num_workers=4,  # if multiple files passed, split in `num_workers` API calls
+    verbose=True,
+    language="en",  # Optionally you can define a language, default=en
+)
+
+file_name = "my_file1.pdf"
+extra_info = {"file_name": file_name}
+
+with open(f"./{file_name}", "rb") as f:
+    # must provide extra_info with file_name key with passing file object
+    documents = parser.load_data(f, extra_info=extra_info)
+
+# you can also pass file bytes directly
+with open(f"./{file_name}", "rb") as f:
+    file_bytes = f.read()
+    # must provide extra_info with file_name key with passing file bytes
+    documents = parser.load_data(file_bytes, extra_info=extra_info)
+```
+
+## Using with `SimpleDirectoryReader`
+
+You can also integrate the parser as the default PDF loader in `SimpleDirectoryReader`:
+
+```python
+import nest_asyncio
+
+nest_asyncio.apply()
+
+from llama_parse import LlamaParse
+from llama_index.core import SimpleDirectoryReader
+
+parser = LlamaParse(
+    api_key="llx-...",  # can also be set in your env as LLAMA_CLOUD_API_KEY
+    result_type="markdown",  # "markdown" and "text" are available
+    verbose=True,
+)
+
+file_extractor = {".pdf": parser}
+documents = SimpleDirectoryReader(
+    "./data", file_extractor=file_extractor
+).load_data()
+```
+
+Full documentation for `SimpleDirectoryReader` can be found on the [LlamaIndex Documentation](https://docs.llamaindex.ai/en/stable/module_guides/loading/simpledirectoryreader.html).
+
+## Examples
+
+Several end-to-end indexing examples can be found in the examples folder
+
+- [Getting Started](examples/demo_basic.ipynb)
+- [Advanced RAG Example](examples/demo_advanced.ipynb)
+- [Raw API Usage](examples/demo_api.ipynb)
 
 ## Documentation
 
-You can see complete SDK and API documentation for each service on [our official docs](https://docs.cloud.llamaindex.ai/).
+[https://docs.cloud.llamaindex.ai/](https://docs.cloud.llamaindex.ai/)
 
 ## Terms of Service
 
@@ -47,4 +160,6 @@ See the [Terms of Service Here](./TOS.pdf).
 
 ## Get in Touch (LlamaCloud)
 
-You can get in touch with us by following our [contact link](https://www.llamaindex.ai/contact).
+LlamaParse is part of LlamaCloud, our e2e enterprise RAG platform that provides out-of-the-box, production-ready connectors, indexing, and retrieval over your complex data sources. We offer SaaS and VPC options.
+
+LlamaCloud is currently available via waitlist (join by [creating an account](https://cloud.llamaindex.ai/)). If you're interested in state-of-the-art quality and in centralizing your RAG efforts, come [get in touch with us](https://www.llamaindex.ai/contact).

examples/advanced_rag/dynamic_section_retrieval.ipynb

@@ -53,7 +53,7 @@
    "source": [
     "!pip install llama-index\n",
     "!pip install llama-index-core\n",
-    "!pip install llama-cloud-services"
+    "!pip install llama-parse"
    ]
   },
   {
@@ -190,7 +190,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from llama_cloud_services import LlamaParse\n",
+    "from llama_parse import LlamaParse\n",
     "\n",
     "parser = LlamaParse(result_type=\"markdown\")"
    ]

examples/agents/demo_simple_openai_agent.ipynb

@@ -22,7 +22,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "!pip install llama-cloud-services llama-index llama-index-postprocessor-sbert-rerank"
+    "!pip install llama-parse llama-index llama-index-postprocessor-sbert-rerank"
    ]
   },
   {
@@ -82,7 +82,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from llama_cloud_services import LlamaParse\n",
+    "from llama_parse import LlamaParse\n",
     "\n",
     "parser = LlamaParse(\n",
     "    result_type=\"markdown\",\n",

examples/caltrain/caltrain_text_mode.ipynb

@@ -81,7 +81,7 @@
     }
    ],
    "source": [
-    "from llama_cloud_services import LlamaParse\n",
+    "from llama_parse import LlamaParse\n",
     "\n",
     "docs = LlamaParse(result_type=\"text\").load_data(\"./caltrain_schedule_weekend.pdf\")"
    ]

examples/demo_advanced.ipynb

@@ -26,7 +26,7 @@
     "!pip install llama-index-embeddings-openai\n",
     "!pip install llama-index-postprocessor-flag-embedding-reranker\n",
     "!pip install git+https://github.com/FlagOpen/FlagEmbedding.git\n",
-    "!pip install llama-cloud-services"
+    "!pip install llama-parse"
    ]
   },
   {
@@ -108,7 +108,7 @@
     }
    ],
    "source": [
-    "from llama_cloud_services import LlamaParse\n",
+    "from llama_parse import LlamaParse\n",
     "\n",
     "documents = LlamaParse(result_type=\"markdown\").load_data(\"./apple_2021_10k.pdf\")"
    ]

examples/demo_advanced_astradb.ipynb

@@ -22,7 +22,7 @@
     "%pip install llama-index-embeddings-openai\n",
     "%pip install llama-index-postprocessor-flag-embedding-reranker\n",
     "%pip install git+https://github.com/FlagOpen/FlagEmbedding.git\n",
-    "%pip install llama-cloud-services\n",
+    "%pip install llama-parse\n",
     "%pip install llama-index-vector-stores-astra-db"
    ]
   },
@@ -107,7 +107,7 @@
     }
    ],
    "source": [
-    "from llama_cloud_services import LlamaParse\n",
+    "from llama_parse import LlamaParse\n",
     "\n",
     "documents = LlamaParse(result_type=\"markdown\").load_data(\"./uber_10q_march_2022.pdf\")"
    ]

examples/demo_advanced_weaviate.ipynb

@@ -176,7 +176,7 @@
     }
    ],
    "source": [
-    "from llama_cloud_services import LlamaParse\n",
+    "from llama_parse import LlamaParse\n",
     "\n",
     "documents = LlamaParse(result_type=\"markdown\").load_data(\"./uber_10q_march_2022.pdf\")"
    ]

examples/demo_astradb.ipynb

@@ -130,7 +130,7 @@
     }
    ],
    "source": [
-    "from llama_cloud_services import LlamaParse\n",
+    "from llama_parse import LlamaParse\n",
     "\n",
     "documents = LlamaParse(result_type=\"text\").load_data(file_path)"
    ]

examples/demo_basic.ipynb

@@ -73,7 +73,7 @@
     }
    ],
    "source": [
-    "from llama_cloud_services import LlamaParse\n",
+    "from llama_parse import LlamaParse\n",
     "\n",
     "documents = LlamaParse(result_type=\"text\").load_data(\"./attention.pdf\")"
    ]
@@ -120,7 +120,7 @@
     }
    ],
    "source": [
-    "from llama_cloud_services import LlamaParse\n",
+    "from llama_parse import LlamaParse\n",
     "\n",
     "documents = LlamaParse(result_type=\"markdown\").load_data(\"./attention.pdf\")"
    ]

examples/demo_elasticsearch_vectordb.ipynb

@@ -142,7 +142,7 @@
     }
    ],
    "source": [
-    "from llama_cloud_services import LlamaParse\n",
+    "from llama_parse import LlamaParse\n",
     "\n",
     "documents = LlamaParse(result_type=\"text\").load_data(file_path)"
    ]

examples/demo_excel.ipynb

@@ -21,7 +21,7 @@
    "outputs": [],
    "source": [
     "%pip install llama-index\n",
-    "%pip install llama-cloud-services"
+    "%pip install llama-parse"
    ]
   },
   {
@@ -41,7 +41,7 @@
     "\n",
     "nest_asyncio.apply()\n",
     "\n",
-    "from llama_cloud_services import LlamaParse\n",
+    "from llama_parse import LlamaParse\n",
     "\n",
     "api_key = \"llx-\"  # get from cloud.llamaindex.ai"
    ]

examples/demo_insurance.ipynb

@@ -116,7 +116,7 @@
     }
    ],
    "source": [
-    "from llama_cloud_services import LlamaParse\n",
+    "from llama_parse import LlamaParse\n",
     "\n",
     "documents = LlamaParse(result_type=\"markdown\").load_data(\"./policy.pdf\")"
    ]

examples/demo_json.ipynb

@@ -35,7 +35,7 @@
     "!pip install llama-index-core\n",
     "!pip install llama-index-llms-anthropic llama-index-multi-modal-llms-anthropic\n",
     "!pip install llama-index-embeddings-huggingface\n",
-    "!pip install llama-cloud-services"
+    "!pip install llama-parse"
    ]
   },
   {
@@ -129,7 +129,7 @@
     }
    ],
    "source": [
-    "from llama_cloud_services import LlamaParse\n",
+    "from llama_parse import LlamaParse\n",
     "\n",
     "parser = LlamaParse(verbose=True)\n",
     "json_objs = parser.get_json_result(\"./uber_10q_march_2022.pdf\")\n",

examples/demo_json_parsing.ipynb

@@ -37,7 +37,7 @@
     "%pip install llama-index-core\n",
     "%pip install llama-index-llms-anthropic llama-index-multi-modal-llms-anthropic\n",
     "%pip install llama-index-embeddings-huggingface\n",
-    "%pip install llama-cloud-services"
+    "%pip install llama-parse"
    ]
   },
   {
@@ -110,7 +110,7 @@
     }
    ],
    "source": [
-    "from llama_cloud_services import LlamaParse\n",
+    "from llama_parse import LlamaParse\n",
     "\n",
     "parser = LlamaParse(verbose=True)\n",
     "json_objs = parser.get_json_result(\"./uber_10q_march_2022.pdf\")\n",

examples/demo_languages.ipynb

@@ -77,7 +77,7 @@
     }
    ],
    "source": [
-    "from llama_cloud_services import LlamaParse\n",
+    "from llama_parse import LlamaParse\n",
     "\n",
     "parser = LlamaParse(result_type=\"text\", language=\"fr\")\n",
     "documents = parser.load_data(\"./treasury_report.pdf\")"
@@ -250,7 +250,7 @@
     }
    ],
    "source": [
-    "from llama_cloud_services import LlamaParse\n",
+    "from llama_parse import LlamaParse\n",
     "\n",
     "parser = LlamaParse(result_type=\"text\", language=\"ch_sim\")\n",
     "documents = parser.load_data(\"./chinese_pdf.pdf\")"
@@ -404,7 +404,7 @@
     }
    ],
    "source": [
-    "from llama_cloud_services import LlamaParse\n",
+    "from llama_parse import LlamaParse\n",
     "\n",
     "base_parser = LlamaParse(result_type=\"text\", language=\"en\")\n",
     "base_documents = parser.load_data(\"./chinese_pdf2.pdf\")"

examples/demo_mongodb.ipynb

@@ -69,7 +69,7 @@
     "import pymongo\n",
     "\n",
     "from llama_index.vector_stores.mongodb import MongoDBAtlasVectorSearch\n",
-    "from llama_cloud_services import LlamaParse\n",
+    "from llama_parse import LlamaParse\n",
     "from llama_index.embeddings.openai import OpenAIEmbedding\n",
     "from llama_index.core import VectorStoreIndex, StorageContext\n",
     "from llama_index.core.node_parser import SimpleNodeParser"

examples/demo_parsing_instructions.ipynb

@@ -114,7 +114,7 @@
     }
    ],
    "source": [
-    "%pip install llama-cloud-services"
+    "%pip install llama-parse"
    ]
   },
   {
@@ -169,7 +169,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from llama_cloud_services import LlamaParse"
+    "from llama_parse import LlamaParse"
    ]
   },
   {

examples/demo_starter_multimodal.ipynb

@@ -0,0 +1,415 @@
+{
+  "cells": [
+    {
+      "cell_type": "markdown",
+      "id": "97c79c38-38a3-40f3-ba2e-250649347d63",
+      "metadata": {
+        "id": "97c79c38-38a3-40f3-ba2e-250649347d63"
+      },
+      "source": [
+        "<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/demo_starter_multimodal.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "id": "4e081457",
+      "metadata": {},
+      "source": [
+        "# Multimodal Parsing using LlamaParse\n",
+        "\n",
+        "This cookbook shows you how to use LlamaParse to parse any document with the multimodal capabilities of Multi-Modal LLMs from Anthropic/ OpenAI.\n",
+        "\n",
+        "LlamaParse allows you to plug in external, multimodal model vendors for parsing - we handle the error correction, validation, and scalability/reliability for you.\n"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "id": "qOdqBxCS51Ow",
+      "metadata": {
+        "id": "qOdqBxCS51Ow"
+      },
+      "source": [
+        "### Installation"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "id": "H_Vqcylb50vm",
+      "metadata": {
+        "id": "H_Vqcylb50vm"
+      },
+      "outputs": [],
+      "source": [
+        "!pip install llama-parse"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "id": "15e60ecf-519c-41fc-911b-765adaf8bad4",
+      "metadata": {
+        "id": "15e60ecf-519c-41fc-911b-765adaf8bad4"
+      },
+      "source": [
+        "### Setup\n",
+        "\n",
+        "Here we setup `LLAMA_CLOUD_API_KEY` for using `LlamaParse`."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": 1,
+      "id": "91a9e532-1454-40e0-bbf0-fd442c350121",
+      "metadata": {
+        "id": "91a9e532-1454-40e0-bbf0-fd442c350121"
+      },
+      "outputs": [],
+      "source": [
+        "import nest_asyncio\n",
+        "\n",
+        "nest_asyncio.apply()\n",
+        "\n",
+        "import os\n",
+        "\n",
+        "# API access to llama-cloud\n",
+        "os.environ[\"LLAMA_CLOUD_API_KEY\"] = \"<YOUR LLAMACLOUD API KEY>\""
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "id": "LGwBNPNotZRQ",
+      "metadata": {
+        "id": "LGwBNPNotZRQ"
+      },
+      "source": [
+        "## Download Data\n",
+        "\n",
+        "For this demonstration, we will use OpenAI's recent paper `Evaluation of OpenAI o1: Opportunities and Challenges of AGI`."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": 2,
+      "id": "IjtKDQRLrylI",
+      "metadata": {
+        "colab": {
+          "base_uri": "https://localhost:8080/"
+        },
+        "id": "IjtKDQRLrylI",
+        "outputId": "31df0fac-51f2-4697-f78b-0b7c0b8cd145"
+      },
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "--2024-12-05 18:54:24--  https://arxiv.org/pdf/2409.18486\n",
+            "Resolving arxiv.org (arxiv.org)... 151.101.67.42, 151.101.131.42, 151.101.3.42, ...\n",
+            "Connecting to arxiv.org (arxiv.org)|151.101.67.42|:443... connected.\n",
+            "HTTP request sent, awaiting response... 200 OK\n",
+            "Length: 13986265 (13M) [application/pdf]\n",
+            "Saving to: ‘o1.pdf’\n",
+            "\n",
+            "o1.pdf              100%[===================>]  13.34M  11.8MB/s    in 1.1s    \n",
+            "\n",
+            "2024-12-05 18:54:26 (11.8 MB/s) - ‘o1.pdf’ saved [13986265/13986265]\n",
+            "\n"
+          ]
+        }
+      ],
+      "source": [
+        "!wget \"https://arxiv.org/pdf/2409.18486\" -O \"o1.pdf\""
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "id": "4e29a9d7-5bd9-4fb8-8ec1-4c128a748662",
+      "metadata": {
+        "id": "4e29a9d7-5bd9-4fb8-8ec1-4c128a748662"
+      },
+      "source": [
+        "## Initialize LlamaParse\n",
+        "\n",
+        "Initialize LlamaParse in multimodal mode, and specify the vendor.\n",
+        "\n",
+        "**NOTE**: optionally you can specify the Anthropic/ OpenAI API key. If you choose to do so LlamaParse will only charge you 1 credit (0.3c) per page. \n",
+        "\n",
+        "\n",
+        "Using your own API key may incur additional costs from your model provider and could result in failed pages or documents if you do not have sufficient usage limits."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": 3,
+      "id": "dc921729-3446-42ca-8e1b-a6fd26195ed9",
+      "metadata": {
+        "id": "dc921729-3446-42ca-8e1b-a6fd26195ed9"
+      },
+      "outputs": [],
+      "source": [
+        "from llama_index.core.schema import TextNode\n",
+        "from typing import List\n",
+        "\n",
+        "def get_text_nodes(json_list: List[dict]):\n",
+        "    text_nodes = []\n",
+        "    for idx, page in enumerate(json_list):\n",
+        "        text_node = TextNode(text=page[\"md\"], metadata={\"page\": page[\"page\"]})\n",
+        "        text_nodes.append(text_node)\n",
+        "    return text_nodes"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "id": "1b5d6da6",
+      "metadata": {},
+      "source": [
+        "### With anthropic-sonnet-3.5"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": 5,
+      "id": "f2e9d9cf-8189-4fcb-b34f-cde6cc0b59c8",
+      "metadata": {
+        "colab": {
+          "base_uri": "https://localhost:8080/"
+        },
+        "id": "f2e9d9cf-8189-4fcb-b34f-cde6cc0b59c8",
+        "outputId": "a337cbdd-60db-4a73-b66b-2bd6159e81f2"
+      },
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "Started parsing the file under job_id dd9d5e0f-160e-486a-89a2-6005e5a1c2ac\n"
+          ]
+        }
+      ],
+      "source": [
+        "from llama_parse import LlamaParse\n",
+        "\n",
+        "parser = LlamaParse(\n",
+        "    result_type=\"markdown\",\n",
+        "    use_vendor_multimodal_model=True,\n",
+        "    vendor_multimodal_model_name=\"anthropic-sonnet-3.5\",\n",
+        "    target_pages=\"24\"\n",
+        "    # invalidate_cache=True\n",
+        ")\n",
+        "json_objs = parser.get_json_result(\"o1.pdf\")\n",
+        "json_list = json_objs[0][\"pages\"]\n",
+        "docs = get_text_nodes(json_list)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "id": "4f3c51b0-7878-48d7-9bc3-02b516500128",
+      "metadata": {
+        "id": "4f3c51b0-7878-48d7-9bc3-02b516500128"
+      },
+      "source": [
+        "### With GPT-4o\n",
+        "\n",
+        "For comparison, we will also parse the document using GPT-4o."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": 6,
+      "id": "6fc3f258-50ae-4988-b904-c105463a498f",
+      "metadata": {
+        "colab": {
+          "base_uri": "https://localhost:8080/"
+        },
+        "id": "6fc3f258-50ae-4988-b904-c105463a498f",
+        "outputId": "89c525c4-2b93-4909-9657-55646e034637"
+      },
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "Started parsing the file under job_id 6a4dea44-4f90-406b-b290-9e98620b1232\n"
+          ]
+        }
+      ],
+      "source": [
+        "from llama_parse import LlamaParse\n",
+        "\n",
+        "parser_gpt4o = LlamaParse(\n",
+        "    result_type=\"markdown\",\n",
+        "    use_vendor_multimodal_model=True,\n",
+        "    vendor_multimodal_model=\"openai-gpt4o\",\n",
+        "    target_pages=\"24\",\n",
+        "    # invalidate_cache=True\n",
+        ")\n",
+        "json_objs_gpt4o = parser_gpt4o.get_json_result(\"o1.pdf\")\n",
+        "json_list_gpt4o = json_objs_gpt4o[0][\"pages\"]\n",
+        "docs_gpt4o = get_text_nodes(json_list_gpt4o)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "id": "44c20f7a-2901-4dd0-b635-a4b33c5664c1",
+      "metadata": {
+        "id": "44c20f7a-2901-4dd0-b635-a4b33c5664c1"
+      },
+      "source": [
+        "### View Results\n",
+        "\n",
+        "Let's visualize the results along with the original document page."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": 7,
+      "id": "778698aa-da7e-4081-b3b5-0372f228536f",
+      "metadata": {
+        "colab": {
+          "base_uri": "https://localhost:8080/"
+        },
+        "id": "778698aa-da7e-4081-b3b5-0372f228536f",
+        "outputId": "bb89e323-7041-4fc3-d835-95e373189d02"
+      },
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "page: 25\n",
+            "\n",
+            "| Participant_ID | clinical Description Reference |\n",
+            "|-----------------|----------------------------------|\n",
+            "| Attribute | Value | Basic Personal Information: Subject 098_S_0896 is a 72.0-year-old Female who has completed 15 years of education. The ethnicity is Not Hisp/Latino and race is White. Marital status is Married. Initially diagnosed as AD, as of the date 2007-10-24, the final diagnosis was Dementia. |\n",
+            "| Age | 72.0 |\n",
+            "| Sex | Female |\n",
+            "| Education | 15 |\n",
+            "| Race | White | Biomarker Measurements: The subject's genetic profile includes an ApoE4 status of 0.0... |\n",
+            "| DX_bl | AD |\n",
+            "| DX | Dementia |\n",
+            "| ... | ... | Cognitive and Neurofunctional Assessments: The Mini-Mental State Examination score stands at 29.0. The Clinical Dementia Rating, sum of boxes, is 1.0. ADAS 11 and 13 scores are 4.67 and 4.67 respectively, with a score of 1.0 in delayed word recall... |\n",
+            "| APOE4 | 1.0 |\n",
+            "| TAU | 212.5 |\n",
+            "| ... | ... |\n",
+            "| MMSE | 29.0 | Volumetric Data: Under MRI conditions at a field strength of 1.5 Tesla MRI Tesla, using Cross Sectional FreeSurfer (FreeSurfer Version 4.3), the imaging data recorded includes ventricles volume at 54422.0, hippocampus volume at 6677.0, whole brain volume at 1147980.0, entorhinal cortex volume at 2782.0, fusiform gyrus volume at 19432.0, and middle temporal area volume at 24951.0. The intracranial volume measured is 1799580.0.... |\n",
+            "| CDRSB | 0.0 |\n",
+            "| ... | ... |\n",
+            "| FLDSTRENG | 1.5 Tesla MRI |\n",
+            "| Ventricles | 84599 |\n",
+            "| Hippocampus | 5319 |\n",
+            "| ... | ... |\n",
+            "\n",
+            "Figure 2: An example of a patient table and its corresponding clinical description.\n",
+            "\n",
+            "skills. Mathematics, as a highly structured and logic-driven discipline, provides an ideal testing ground for evaluating this reasoning ability. To investigate o1-preview's performance, we designed a series of tests covering various difficulty levels. We begin with high school-level math competition problems in this section, followed by college-level mathematics problems in the next section, allowing us to observe the model's logical reasoning across varying levels of complexity.\n",
+            "\n",
+            "In this section, we selected two primary areas of mathematics: algebra and counting and probability in this section. We chose these two topics because of their heavy reliance on problem-solving skills and their frequent use in assessing logical and abstract thinking [46]. The dataset used in testing is from the MATH dataset [46]. The problems in the dataset cover a wide range of subjects, including Prealgebra, Intermediate Algebra, Algebra, Geometry, Counting and Probability, Number Theory, and Precalculus. Each problem is categorized based on difficulty, ranked from level 1 to 5, according to the Art of Problem Solving (AoPS). The dataset mainly comprises problems from various high school math competitions, including the American Mathematics Competitions (AMC) 10 and 12, as well as the American Invitational Mathematics Examination (AIME), and other similar contests. Each problem comes with detailed reference solutions, allowing for a comprehensive comparison of o1-preview's solutions.\n",
+            "\n",
+            "In addition to evaluating the final answers produced by o1-preview, our analysis delves into the step-by-step reasoning process of the o1-preview's solutions. By comparing o1-preview's solutions with the dataset's solutions, we assess its ability to engage in logical reasoning, handle abstract problem-solving tasks, and apply structured approaches to reach correct answers. This deeper analysis offers insights into o1-preview's overall reasoning capabilities, using mathematics as a reliable indicator for logical and structured thought processes.\n"
+          ]
+        }
+      ],
+      "source": [
+        "# using Sonnet-3.5\n",
+        "print(docs[0].get_content(metadata_mode=\"all\"))"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": 8,
+      "id": "1511a30f-3efc-4142-9668-7dc056a24d0c",
+      "metadata": {
+        "colab": {
+          "base_uri": "https://localhost:8080/"
+        },
+        "id": "1511a30f-3efc-4142-9668-7dc056a24d0c",
+        "outputId": "2e5e8e20-2b41-4183-f21f-dff503a03089"
+      },
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "page: 25\n",
+            "\n",
+            "\n",
+            "| Participant_ID | clinical Description Reference |\n",
+            "|----------------|--------------------------------|\n",
+            "| **Attribute**  | **Value**                      |\n",
+            "| Age            | 72.0                           |\n",
+            "| Sex            | Female                         |\n",
+            "| Education      | 15                             |\n",
+            "| Race           | White                          |\n",
+            "| DX_bl          | AD                             |\n",
+            "| DX             | Dementia                       |\n",
+            "| ...            | ...                            |\n",
+            "| APOE4          | 1.0                            |\n",
+            "| TAU            | 212.5                          |\n",
+            "| ...            | ...                            |\n",
+            "| MMSE           | 29.0                           |\n",
+            "| CDRSB          | 0.0                            |\n",
+            "| ...            | ...                            |\n",
+            "| FLDSTRENG      | 1.5 Tesla MRI                  |\n",
+            "| Ventricles     | 84599                          |\n",
+            "| Hippocampus    | 5319                           |\n",
+            "| ...            | ...                            |\n",
+            "\n",
+            "**Basic Personal Information:** Subject 098_S_0896 is a 72.0-year-old Female who has completed 15 years of education. The ethnicity is Not Hisp/Latino and race is White. Marital status is Married. Initially diagnosed as AD, as of the date 2007-10-24, the final diagnosis was Dementia.\n",
+            "\n",
+            "**Biomarker Measurements:** The subject's genetic profile includes an ApoE4 status of 0.0...\n",
+            "\n",
+            "**Cognitive and Neurofunctional Assessments:** The Mini-Mental State Examination score stands at 29.0. The Clinical Dementia Rating, sum of boxes, is 1.0. ADAS 11 and 13 scores are 4.67 and 4.67 respectively, with a score of 1.0 in delayed word recall...\n",
+            "\n",
+            "**Volumetric Data:** Under MRI conditions at a field strength of 1.5 Tesla MRI Tesla, using Cross-Sectional FreeSurfer (FreeSurfer Version 4.3), the imaging data recorded includes ventricles volume at 84422.0, hippocampus volume at 6677.0, whole brain volume at 1147980.0, entorhinal cortex volume at 27820.0, fusiform gyrus volume at 19432.0, and middle temporal area volume at 24951.0. The intracranial volume measured is 1799580.0...\n",
+            "\n",
+            "Figure 2: An example of a patient table and its corresponding clinical description.\n",
+            "\n",
+            "----\n",
+            "\n",
+            "Skills. Mathematics, as a highly structured and logic-driven discipline, provides an ideal testing ground for evaluating this reasoning ability. To investigate o1-preview’s performance, we designed a series of tests covering various difficulty levels. We begin with high school-level math competition problems in this section, followed by college-level mathematics problems in the next section, allowing us to observe the model’s logical reasoning across varying levels of complexity.\n",
+            "\n",
+            "In this section, we selected two primary areas of mathematics: algebra and counting and probability in this section. We chose these two topics because of their heavy reliance on problem-solving skills and their frequent use in assessing logical and abstract thinking [46]. The dataset used in testing is from the MATH dataset [46]. The problems in the dataset cover a wide range of subjects, including Prealgebra, Intermediate Algebra, Algebra, Geometry, Counting and Probability, Number Theory, and Precalculus. Each problem is categorized based on difficulty, ranked from level 1 to 5, according to the Art of Problem Solving (AoPS). The dataset mainly comprises problems from various high school math competitions, including the American Mathematics Competitions (AMC) 10 and 12, as well as the American Invitational Mathematics Examination (AIME), and other similar contests. Each problem comes with detailed reference solutions, allowing for a comprehensive comparison of o1-preview’s solutions.\n",
+            "\n",
+            "In addition to evaluating the final answers produced by o1-preview, our analysis delves into the step-by-step reasoning process of the o1-preview’s solutions. By comparing o1-preview’s solutions with the dataset’s solutions, we assess its ability to engage in logical reasoning, handle abstract problem-solving tasks, and apply structured approaches to reach correct answers. This deeper analysis offers insights into o1-preview’s overall reasoning capabilities, using mathematics as a reliable indicator for logical and structured thought processes.\n"
+          ]
+        }
+      ],
+      "source": [
+        "# using GPT-4o\n",
+        "print(docs_gpt4o[0].get_content(metadata_mode=\"all\"))"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "id": "1c75bb85",
+      "metadata": {},
+      "outputs": [],
+      "source": []
+    }
+  ],
+  "metadata": {
+    "colab": {
+      "provenance": []
+    },
+    "kernelspec": {
+      "display_name": "llamacloud",
+      "language": "python",
+      "name": "llamacloud"
+    },
+    "language_info": {
+      "codemirror_mode": {
+        "name": "ipython",
+        "version": 3
+      },
+      "file_extension": ".py",
+      "mimetype": "text/x-python",
+      "name": "python",
+      "nbconvert_exporter": "python",
+      "pygments_lexer": "ipython3",
+      "version": "3.12.4"
+    }
+  },
+  "nbformat": 4,
+  "nbformat_minor": 5
+}

examples/demo_starter_parse_selected_pages.ipynb

@@ -31,7 +31,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "!pip install llama-cloud-services"
+    "!pip install llama-parse"
    ]
   },
   {
@@ -43,7 +43,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 1,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -69,7 +69,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 2,
    "metadata": {},
    "outputs": [
     {
@@ -105,7 +105,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 3,
    "metadata": {},
    "outputs": [
     {
@@ -117,16 +117,19 @@
     }
    ],
    "source": [
-    "from llama_cloud_services import LlamaParse\n",
+    "from llama_parse import LlamaParse\n",
     "\n",
-    "parser = LlamaParse(target_pages=\"0,1,2\", result_type=\"markdown\")\n",
+    "parser = LlamaParse(\n",
+    "  target_pages=\"0,1,2\",\n",
+    "  result_type=\"markdown\"\n",
+    ")\n",
     "\n",
-    "documents = parser.load_data(\"./uber_2021.pdf\")"
+    "documents = parser.load_data('./uber_2021.pdf')"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 4,
    "metadata": {},
    "outputs": [
     {
@@ -137,7 +140,7 @@
        " Document(id_='ad988239-3ab5-498d-85ba-a29241db24d4', embedding=None, metadata={}, excluded_embed_metadata_keys=[], excluded_llm_metadata_keys=[], relationships={}, metadata_template='{key}: {value}', metadata_separator='\\n', text='# UBER TECHNOLOGIES, INC.\\n\\n# TABLE OF CONTENTS\\n\\n|Special Note Regarding Forward-Looking Statements|2|\\n|---|---|\\n|PART I|PART I|\\n|Item 1. Business|4|\\n|Item 1A. Risk Factors|11|\\n|Item 1B. Unresolved Staff Comments|46|\\n|Item 2. Properties|46|\\n|Item 3. Legal Proceedings|46|\\n|Item 4. Mine Safety Disclosures|47|\\n|PART II|PART II|\\n|Item 5. Market for Registrant’s Common Equity, Related Stockholder Matters and Issuer Purchases of Equity Securities|47|\\n|Item 6. [Reserved]|48|\\n|Item 7. Management’s Discussion and Analysis of Financial Condition and Results of Operations|48|\\n|Item 7A. Quantitative and Qualitative Disclosures About Market Risk|69|\\n|Item 8. Financial Statements and Supplementary Data|70|\\n|Item 9. Changes in and Disagreements with Accountants on Accounting and Financial Disclosure|146|\\n|Item 9A. Controls and Procedures|147|\\n|Item 9B. Other Information|147|\\n|Item 9C. Disclosure Regarding Foreign Jurisdictions that Prevent Inspections|147|\\n|PART III|PART III|\\n|Item 10. Directors, Executive Officers and Corporate Governance|147|\\n|Item 11. Executive Compensation|147|\\n|Item 12. Security Ownership of Certain Beneficial Owners and Management and Related Stockholder Matters|148|\\n|Item 13. Certain Relationships and Related Transactions, and Director Independence|148|\\n|Item 14. Principal Accounting Fees and Services|148|\\n|PART IV|PART IV|\\n|Item 15. Exhibits, Financial Statement Schedules|148|\\n|Item 16. Form 10-K Summary|148|\\n|Exhibit Index|149|\\n|Signatures|152|', mimetype='text/plain', start_char_idx=None, end_char_idx=None, metadata_seperator='\\n', text_template='{metadata_str}\\n\\n{content}')]"
       ]
      },
-     "execution_count": null,
+     "execution_count": 4,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -145,6 +148,13 @@
    "source": [
     "documents"
    ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
   }
  ],
  "metadata": {
@@ -162,7 +172,8 @@
    "mimetype": "text/x-python",
    "name": "python",
    "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3"
+   "pygments_lexer": "ipython3",
+   "version": "3.12.4"
   }
  },
  "nbformat": 4,

examples/demo_table_comparisons.ipynb

@@ -34,7 +34,7 @@
     "%pip install llama-index-question-gen-openai\n",
     "%pip install llama-index-postprocessor-flag-embedding-reranker\n",
     "%pip install git+https://github.com/FlagOpen/FlagEmbedding.git\n",
-    "%pip install llama-cloud-services"
+    "%pip install llama-parse"
    ]
   },
   {
@@ -109,7 +109,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from llama_cloud_services import LlamaParse\n",
+    "from llama_parse import LlamaParse\n",
     "\n",
     "docs_2021 = LlamaParse(result_type=\"markdown\").load_data(\"./apple_2021_10k.pdf\")\n",
     "docs_2020 = LlamaParse(result_type=\"markdown\").load_data(\"./apple_2020_10k.pdf\")"

examples/excel/dcf_rag.ipynb

@@ -31,7 +31,7 @@
    "outputs": [],
    "source": [
     "%pip install llama-index\n",
-    "%pip install llama-cloud-services"
+    "%pip install llama-parse"
    ]
   },
   {
@@ -53,7 +53,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from llama_cloud_services import LlamaParse\n",
+    "from llama_parse import LlamaParse\n",
     "\n",
     "# api_key = \"llx-\"  # get from cloud.llamaindex.ai"
    ]

examples/excel/o1_excel_rag.ipynb

@@ -37,7 +37,7 @@
    "outputs": [],
    "source": [
     "# !pip install llama-index\n",
-    "# !pip install llama-cloud-services"
+    "# !pip install llama-parse"
    ]
   },
   {
@@ -59,7 +59,7 @@
     "from llama_index.core import VectorStoreIndex\n",
     "from IPython.display import Image, Markdown\n",
     "\n",
-    "from llama_cloud_services import LlamaParse\n",
+    "from llama_parse import LlamaParse\n",
     "\n",
     "from llama_index.core.node_parser import MarkdownElementNodeParser"
    ]

examples/knowledge_graphs/kg_agent.ipynb

@@ -33,7 +33,7 @@
     "!pip install llama-index-postprocessor-flag-embedding-reranker\n",
     "!pip install git+https://github.com/FlagOpen/FlagEmbedding.git\n",
     "!pip install llama-index-graph-stores-neo4j\n",
-    "!pip install llama-cloud-services"
+    "!pip install llama-parse"
    ]
   },
   {
@@ -125,7 +125,7 @@
     }
    ],
    "source": [
-    "from llama_cloud_services import LlamaParse\n",
+    "from llama_parse import LlamaParse\n",
     "\n",
     "docs = LlamaParse(result_type=\"text\").load_data(\"./data/budget_2023.pdf\")"
    ]

examples/multimodal/claude_parse.ipynb

@@ -141,7 +141,7 @@
     }
    ],
    "source": [
-    "from llama_cloud_services import LlamaParse\n",
+    "from llama_parse import LlamaParse\n",
     "\n",
     "parser = LlamaParse(\n",
     "    result_type=\"markdown\",\n",
@@ -205,7 +205,7 @@
     }
    ],
    "source": [
-    "from llama_cloud_services import LlamaParse\n",
+    "from llama_parse import LlamaParse\n",
     "\n",
     "parser_gpt4o = LlamaParse(\n",
     "    result_type=\"markdown\",\n",

examples/multimodal/gpt4o_mini.ipynb

@@ -118,7 +118,7 @@
     }
    ],
    "source": [
-    "from llama_cloud_services import LlamaParse\n",
+    "from llama_parse import LlamaParse\n",
     "\n",
     "parser = LlamaParse(\n",
     "    result_type=\"markdown\",\n",
@@ -181,7 +181,7 @@
     }
    ],
    "source": [
-    "from llama_cloud_services import LlamaParse\n",
+    "from llama_parse import LlamaParse\n",
     "\n",
     "parser_gpt4o = LlamaParse(\n",
     "    result_type=\"markdown\",\n",

examples/multimodal/insurance_rag.ipynb

@@ -99,7 +99,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from llama_cloud_services import LlamaParse\n",
+    "from llama_parse import LlamaParse\n",
     "\n",
     "parser = LlamaParse(\n",
     "    result_type=\"markdown\",\n",

examples/multimodal/legal_rag.ipynb

@@ -102,7 +102,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from llama_cloud_services import LlamaParse\n",
+    "from llama_parse import LlamaParse\n",
     "\n",
     "parser = LlamaParse(\n",
     "    result_type=\"markdown\",\n",

examples/multimodal/multimodal_contextual_retrieval_rag.ipynb

@@ -169,7 +169,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from llama_cloud_services import LlamaParse\n",
+    "from llama_parse import LlamaParse\n",
     "\n",
     "\n",
     "parser = LlamaParse(\n",

examples/multimodal/multimodal_rag_slide_deck.ipynb

@@ -153,7 +153,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from llama_cloud_services import LlamaParse\n",
+    "from llama_parse import LlamaParse\n",
     "\n",
     "\n",
     "parser_text = LlamaParse(result_type=\"text\")\n",

examples/multimodal/multimodal_report_generation.ipynb

@@ -143,7 +143,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from llama_cloud_services import LlamaParse\n",
+    "from llama_parse import LlamaParse\n",
     "\n",
     "parser = LlamaParse(\n",
     "    result_type=\"markdown\",\n",

examples/multimodal/multimodal_report_generation_agent.ipynb

@@ -172,7 +172,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from llama_cloud_services import LlamaParse\n",
+    "from llama_parse import LlamaParse\n",
     "\n",
     "parser = LlamaParse(\n",
     "    result_type=\"markdown\",\n",

examples/multimodal/product_manual_rag.ipynb

@@ -104,7 +104,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from llama_cloud_services import LlamaParse\n",
+    "from llama_parse import LlamaParse\n",
     "\n",
     "parser = LlamaParse(\n",
     "    result_type=\"markdown\",\n",

examples/other_files/demo_ppt_basic.ipynb

@@ -25,7 +25,7 @@
     "\n",
     "nest_asyncio.apply()\n",
     "\n",
-    "from llama_cloud_services import LlamaParse"
+    "from llama_parse import LlamaParse"
    ]
   },
   {

examples/other_files/demo_ppt_financial.ipynb

@@ -27,7 +27,7 @@
    "outputs": [],
    "source": [
     "%pip install llama-index\n",
-    "%pip install llama-cloud-services\n",
+    "%pip install llama-parse\n",
     "%pip install torch transformers python-pptx Pillow"
    ]
   },
@@ -85,7 +85,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from llama_cloud_services import LlamaParse"
+    "from llama_parse import LlamaParse"
    ]
   },
   {

examples/parse/demo_starter_multimodal.ipynb

@@ -1,357 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "id": "97c79c38-38a3-40f3-ba2e-250649347d63",
-   "metadata": {},
-   "source": [
-    "<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/demo_starter_multimodal.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "4e081457",
-   "metadata": {},
-   "source": [
-    "# Multimodal Parsing using LlamaParse\n",
-    "\n",
-    "This cookbook shows you how to use LlamaParse to parse any document with the multimodal capabilities of Multi-Modal LLMs from Anthropic/ OpenAI.\n",
-    "\n",
-    "LlamaParse allows you to plug in external, multimodal model vendors for parsing - we handle the error correction, validation, and scalability/reliability for you.\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "qOdqBxCS51Ow",
-   "metadata": {},
-   "source": [
-    "### Installation"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "H_Vqcylb50vm",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "!pip install llama-cloud-services"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "15e60ecf-519c-41fc-911b-765adaf8bad4",
-   "metadata": {},
-   "source": [
-    "### Setup\n",
-    "\n",
-    "Here we setup `LLAMA_CLOUD_API_KEY` for using `LlamaParse`."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "91a9e532-1454-40e0-bbf0-fd442c350121",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "import nest_asyncio\n",
-    "\n",
-    "nest_asyncio.apply()\n",
-    "\n",
-    "import os\n",
-    "\n",
-    "# API access to llama-cloud\n",
-    "os.environ[\"LLAMA_CLOUD_API_KEY\"] = \"<YOUR LLAMACLOUD API KEY>\""
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "LGwBNPNotZRQ",
-   "metadata": {},
-   "source": [
-    "## Download Data\n",
-    "\n",
-    "For this demonstration, we will use OpenAI's recent paper `Evaluation of OpenAI o1: Opportunities and Challenges of AGI`."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "IjtKDQRLrylI",
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "--2024-12-05 18:54:24--  https://arxiv.org/pdf/2409.18486\n",
-      "Resolving arxiv.org (arxiv.org)... 151.101.67.42, 151.101.131.42, 151.101.3.42, ...\n",
-      "Connecting to arxiv.org (arxiv.org)|151.101.67.42|:443... connected.\n",
-      "HTTP request sent, awaiting response... 200 OK\n",
-      "Length: 13986265 (13M) [application/pdf]\n",
-      "Saving to: ‘o1.pdf’\n",
-      "\n",
-      "o1.pdf              100%[===================>]  13.34M  11.8MB/s    in 1.1s    \n",
-      "\n",
-      "2024-12-05 18:54:26 (11.8 MB/s) - ‘o1.pdf’ saved [13986265/13986265]\n",
-      "\n"
-     ]
-    }
-   ],
-   "source": [
-    "!wget \"https://arxiv.org/pdf/2409.18486\" -O \"o1.pdf\""
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "4e29a9d7-5bd9-4fb8-8ec1-4c128a748662",
-   "metadata": {},
-   "source": [
-    "## Initialize LlamaParse\n",
-    "\n",
-    "Initialize LlamaParse in multimodal mode, and specify the vendor.\n",
-    "\n",
-    "**NOTE**: optionally you can specify the Anthropic/ OpenAI API key. If you choose to do so LlamaParse will only charge you 1 credit (0.3c) per page. \n",
-    "\n",
-    "\n",
-    "Using your own API key may incur additional costs from your model provider and could result in failed pages or documents if you do not have sufficient usage limits."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "dc921729-3446-42ca-8e1b-a6fd26195ed9",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from llama_index.core.schema import TextNode\n",
-    "from typing import List\n",
-    "\n",
-    "\n",
-    "def get_text_nodes(json_list: List[dict]):\n",
-    "    text_nodes = []\n",
-    "    for idx, page in enumerate(json_list):\n",
-    "        text_node = TextNode(text=page[\"md\"], metadata={\"page\": page[\"page\"]})\n",
-    "        text_nodes.append(text_node)\n",
-    "    return text_nodes"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "1b5d6da6",
-   "metadata": {},
-   "source": [
-    "### With anthropic-sonnet-3.5"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "f2e9d9cf-8189-4fcb-b34f-cde6cc0b59c8",
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "Started parsing the file under job_id dd9d5e0f-160e-486a-89a2-6005e5a1c2ac\n"
-     ]
-    }
-   ],
-   "source": [
-    "from llama_cloud_services import LlamaParse\n",
-    "\n",
-    "parser = LlamaParse(\n",
-    "    result_type=\"markdown\",\n",
-    "    use_vendor_multimodal_model=True,\n",
-    "    vendor_multimodal_model_name=\"anthropic-sonnet-3.5\",\n",
-    "    target_pages=\"24\"\n",
-    "    # invalidate_cache=True\n",
-    ")\n",
-    "json_objs = parser.get_json_result(\"o1.pdf\")\n",
-    "json_list = json_objs[0][\"pages\"]\n",
-    "docs = get_text_nodes(json_list)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "4f3c51b0-7878-48d7-9bc3-02b516500128",
-   "metadata": {},
-   "source": [
-    "### With GPT-4o\n",
-    "\n",
-    "For comparison, we will also parse the document using GPT-4o."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "6fc3f258-50ae-4988-b904-c105463a498f",
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "Started parsing the file under job_id 6a4dea44-4f90-406b-b290-9e98620b1232\n"
-     ]
-    }
-   ],
-   "source": [
-    "from llama_cloud_services import LlamaParse\n",
-    "\n",
-    "parser_gpt4o = LlamaParse(\n",
-    "    result_type=\"markdown\",\n",
-    "    use_vendor_multimodal_model=True,\n",
-    "    vendor_multimodal_model=\"openai-gpt4o\",\n",
-    "    target_pages=\"24\",\n",
-    "    # invalidate_cache=True\n",
-    ")\n",
-    "json_objs_gpt4o = parser_gpt4o.get_json_result(\"o1.pdf\")\n",
-    "json_list_gpt4o = json_objs_gpt4o[0][\"pages\"]\n",
-    "docs_gpt4o = get_text_nodes(json_list_gpt4o)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "44c20f7a-2901-4dd0-b635-a4b33c5664c1",
-   "metadata": {},
-   "source": [
-    "### View Results\n",
-    "\n",
-    "Let's visualize the results along with the original document page."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "778698aa-da7e-4081-b3b5-0372f228536f",
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "page: 25\n",
-      "\n",
-      "| Participant_ID | clinical Description Reference |\n",
-      "|-----------------|----------------------------------|\n",
-      "| Attribute | Value | Basic Personal Information: Subject 098_S_0896 is a 72.0-year-old Female who has completed 15 years of education. The ethnicity is Not Hisp/Latino and race is White. Marital status is Married. Initially diagnosed as AD, as of the date 2007-10-24, the final diagnosis was Dementia. |\n",
-      "| Age | 72.0 |\n",
-      "| Sex | Female |\n",
-      "| Education | 15 |\n",
-      "| Race | White | Biomarker Measurements: The subject's genetic profile includes an ApoE4 status of 0.0... |\n",
-      "| DX_bl | AD |\n",
-      "| DX | Dementia |\n",
-      "| ... | ... | Cognitive and Neurofunctional Assessments: The Mini-Mental State Examination score stands at 29.0. The Clinical Dementia Rating, sum of boxes, is 1.0. ADAS 11 and 13 scores are 4.67 and 4.67 respectively, with a score of 1.0 in delayed word recall... |\n",
-      "| APOE4 | 1.0 |\n",
-      "| TAU | 212.5 |\n",
-      "| ... | ... |\n",
-      "| MMSE | 29.0 | Volumetric Data: Under MRI conditions at a field strength of 1.5 Tesla MRI Tesla, using Cross Sectional FreeSurfer (FreeSurfer Version 4.3), the imaging data recorded includes ventricles volume at 54422.0, hippocampus volume at 6677.0, whole brain volume at 1147980.0, entorhinal cortex volume at 2782.0, fusiform gyrus volume at 19432.0, and middle temporal area volume at 24951.0. The intracranial volume measured is 1799580.0.... |\n",
-      "| CDRSB | 0.0 |\n",
-      "| ... | ... |\n",
-      "| FLDSTRENG | 1.5 Tesla MRI |\n",
-      "| Ventricles | 84599 |\n",
-      "| Hippocampus | 5319 |\n",
-      "| ... | ... |\n",
-      "\n",
-      "Figure 2: An example of a patient table and its corresponding clinical description.\n",
-      "\n",
-      "skills. Mathematics, as a highly structured and logic-driven discipline, provides an ideal testing ground for evaluating this reasoning ability. To investigate o1-preview's performance, we designed a series of tests covering various difficulty levels. We begin with high school-level math competition problems in this section, followed by college-level mathematics problems in the next section, allowing us to observe the model's logical reasoning across varying levels of complexity.\n",
-      "\n",
-      "In this section, we selected two primary areas of mathematics: algebra and counting and probability in this section. We chose these two topics because of their heavy reliance on problem-solving skills and their frequent use in assessing logical and abstract thinking [46]. The dataset used in testing is from the MATH dataset [46]. The problems in the dataset cover a wide range of subjects, including Prealgebra, Intermediate Algebra, Algebra, Geometry, Counting and Probability, Number Theory, and Precalculus. Each problem is categorized based on difficulty, ranked from level 1 to 5, according to the Art of Problem Solving (AoPS). The dataset mainly comprises problems from various high school math competitions, including the American Mathematics Competitions (AMC) 10 and 12, as well as the American Invitational Mathematics Examination (AIME), and other similar contests. Each problem comes with detailed reference solutions, allowing for a comprehensive comparison of o1-preview's solutions.\n",
-      "\n",
-      "In addition to evaluating the final answers produced by o1-preview, our analysis delves into the step-by-step reasoning process of the o1-preview's solutions. By comparing o1-preview's solutions with the dataset's solutions, we assess its ability to engage in logical reasoning, handle abstract problem-solving tasks, and apply structured approaches to reach correct answers. This deeper analysis offers insights into o1-preview's overall reasoning capabilities, using mathematics as a reliable indicator for logical and structured thought processes.\n"
-     ]
-    }
-   ],
-   "source": [
-    "# using Sonnet-3.5\n",
-    "print(docs[0].get_content(metadata_mode=\"all\"))"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "1511a30f-3efc-4142-9668-7dc056a24d0c",
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "page: 25\n",
-      "\n",
-      "\n",
-      "| Participant_ID | clinical Description Reference |\n",
-      "|----------------|--------------------------------|\n",
-      "| **Attribute**  | **Value**                      |\n",
-      "| Age            | 72.0                           |\n",
-      "| Sex            | Female                         |\n",
-      "| Education      | 15                             |\n",
-      "| Race           | White                          |\n",
-      "| DX_bl          | AD                             |\n",
-      "| DX             | Dementia                       |\n",
-      "| ...            | ...                            |\n",
-      "| APOE4          | 1.0                            |\n",
-      "| TAU            | 212.5                          |\n",
-      "| ...            | ...                            |\n",
-      "| MMSE           | 29.0                           |\n",
-      "| CDRSB          | 0.0                            |\n",
-      "| ...            | ...                            |\n",
-      "| FLDSTRENG      | 1.5 Tesla MRI                  |\n",
-      "| Ventricles     | 84599                          |\n",
-      "| Hippocampus    | 5319                           |\n",
-      "| ...            | ...                            |\n",
-      "\n",
-      "**Basic Personal Information:** Subject 098_S_0896 is a 72.0-year-old Female who has completed 15 years of education. The ethnicity is Not Hisp/Latino and race is White. Marital status is Married. Initially diagnosed as AD, as of the date 2007-10-24, the final diagnosis was Dementia.\n",
-      "\n",
-      "**Biomarker Measurements:** The subject's genetic profile includes an ApoE4 status of 0.0...\n",
-      "\n",
-      "**Cognitive and Neurofunctional Assessments:** The Mini-Mental State Examination score stands at 29.0. The Clinical Dementia Rating, sum of boxes, is 1.0. ADAS 11 and 13 scores are 4.67 and 4.67 respectively, with a score of 1.0 in delayed word recall...\n",
-      "\n",
-      "**Volumetric Data:** Under MRI conditions at a field strength of 1.5 Tesla MRI Tesla, using Cross-Sectional FreeSurfer (FreeSurfer Version 4.3), the imaging data recorded includes ventricles volume at 84422.0, hippocampus volume at 6677.0, whole brain volume at 1147980.0, entorhinal cortex volume at 27820.0, fusiform gyrus volume at 19432.0, and middle temporal area volume at 24951.0. The intracranial volume measured is 1799580.0...\n",
-      "\n",
-      "Figure 2: An example of a patient table and its corresponding clinical description.\n",
-      "\n",
-      "----\n",
-      "\n",
-      "Skills. Mathematics, as a highly structured and logic-driven discipline, provides an ideal testing ground for evaluating this reasoning ability. To investigate o1-preview’s performance, we designed a series of tests covering various difficulty levels. We begin with high school-level math competition problems in this section, followed by college-level mathematics problems in the next section, allowing us to observe the model’s logical reasoning across varying levels of complexity.\n",
-      "\n",
-      "In this section, we selected two primary areas of mathematics: algebra and counting and probability in this section. We chose these two topics because of their heavy reliance on problem-solving skills and their frequent use in assessing logical and abstract thinking [46]. The dataset used in testing is from the MATH dataset [46]. The problems in the dataset cover a wide range of subjects, including Prealgebra, Intermediate Algebra, Algebra, Geometry, Counting and Probability, Number Theory, and Precalculus. Each problem is categorized based on difficulty, ranked from level 1 to 5, according to the Art of Problem Solving (AoPS). The dataset mainly comprises problems from various high school math competitions, including the American Mathematics Competitions (AMC) 10 and 12, as well as the American Invitational Mathematics Examination (AIME), and other similar contests. Each problem comes with detailed reference solutions, allowing for a comprehensive comparison of o1-preview’s solutions.\n",
-      "\n",
-      "In addition to evaluating the final answers produced by o1-preview, our analysis delves into the step-by-step reasoning process of the o1-preview’s solutions. By comparing o1-preview’s solutions with the dataset’s solutions, we assess its ability to engage in logical reasoning, handle abstract problem-solving tasks, and apply structured approaches to reach correct answers. This deeper analysis offers insights into o1-preview’s overall reasoning capabilities, using mathematics as a reliable indicator for logical and structured thought processes.\n"
-     ]
-    }
-   ],
-   "source": [
-    "# using GPT-4o\n",
-    "print(docs_gpt4o[0].get_content(metadata_mode=\"all\"))"
-   ]
-  }
- ],
- "metadata": {
-  "colab": {
-   "provenance": []
-  },
-  "kernelspec": {
-   "display_name": "llamacloud",
-   "language": "python",
-   "name": "llamacloud"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 5
-}

examples/parse/parsing_instructions/parsing_instructions.ipynb

@@ -1,602 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/parsing_instructions.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
-    "\n",
-    "# Parsing documents with Instructions\n",
-    "\n",
-    "Parsing instructions allow you to guide our parsing model in the same way you would instruct an LLM.\n",
-    "\n",
-    "These instructions can be useful for improving the parser's performance on complex document layouts, extracting data in a specific format, or transforming the document in other ways.\n",
-    "\n",
-    "### Why This Matters:\n",
-    "Traditional document parsing can be rigid and error-prone, often missing crucial context and nuances in complex layouts. Our instruction-based parsing allows you to:\n",
-    "\n",
-    "1. Extract specific information with pinpoint accuracy\n",
-    "2. Handle complex document layouts with ease\n",
-    "3. Transform unstructured data into structured formats effortlessly\n",
-    "4. Save hours of manual data entry and verification\n",
-    "5. Reduce errors in document processing workflows\n",
-    "\n",
-    "In this demonstration, we showcase how parsing instructions can be used to extract specific information from unstructured documents. Below are the documents we use for testing:\n",
-    "\n",
-    "1. McDonald's Receipt - Extracting the price of each order and the final amount to be paid.\n",
-    "\n",
-    "2. Expense Report Document - Extracting employee name, employee ID, position, department, date ranges, individual expense items with dates, categories, and amounts.\n",
-    "\n",
-    "3. Purchase Order Document - Identifying the PO number, vendor details, shipping terms, and an itemized list of products with quantities and unit prices.\n",
-    "\n",
-    "Let's jump into these real-world examples and see how parsing instructions can help us extract specific information."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "### Installation"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "!pip install llama-cloud-services"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "### Setup API Key"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "import nest_asyncio\n",
-    "\n",
-    "nest_asyncio.apply()\n",
-    "\n",
-    "import os\n",
-    "\n",
-    "os.environ[\"LLAMA_CLOUD_API_KEY\"] = \"llx-...\""
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "### McDonald's Receipt\n",
-    "\n",
-    "Here we extract the price of each order and the final amount to be paid."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "<img src=\"mcdonalds_receipt.png\" alt=\"Alt Text\" width=\"500\">"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "Started parsing the file under job_id 66643b81-e2f4-408b-890b-8e116472210b\n"
-     ]
-    }
-   ],
-   "source": [
-    "from llama_cloud_services import LlamaParse\n",
-    "\n",
-    "vanilaParsing = LlamaParse(result_type=\"markdown\").load_data(\"./mcdonalds_receipt.png\")"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "# Rate us HIGHLY SATISFIED\n",
-      "\n",
-      "Purchase any sandwich and receive a FREE ITEM\n",
-      "\n",
-      "Go to WWW.mcdvoice.com within 7 days of purchase of equal or lesser value and tell us about your visit.\n",
-      "\n",
-      "Validation Code: 31278-01121-21018-20481-00081-0\n",
-      "\n",
-      "Valid at participating US McDonald's\n",
-      "\n",
-      "Expires 30 days after receipt date\n",
-      "\n",
-      "# McDonald's Restaurant #312782378\n",
-      "\n",
-      "PINE RD NW\n",
-      "\n",
-      "RICE MN 56367-9740\n",
-      "\n",
-      "TEL# 320 393 4600\n",
-      "\n",
-      "KS# 12/08/2022 08:48 PM\n",
-      "\n",
-      "# Order\n",
-      "\n",
-      "|Happy Meal 6 Pc|$4.89|\n",
-      "|---|---|\n",
-      "|Creamy Ranch Cup| |\n",
-      "|Extra Kids Fry| |\n",
-      "|Wreck It Ralph 2 Snack| |\n",
-      "|Oreo McFlurry|$2.69|\n",
-      "\n",
-      "# Summary\n",
-      "\n",
-      "|Subtotal|$7.58|\n",
-      "|---|---|\n",
-      "|Tax|$0.52|\n",
-      "|Take-Out Total|$8.10|\n",
-      "|Cash Tendered|$10.00|\n",
-      "|Change|$1.90|\n",
-      "\n",
-      "### Not ACCEPTING APPLICATIONS *++ McDonald's Restaurant Rice\n",
-      "\n",
-      "Text to #36453 apply 31278\n"
-     ]
-    }
-   ],
-   "source": [
-    "print(vanilaParsing[0].text)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "Started parsing the file under job_id 1a04fdbb-5415-4a36-a1bd-26bfb5d618fa\n"
-     ]
-    }
-   ],
-   "source": [
-    "parsingInstruction = \"\"\"The provided document is a McDonald's receipt.\n",
-    " Provide the price of each order and final amount to be paid.\"\"\"\n",
-    "withInstructionParsing = LlamaParse(\n",
-    "    result_type=\"markdown\", parsing_instruction=parsingInstruction\n",
-    ").load_data(\"./mcdonalds_receipt.png\")"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "Here are the prices for each order from the McDonald's receipt:\n",
-      "\n",
-      "1. Happy Meal 6 Pc: $4.89\n",
-      "2. Snack Oreo McFlurry: $2.69\n",
-      "\n",
-      "**Subtotal:** $7.58\n",
-      "**Tax:** $0.52\n",
-      "**Total Amount to be Paid:** $8.10\n",
-      "\n",
-      "The cash tendered was $10.00, and the change given was $1.90.\n"
-     ]
-    }
-   ],
-   "source": [
-    "print(withInstructionParsing[0].text)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "### Expense Report Document\n",
-    "\n",
-    "Here we extract employee name, employee ID, position, department, date ranges, individual expense items with dates, categories, and amounts."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "<img src=\"expense_report_document.png\" alt=\"Alt Text\" width=\"500\">"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "Started parsing the file under job_id b6bcc6e1-7d30-4522-9abd-ace196781a70\n"
-     ]
-    }
-   ],
-   "source": [
-    "vanilaParsing = LlamaParse(result_type=\"markdown\").load_data(\n",
-    "    \"./expense_report_document.pdf\"\n",
-    ")"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "# QUANTUM DYNAMICS CORPORATION\n",
-      "\n",
-      "# EMPLOYEE EXPENSE REPORT\n",
-      "\n",
-      "# FISCAL YEAR 2024\n",
-      "\n",
-      "# EMPLOYEE INFORMATION:\n",
-      "\n",
-      "Name: Dr. Alexandra Chen-Martinez, PhD\n",
-      "\n",
-      "Employee ID: QD-2022-1457\n",
-      "\n",
-      "Department: Advanced Research & Development\n",
-      "\n",
-      "Cost Center: CC-ARD-NA-003\n",
-      "\n",
-      "Project Codes: QD-QUANTUM-2024-01, QD-AI-2024-03\n",
-      "\n",
-      "Position: Principal Research Scientist\n",
-      "\n",
-      "Reporting Manager: Dr. James Thompson\n",
-      "\n",
-      "# TRIP/EXPENSE PERIOD:\n",
-      "\n",
-      "Start Date: November 15, 2024\n",
-      "\n",
-      "End Date: December 10, 2024\n",
-      "\n",
-      "Purpose: International Conference Attendance & Client Meetings\n",
-      "\n",
-      "Locations: Tokyo, Japan → Singapore → Sydney, Australia\n",
-      "\n",
-      "# CURRENCY CONVERSION RATES APPLIED:\n",
-      "\n",
-      "JPY (¥) → USD: 0.0068 (as of 11/15/2024)\n",
-      "\n",
-      "SGD (S$) → USD: 0.74 (as of 11/28/2024)\n",
-      "\n",
-      "AUD (A$) → USD: 0.65 (as of 12/03/2024)\n",
-      "\n",
-      "# ITEMIZED EXPENSES:\n",
-      "\n",
-      "|Date|Category|Description|Original|Currency|USD|\n",
-      "|---|---|---|---|---|---|\n",
-      "|11/15/2024|Transportation|JFK → NRT Business Class|4,250.00|USD|4,250.00|\n",
-      "|Booking Ref: QF78956 - Corporate Rate Applied|Booking Ref: QF78956 - Corporate Rate Applied|Booking Ref: QF78956 - Corporate Rate Applied|Booking Ref: QF78956 - Corporate Rate Applied|Booking Ref: QF78956 - Corporate Rate Applied|Booking Ref: QF78956 - Corporate Rate Applied|\n",
-      "|Project Code: QD-QUANTUM-2024-01|Project Code: QD-QUANTUM-2024-01|Project Code: QD-QUANTUM-2024-01|Project Code: QD-QUANTUM-2024-01|Project Code: QD-QUANTUM-2024-01|Project Code: QD-QUANTUM-2024-01|\n",
-      "|11/16/2024|Accommodation|Hilton Tokyo - 5 nights|225,000|JPY|1,530.00|\n",
-      "|Confirmation: HTK-2024-78956|Confirmation: HTK-2024-78956|Confirmation: HTK-2024-78956|Confirmation: HTK-2024-78956|Confirmation: HTK-2024-78956|Confirmation: HTK-2024-78956|\n"
-     ]
-    }
-   ],
-   "source": [
-    "print(vanilaParsing[0].text)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "Started parsing the file under job_id 7b0d05bb-947b-4475-8d0f-f10386f7446e\n"
-     ]
-    }
-   ],
-   "source": [
-    "parsingInstruction = \"\"\"You are provided with an expense report. \n",
-    "Extract employee name, employee id, position, department, date ranges, individual expense items with dates, categories, and amounts.\"\"\"\n",
-    "\n",
-    "withInstructionParsing = LlamaParse(\n",
-    "    result_type=\"markdown\", parsing_instruction=parsingInstruction\n",
-    ").load_data(\"./expense_report_document.pdf\")"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "**Employee Information:**\n",
-      "- **Name:** Dr. Alexandra Chen-Martinez, PhD\n",
-      "- **Employee ID:** QD-2022-1457\n",
-      "- **Position:** Principal Research Scientist\n",
-      "- **Department:** Advanced Research & Development\n",
-      "\n",
-      "**Trip/Expense Period:**\n",
-      "- **Start Date:** November 15, 2024\n",
-      "- **End Date:** December 10, 2024\n",
-      "\n",
-      "**Expense Items:**\n",
-      "1. **Date:** 11/15/2024\n",
-      "- **Category:** Transportation\n",
-      "- **Description:** JFK → NRT Business Class\n",
-      "- **Original Amount:** $4,250.00\n",
-      "- **Currency:** USD\n",
-      "- **USD Amount:** $4,250.00\n",
-      "- **Booking Reference:** QF78956 - Corporate Rate Applied\n",
-      "- **Project Code:** QD-QUANTUM-2024-01\n",
-      "\n",
-      "2. **Date:** 11/16/2024\n",
-      "- **Category:** Accommodation\n",
-      "- **Description:** Hilton Tokyo - 5 nights\n",
-      "- **Original Amount:** ¥225,000\n",
-      "- **Currency:** JPY\n",
-      "- **USD Amount:** $1,530.00\n",
-      "- **Confirmation:** HTK-2024-78956\n",
-      "\n",
-      "**Locations:**\n",
-      "- Tokyo, Japan\n",
-      "- Singapore\n",
-      "- Sydney, Australia\n",
-      "\n",
-      "**Currency Conversion Rates Applied:**\n",
-      "- JPY (¥) → USD: 0.0068 (as of 11/15/2024)\n",
-      "- SGD (S$) → USD: 0.74 (as of 11/28/2024)\n",
-      "- AUD (A$) → USD: 0.65 (as of 12/03/2024)\n"
-     ]
-    }
-   ],
-   "source": [
-    "print(withInstructionParsing[0].text)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "### Purchase Order Document \n",
-    "\n",
-    "Here we identify the PO number, vendor details, shipping terms, and an itemized list of products with quantities and unit prices."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "<img src=\"purchase_order_document.png\" alt=\"Alt Text\" width=\"500\">"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "Started parsing the file under job_id b8cb11c3-7dce-4e6a-94bb-1a4e50e45e55\n"
-     ]
-    }
-   ],
-   "source": [
-    "vanilaParsing = LlamaParse(result_type=\"markdown\").load_data(\n",
-    "    \"./purchase_order_document.pdf\"\n",
-    ")"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "# GLOBAL TECH SOLUTIONS, INC.\n",
-      "\n",
-      "# PURCHASE ORDER\n",
-      "\n",
-      "Document Reference: PO-2024-GT-9876/REV.2\n",
-      "\n",
-      "[Original: PO-2024-GT-9876]\n",
-      "\n",
-      "Amendment Date: 12/10/2024\n",
-      "\n",
-      "# VENDOR INFORMATION:\n",
-      "\n",
-      "Quantum Electronics Manufacturing\n",
-      "\n",
-      "DUNS: 78-456-7890\n",
-      "\n",
-      "Tax ID: EU8976543210\n",
-      "\n",
-      "Hoofdorp, Netherlands\n",
-      "\n",
-      "Vendor #: QEM-EU-2024-001\n",
-      "\n",
-      "# SHIP TO:\n",
-      "\n",
-      "Global Tech Solutions, Inc.\n",
-      "\n",
-      "Building 7A, Innovation Park\n",
-      "\n",
-      "2100 Technology Drive\n",
-      "\n",
-      "Austin, TX 78701\n",
-      "\n",
-      "USA\n",
-      "\n",
-      "Attn: Sarah Martinez, Receiving Manager\n",
-      "\n",
-      "Tel: +1 (512) 555-0123\n",
-      "\n",
-      "# PAYMENT TERMS:\n",
-      "\n",
-      "Net 45\n",
-      "\n",
-      "2% discount if paid within 15 days\n",
-      "\n",
-      "# SHIPPING TERMS:\n",
-      "\n",
-      "DDP (Delivered Duty Paid) - Incoterms 2020\n",
-      "\n",
-      "Insurance Required: Yes\n",
-      "\n",
-      "Preferred Carrier: DHL/FedEx\n",
-      "\n",
-      "Required Delivery Date: 01/15/2025\n",
-      "\n",
-      "# SPECIAL INSTRUCTIONS:\n",
-      "\n",
-      "1. All shipments must include Certificate of Conformance\n",
-      "2. ESD-sensitive items must be properly packaged\n",
-      "3. Temperature logging required for items marked with *\n",
-      "4. Partial shipments accepted with prior approval\n",
-      "5. Quote PO number on all correspondence\n",
-      "\n",
-      "# ITEM DETAILS:\n",
-      "\n",
-      "|Line|Part Number|Description|Qty|UOM|Unit Price|Total|\n",
-      "|---|---|---|---|---|---|---|\n",
-      "|1|QE-MCU-5590|Microcontroller Unit|500|EA|$12.50|$6,250.00|\n"
-     ]
-    }
-   ],
-   "source": [
-    "print(vanilaParsing[0].text)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "Started parsing the file under job_id d2731305-984d-4633-8a52-0493748cf10b\n"
-     ]
-    }
-   ],
-   "source": [
-    "parsingInstruction = \"\"\"You are provided with a purchase order. \n",
-    "Identify the PO number, vendor details, shipping terms, and itemized list of products with quantities and unit prices.\"\"\"\n",
-    "\n",
-    "withInstructionParsing = LlamaParse(\n",
-    "    result_type=\"markdown\", parsing_instruction=parsingInstruction\n",
-    ").load_data(\"./purchase_order_document.pdf\")"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "Here are the details extracted from the purchase order:\n",
-      "\n",
-      "**PO Number:** PO-2024-GT-9876/REV.2\n",
-      "\n",
-      "**Vendor Details:**\n",
-      "- **Vendor Name:** Quantum Electronics Manufacturing\n",
-      "- **DUNS:** 78-456-7890\n",
-      "- **Tax ID:** EU8976543210\n",
-      "- **Address:** Hoofdorp, Netherlands\n",
-      "- **Vendor Number:** QEM-EU-2024-001\n",
-      "- **Contact Person:** Sarah Martinez, Receiving Manager\n",
-      "- **Phone:** +1 (512) 555-0123\n",
-      "\n",
-      "**Shipping Terms:**\n",
-      "- **Terms:** DDP (Delivered Duty Paid) - Incoterms 2020\n",
-      "- **Insurance Required:** Yes\n",
-      "- **Preferred Carrier:** DHL/FedEx\n",
-      "- **Required Delivery Date:** 01/15/2025\n",
-      "\n",
-      "**Itemized List of Products:**\n",
-      "1. **Part Number:** QE-MCU-5590\n",
-      "- **Description:** Microcontroller Unit\n",
-      "- **Quantity:** 500 EA\n",
-      "- **Unit Price:** $12.50\n",
-      "- **Total:** $6,250.00\n",
-      "\n",
-      "**Payment Terms:**\n",
-      "- Net 45\n",
-      "- 2% discount if paid within 15 days\n",
-      "\n",
-      "**Special Instructions:**\n",
-      "1. All shipments must include Certificate of Conformance\n",
-      "2. ESD-sensitive items must be properly packaged\n",
-      "3. Temperature logging required for items marked with *\n",
-      "4. Partial shipments accepted with prior approval\n",
-      "5. Quote PO number on all correspondence\n"
-     ]
-    }
-   ],
-   "source": [
-    "print(withInstructionParsing[0].text)"
-   ]
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "llamacloud",
-   "language": "python",
-   "name": "llamacloud"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 2
-}

examples/report/basic_report.ipynb

@@ -1,762 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "# Report Generation with LlamaReport\n",
-    "\n",
-    "In this notebook, we'll walk through the basic process of generating a report with LlamaReport, and highlight some of the key features of the library.\n",
-    "\n",
-    "TLDR:\n",
-    "1. Download source data to use as knowledge base for the report\n",
-    "2. Kick off report generation with a template\n",
-    "3. Get the plan and review/accept/reject suggestions\n",
-    "4. Get the final report\n",
-    "5. Review/accept/reject suggestions to edit the final report\n",
-    "6. Print the final report"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "%pip install llama-cloud-services"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## 1. Download Source Data\n",
-    "\n",
-    "Here, we download the `Attention is All You Need` paper as a PDF.\n",
-    "\n",
-    "LlamaReport currently supports up to 5 files as input, and essentially any file type that can be parsed by LlamaParse.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "!wget \"https://arxiv.org/pdf/1706.03762.pdf\" -O \"./attention.pdf\""
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## 2. Kick off Report Generation\n",
-    "\n",
-    "Here, we kick off report generation with a template.\n",
-    "\n",
-    "The template can either be a string or a file path, but here we'll use a string.\n",
-    "\n",
-    "In our experiments, anything works as a template, but some general guidelines:\n",
-    "\n",
-    "- Use markdown formatting + instructions in each section to guide the report generation\n",
-    "- If using an existing file as a template, provide extra instructions to guide the report generation\n",
-    "\n",
-    "**NOTE:** Since we are in a notebook, we will use async functions and `await` throughout. Synchronous methods that work without `await` are available by just removing the `a` from the method name and removing the `await` keyword."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from llama_cloud_services import LlamaReport\n",
-    "\n",
-    "llama_report = LlamaReport(\n",
-    "    api_key=\"llx-...\",\n",
-    ")\n",
-    "\n",
-    "report_client = await llama_report.acreate_report(\n",
-    "    name=\"my_cool_report_on_attention\",\n",
-    "    # can pass in file paths or bytes\n",
-    "    input_files=[\"./attention.pdf\"],\n",
-    "    template_text=\"\"\"\\\n",
-    "# [Some title]\\n\\n\n",
-    "## TLDR\\n\n",
-    "A quick summary of the paper.\\n\\n\n",
-    "## Details\\n\n",
-    "More details about the paper, possibly more than one section here.\\n\n",
-    "\"\"\",\n",
-    "    # optional additional instructions for the report generation\n",
-    "    # template_instructions=None,\n",
-    "    # optional file path to an existing template instead of template_text\n",
-    "    # template_file=None,\n",
-    ")"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "The returned `ReportClient` object is used to interact with the report generation process for this specific report."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "Report(id=0a394b33-1a3e-463c-b5cb-7ff8ab827d0a, name=my_cool_report_on_attention)\n"
-     ]
-    }
-   ],
-   "source": [
-    "print(report_client)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## 3. Get the plan\n",
-    "\n",
-    "The first phases of report generation involve ingesting the source data and generating a plan.\n",
-    "\n",
-    "The plan is a list of instructions for the report generation, and can be reviewed/accepted/rejected by the user.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "plan = await report_client.await_for_plan(\n",
-    "    timeout=10000,\n",
-    "    poll_interval=10,\n",
-    ")"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "# {title}\n",
-      "[ReportQuery(field='title', prompt='Generate a clear and concise title for this paper about the Transformer model and attention mechanisms', context='The paper discusses the Transformer architecture for sequence transduction using attention mechanisms, focusing on machine translation applications')]\n",
-      "==================\n",
-      "## TLDR\n",
-      "\n",
-      "{tldr_content}\n",
-      "[ReportQuery(field='tldr_content', prompt='Write a brief, clear summary of the key points about the Transformer model', context='Focus on the main innovations: attention mechanisms, efficiency improvements, and state-of-the-art results in machine translation')]\n",
-      "==================\n",
-      "## Details\n",
-      "\n",
-      "{details_content}\n",
-      "[ReportQuery(field='details_content', prompt='Provide detailed information about the Transformer model architecture and its applications', context='Include information about:\\n- The attention mechanism implementation\\n- Advantages over recurrent and convolutional models\\n- Performance in machine translation tasks\\n- Training efficiency improvements')]\n",
-      "==================\n"
-     ]
-    }
-   ],
-   "source": [
-    "for plan_block in plan.blocks:\n",
-    "    print(plan_block.block.template)\n",
-    "    print(plan_block.queries)\n",
-    "    print(\"==================\")"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "With the plan, we can either use it to kick off generation of the final report, or we can edit the plan and adjust it as needed.\n",
-    "\n",
-    "While we could manually edit the objects here and use `await report_client.aupdate_plan(action=\"edit\", updated_plan=plan)`, we can also use `LlamaReport` to agentically edit the plan."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "suggestions = await report_client.asuggest_edits(\n",
-    "    \"Can you split the details section into two sections?\"\n",
-    ")"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "Justification for change: \n",
-      "I'll help you break down the details section into two distinct parts - one focusing on the architecture and another on the practical applications and performance. This will make the content more organized and easier to follow. The original block at index 2 will be replaced with these two new sections.\n",
-      "\n",
-      "Proposed changes:\n",
-      "\n",
-      "## Architecture Details\n",
-      "\n",
-      "{architecture_content}\n",
-      "\n",
-      "[ReportQuery(field='architecture_content', prompt='Describe the technical details of the Transformer model architecture', context='Focus on:\\n- Core components of the Transformer architecture\\n- Self-attention mechanism implementation\\n- Multi-head attention details\\n- Position encoding approach\\n- Feed-forward network structure')]\n",
-      "==================\n",
-      "\n",
-      "## Performance and Applications\n",
-      "\n",
-      "{applications_content}\n",
-      "\n",
-      "[ReportQuery(field='applications_content', prompt='Explain the practical applications and performance advantages of the Transformer model', context='Cover:\\n- Comparison with RNN and CNN models\\n- Machine translation results and benchmarks\\n- Training efficiency improvements\\n- Real-world applications and use cases\\n- Scalability benefits')]\n",
-      "==================\n"
-     ]
-    }
-   ],
-   "source": [
-    "for suggestion in suggestions:\n",
-    "    print(\"Justification for change:\", suggestion.justification)\n",
-    "    print(\"Proposed changes:\")\n",
-    "    for plan_block in suggestion.blocks:\n",
-    "        print(plan_block.block.template)\n",
-    "        print(plan_block.queries)\n",
-    "        print(\"==================\")"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "This looks pretty good! We can also use the client to automatically accept and apply, or reject, these suggestions.\n",
-    "\n",
-    "This will (locally) keep track of the history of changes, so that future suggestions can be based on the previous changes."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "for suggestion in suggestions:\n",
-    "    await report_client.aaccept_edit(suggestion)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "What effect did that have on the tracked local history? Let's see!"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "[EditAction(block_idx=2, old_content='## Details\\n\\n{details_content}\\n\\nField: details_content, Prompt: Provide detailed information about the Transformer model architecture and its applications, Context: Include information about:\\n- The attention mechanism implementation\\n- Advantages over recurrent and convolutional models\\n- Performance in machine translation tasks\\n- Training efficiency improvements\\nDepends on: none', new_content='\\n## Architecture Details\\n\\n{architecture_content}\\n\\n\\nField: architecture_content, Prompt: Describe the technical details of the Transformer model architecture, Context: Focus on:\\n- Core components of the Transformer architecture\\n- Self-attention mechanism implementation\\n- Multi-head attention details\\n- Position encoding approach\\n- Feed-forward network structure\\nDepends on: none', action='approved', timestamp=datetime.datetime(2025, 2, 4, 20, 59, 55, 773558)),\n",
-       " EditAction(block_idx=3, old_content='[No old content]', new_content='\\n## Performance and Applications\\n\\n{applications_content}\\n\\n\\nField: applications_content, Prompt: Explain the practical applications and performance advantages of the Transformer model, Context: Cover:\\n- Comparison with RNN and CNN models\\n- Machine translation results and benchmarks\\n- Training efficiency improvements\\n- Real-world applications and use cases\\n- Scalability benefits\\nDepends on: previous', action='approved', timestamp=datetime.datetime(2025, 2, 4, 20, 59, 55, 773687))]"
-      ]
-     },
-     "execution_count": null,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "report_client.edit_history"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "[Message(role=<MessageRole.USER: 'user'>, content='Can you split the details section into two sections?', timestamp=datetime.datetime(2025, 2, 4, 20, 59, 47, 754848)),\n",
-       " Message(role=<MessageRole.ASSISTANT: 'assistant'>, content=\"\\nI'll help you break down the details section into two distinct parts - one focusing on the architecture and another on the practical applications and performance. This will make the content more organized and easier to follow. The original block at index 2 will be replaced with these two new sections.\\n\", timestamp=datetime.datetime(2025, 2, 4, 20, 59, 55, 482070))]"
-      ]
-     },
-     "execution_count": null,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "report_client.chat_history"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "These two items are used to provide context for future suggestions! You can always clear this, or provide your own history."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# report_client.suggest_edits(\"....\", chat_history=[{\"role\": \"user\", \"content\": \"...\"}, ...])"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## 4. Get the final report\n",
-    "\n",
-    "Now that we have a plan, we can kick off generation of the final report."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# kicks off report generation\n",
-    "await report_client.aupdate_plan(action=\"approve\")\n",
-    "\n",
-    "# waits for report generation to complete\n",
-    "report = await report_client.await_completion(\n",
-    "    timeout=10000,\n",
-    "    poll_interval=10,\n",
-    ")"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "# Attention Is All You Need: A Pure Attention-Based Architecture for Neural Machine Translation\n",
-      "\n",
-      "## TLDR\n",
-      "\n",
-      "The Transformer introduced a revolutionary architecture that relies entirely on attention mechanisms, eliminating the need for recurrence or convolution in sequence processing. Its key innovations include multi-head self-attention for parallel processing of input sequences, scaled dot-product attention for efficient computation, and positional encodings for sequence order awareness. The model achieved breakthrough results in machine translation (28.4 BLEU on English-to-German, 41.8 BLEU on English-to-French) while requiring significantly less training time than previous approaches, training in 3.5 days on 8 GPUs. This architecture demonstrated that attention mechanisms alone are sufficient for state-of-the-art sequence modeling, setting a new direction for natural language processing.\n",
-      "\n",
-      "\n",
-      "## Architecture Details\n",
-      "\n",
-      "The Transformer architecture represents a groundbreaking approach to sequence processing, built entirely on attention mechanisms without recurrence or convolution. Here are its key technical details:\n",
-      "\n",
-      "Core Components:\n",
-      "- Encoder-decoder architecture with stacked self-attention and point-wise feed-forward layers\n",
-      "- Each layer contains two main sub-layers: multi-head self-attention mechanism and position-wise feed-forward network\n",
-      "- Layer normalization and residual connections between sub-layers\n",
-      "- No recurrent or convolutional elements, enabling parallel processing\n",
-      "\n",
-      "Self-Attention Mechanism:\n",
-      "- Processes relationships between all positions in a sequence simultaneously\n",
-      "- Computes attention weights using queries, keys, and values derived from input representations\n",
-      "- Implements scaled dot-product attention to prevent gradient issues with large input dimensions\n",
-      "- Allows direct modeling of dependencies regardless of positional distance\n",
-      "- Uses masking in decoder to prevent leftward information flow and maintain auto-regressive property\n",
-      "\n",
-      "Multi-Head Attention:\n",
-      "- Employs multiple attention heads operating in parallel\n",
-      "- Each head processes information in different representation subspaces\n",
-      "- Three types of attention applications:\n",
-      "  1. Encoder self-attention (all positions attend to each other)\n",
-      "  2. Decoder self-attention (each position attends to previous positions)\n",
-      "  3. Encoder-decoder attention (decoder queries attend to encoder outputs)\n",
-      "- Counteracts reduced resolution from attention averaging through parallel processing\n",
-      "\n",
-      "Position-wise Feed-Forward Network:\n",
-      "- Applied identically to each position separately\n",
-      "- Consists of two linear transformations with ReLU activation\n",
-      "- Structure: FFN(x) = max(0, xW1 + b1)W2 + b2\n",
-      "- Input and output dimensionality: dmodel = 512\n",
-      "- Inner-layer dimensionality: dff = 2048\n",
-      "- Parameters vary between layers but remain constant across positions\n",
-      "\n",
-      "Position Encoding:\n",
-      "- Adds positional information to input embeddings\n",
-      "- Enables the model to consider sequential order without recurrence\n",
-      "- Implements sinusoidal position encodings to allow model to attend to relative positions\n",
-      "- Maintains constant number of operations between any two positions, unlike convolutional approaches\n",
-      "- Allows effective modeling of both local and long-range dependencies\n",
-      "\n",
-      "\n",
-      "\n",
-      "## Performance and Applications\n",
-      "\n",
-      "The Transformer model demonstrates significant performance advantages and practical applications across multiple domains:\n",
-      "\n",
-      "Performance Advantages over RNN/CNN Models:\n",
-      "- Eliminates sequential computation constraints present in RNNs, enabling superior parallelization\n",
-      "- Reduces operations needed for relating distant positions to a constant number, compared to linear/logarithmic scaling in CNNs\n",
-      "- Processes all input and output positions simultaneously through self-attention mechanisms\n",
-      "- Achieves state-of-the-art results while requiring significantly less computational resources\n",
-      "\n",
-      "Machine Translation Benchmarks:\n",
-      "- WMT 2014 English-to-German: 28.4 BLEU score, exceeding previous best results by over 2 BLEU points\n",
-      "- WMT 2014 English-to-French: 41.8 BLEU score (single-model state-of-the-art)\n",
-      "- Surpasses performance of existing model ensembles in translation tasks\n",
-      "\n",
-      "Training Efficiency:\n",
-      "- Requires only 3.5 days of training on eight GPUs for state-of-the-art performance\n",
-      "- Achieves superior results at \"a small fraction of the training costs\" compared to previous models\n",
-      "- Enables significantly faster training through parallel processing of input/output sequences\n",
-      "- Can reach production-quality performance in as little as twelve hours on modern GPU hardware\n",
-      "\n",
-      "Real-world Applications:\n",
-      "- Machine translation systems\n",
-      "- Natural language understanding tasks\n",
-      "- Reading comprehension\n",
-      "- Abstractive summarization\n",
-      "- Text entailment analysis\n",
-      "- Constituency parsing (achieving 92.7 F1 score in semi-supervised settings)\n",
-      "- Adaptable to both large and limited training data scenarios\n",
-      "\n",
-      "Scalability Benefits:\n",
-      "- Highly parallelizable architecture enables efficient scaling across multiple GPUs\n",
-      "- Constant computational complexity for relating any input/output positions\n",
-      "- Effective handling of long-range dependencies in sequences\n",
-      "- Maintains performance quality while scaling to larger datasets and model sizes\n",
-      "- Generalizes well across different tasks and domains without architectural changes\n",
-      "- Supports efficient inference and deployment in production environments\n",
-      "\n"
-     ]
-    }
-   ],
-   "source": [
-    "report_text = \"\\n\\n\".join([block.template for block in report.blocks])\n",
-    "print(report_text)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## 5. Edit the final report\n",
-    "\n",
-    "Now that we have a report, we can edit it.\n",
-    "\n",
-    "We can use the `asuggest_edits` method to get suggestions for edits, and then use the `aaccept_edit`/`areject_edit` methods to apply them.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "Justification for change: \n",
-      "I'd suggest changing \"TLDR\" to \"Executive Summary\" which is more appropriate for a professional or academic report. This term is widely used in formal documents and better reflects the nature of this concise overview section while maintaining the same function of providing a quick summary of the key points.\n",
-      "\n",
-      "Proposed changes:\n",
-      "## Executive Summary\n",
-      "\n",
-      "The Transformer introduced a revolutionary architecture that relies entirely on attention mechanisms, eliminating the need for recurrence or convolution in sequence processing. Its key innovations include multi-head self-attention for parallel processing of input sequences, scaled dot-product attention for efficient computation, and positional encodings for sequence order awareness. The model achieved breakthrough results in machine translation (28.4 BLEU on English-to-German, 41.8 BLEU on English-to-French) while requiring significantly less training time than previous approaches, training in 3.5 days on 8 GPUs. This architecture demonstrated that attention mechanisms alone are sufficient for state-of-the-art sequence modeling, setting a new direction for natural language processing.\n",
-      "==================\n"
-     ]
-    }
-   ],
-   "source": [
-    "suggestions = await report_client.asuggest_edits(\n",
-    "    \"Can you change the TLDR header to something more professional?\"\n",
-    ")\n",
-    "for suggestion in suggestions:\n",
-    "    print(\"Justification for change:\", suggestion.justification)\n",
-    "    print(\"Proposed changes:\")\n",
-    "    for block in suggestion.blocks:\n",
-    "        print(block.template)\n",
-    "        print(\"==================\")"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Changing to \"Executive Summary\" sounds reasonable, lets accept that!\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "for suggestion in suggestions:\n",
-    "    await report_client.aaccept_edit(suggestion)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## 7. Print the final report\n",
-    "\n",
-    "Now that we have a report, we can print it."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "# Attention Is All You Need: A Pure Attention-Based Architecture for Neural Machine Translation\n",
-      "\n",
-      "## Executive Summary\n",
-      "\n",
-      "The Transformer introduced a revolutionary architecture that relies entirely on attention mechanisms, eliminating the need for recurrence or convolution in sequence processing. Its key innovations include multi-head self-attention for parallel processing of input sequences, scaled dot-product attention for efficient computation, and positional encodings for sequence order awareness. The model achieved breakthrough results in machine translation (28.4 BLEU on English-to-German, 41.8 BLEU on English-to-French) while requiring significantly less training time than previous approaches, training in 3.5 days on 8 GPUs. This architecture demonstrated that attention mechanisms alone are sufficient for state-of-the-art sequence modeling, setting a new direction for natural language processing.\n",
-      "\n",
-      "\n",
-      "## Architecture Details\n",
-      "\n",
-      "The Transformer architecture represents a groundbreaking approach to sequence processing, built entirely on attention mechanisms without recurrence or convolution. Here are its key technical details:\n",
-      "\n",
-      "Core Components:\n",
-      "- Encoder-decoder architecture with stacked self-attention and point-wise feed-forward layers\n",
-      "- Each layer contains two main sub-layers: multi-head self-attention mechanism and position-wise feed-forward network\n",
-      "- Layer normalization and residual connections between sub-layers\n",
-      "- No recurrent or convolutional elements, enabling parallel processing\n",
-      "\n",
-      "Self-Attention Mechanism:\n",
-      "- Processes relationships between all positions in a sequence simultaneously\n",
-      "- Computes attention weights using queries, keys, and values derived from input representations\n",
-      "- Implements scaled dot-product attention to prevent gradient issues with large input dimensions\n",
-      "- Allows direct modeling of dependencies regardless of positional distance\n",
-      "- Uses masking in decoder to prevent leftward information flow and maintain auto-regressive property\n",
-      "\n",
-      "Multi-Head Attention:\n",
-      "- Employs multiple attention heads operating in parallel\n",
-      "- Each head processes information in different representation subspaces\n",
-      "- Three types of attention applications:\n",
-      "  1. Encoder self-attention (all positions attend to each other)\n",
-      "  2. Decoder self-attention (each position attends to previous positions)\n",
-      "  3. Encoder-decoder attention (decoder queries attend to encoder outputs)\n",
-      "- Counteracts reduced resolution from attention averaging through parallel processing\n",
-      "\n",
-      "Position-wise Feed-Forward Network:\n",
-      "- Applied identically to each position separately\n",
-      "- Consists of two linear transformations with ReLU activation\n",
-      "- Structure: FFN(x) = max(0, xW1 + b1)W2 + b2\n",
-      "- Input and output dimensionality: dmodel = 512\n",
-      "- Inner-layer dimensionality: dff = 2048\n",
-      "- Parameters vary between layers but remain constant across positions\n",
-      "\n",
-      "Position Encoding:\n",
-      "- Adds positional information to input embeddings\n",
-      "- Enables the model to consider sequential order without recurrence\n",
-      "- Implements sinusoidal position encodings to allow model to attend to relative positions\n",
-      "- Maintains constant number of operations between any two positions, unlike convolutional approaches\n",
-      "- Allows effective modeling of both local and long-range dependencies\n",
-      "\n",
-      "\n",
-      "\n",
-      "## Performance and Applications\n",
-      "\n",
-      "The Transformer model demonstrates significant performance advantages and practical applications across multiple domains:\n",
-      "\n",
-      "Performance Advantages over RNN/CNN Models:\n",
-      "- Eliminates sequential computation constraints present in RNNs, enabling superior parallelization\n",
-      "- Reduces operations needed for relating distant positions to a constant number, compared to linear/logarithmic scaling in CNNs\n",
-      "- Processes all input and output positions simultaneously through self-attention mechanisms\n",
-      "- Achieves state-of-the-art results while requiring significantly less computational resources\n",
-      "\n",
-      "Machine Translation Benchmarks:\n",
-      "- WMT 2014 English-to-German: 28.4 BLEU score, exceeding previous best results by over 2 BLEU points\n",
-      "- WMT 2014 English-to-French: 41.8 BLEU score (single-model state-of-the-art)\n",
-      "- Surpasses performance of existing model ensembles in translation tasks\n",
-      "\n",
-      "Training Efficiency:\n",
-      "- Requires only 3.5 days of training on eight GPUs for state-of-the-art performance\n",
-      "- Achieves superior results at \"a small fraction of the training costs\" compared to previous models\n",
-      "- Enables significantly faster training through parallel processing of input/output sequences\n",
-      "- Can reach production-quality performance in as little as twelve hours on modern GPU hardware\n",
-      "\n",
-      "Real-world Applications:\n",
-      "- Machine translation systems\n",
-      "- Natural language understanding tasks\n",
-      "- Reading comprehension\n",
-      "- Abstractive summarization\n",
-      "- Text entailment analysis\n",
-      "- Constituency parsing (achieving 92.7 F1 score in semi-supervised settings)\n",
-      "- Adaptable to both large and limited training data scenarios\n",
-      "\n",
-      "Scalability Benefits:\n",
-      "- Highly parallelizable architecture enables efficient scaling across multiple GPUs\n",
-      "- Constant computational complexity for relating any input/output positions\n",
-      "- Effective handling of long-range dependencies in sequences\n",
-      "- Maintains performance quality while scaling to larger datasets and model sizes\n",
-      "- Generalizes well across different tasks and domains without architectural changes\n",
-      "- Supports efficient inference and deployment in production environments\n",
-      "\n"
-     ]
-    }
-   ],
-   "source": [
-    "report_response = await report_client.aget()\n",
-    "report_text = \"\\n\\n\".join([block.template for block in report_response.report.blocks])\n",
-    "print(report_text)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "We can also see the sources for each block!"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "0.99687636\n",
-      "# Abstract\n",
-      "\n",
-      "The dominant sequence transduction models are based on complex recurrent or convolutiona\n",
-      "==================\n",
-      "0.99591404\n",
-      "# 2 Background\n",
-      "\n",
-      "The goal of reducing sequential computation also forms the foundation of the Extende\n",
-      "==================\n",
-      "0.9951325\n",
-      "# 1 Introduction\n",
-      "\n",
-      "Recurrent neural networks, long short-term memory [13] and gated recurrent [7] neu\n",
-      "==================\n",
-      "0.99442345\n",
-      "# 7 Conclusion\n",
-      "\n",
-      "In this work, we presented the Transformer, the first sequence transduction model ba\n",
-      "==================\n",
-      "0.9967649\n",
-      "# 3.2.3 Applications of Attention in our Model\n",
-      "\n",
-      "The Transformer uses multi-head attention in three d\n",
-      "==================\n",
-      "0.99533635\n",
-      "# 2 Background\n",
-      "\n",
-      "The goal of reducing sequential computation also forms the foundation of the Extende\n",
-      "==================\n",
-      "0.9935868\n",
-      "# Abstract\n",
-      "\n",
-      "The dominant sequence transduction models are based on complex recurrent or convolutiona\n",
-      "==================\n",
-      "0.98780584\n",
-      "# Outputs\n",
-      "\n",
-      "(shifted right)\n",
-      "\n",
-      "Figure 1: The Transformer - model architecture.\n",
-      "\n",
-      "The Transformer follows\n",
-      "==================\n",
-      "0.9205043\n",
-      "# 3.3 Position-wise Feed-Forward Networks\n",
-      "\n",
-      "In addition to attention sub-layers, each of the layers i\n",
-      "==================\n",
-      "0.79581684\n",
-      "# 1 Introduction\n",
-      "\n",
-      "Recurrent neural networks, long short-term memory [13] and gated recurrent [7] neu\n",
-      "==================\n",
-      "0.9946774\n",
-      "# Abstract\n",
-      "\n",
-      "The dominant sequence transduction models are based on complex recurrent or convolutiona\n",
-      "==================\n",
-      "0.97079873\n",
-      "# 7 Conclusion\n",
-      "\n",
-      "In this work, we presented the Transformer, the first sequence transduction model ba\n",
-      "==================\n",
-      "0.9535353\n",
-      "# 6.3 English Constituency Parsing\n",
-      "\n",
-      "To evaluate if the Transformer can generalize to other tasks we \n",
-      "==================\n",
-      "0.9514138\n",
-      "# 2 Background\n",
-      "\n",
-      "The goal of reducing sequential computation also forms the foundation of the Extende\n",
-      "==================\n",
-      "0.9790758\n",
-      "# 1 Introduction\n",
-      "\n",
-      "Recurrent neural networks, long short-term memory [13] and gated recurrent [7] neu\n",
-      "==================\n",
-      "0.92262185\n",
-      "# Outputs\n",
-      "\n",
-      "(shifted right)\n",
-      "\n",
-      "Figure 1: The Transformer - model architecture.\n",
-      "\n",
-      "The Transformer follows\n",
-      "==================\n"
-     ]
-    }
-   ],
-   "source": [
-    "for block in report_response.report.blocks:\n",
-    "    # Each block has a list of sources, which are the nodes that were used to generate the block\n",
-    "    for source in block.sources:\n",
-    "        print(source.score)\n",
-    "        print(source.node.text[:100])\n",
-    "        print(\"==================\")"
-   ]
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "llama-parse-aNC435Vv-py3.10",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 2
-}

examples/report_generation/rfp_response/generate_rfp.ipynb

@@ -95,7 +95,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from llama_cloud_services import LlamaParse\n",
+    "from llama_parse import LlamaParse\n",
     "\n",
     "# use our multimodal models for extractions\n",
     "parser = LlamaParse(\n",

examples/test_tesla_impact_report/test_gpt4o.ipynb

@@ -107,7 +107,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from llama_cloud_services import LlamaParse\n",
+    "from llama_parse import LlamaParse\n",
     "\n",
     "parser_gpt4o = LlamaParse(\n",
     "    result_type=\"markdown\",\n",

llama_cloud_services/__init__.py

@@ -1,8 +0,0 @@
-from llama_cloud_services.parse import LlamaParse
-from llama_cloud_services.report import ReportClient, LlamaReport
-
-__all__ = [
-    "LlamaParse",
-    "ReportClient",
-    "LlamaReport",
-]

llama_cloud_services/parse/__init__.py

@@ -1,3 +0,0 @@
-from llama_cloud_services.parse.base import LlamaParse, ResultType
-
-__all__ = ["LlamaParse", "ResultType"]

llama_cloud_services/report/__init__.py

@@ -1,4 +0,0 @@
-from llama_cloud_services.report.report import ReportClient
-from llama_cloud_services.report.base import LlamaReport
-
-__all__ = ["ReportClient", "LlamaReport"]

llama_cloud_services/report/base.py

@@ -1,269 +0,0 @@
-import asyncio
-import httpx
-import os
-import io
-from concurrent.futures import ThreadPoolExecutor
-from typing import Optional, List, Union, Any, Coroutine, TypeVar
-from urllib.parse import urljoin
-
-from llama_cloud.types import ReportMetadata
-from llama_cloud_services.report.report import ReportClient
-
-T = TypeVar("T")
-
-
-class LlamaReport:
-    """Client for managing reports and general report operations."""
-
-    def __init__(
-        self,
-        api_key: Optional[str] = None,
-        project_id: Optional[str] = None,
-        organization_id: Optional[str] = None,
-        base_url: Optional[str] = None,
-        timeout: Optional[int] = None,
-        async_httpx_client: Optional[httpx.AsyncClient] = None,
-    ):
-        self.api_key = api_key or os.getenv("LLAMA_CLOUD_API_KEY", None)
-        if not self.api_key:
-            raise ValueError("No API key provided.")
-
-        self.base_url = base_url or os.getenv(
-            "LLAMA_CLOUD_BASE_URL", "https://api.cloud.llamaindex.ai"
-        )
-        self.timeout = timeout or 60
-
-        # Initialize HTTP clients
-        self._aclient = async_httpx_client or httpx.AsyncClient(timeout=self.timeout)
-
-        # Set auth headers
-        self.headers = {
-            "Authorization": f"Bearer {self.api_key}",
-        }
-
-        self.organization_id = organization_id
-        self.project_id = project_id
-        self._client_params = {
-            "timeout": self._aclient.timeout,
-            "headers": self._aclient.headers,
-            "base_url": self._aclient.base_url,
-            "auth": self._aclient.auth,
-            "event_hooks": self._aclient.event_hooks,
-            "cookies": self._aclient.cookies,
-            "max_redirects": self._aclient.max_redirects,
-            "params": self._aclient.params,
-            "trust_env": self._aclient.trust_env,
-        }
-        self._thread_pool = ThreadPoolExecutor(
-            max_workers=min(10, (os.cpu_count() or 1) + 4)
-        )
-
-    @property
-    def aclient(self) -> httpx.AsyncClient:
-        if self._aclient is None:
-            self._aclient = httpx.AsyncClient(**self._client_params)
-        return self._aclient
-
-    def _run_sync(self, coro: Coroutine[Any, Any, T]) -> T:
-        """Run coroutine in a separate thread to avoid event loop issues"""
-
-        # force a new client for this thread/event loop
-        original_client = self._aclient
-        self._aclient = None
-
-        def run_coro() -> T:
-            async def wrapped_coro() -> T:
-                return await coro
-
-            return asyncio.run(wrapped_coro())
-
-        result = self._thread_pool.submit(run_coro).result()
-
-        # restore the original client
-        self._aclient = original_client
-
-        return result
-
-    async def _get_default_project(self) -> str:
-        response = await self.aclient.get(
-            urljoin(str(self.base_url), "/api/v1/projects"), headers=self.headers
-        )
-        response.raise_for_status()
-        projects = response.json()
-        default_project = [p for p in projects if p.get("is_default")]
-        return default_project[0]["id"]
-
-    async def _build_url(
-        self, endpoint: str, extra_params: Optional[List[str]] = None
-    ) -> str:
-        """Helper method to build URLs with common query parameters."""
-        url = urljoin(str(self.base_url), endpoint)
-
-        if not self.project_id:
-            self.project_id = await self._get_default_project()
-
-        query_params = []
-        if self.organization_id:
-            query_params.append(f"organization_id={self.organization_id}")
-        if self.project_id:
-            query_params.append(f"project_id={self.project_id}")
-        if extra_params:
-            query_params.extend([p for p in extra_params if p is not None])
-
-        if query_params:
-            url += "?" + "&".join(query_params)
-
-        return url
-
-    async def acreate_report(
-        self,
-        name: str,
-        template_instructions: Optional[str] = None,
-        template_text: Optional[str] = None,
-        template_file: Optional[Union[str, tuple[str, bytes]]] = None,
-        input_files: Optional[List[Union[str, tuple[str, bytes]]]] = None,
-        existing_retriever_id: Optional[str] = None,
-    ) -> ReportClient:
-        """Create a new report asynchronously."""
-        url = await self._build_url("/api/v1/reports/")
-        open_files: List[io.BufferedReader] = []
-
-        data = {"name": name}
-        if template_instructions:
-            data["template_instructions"] = template_instructions
-        if template_text:
-            data["template_text"] = template_text
-        if existing_retriever_id:
-            data["existing_retriever_id"] = str(existing_retriever_id)
-
-        files: List[tuple[str, io.BufferedReader | bytes]] = []
-        if template_file:
-            if isinstance(template_file, str):
-                open_files.append(open(template_file, "rb"))
-                files.append(("template_file", open_files[-1]))
-            else:
-                files.append(("template_file", template_file[1]))
-
-        if input_files:
-            for f in input_files:
-                if isinstance(f, str):
-                    open_files.append(open(f, "rb"))
-                    files.append(("files", open_files[-1]))
-                else:
-                    files.append(("files", f[1]))
-
-        response = await self.aclient.post(
-            url, headers=self.headers, data=data, files=files
-        )
-        try:
-            response.raise_for_status()
-            report_id = response.json()["id"]
-            return ReportClient(report_id, name, self)
-        except httpx.HTTPStatusError as e:
-            raise ValueError(
-                f"Failed to create report: {e.response.text}\nError Code: {e.response.status_code}"
-            )
-        finally:
-            for open_file in open_files:
-                open_file.close()
-
-    def create_report(
-        self,
-        name: str,
-        template_instructions: Optional[str] = None,
-        template_text: Optional[str] = None,
-        template_file: Optional[Union[str, tuple[str, bytes]]] = None,
-        input_files: Optional[List[Union[str, tuple[str, bytes]]]] = None,
-        existing_retriever_id: Optional[str] = None,
-    ) -> ReportClient:
-        """Create a new report."""
-        return self._run_sync(
-            self.acreate_report(
-                name=name,
-                template_instructions=template_instructions,
-                template_text=template_text,
-                template_file=template_file,
-                input_files=input_files,
-                existing_retriever_id=existing_retriever_id,
-            )
-        )
-
-    async def alist_reports(
-        self, state: Optional[str] = None, limit: int = 100, offset: int = 0
-    ) -> List[ReportClient]:
-        """List all reports asynchronously."""
-        params = []
-        if state:
-            params.append(f"state={state}")
-        if limit:
-            params.append(f"limit={limit}")
-        if offset:
-            params.append(f"offset={offset}")
-
-        url = await self._build_url(
-            "/api/v1/reports/list",
-            extra_params=params,
-        )
-
-        response = await self.aclient.get(url, headers=self.headers)
-        response.raise_for_status()
-        data = response.json()
-
-        return [
-            ReportClient(r["report_id"], r["name"], self)
-            for r in data["report_responses"]
-        ]
-
-    def list_reports(
-        self, state: Optional[str] = None, limit: int = 100, offset: int = 0
-    ) -> List[ReportClient]:
-        """Synchronous wrapper for listing reports."""
-        return self._run_sync(self.alist_reports(state, limit, offset))
-
-    async def aget_report(self, report_id: str) -> ReportClient:
-        """Get a Report instance for working with a specific report."""
-        url = await self._build_url(f"/api/v1/reports/{report_id}")
-
-        response = await self.aclient.get(url, headers=self.headers)
-        response.raise_for_status()
-        data = response.json()
-
-        return ReportClient(data["report_id"], data["name"], self)
-
-    def get_report(self, report_id: str) -> ReportClient:
-        """Synchronous wrapper for getting a report."""
-        return self._run_sync(self.aget_report(report_id))
-
-    async def aget_report_metadata(self, report_id: str) -> ReportMetadata:
-        """Get metadata for a specific report asynchronously.
-
-        Returns:
-            dict containing:
-            - id: Report ID
-            - name: Report name
-            - state: Current report state
-            - report_metadata: Additional metadata
-            - template_file: Name of template file if used
-            - template_instructions: Template instructions if provided
-            - input_files: List of input file names
-        """
-        url = await self._build_url(f"/api/v1/reports/{report_id}/metadata")
-
-        response = await self.aclient.get(url, headers=self.headers)
-        response.raise_for_status()
-        return ReportMetadata(**response.json())
-
-    def get_report_metadata(self, report_id: str) -> ReportMetadata:
-        """Synchronous wrapper for getting report metadata."""
-        return self._run_sync(self.aget_report_metadata(report_id))
-
-    async def adelete_report(self, report_id: str) -> None:
-        """Delete a specific report asynchronously."""
-        url = await self._build_url(f"/api/v1/reports/{report_id}")
-
-        response = await self.aclient.delete(url, headers=self.headers)
-        response.raise_for_status()
-
-    def delete_report(self, report_id: str) -> None:
-        """Synchronous wrapper for deleting a report."""
-        return self._run_sync(self.adelete_report(report_id))

llama_cloud_services/report/report.py

@@ -1,527 +0,0 @@
-import asyncio
-import httpx
-import time
-from typing import Optional, List, Literal, Union, TYPE_CHECKING
-from dataclasses import dataclass
-from datetime import datetime
-from enum import Enum
-
-from llama_cloud.types import (
-    ReportEventItemEventData_Progress,
-    ReportMetadata,
-    EditSuggestion,
-    ReportResponse,
-    ReportPlan,
-    ReportBlock,
-    ReportPlanBlock,
-    Report,
-)
-
-if TYPE_CHECKING:
-    from llama_cloud_services.report.base import LlamaReport
-
-
-class MessageRole(str, Enum):
-    USER = "user"
-    ASSISTANT = "assistant"
-
-
-@dataclass
-class Message:
-    role: MessageRole
-    content: str
-    timestamp: datetime
-
-
-@dataclass
-class EditAction:
-    block_idx: int
-    old_content: str
-    new_content: Optional[str]
-    action: Literal["approved", "rejected"]
-    timestamp: datetime
-
-
-DEFAULT_POLL_INTERVAL = 5
-DEFAULT_TIMEOUT = 600
-
-
-class ReportClient:
-    """Client for operations on a specific report."""
-
-    def __init__(self, report_id: str, name: str, parent_client: "LlamaReport"):
-        self.report_id = report_id
-        self.name = name
-        self._client = parent_client
-        self._headers = parent_client.headers
-        self._run_sync = parent_client._run_sync
-        self._build_url = parent_client._build_url
-        self.chat_history: List[Message] = []
-        self.edit_history: List[EditAction] = []
-
-    @property
-    def aclient(self) -> httpx.AsyncClient:
-        return self._client.aclient
-
-    def __str__(self) -> str:
-        return f"Report(id={self.report_id}, name={self.name})"
-
-    def __repr__(self) -> str:
-        return f"Report(id={self.report_id}, name={self.name})"
-
-    def _get_block_content(self, block: Union[ReportBlock, ReportPlanBlock]) -> str:
-        if isinstance(block, ReportBlock):
-            return block.template
-        elif isinstance(block, ReportPlanBlock):
-            return block.block.template
-        else:
-            raise ValueError(f"Invalid block type: {type(block)}")
-
-    def _get_block_idx(self, block: Union[ReportBlock, ReportPlanBlock]) -> int:
-        if isinstance(block, ReportBlock):
-            return block.idx
-        elif isinstance(block, ReportPlanBlock):
-            return block.block.idx
-        else:
-            raise ValueError(f"Invalid block type: {type(block)}")
-
-    async def aget(self, version: Optional[int] = None) -> ReportResponse:
-        """Get this report's details asynchronously."""
-        extra_params = []
-        if version is not None:
-            extra_params.append(f"version={version}")
-
-        url = await self._build_url(f"/api/v1/reports/{self.report_id}", extra_params)
-
-        response = await self.aclient.get(url, headers=self._headers)
-        response.raise_for_status()
-        return ReportResponse(**response.json())
-
-    def get(self, version: Optional[int] = None) -> ReportResponse:
-        """Synchronous wrapper for getting this report's details."""
-        return self._run_sync(self.aget(version))
-
-    async def aupdate_report(self, updated_report: Report) -> ReportResponse:
-        """Update this report's content asynchronously."""
-        url = await self._build_url(f"/api/v1/reports/{self.report_id}")
-        response = await self.aclient.patch(
-            url, headers=self._headers, json={"content": updated_report.dict()}
-        )
-        response.raise_for_status()
-        return ReportResponse(**response.json())
-
-    def update_report(self, updated_report: Report) -> ReportResponse:
-        """Synchronous wrapper for updating this report's content."""
-        return self._run_sync(self.aupdate_report(updated_report))
-
-    async def aupdate_plan(
-        self,
-        action: Literal["approve", "reject", "edit"],
-        updated_plan: Optional[ReportPlan] = None,
-    ) -> ReportResponse:
-        """Update this report's plan asynchronously."""
-        if action == "edit" and not updated_plan:
-            raise ValueError("updated_plan is required when action is 'edit'")
-
-        url = await self._build_url(
-            f"/api/v1/reports/{self.report_id}/plan", [f"action={action}"]
-        )
-
-        data = None
-        if updated_plan is not None:
-            plan_dict = updated_plan.dict()
-            plan_dict.pop("generated_at", None)
-            data = plan_dict
-
-        if updated_plan is None and action == "edit":
-            raise ValueError("updated_plan is required when action is 'edit'")
-
-        response = await self.aclient.patch(url, headers=self._headers, json=data)
-        response.raise_for_status()
-        return ReportResponse(**response.json())
-
-    def update_plan(
-        self,
-        action: Literal["approve", "reject", "edit"],
-        updated_plan: Optional[ReportPlan] = None,
-    ) -> ReportResponse:
-        """Synchronous wrapper for updating this report's plan."""
-        return self._run_sync(self.aupdate_plan(action, updated_plan))
-
-    async def asuggest_edits(
-        self,
-        user_query: str,
-        auto_history: bool = True,
-        chat_history: Optional[List[dict]] = None,
-    ) -> List[EditSuggestion]:
-        """Get AI suggestions for edits to this report asynchronously.
-
-        Args:
-            user_query: The user's request/question about what to edit
-            auto_history: Whether to automatically add the user's message to the chat history
-            chat_history:
-                A list of chat messages to include in the chat history.
-                The format being a list of dictionaries with "role" and "content" keys.
-        """
-        # Add user message to history
-        self.chat_history.append(
-            Message(role=MessageRole.USER, content=user_query, timestamp=datetime.now())
-        )
-
-        # Format chat history with edit summaries
-        chat_history_dicts = []
-        for msg in self.chat_history[:-1]:  # Exclude current message
-            content = msg.content
-            if msg.role == MessageRole.USER:
-                # Add edit summary for user messages
-                edit_summary = self._get_edit_summary_after_message(msg.timestamp)
-                if edit_summary:
-                    content = f"{content}\n\nActions taken:\n{edit_summary}"
-
-            chat_history_dicts.append({"role": msg.role.value, "content": content})
-
-        # decide whether to include chat history or not
-        if chat_history:
-            chat_history_dicts = chat_history
-        elif auto_history:
-            chat_history_dicts = chat_history_dicts
-        else:
-            chat_history_dicts = []
-
-        # Make the API call
-        url = await self._build_url(f"/api/v1/reports/{self.report_id}/suggest_edits")
-        data = {"user_query": user_query, "chat_history": chat_history_dicts}
-
-        response = await self.aclient.post(url, headers=self._headers, json=data)
-        response.raise_for_status()
-        suggestions = response.json()
-        suggestions = [EditSuggestion(**suggestion) for suggestion in suggestions]
-
-        # Add assistant response to history
-        if suggestions:
-            for suggestion in suggestions:
-                self.chat_history.append(
-                    Message(
-                        role=MessageRole.ASSISTANT,
-                        content=suggestion.justification,
-                        timestamp=datetime.now(),
-                    )
-                )
-
-        return suggestions
-
-    def suggest_edits(
-        self,
-        user_query: str,
-        auto_history: bool = True,
-        chat_history: Optional[List[dict]] = None,
-    ) -> List[EditSuggestion]:
-        """Synchronous wrapper for getting edit suggestions."""
-        return self._run_sync(
-            self.asuggest_edits(user_query, auto_history, chat_history)
-        )
-
-    async def await_completion(
-        self, timeout: int = DEFAULT_TIMEOUT, poll_interval: int = DEFAULT_POLL_INTERVAL
-    ) -> Report:
-        """Wait for this report to complete processing."""
-        start_time = time.time()
-        while True:
-            report_response = await self.aget()
-            status = report_response.status
-
-            if status == "completed":
-                return report_response.report
-            elif status == "error":
-                events = await self.aget_events()
-                raise ValueError(f"Report entered error state: {events[-1].msg}")
-            elif time.time() - start_time > timeout:
-                raise TimeoutError(f"Report did not complete within {timeout} seconds")
-
-            await asyncio.sleep(poll_interval)
-
-    def wait_for_completion(
-        self, timeout: int = DEFAULT_TIMEOUT, poll_interval: int = DEFAULT_POLL_INTERVAL
-    ) -> Report:
-        """Synchronous wrapper for awaiting report completion."""
-        return self._run_sync(self.await_completion(timeout, poll_interval))
-
-    async def await_for_plan(
-        self, timeout: int = DEFAULT_TIMEOUT, poll_interval: int = DEFAULT_POLL_INTERVAL
-    ) -> ReportPlan:
-        """Wait for this report's plan to be ready for review."""
-        start_time = time.time()
-        while True:
-            report_metadata = await self.aget_metadata()
-            state = report_metadata.state
-
-            if state == "waiting_approval":
-                report_response = await self.aget()
-                return report_response.plan
-            elif state == "error":
-                events = await self.aget_events()
-                raise ValueError(f"Report entered error state: {events[-1].msg}")
-            elif time.time() - start_time > timeout:
-                raise TimeoutError(f"Plan was not ready within {timeout} seconds")
-
-            await asyncio.sleep(poll_interval)
-
-    def wait_for_plan(
-        self, timeout: int = DEFAULT_TIMEOUT, poll_interval: int = DEFAULT_POLL_INTERVAL
-    ) -> ReportPlan:
-        """Synchronous wrapper for awaiting plan readiness."""
-        return self._run_sync(self.await_for_plan(timeout, poll_interval))
-
-    async def aget_metadata(self) -> ReportMetadata:
-        """Get this report's metadata asynchronously."""
-        return await self._client.aget_report_metadata(self.report_id)
-
-    def get_metadata(self) -> ReportMetadata:
-        """Synchronous wrapper for getting this report's metadata."""
-        return self._run_sync(self.aget_metadata())
-
-    async def adelete(self) -> None:
-        """Delete this report asynchronously."""
-        return await self._client.adelete_report(self.report_id)
-
-    def delete(self) -> None:
-        """Synchronous wrapper for deleting this report."""
-        return self._run_sync(self.adelete())
-
-    async def aaccept_edit(self, suggestion: EditSuggestion) -> None:
-        """Accept a suggested edit.
-
-        Args:
-            suggestion: The EditSuggestion to accept, typically from suggest_edits()
-        """
-        if len(suggestion.blocks) == 0:
-            return
-
-        # Determine if we're editing a plan or report based on first block type
-        is_plan_edit = isinstance(suggestion.blocks[0], ReportPlanBlock)
-
-        # Get current content
-        report_response = await self.aget()
-        current_blocks = (
-            report_response.plan.blocks
-            if is_plan_edit
-            else report_response.report.blocks
-        )
-
-        # Track the edit
-        new_blocks = []
-        for edit_block in suggestion.blocks:
-            # Find matching block in current content
-            old_block = next(
-                (
-                    b
-                    for b in current_blocks
-                    if self._get_block_idx(b) == self._get_block_idx(edit_block)
-                ),
-                None,
-            )
-
-            old_content = (
-                self._get_block_content(old_block) if old_block else "[No old content]"
-            )
-            new_content = self._get_block_content(edit_block)
-
-            if is_plan_edit:
-                new_queries_str = "\n".join(
-                    [
-                        f"Field: {q.field}, Prompt: {q.prompt}, Context: {q.context}"
-                        for q in edit_block.queries
-                    ]
-                )
-                new_dependency_str = (
-                    f"Depends on: {edit_block.dependency}"
-                    if edit_block.dependency
-                    else ""
-                )
-                new_content += f"\n\n{new_queries_str}\n{new_dependency_str}"
-
-                if old_block:
-                    old_queries_str = "\n".join(
-                        [
-                            f"Field: {q.field}, Prompt: {q.prompt}, Context: {q.context}"
-                            for q in old_block.queries
-                        ]
-                    )
-                    old_dependency_str = (
-                        f"Depends on: {old_block.dependency}"
-                        if old_block.dependency
-                        else ""
-                    )
-                    old_content += f"\n\n{old_queries_str}\n{old_dependency_str}"
-
-            self.edit_history.append(
-                EditAction(
-                    block_idx=self._get_block_idx(edit_block),
-                    old_content=old_content,
-                    new_content=new_content,
-                    action="approved",
-                    timestamp=datetime.now(),
-                )
-            )
-
-            # Create updated block
-            if is_plan_edit:
-                new_blocks.append(
-                    ReportPlanBlock(
-                        block=ReportBlock(
-                            idx=edit_block.block.idx,
-                            template=self._get_block_content(edit_block),
-                            sources=edit_block.block.sources,
-                        ),
-                        queries=edit_block.queries,
-                        dependency=edit_block.dependency,
-                    )
-                )
-            else:
-                new_blocks.append(
-                    ReportBlock(
-                        idx=edit_block.idx,
-                        template=self._get_block_content(edit_block),
-                        sources=edit_block.sources,
-                    )
-                )
-
-        if new_blocks:
-            if is_plan_edit:
-                # Update plan in place
-                plan = report_response.plan
-
-                # Replace edited blocks and add new ones
-                for new_block in new_blocks:
-                    block_idx = self._get_block_idx(new_block)
-                    existing_block_idx = next(
-                        (
-                            i
-                            for i, b in enumerate(plan.blocks)
-                            if b.block.idx == block_idx
-                        ),
-                        None,
-                    )
-
-                    if existing_block_idx is not None:
-                        # Replace existing block
-                        plan.blocks[existing_block_idx] = new_block
-                    else:
-                        # Add new block to end
-                        plan.blocks.append(new_block)
-
-                await self.aupdate_plan("edit", plan)
-            else:
-                # Update report in place
-                report = report_response.report
-
-                # Replace edited blocks and add new ones
-                for new_block in new_blocks:
-                    block_idx = self._get_block_idx(new_block)
-                    existing_block_idx = next(
-                        (i for i, b in enumerate(report.blocks) if b.idx == block_idx),
-                        None,
-                    )
-
-                    if existing_block_idx is not None:
-                        # Replace existing block
-                        report.blocks[existing_block_idx] = new_block
-                    else:
-                        # Add new block to end
-                        report.blocks.append(new_block)
-
-                await self.aupdate_report(report)
-
-    def accept_edit(self, suggestion: EditSuggestion) -> None:
-        """Synchronous wrapper for accepting an edit."""
-        return self._run_sync(self.aaccept_edit(suggestion))
-
-    async def areject_edit(self, suggestion: EditSuggestion) -> None:
-        """Reject a suggested edit.
-
-        Args:
-            suggestion: The EditSuggestion to reject, typically from suggest_edits()
-        """
-        # Track the rejections
-        for edit_block in suggestion.blocks:
-            self.edit_history.append(
-                EditAction(
-                    block_idx=self._get_block_idx(edit_block),
-                    old_content=self._get_block_content(edit_block),
-                    new_content=None,
-                    action="rejected",
-                    timestamp=datetime.now(),
-                )
-            )
-
-    def reject_edit(self, suggestion: EditSuggestion) -> None:
-        """Synchronous wrapper for rejecting an edit."""
-        return self._run_sync(self.areject_edit(suggestion))
-
-    def _get_edit_summary_after_message(
-        self, message_timestamp: datetime
-    ) -> Optional[str]:
-        """Get a summary of edits that occurred after a specific message."""
-        relevant_edits = [
-            edit for edit in self.edit_history if edit.timestamp > message_timestamp
-        ]
-
-        if not relevant_edits:
-            return None
-
-        approved = [edit for edit in relevant_edits if edit.action == "approved"]
-        rejected = [edit for edit in relevant_edits if edit.action == "rejected"]
-
-        summary = []
-
-        if approved:
-            summary.append("Approved edits:")
-            for edit in approved:
-                summary.append(
-                    f'Block {edit.block_idx}: "{edit.old_content}" -> "{edit.new_content}"'
-                )
-
-        if rejected:
-            if approved:  # Add spacing if we had approved edits
-                summary.append("")
-            summary.append("Rejected edits:")
-            for edit in rejected:
-                summary.append(f'Block {edit.block_idx}: "{edit.old_content}"')
-
-        return "\n".join(summary)
-
-    async def aget_events(
-        self, last_sequence: Optional[int] = None
-    ) -> List[ReportEventItemEventData_Progress]:
-        """Get all events for this report asynchronously.
-
-        Args:
-            last_sequence: If provided, only get events after this sequence number
-
-        Returns:
-            List of ReportEvent objects
-        """
-        extra_params = []
-        if last_sequence is not None:
-            extra_params.append(f"last_sequence={last_sequence}")
-
-        url = await self._build_url(
-            f"/api/v1/reports/{self.report_id}/events", extra_params
-        )
-
-        response = await self.aclient.get(url, headers=self._headers)
-        response.raise_for_status()
-        progress_events = []
-        for event in response.json():
-            if event["event_type"] == "progress":
-                progress_events.append(
-                    ReportEventItemEventData_Progress(**event["event_data"])
-                )
-
-        return progress_events
-
-    def get_events(
-        self, last_sequence: Optional[int] = None
-    ) -> List[ReportEventItemEventData_Progress]:
-        """Synchronous wrapper for getting report events."""
-        return self._run_sync(self.aget_events(last_sequence))

llama_parse/README.md

@@ -1,165 +0,0 @@
-# LlamaParse
-
-[![PyPI - Downloads](https://img.shields.io/pypi/dm/llama-parse)](https://pypi.org/project/llama-parse/)
-[![GitHub contributors](https://img.shields.io/github/contributors/run-llama/llama_parse)](https://github.com/run-llama/llama_parse/graphs/contributors)
-[![Discord](https://img.shields.io/discord/1059199217496772688)](https://discord.gg/dGcwcsnxhU)
-
-LlamaParse is a **GenAI-native document parser** that can parse complex document data for any downstream LLM use case (RAG, agents).
-
-It is really good at the following:
-
-- ✅ **Broad file type support**: Parsing a variety of unstructured file types (.pdf, .pptx, .docx, .xlsx, .html) with text, tables, visual elements, weird layouts, and more.
-- ✅ **Table recognition**: Parsing embedded tables accurately into text and semi-structured representations.
-- ✅ **Multimodal parsing and chunking**: Extracting visual elements (images/diagrams) into structured formats and return image chunks using the latest multimodal models.
-- ✅ **Custom parsing**: Input custom prompt instructions to customize the output the way you want it.
-
-LlamaParse directly integrates with [LlamaIndex](https://github.com/run-llama/llama_index).
-
-The free plan is up to 1000 pages a day. Paid plan is free 7k pages per week + 0.3c per additional page by default. There is a sandbox available to test the API [**https://cloud.llamaindex.ai/parse ↗**](https://cloud.llamaindex.ai/parse).
-
-Read below for some quickstart information, or see the [full documentation](https://docs.cloud.llamaindex.ai/).
-
-If you're a company interested in enterprise RAG solutions, and/or high volume/on-prem usage of LlamaParse, come [talk to us](https://www.llamaindex.ai/contact).
-
-## Getting Started
-
-First, login and get an api-key from [**https://cloud.llamaindex.ai/api-key ↗**](https://cloud.llamaindex.ai/api-key).
-
-Then, make sure you have the latest LlamaIndex version installed.
-
-**NOTE:** If you are upgrading from v0.9.X, we recommend following our [migration guide](https://pretty-sodium-5e0.notion.site/v0-10-0-Migration-Guide-6ede431dcb8841b09ea171e7f133bd77), as well as uninstalling your previous version first.
-
-```
-pip uninstall llama-index  # run this if upgrading from v0.9.x or older
-pip install -U llama-index --upgrade --no-cache-dir --force-reinstall
-```
-
-Lastly, install the package:
-
-`pip install llama-parse`
-
-Now you can parse your first PDF file using the command line interface. Use the command `llama-parse [file_paths]`. See the help text with `llama-parse --help`.
-
-```bash
-export LLAMA_CLOUD_API_KEY='llx-...'
-
-# output as text
-llama-parse my_file.pdf --result-type text --output-file output.txt
-
-# output as markdown
-llama-parse my_file.pdf --result-type markdown --output-file output.md
-
-# output as raw json
-llama-parse my_file.pdf --output-raw-json --output-file output.json
-```
-
-You can also create simple scripts:
-
-```python
-import nest_asyncio
-
-nest_asyncio.apply()
-
-from llama_parse import LlamaParse
-
-parser = LlamaParse(
-    api_key="llx-...",  # can also be set in your env as LLAMA_CLOUD_API_KEY
-    result_type="markdown",  # "markdown" and "text" are available
-    num_workers=4,  # if multiple files passed, split in `num_workers` API calls
-    verbose=True,
-    language="en",  # Optionally you can define a language, default=en
-)
-
-# sync
-documents = parser.load_data("./my_file.pdf")
-
-# sync batch
-documents = parser.load_data(["./my_file1.pdf", "./my_file2.pdf"])
-
-# async
-documents = await parser.aload_data("./my_file.pdf")
-
-# async batch
-documents = await parser.aload_data(["./my_file1.pdf", "./my_file2.pdf"])
-```
-
-## Using with file object
-
-You can parse a file object directly:
-
-```python
-import nest_asyncio
-
-nest_asyncio.apply()
-
-from llama_parse import LlamaParse
-
-parser = LlamaParse(
-    api_key="llx-...",  # can also be set in your env as LLAMA_CLOUD_API_KEY
-    result_type="markdown",  # "markdown" and "text" are available
-    num_workers=4,  # if multiple files passed, split in `num_workers` API calls
-    verbose=True,
-    language="en",  # Optionally you can define a language, default=en
-)
-
-file_name = "my_file1.pdf"
-extra_info = {"file_name": file_name}
-
-with open(f"./{file_name}", "rb") as f:
-    # must provide extra_info with file_name key with passing file object
-    documents = parser.load_data(f, extra_info=extra_info)
-
-# you can also pass file bytes directly
-with open(f"./{file_name}", "rb") as f:
-    file_bytes = f.read()
-    # must provide extra_info with file_name key with passing file bytes
-    documents = parser.load_data(file_bytes, extra_info=extra_info)
-```
-
-## Using with `SimpleDirectoryReader`
-
-You can also integrate the parser as the default PDF loader in `SimpleDirectoryReader`:
-
-```python
-import nest_asyncio
-
-nest_asyncio.apply()
-
-from llama_parse import LlamaParse
-from llama_index.core import SimpleDirectoryReader
-
-parser = LlamaParse(
-    api_key="llx-...",  # can also be set in your env as LLAMA_CLOUD_API_KEY
-    result_type="markdown",  # "markdown" and "text" are available
-    verbose=True,
-)
-
-file_extractor = {".pdf": parser}
-documents = SimpleDirectoryReader(
-    "./data", file_extractor=file_extractor
-).load_data()
-```
-
-Full documentation for `SimpleDirectoryReader` can be found on the [LlamaIndex Documentation](https://docs.llamaindex.ai/en/stable/module_guides/loading/simpledirectoryreader.html).
-
-## Examples
-
-Several end-to-end indexing examples can be found in the examples folder
-
-- [Getting Started](examples/demo_basic.ipynb)
-- [Advanced RAG Example](examples/demo_advanced.ipynb)
-- [Raw API Usage](examples/demo_api.ipynb)
-
-## Documentation
-
-[https://docs.cloud.llamaindex.ai/](https://docs.cloud.llamaindex.ai/)
-
-## Terms of Service
-
-See the [Terms of Service Here](./TOS.pdf).
-
-## Get in Touch (LlamaCloud)
-
-LlamaParse is part of LlamaCloud, our e2e enterprise RAG platform that provides out-of-the-box, production-ready connectors, indexing, and retrieval over your complex data sources. We offer SaaS and VPC options.
-
-LlamaCloud is currently available via waitlist (join by [creating an account](https://cloud.llamaindex.ai/)). If you're interested in state-of-the-art quality and in centralizing your RAG efforts, come [get in touch with us](https://www.llamaindex.ai/contact).

llama_parse/__init__.py

@@ -0,0 +1,3 @@
+from llama_parse.base import LlamaParse, ResultType
+
+__all__ = ["LlamaParse", "ResultType"]

llama_parse/base.py

@@ -1,29 +1,29 @@
-import asyncio
-import mimetypes
 import os
-import time
-from contextlib import asynccontextmanager
-from copy import deepcopy
-from io import BufferedIOBase
-from pathlib import Path, PurePath, PurePosixPath
-from typing import Any, AsyncGenerator, Dict, List, Optional, Union
+import asyncio
 from urllib.parse import urlparse
 
 import httpx
+import mimetypes
+import time
+from pathlib import Path, PurePath, PurePosixPath
+from typing import AsyncGenerator, Any, Dict, List, Optional, Union
+from contextlib import asynccontextmanager
+from io import BufferedIOBase
+
 from fsspec import AbstractFileSystem
 from llama_index.core.async_utils import asyncio_run, run_jobs
-from llama_index.core.bridge.pydantic import Field, PrivateAttr, field_validator
+from llama_index.core.bridge.pydantic import Field, field_validator
 from llama_index.core.constants import DEFAULT_BASE_URL
 from llama_index.core.readers.base import BasePydanticReader
 from llama_index.core.readers.file.base import get_default_fs
 from llama_index.core.schema import Document
-
-from llama_cloud_services.parse.utils import (
-    SUPPORTED_FILE_TYPES,
-    ResultType,
+from llama_parse.utils import (
     nest_asyncio_err,
     nest_asyncio_msg,
+    ResultType,
+    SUPPORTED_FILE_TYPES,
 )
+from copy import deepcopy
 
 # can put in a path to the file or the file bytes itself
 # if passing as bytes or a buffer, must provide the file_name in extra_info
@@ -32,11 +32,6 @@ FileInput = Union[str, bytes, BufferedIOBase]
 _DEFAULT_SEPARATOR = "\n---\n"
 
 
-JOB_RESULT_URL = "/api/parsing/job/{job_id}/result/{result_type}"
-JOB_STATUS_ROUTE = "/api/parsing/job/{job_id}"
-JOB_UPLOAD_ROUTE = "/api/parsing/upload"
-
-
 class LlamaParse(BasePydanticReader):
     """A smart-parser for files."""
 
@@ -54,11 +49,9 @@ class LlamaParse(BasePydanticReader):
         default=1,
         description="The interval in seconds to check if the parsing is done.",
     )
-
     custom_client: Optional[httpx.AsyncClient] = Field(
         default=None, description="A custom HTTPX client to use for sending requests."
     )
-
     ignore_errors: bool = Field(
         default=True,
         description="Whether or not to ignore and skip errors raised during parsing.",
@@ -140,14 +133,6 @@ class LlamaParse(BasePydanticReader):
         default=None,
         description="The top margin of the bounding box to use to extract text from documents expressed as a float between 0 and 1 representing the percentage of the page height.",
     )
-    complemental_formatting_instruction: Optional[str] = Field(
-        default=None,
-        description="The complemental formatting instruction for the parser. Tell llamaParse how some thing should to be formatted, while retaining the markdown output.",
-    )
-    content_guideline_instruction: Optional[str] = Field(
-        default=None,
-        description="The content guideline for the parser. Tell LlamaParse how the content should be changed / transformed.",
-    )
     continuous_mode: Optional[bool] = Field(
         default=False,
         description="Parse documents continuously, leading to better results on documents where tables span across two pages.",
@@ -172,18 +157,10 @@ class LlamaParse(BasePydanticReader):
         default=False,
         description="If set to true, the parser will extract/tag charts from the document.",
     )
-    extract_layout: Optional[bool] = Field(
-        default=False,
-        description="If set to true, the parser will extract the layout information of the document. Cost 1 credit per page.",
-    )
     fast_mode: Optional[bool] = Field(
         default=False,
         description="Note: Non compatible with gpt-4o. If set to true, the parser will use a faster mode to extract text from documents. This mode will skip OCR of images, and table/heading reconstruction.",
     )
-    formatting_instruction: Optional[str] = Field(
-        default=None,
-        description="The Formatting instruction for the parser. Override default llamaParse behavior. In most case you want to use complemental_formatting_instruction instead.",
-    )
     guess_xlsx_sheet_names: Optional[bool] = Field(
         default=False,
         description="Whether to guess the sheet names of the xlsx file.",
@@ -204,25 +181,13 @@ class LlamaParse(BasePydanticReader):
         default=None,
         description="(optional) If set with input_url will use the specified http proxy to download the file.",
     )
-    ignore_document_elements_for_layout_detection: Optional[bool] = Field(
-        default=False,
-        description="If set to true, the parser will ignore document elements for layout detection and only rely on a vision model.",
-    )
-    input_s3_region: Optional[str] = Field(
-        default=None,
-        description="The region of the input S3 bucket if input_s3_path is specified.",
-    )
     invalidate_cache: Optional[bool] = Field(
         default=False,
         description="If set to true, the cache will be ignored and the document re-processes. All document are kept in cache for 48hours after the job was completed to avoid processing the same document twice.",
     )
-    job_timeout_extra_time_per_page_in_seconds: Optional[float] = Field(
-        default=None,
-        description="The extra time in seconds to wait for the parsing to finish per page. Get added to job_timeout_in_seconds.",
-    )
-    job_timeout_in_seconds: Optional[float] = Field(
-        default=None,
-        description="The maximum timeout in seconds to wait for the parsing to finish. Override default timeout of 30 minutes. Minimum is 120 seconds.",
+    is_formatting_instruction: Optional[bool] = Field(
+        default=False,
+        description="Allow the parsing instruction to also format the output. Disable to have a cleaner markdown output.",
     )
     language: Optional[str] = Field(
         default="en", description="The language of the text to parse."
@@ -239,14 +204,6 @@ class LlamaParse(BasePydanticReader):
         default=None,
         description="An S3 path prefix to store the output of the parsing job. If set, the parser will upload the output to S3. The bucket need to be accessible from the LlamaIndex organization.",
     )
-    output_s3_region: Optional[str] = Field(
-        default=None,
-        description="The AWS region of the output S3 bucket defined in output_s3_path_prefix.",
-    )
-    output_tables_as_HTML: Optional[bool] = Field(
-        default=False,
-        description="If set to true, the parser will output tables as HTML in the markdown.",
-    )
     page_prefix: Optional[str] = Field(
         default=None,
         description="A templated prefix to add to the beginning of each page. If it contain `{page_number}`, it will be replaced by the page number.",
@@ -259,6 +216,9 @@ class LlamaParse(BasePydanticReader):
         default=None,
         description="A templated suffix to add to the beginning of each page. If it contain `{page_number}`, it will be replaced by the page number.",
     )
+    parsing_instruction: Optional[str] = Field(
+        default="", description="The parsing instruction for the parser."
+    )
     premium_mode: Optional[bool] = Field(
         default=False,
         description="Use our best parser mode if set to True.",
@@ -267,31 +227,6 @@ class LlamaParse(BasePydanticReader):
         default=False,
         description="If set to true, the parser will ignore diagonal text (when the text rotation in degrees modulo 90 is not 0).",
     )
-    spreadsheet_extract_sub_tables: Optional[bool] = Field(
-        default=False,
-        description="If set to true, the parser will extract sub-tables from the spreadsheet when possible (more than one table per sheet).",
-    )
-
-    strict_mode_buggy_font: Optional[bool] = Field(
-        default=False,
-        description="If set to true, the parser will fail if it can't extract text from a document because of a buggy font.",
-    )
-
-    strict_mode_image_extraction: Optional[bool] = Field(
-        default=False,
-        description="If set to true, the parser will fail if it can't extract an image from the document.",
-    )
-
-    strict_mode_image_ocr: Optional[bool] = Field(
-        default=False,
-        description="If set to true, the parser will fail if it can't OCR an image from the document.",
-    )
-
-    strict_mode_reconstruction: Optional[bool] = Field(
-        default=False,
-        description="If set to true, the parser will fail if it can't reconstruct a table or a heading from the document.",
-    )
-
     structured_output: Optional[bool] = Field(
         default=False,
         description="If set to true, the parser will output structured data based on the provided JSON Schema.",
@@ -342,13 +277,6 @@ class LlamaParse(BasePydanticReader):
         default=None,
         description="The API key for the GPT-4o API. Lowers the cost of parsing.",
     )
-    is_formatting_instruction: Optional[bool] = Field(
-        default=False,
-        description="Allow the parsing instruction to also format the output. Disable to have a cleaner markdown output.",
-    )
-    parsing_instruction: Optional[str] = Field(
-        default="", description="The parsing instruction for the parser."
-    )
 
     @field_validator("api_key", mode="before", check_fields=True)
     @classmethod
@@ -371,25 +299,6 @@ class LlamaParse(BasePydanticReader):
         url = os.getenv("LLAMA_CLOUD_BASE_URL", None)
         return url or v or DEFAULT_BASE_URL
 
-    _aclient: Union[httpx.AsyncClient, None] = PrivateAttr(default=None, init=False)
-
-    @property
-    def aclient(self) -> httpx.AsyncClient:
-        if not self._aclient:
-            self._aclient = self.custom_client or httpx.AsyncClient()
-
-        # need to do this outside instantiation in case user
-        # updates base_url, api_key, or max_timeout later
-        # ... you wouldn't usually expect that, except
-        # if someone does do it and it doesn't reflect on
-        # the client they'll end up pretty confused, so
-        # for the sake of ergonomics...
-        self._aclient.base_url = self.base_url
-        self._aclient.headers["Authorization"] = f"Bearer {self.api_key}"
-        self._aclient.timeout = self.max_timeout
-
-        return self._aclient
-
     @asynccontextmanager
     async def client_context(self) -> AsyncGenerator[httpx.AsyncClient, None]:
         """Create a context for the HTTPX client."""
@@ -438,6 +347,8 @@ class LlamaParse(BasePydanticReader):
         extra_info: Optional[dict] = None,
         fs: Optional[AbstractFileSystem] = None,
     ) -> str:
+        headers = {"Authorization": f"Bearer {self.api_key}"}
+        url = f"{self.base_url}/api/parsing/upload"
         files = None
         file_handle = None
         input_url = file_input if self._is_input_url(file_input) else None
@@ -528,14 +439,6 @@ class LlamaParse(BasePydanticReader):
         if self.bbox_top is not None:
             data["bbox_top"] = self.bbox_top
 
-        if self.complemental_formatting_instruction:
-            data[
-                "complemental_formatting_instruction"
-            ] = self.complemental_formatting_instruction
-
-        if self.content_guideline_instruction:
-            data["content_guideline_instruction"] = self.content_guideline_instruction
-
         if self.continuous_mode:
             data["continuous_mode"] = self.continuous_mode
 
@@ -554,15 +457,9 @@ class LlamaParse(BasePydanticReader):
         if self.extract_charts:
             data["extract_charts"] = self.extract_charts
 
-        if self.extract_layout:
-            data["extract_layout"] = self.extract_layout
-
         if self.fast_mode:
             data["fast_mode"] = self.fast_mode
 
-        if self.formatting_instruction:
-            data["formatting_instruction"] = self.formatting_instruction
-
         if self.guess_xlsx_sheet_names:
             data["guess_xlsx_sheet_names"] = self.guess_xlsx_sheet_names
 
@@ -580,11 +477,6 @@ class LlamaParse(BasePydanticReader):
         if self.http_proxy is not None:
             data["http_proxy"] = self.http_proxy
 
-        if self.ignore_document_elements_for_layout_detection:
-            data[
-                "ignore_document_elements_for_layout_detection"
-            ] = self.ignore_document_elements_for_layout_detection
-
         if input_url is not None:
             files = None
             data["input_url"] = str(input_url)
@@ -593,23 +485,12 @@ class LlamaParse(BasePydanticReader):
             files = None
             data["input_s3_path"] = str(input_s3_path)
 
-        if self.input_s3_region is not None:
-            data["input_s3_region"] = self.input_s3_region
-
         if self.invalidate_cache:
             data["invalidate_cache"] = self.invalidate_cache
 
         if self.is_formatting_instruction:
             data["is_formatting_instruction"] = self.is_formatting_instruction
 
-        if self.job_timeout_extra_time_per_page_in_seconds is not None:
-            data[
-                "job_timeout_extra_time_per_page_in_seconds"
-            ] = self.job_timeout_extra_time_per_page_in_seconds
-
-        if self.job_timeout_in_seconds is not None:
-            data["job_timeout_in_seconds"] = self.job_timeout_in_seconds
-
         if self.language:
             data["language"] = self.language
 
@@ -622,12 +503,6 @@ class LlamaParse(BasePydanticReader):
         if self.output_s3_path_prefix is not None:
             data["output_s3_path_prefix"] = self.output_s3_path_prefix
 
-        if self.output_s3_region is not None:
-            data["output_s3_region"] = self.output_s3_region
-
-        if self.output_tables_as_HTML:
-            data["output_tables_as_HTML"] = self.output_tables_as_HTML
-
         if self.page_prefix is not None:
             data["page_prefix"] = self.page_prefix
 
@@ -640,9 +515,6 @@ class LlamaParse(BasePydanticReader):
             data["page_suffix"] = self.page_suffix
 
         if self.parsing_instruction is not None:
-            print(
-                "WARNING: parsing_instruction is deprecated. Use complemental_formatting_instruction or content_guideline_instruction instead."
-            )
             data["parsing_instruction"] = self.parsing_instruction
 
         if self.premium_mode:
@@ -651,21 +523,6 @@ class LlamaParse(BasePydanticReader):
         if self.skip_diagonal_text:
             data["skip_diagonal_text"] = self.skip_diagonal_text
 
-        if self.spreadsheet_extract_sub_tables:
-            data["spreadsheet_extract_sub_tables"] = self.spreadsheet_extract_sub_tables
-
-        if self.strict_mode_buggy_font:
-            data["strict_mode_buggy_font"] = self.strict_mode_buggy_font
-
-        if self.strict_mode_image_extraction:
-            data["strict_mode_image_extraction"] = self.strict_mode_image_extraction
-
-        if self.strict_mode_image_ocr:
-            data["strict_mode_image_ocr"] = self.strict_mode_image_ocr
-
-        if self.strict_mode_reconstruction:
-            data["strict_mode_reconstruction"] = self.strict_mode_reconstruction
-
         if self.structured_output:
             data["structured_output"] = self.structured_output
 
@@ -706,12 +563,17 @@ class LlamaParse(BasePydanticReader):
             data["gpt4o_api_key"] = self.gpt4o_api_key
 
         try:
-            resp = await self.aclient.post(JOB_UPLOAD_ROUTE, files=files, data=data)  # type: ignore
-            resp.raise_for_status()  # this raises if status is not 2xx
-            return resp.json()["id"]
-        except httpx.HTTPStatusError as err:  # this catches it
-            msg = f"Failed to parse the file: {err.response.text}"
-            raise Exception(msg) from err  # this preserves the exception context
+            async with self.client_context() as client:
+                response = await client.post(
+                    url,
+                    files=files,
+                    headers=headers,
+                    data=data,
+                )
+                if not response.is_success:
+                    raise Exception(f"Failed to parse the file: {response.text}")
+                job_id = response.json()["id"]
+                return job_id
         finally:
             if file_handle is not None:
                 file_handle.close()
@@ -719,51 +581,52 @@ class LlamaParse(BasePydanticReader):
     async def _get_job_result(
         self, job_id: str, result_type: str, verbose: bool = False
     ) -> Dict[str, Any]:
+        result_url = f"{self.base_url}/api/parsing/job/{job_id}/result/{result_type}"
+        status_url = f"{self.base_url}/api/parsing/job/{job_id}"
+        headers = {"Authorization": f"Bearer {self.api_key}"}
+
         start = time.time()
         tries = 0
-
-        # so we're not re-setting the headers & stuff on each
-        # usage... assume that there is not some other
-        # coro also modifying base_url and the other client related configs.
-        client = self.aclient
         while True:
             await asyncio.sleep(self.check_interval)
-            tries += 1
-            result = await client.get(JOB_STATUS_ROUTE.format(job_id=job_id))
-            if result.status_code != 200:
-                end = time.time()
-                if end - start > self.max_timeout:
-                    raise Exception(f"Timeout while parsing the file: {job_id}")
-                if verbose and tries % 10 == 0:
-                    print(".", end="", flush=True)
-                await asyncio.sleep(self.check_interval)
-                continue
+            async with self.client_context() as client:
+                tries += 1
 
-            # Allowed values "PENDING", "SUCCESS", "ERROR", "CANCELED"
-            result_json = result.json()
-            status = result_json["status"]
-            if status == "SUCCESS":
-                parsed_result = await client.get(
-                    JOB_RESULT_URL.format(job_id=job_id, result_type=result_type),
-                )
-                return parsed_result.json()
+                result = await client.get(status_url, headers=headers)
 
-            elif status == "PENDING":
-                end = time.time()
-                if end - start > self.max_timeout:
-                    raise Exception(f"Timeout while parsing the file: {job_id}")
-                if verbose and tries % 10 == 0:
-                    print(".", end="", flush=True)
-                await asyncio.sleep(self.check_interval)
+                if result.status_code != 200:
+                    end = time.time()
+                    if end - start > self.max_timeout:
+                        raise Exception(f"Timeout while parsing the file: {job_id}")
+                    if verbose and tries % 10 == 0:
+                        print(".", end="", flush=True)
 
-            else:
-                error_code = result_json.get("error_code", "No error code found")
-                error_message = result_json.get(
-                    "error_message", "No error message found"
-                )
+                    await asyncio.sleep(self.check_interval)
 
-                exception_str = f"Job ID: {job_id} failed with status: {status}, Error code: {error_code}, Error message: {error_message}"
-                raise Exception(exception_str)
+                    continue
+
+                # Allowed values "PENDING", "SUCCESS", "ERROR", "CANCELED"
+                result_json = result.json()
+                status = result_json["status"]
+                if status == "SUCCESS":
+                    parsed_result = await client.get(result_url, headers=headers)
+                    return parsed_result.json()
+                elif status == "PENDING":
+                    end = time.time()
+                    if end - start > self.max_timeout:
+                        raise Exception(f"Timeout while parsing the file: {job_id}")
+                    if verbose and tries % 10 == 0:
+                        print(".", end="", flush=True)
+
+                    await asyncio.sleep(self.check_interval)
+                else:
+                    error_code = result_json.get("error_code", "No error code found")
+                    error_message = result_json.get(
+                        "error_message", "No error message found"
+                    )
+
+                    exception_str = f"Job ID: {job_id} failed with status: {status}, Error code: {error_code}, Error message: {error_message}"
+                    raise Exception(exception_str)
 
     async def _aload_data(
         self,
@@ -924,77 +787,54 @@ class LlamaParse(BasePydanticReader):
             else:
                 raise e
 
-    async def aget_assets(
-        self, json_result: List[dict], download_path: str, asset_key: str
-    ) -> List[dict]:
-        """Download assets (images or charts) from the parsed result."""
-        # Make the download path
-        if not os.path.exists(download_path):
-            os.makedirs(download_path)
-
-        client = self.aclient
-        try:
-            assets = []
-            for result in json_result:
-                job_id = result["job_id"]
-                for page in result["pages"]:
-                    if self.verbose:
-                        print(
-                            f"> {asset_key.capitalize()} for page {page['page']}: {page[asset_key]}"
-                        )
-                    for asset in page[asset_key]:
-                        asset_name = asset["name"]
-
-                        # Get the full path
-                        asset_path = os.path.join(
-                            download_path, f"{job_id}-{asset_name}"
-                        )
-
-                        # Get a valid asset path
-                        if not asset_path.endswith(".png"):
-                            if not asset_path.endswith(".jpg"):
-                                asset_path += ".png"
-
-                        asset["path"] = asset_path
-                        asset["job_id"] = job_id
-                        asset["original_file_path"] = result.get("file_path", None)
-                        asset["page_number"] = page["page"]
-
-                        with open(asset_path, "wb") as f:
-                            asset_url = f"{self.base_url}/api/parsing/job/{job_id}/result/image/{asset_name}"
-                            resp = await client.get(asset_url)
-                            resp.raise_for_status()
-                            f.write(resp.content)
-                        assets.append(asset)
-            return assets
-        except Exception as e:
-            print(f"Error while downloading {asset_key} from the parsed result:", e)
-            if self.ignore_errors:
-                return []
-            else:
-                raise e
-
     async def aget_images(
         self, json_result: List[dict], download_path: str
     ) -> List[dict]:
         """Download images from the parsed result."""
-        try:
-            return await self.aget_assets(json_result, download_path, "images")
-        except Exception as e:
-            print("Error while downloading images:", e)
-            if self.ignore_errors:
-                return []
-            else:
-                raise e
+        headers = {"Authorization": f"Bearer {self.api_key}"}
+
+        # make the download path
+        if not os.path.exists(download_path):
+            os.makedirs(download_path)
 
-    async def aget_charts(
-        self, json_result: List[dict], download_path: str
-    ) -> List[dict]:
-        """Download charts from the parsed result."""
         try:
-            return await self.aget_assets(json_result, download_path, "charts")
+            images = []
+            for result in json_result:
+                job_id = result["job_id"]
+                for page in result["pages"]:
+                    if self.verbose:
+                        print(f"> Image for page {page['page']}: {page['images']}")
+                    for image in page["images"]:
+                        image_name = image["name"]
+
+                        # get the full path
+                        image_path = os.path.join(
+                            download_path, f"{job_id}-{image_name}"
+                        )
+
+                        # get a valid image path
+                        if not image_path.endswith(".png"):
+                            if not image_path.endswith(".jpg"):
+                                image_path += ".png"
+
+                        image["path"] = image_path
+                        image["job_id"] = job_id
+
+                        image["original_file_path"] = result.get("file_path", None)
+
+                        image["page_number"] = page["page"]
+                        with open(image_path, "wb") as f:
+                            image_url = f"{self.base_url}/api/parsing/job/{job_id}/result/image/{image_name}"
+                            async with self.client_context() as client:
+                                res = await client.get(
+                                    image_url, headers=headers, timeout=self.max_timeout
+                                )
+                                res.raise_for_status()
+                                f.write(res.content)
+                        images.append(image)
+            return images
         except Exception as e:
-            print("Error while downloading charts:", e)
+            print("Error while downloading images from the parsed result:", e)
             if self.ignore_errors:
                 return []
             else:
@@ -1010,24 +850,15 @@ class LlamaParse(BasePydanticReader):
             else:
                 raise e
 
-    def get_charts(self, json_result: List[dict], download_path: str) -> List[dict]:
-        """Download charts from the parsed result."""
-        try:
-            return asyncio_run(self.aget_charts(json_result, download_path))
-        except RuntimeError as e:
-            if nest_asyncio_err in str(e):
-                raise RuntimeError(nest_asyncio_msg)
-            else:
-                raise e
-
     async def aget_xlsx(
         self, json_result: List[dict], download_path: str
     ) -> List[dict]:
-        """Download xlsx from the parsed result."""
+        """Download images from the parsed result."""
+        headers = {"Authorization": f"Bearer {self.api_key}"}
+
         # make the download path
         if not os.path.exists(download_path):
             os.makedirs(download_path)
-        client = self.aclient
         try:
             xlsx_list = []
             for result in json_result:
@@ -1047,9 +878,12 @@ class LlamaParse(BasePydanticReader):
                     xlsx_url = (
                         f"{self.base_url}/api/parsing/job/{job_id}/result/raw/xlsx"
                     )
-                    res = await client.get(xlsx_url)
-                    res.raise_for_status()
-                    f.write(res.content)
+                    async with self.client_context() as client:
+                        res = await client.get(
+                            xlsx_url, headers=headers, timeout=self.max_timeout
+                        )
+                        res.raise_for_status()
+                        f.write(res.content)
                 xlsx_list.append(xlsx)
             return xlsx_list
 

llama_parse/cli/main.py

@@ -5,7 +5,7 @@ from pathlib import Path
 from pydantic.fields import FieldInfo
 from typing import Any, Callable, List
 
-from llama_cloud_services.parse.base import LlamaParse
+from llama_parse.base import LlamaParse
 
 
 def pydantic_field_to_click_option(name: str, field: FieldInfo) -> click.Option:

llama_parse/llama_parse/__init__.py

@@ -1,3 +0,0 @@
-from llama_cloud_services.parse import LlamaParse, ResultType
-
-__all__ = ["LlamaParse", "ResultType"]

llama_parse/llama_parse/base.py

@@ -1,19 +0,0 @@
-from llama_cloud_services.parse.base import (
-    LlamaParse,
-    ResultType,
-    FileInput,
-    _DEFAULT_SEPARATOR,
-    JOB_RESULT_URL,
-    JOB_STATUS_ROUTE,
-    JOB_UPLOAD_ROUTE,
-)
-
-__all__ = [
-    "LlamaParse",
-    "ResultType",
-    "FileInput",
-    "_DEFAULT_SEPARATOR",
-    "JOB_RESULT_URL",
-    "JOB_STATUS_ROUTE",
-    "JOB_UPLOAD_ROUTE",
-]

llama_parse/llama_parse/cli/main.py

@@ -1,4 +0,0 @@
-from llama_cloud_services.parse.cli.main import parse
-
-if __name__ == "__main__":
-    parse()

llama_parse/llama_parse/utils.py

@@ -1,11 +0,0 @@
-from llama_cloud_services.parse.utils import (
-    SUPPORTED_FILE_TYPES,
-    Language,
-    ResultType,
-)
-
-__all__ = [
-    "SUPPORTED_FILE_TYPES",
-    "Language",
-    "ResultType",
-]

llama_parse/pyproject.toml

@@ -1,24 +0,0 @@
-[build-system]
-requires = ["poetry-core"]
-build-backend = "poetry.core.masonry.api"
-
-[tool.poetry]
-name = "llama-parse"
-version = "0.5.21"
-description = "Parse files into RAG-Optimized formats."
-authors = ["Logan Markewich <logan@llamaindex.ai>"]
-license = "MIT"
-readme = "README.md"
-packages = [{include = "llama_parse"}]
-
-[tool.poetry.dependencies]
-python = ">=3.9,<4.0"
-llama-cloud-services = "*"
-
-[tool.poetry.group.dev.dependencies]
-pytest = "^8.0.0"
-pytest-asyncio = "*"
-ipykernel = "^6.29.0"
-
-[tool.poetry.scripts]
-llama-parse = "llama_parse.cli.main:parse"

llama_parse/utils.py

@@ -191,11 +191,4 @@ SUPPORTED_FILE_TYPES = [
     ".xlr",
     ".eth",
     ".tsv",
-    ".mp3",
-    ".mp4",
-    ".mpeg",
-    ".mpga",
-    ".m4a",
-    ".wav",
-    ".webm",
 ]

parse.md

@@ -1,142 +0,0 @@
-# LlamaParse
-
-LlamaParse is a **GenAI-native document parser** that can parse complex document data for any downstream LLM use case (RAG, agents).
-
-It is really good at the following:
-
-- ✅ **Broad file type support**: Parsing a variety of unstructured file types (.pdf, .pptx, .docx, .xlsx, .html) with text, tables, visual elements, weird layouts, and more.
-- ✅ **Table recognition**: Parsing embedded tables accurately into text and semi-structured representations.
-- ✅ **Multimodal parsing and chunking**: Extracting visual elements (images/diagrams) into structured formats and return image chunks using the latest multimodal models.
-- ✅ **Custom parsing**: Input custom prompt instructions to customize the output the way you want it.
-
-LlamaParse directly integrates with [LlamaIndex](https://github.com/run-llama/llama_index).
-
-The free plan is up to 1000 pages a day. Paid plan is free 7k pages per week + 0.3c per additional page by default. There is a sandbox available to test the API [**https://cloud.llamaindex.ai/parse ↗**](https://cloud.llamaindex.ai/parse).
-
-Read below for some quickstart information, or see the [full documentation](https://docs.cloud.llamaindex.ai/).
-
-If you're a company interested in enterprise RAG solutions, and/or high volume/on-prem usage of LlamaParse, come [talk to us](https://www.llamaindex.ai/contact).
-
-## Getting Started
-
-First, login and get an api-key from [**https://cloud.llamaindex.ai/api-key ↗**](https://cloud.llamaindex.ai/api-key).
-
-Then, install the package:
-
-`pip install llama-cloud-services`
-
-Now you can parse your first PDF file using the command line interface. Use the command `llama-parse [file_paths]`. See the help text with `llama-parse --help`.
-
-```bash
-export LLAMA_CLOUD_API_KEY='llx-...'
-
-# output as text
-llama-parse my_file.pdf --result-type text --output-file output.txt
-
-# output as markdown
-llama-parse my_file.pdf --result-type markdown --output-file output.md
-
-# output as raw json
-llama-parse my_file.pdf --output-raw-json --output-file output.json
-```
-
-You can also create simple scripts:
-
-```python
-import nest_asyncio
-
-nest_asyncio.apply()
-
-from llama_cloud_services import LlamaParse
-
-parser = LlamaParse(
-    api_key="llx-...",  # can also be set in your env as LLAMA_CLOUD_API_KEY
-    result_type="markdown",  # "markdown" and "text" are available
-    num_workers=4,  # if multiple files passed, split in `num_workers` API calls
-    verbose=True,
-    language="en",  # Optionally you can define a language, default=en
-)
-
-# sync
-documents = parser.load_data("./my_file.pdf")
-
-# sync batch
-documents = parser.load_data(["./my_file1.pdf", "./my_file2.pdf"])
-
-# async
-documents = await parser.aload_data("./my_file.pdf")
-
-# async batch
-documents = await parser.aload_data(["./my_file1.pdf", "./my_file2.pdf"])
-```
-
-## Using with file object
-
-You can parse a file object directly:
-
-```python
-import nest_asyncio
-
-nest_asyncio.apply()
-
-from llama_cloud_services import LlamaParse
-
-parser = LlamaParse(
-    api_key="llx-...",  # can also be set in your env as LLAMA_CLOUD_API_KEY
-    result_type="markdown",  # "markdown" and "text" are available
-    num_workers=4,  # if multiple files passed, split in `num_workers` API calls
-    verbose=True,
-    language="en",  # Optionally you can define a language, default=en
-)
-
-file_name = "my_file1.pdf"
-extra_info = {"file_name": file_name}
-
-with open(f"./{file_name}", "rb") as f:
-    # must provide extra_info with file_name key with passing file object
-    documents = parser.load_data(f, extra_info=extra_info)
-
-# you can also pass file bytes directly
-with open(f"./{file_name}", "rb") as f:
-    file_bytes = f.read()
-    # must provide extra_info with file_name key with passing file bytes
-    documents = parser.load_data(file_bytes, extra_info=extra_info)
-```
-
-## Using with `SimpleDirectoryReader`
-
-You can also integrate the parser as the default PDF loader in `SimpleDirectoryReader`:
-
-```python
-import nest_asyncio
-
-nest_asyncio.apply()
-
-from llama_cloud_services import LlamaParse
-from llama_index.core import SimpleDirectoryReader
-
-parser = LlamaParse(
-    api_key="llx-...",  # can also be set in your env as LLAMA_CLOUD_API_KEY
-    result_type="markdown",  # "markdown" and "text" are available
-    verbose=True,
-)
-
-file_extractor = {".pdf": parser}
-documents = SimpleDirectoryReader(
-    "./data", file_extractor=file_extractor
-).load_data()
-```
-
-Full documentation for `SimpleDirectoryReader` can be found on the [LlamaIndex Documentation](https://docs.llamaindex.ai/en/stable/module_guides/loading/simpledirectoryreader.html).
-
-## Examples
-
-Several end-to-end indexing examples can be found in the examples folder
-
-- [Getting Started](examples/parse/demo_basic.ipynb)
-- [Advanced RAG Example](examples/parse/demo_advanced.ipynb)
-- [Raw API Usage](examples/parse/demo_api.ipynb)
-
-## Documentation
-
-[https://docs.cloud.llamaindex.ai/](https://docs.cloud.llamaindex.ai/)