Update extract.md

* cr

* cr

* cr

README.md

@@ -36,7 +36,26 @@ See the quickstart guides for each service for more information:
 
 - [LlamaParse](./parse.md)
 - [LlamaReport (beta/invite-only)](./report.md)
-- [LlamaExtract (coming soon!)]()
+- [LlamaExtract (beta/invite-only)](./extract.md)
+
+## Switch to EU SaaS 🇪🇺
+
+If you are interested in using LlamaCloud services in the EU, you can adjust your base URL to `https://api.cloud.eu.llamaindex.ai`.
+
+You can also create your API key in the EU region [here](https://cloud.eu.llamaindex.ai).
+
+```python
+from llama_cloud_services import (
+    LlamaParse,
+    LlamaReport,
+    LlamaExtract,
+    EU_BASE_URL,
+)
+
+parser = LlamaParse(api_key="YOUR_API_KEY", base_url=EU_BASE_URL)
+report = LlamaReport(api_key="YOUR_API_KEY", base_url=EU_BASE_URL)
+extract = LlamaExtract(api_key="YOUR_API_KEY", base_url=EU_BASE_URL)
+```
 
 ## Documentation
 

examples/extract/lm317_structured_extraction.ipynb

@@ -0,0 +1,318 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "1f6bd03d-1b8b-45a0-bc2c-5a13f1a5d8d3",
+   "metadata": {},
+   "source": [
+    "# LM317 Voltage Regulator Datasheet Structured Extraction\n",
+    "\n",
+    "<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/extract/lm317_structured_extraction.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
+    "\n",
+    "This notebook demonstrates an agentic document workflow using LlamaExtract to process an LM317 voltage regulator datasheet. In this example, we define a structured extraction schema that converts key technical fields into standardized subfields. For instance, the output voltage is split into a minimum and maximum value with a defined unit, and we capture page citations for each extracted field.\n",
+    "\n",
+    "The target user is an electronics engineer at a component manufacturing company who needs to consolidate datasheet information into a standardized specification sheet for design and quality control.\n",
+    "\n",
+    "This approach reduces manual data entry, improves extraction accuracy and standardization, and provides traceability for each technical detail."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "a3b8c8d5-ff3e-48ce-b0b8-29b6b1f517f8",
+   "metadata": {},
+   "source": [
+    "## Use Case Overview\n",
+    "\n",
+    "### Problem\n",
+    "Datasheets like that for the LM317 regulator are often distributed as PDFs containing multiple tables, charts, and complex textual descriptions. Engineers must manually extract technical details such as voltage ranges, dropout voltage, maximum current, input voltage range, and pin configurations. This process is error-prone and time-consuming.\n",
+    "\n",
+    "### Agent Workflow (Combination of Automation and Chat)\n",
+    "1. **Upload Datasheet:** The engineer uploads the LM317 datasheet PDF.  \n",
+    "2. **Structured Extraction:** An automated agent processes the PDF and extracts key technical details into structured fields (e.g., output voltage as a range with separate min/max values).\n",
+    "3. **Interactive Verification:** The engineer can query the agent (via chat) for further details or clarification (e.g., \"Show me the detailed pin configuration extraction\") and review the cited pages.\n",
+    "\n",
+    "**Value Delivered:**\n",
+    "- Up to 70% reduction in manual data extraction time.\n",
+    "- Increased accuracy and standardization with structured fields."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "a704e843-54be-4969-842b-713584cb3c35",
+   "metadata": {},
+   "source": [
+    "## Setup and Download Data\n",
+    "\n",
+    "Download the [LM317 Datasheet](https://www.ti.com/lit/ds/symlink/lm317.pdf) and setup LlamaExtract."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "6e5b1f91-8785-44d4-a710-8be1b48b76de",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!mkdir -p data/lm317_structured_extraction\n",
+    "!wget https://www.ti.com/lit/ds/symlink/lm317.pdf -O data/lm317_structured_extraction/lm317.pdf"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "f17b914a-00ed-4b63-8198-69fd7c4a7c62",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from dotenv import load_dotenv\n",
+    "from llama_cloud_services import LlamaExtract\n",
+    "from llama_cloud.core.api_error import ApiError\n",
+    "\n",
+    "# Load environment variables (ensure LLAMA_CLOUD_API_KEY is set in your .env file)\n",
+    "load_dotenv(override=True)\n",
+    "\n",
+    "# Initialize the LlamaExtract client\n",
+    "llama_extract = LlamaExtract(\n",
+    "    project_id=\"<project_id>\",\n",
+    "    organization_id=\"<organization_id>\",\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "ed9f6e9a-96c8-4ee1-8b45-0b6a4f7dbbf1",
+   "metadata": {},
+   "source": [
+    "## Defining a Structured Extraction Schema\n",
+    "\n",
+    "We now define a rich Pydantic schema to extract technical specifications from the LM317 datasheet. In this schema:\n",
+    "\n",
+    "- The **output_voltage** and **input_voltage** fields are structured as ranges with separate minimum and maximum values and a unit.\n",
+    "- The **pin_configuration** field is structured to include a pin count and a descriptive layout.\n",
+    "- Additional technical fields (e.g., dropout voltage, max current) are captured as numbers.\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "4f7e9b44-5e69-4b30-9864-cd98f1e2a7d4",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from pydantic import BaseModel, Field\n",
+    "from typing import List\n",
+    "\n",
+    "\n",
+    "class VoltageRange(BaseModel):\n",
+    "    min_voltage: float = Field(..., description=\"Minimum voltage in volts\")\n",
+    "    max_voltage: float = Field(..., description=\"Maximum voltage in volts\")\n",
+    "    unit: str = Field(\"V\", description=\"Voltage unit\")\n",
+    "\n",
+    "\n",
+    "class PinConfiguration(BaseModel):\n",
+    "    pin_count: int = Field(..., description=\"Number of pins\")\n",
+    "    layout: str = Field(..., description=\"Detailed pin layout description\")\n",
+    "\n",
+    "\n",
+    "class LM317Spec(BaseModel):\n",
+    "    component_name: str = Field(..., description=\"Name of the component\")\n",
+    "    output_voltage: VoltageRange = Field(\n",
+    "        ..., description=\"Output voltage range specification\"\n",
+    "    )\n",
+    "    dropout_voltage: float = Field(..., description=\"Dropout voltage in volts\")\n",
+    "    max_current: float = Field(..., description=\"Maximum current rating in amperes\")\n",
+    "    input_voltage: VoltageRange = Field(\n",
+    "        ..., description=\"Input voltage range specification\"\n",
+    "    )\n",
+    "    pin_configuration: PinConfiguration = Field(\n",
+    "        ..., description=\"Pin configuration details\"\n",
+    "    )\n",
+    "    features: List[str] = Field([], description=\"List of additional technical features\")\n",
+    "\n",
+    "\n",
+    "class LM317Schema(BaseModel):\n",
+    "    specs: List[LM317Spec] = Field(\n",
+    "        ..., description=\"List of extracted LM317 technical specifications\"\n",
+    "    )"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "e0508e38-35be-446c-afe7-129e39553281",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "try:\n",
+    "    existing_agent = llama_extract.get_agent(name=\"lm317-datasheet\")\n",
+    "    if existing_agent:\n",
+    "        llama_extract.delete_agent(existing_agent.id)\n",
+    "except ApiError as e:\n",
+    "    if e.status_code == 404:\n",
+    "        pass\n",
+    "    else:\n",
+    "        raise"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "bb197dfd-dd37-459e-8953-cc1b12f25bdd",
+   "metadata": {},
+   "source": [
+    "Here we use our balanced extraction mode."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "e3defc0a-c685-4fbd-bbb1-1270f1442e72",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from llama_cloud import ExtractConfig\n",
+    "\n",
+    "extract_config = ExtractConfig(\n",
+    "    extraction_mode=\"BALANCED\",\n",
+    ")\n",
+    "\n",
+    "agent = llama_extract.create_agent(\n",
+    "    name=\"lm317-datasheet\", data_schema=LM317Schema, config=extract_config\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "c0a0f9f9-2ef3-4a38-bd74-68d2c2e9e2d8",
+   "metadata": {},
+   "source": [
+    "## Extracting Information from the LM317 Datasheet\n",
+    "\n",
+    "For this demonstration, please download a publicly available LM317 voltage regulator datasheet (for example, from Texas Instruments) and save it as `lm317.pdf` in the `./data` directory. Then run the cell below to extract the structured technical specifications."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "c58e8b7a-8f9b-46f3-8f72-3c2f96b49e8f",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "Uploading files: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:01<00:00,  1.08s/it]\n",
+      "Creating extraction jobs: 100%|████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.96it/s]\n",
+      "Extracting files: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [01:27<00:00, 87.38s/it]\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Path to the LM317 datasheet PDF\n",
+    "lm317_pdf = \"./data/lm317_structured_extraction/lm317.pdf\"\n",
+    "\n",
+    "# Extract structured technical specifications from the datasheet\n",
+    "lm317_extract = agent.extract(lm317_pdf)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "1a2e2e44-6c48-4a38-a6de-5f2f3c7d4d8b",
+   "metadata": {},
+   "source": [
+    "## Assessing the Extraction Results\n",
+    "\n",
+    "The output will be a consolidated list of LM317 technical specifications. For each entry, you should see structured fields including:\n",
+    "\n",
+    "- **component_name**\n",
+    "- **output_voltage** as a range (with separate `min_voltage` and `max_voltage` plus `unit`)\n",
+    "- **dropout_voltage** and **max_current** as numbers\n",
+    "- **input_voltage** as a structured range\n",
+    "- **pin_configuration** with a `pin_count` and `layout`\n",
+    "- **features** (if available)\n",
+    "\n",
+    "This structured approach makes it easier to standardize the information for downstream integration and verification. Engineers can click on the cited page numbers (in a UI that supports it) to validate the extraction."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "fb2abc44-7c9b-4b19-958e-d0d7b390ae57",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "{'specs': [{'component_name': 'LM317',\n",
+       "   'output_voltage': {'min_voltage': 1.25, 'max_voltage': 37.0, 'unit': 'V'},\n",
+       "   'dropout_voltage': 0.0,\n",
+       "   'max_current': 1.5,\n",
+       "   'input_voltage': {'min_voltage': 4.25, 'max_voltage': 40.0, 'unit': 'V'},\n",
+       "   'pin_configuration': {'pin_count': 3,\n",
+       "    'layout': '1: ADJUST, 2: OUTPUT, 3: INPUT'},\n",
+       "   'features': ['Output voltage range adjustable from 1.25 V to 37 V',\n",
+       "    'Output current greater than 1.5 A',\n",
+       "    'Internal short-circuit current limiting',\n",
+       "    'Thermal overload protection',\n",
+       "    'Output safe-area compensation']}]}"
+      ]
+     },
+     "execution_count": null,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "# Display the extraction results\n",
+    "lm317_extract.data"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "c7a2a523-095e-40bf-b713-f509c13a7747",
+   "metadata": {},
+   "source": [
+    "You can also see the output result in the UI."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "dc22dfa5-b667-4fb0-8dbe-24e401b12389",
+   "metadata": {},
+   "source": [
+    "![](data/lm317_structured_extraction/lm317_extraction.png)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "e0e0c12a-9f89-4bb3-b40d-3e9f7c6d2fef",
+   "metadata": {},
+   "source": [
+    "## Conclusion\n",
+    "\n",
+    "This notebook demonstrated how to use LlamaExtract with a structured extraction schema for the LM317 voltage regulator datasheet. By defining detailed subfields (such as splitting voltage ranges into minimum and maximum values, and structuring the pin configuration), we ensure that the extracted data is standardized and traceable through page citations. This approach minimizes manual effort and improves accuracy, providing a robust example of an agentic document workflow for technical documentation processing.\n",
+    "\n",
+    "Feel free to modify or extend the schema to capture additional technical details or to suit your own use cases."
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "llama_parse",
+   "language": "python",
+   "name": "llama_parse"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

examples/extract/solar_panel_e2e_comparison.ipynb

@@ -0,0 +1,452 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "00f6713b-2a32-4f8f-80e5-9a7d9b6e3b90",
+   "metadata": {},
+   "source": [
+    "# Solar Panel Datasheet Comparison Workflow\n",
+    "\n",
+    "<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/extract/solar_panel_e2e_comparison.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
+    "\n",
+    "\n",
+    "This notebook demonstrates an end‑to‑end agentic workflow using LlamaExtract and the LlamaIndex event‑driven workflow framework. In this workflow, we:\n",
+    "\n",
+    "1. **Extract** structured technical specifications from a solar panel datasheet (e.g. a PDF downloaded from a vendor).\n",
+    "2. **Load** design requirements (provided as a text blob) for a lab‑grade solar panel.\n",
+    "3. **Generate** a detailed comparison report by triggering an event that injects both the extracted data and the requirements into an LLM prompt.\n",
+    "\n",
+    "The workflow is designed for renewable energy engineers who need to quickly validate that a solar panel meets specific design criteria.\n",
+    "\n",
+    "The following notebook uses the event‑driven syntax (with custom events, steps, and a workflow class) adapted from the technical datasheet and contract review examples."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "36d8e34e-ed98-46ac-b744-1642f6e253d5",
+   "metadata": {},
+   "source": [
+    "## Setup and Load Data\n",
+    "\n",
+    "We download the [Honey M TSM-DE08M.08(II) datasheet](https://static.trinasolar.com/sites/default/files/EU_Datasheet_HoneyM_DE08M.08%28II%29_2021_A.pdf) as a PDF.\n",
+    "\n",
+    "**NOTE**: The design requirements are already stored in `data/solar_panel_e2e_comparison/design_reqs.txt`."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "1de7b1b3-c285-492c-8b2e-b37974b4fc63",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "--2025-04-01 14:47:56--  https://static.trinasolar.com/sites/default/files/EU_Datasheet_HoneyM_DE08M.08%28II%29_2021_A.pdf\n",
+      "Resolving static.trinasolar.com (static.trinasolar.com)... 47.246.23.232, 47.246.23.234, 47.246.23.227, ...\n",
+      "Connecting to static.trinasolar.com (static.trinasolar.com)|47.246.23.232|:443... connected.\n",
+      "WARNING: cannot verify static.trinasolar.com's certificate, issued by ‘CN=DigiCert Global G2 TLS RSA SHA256 2020 CA1,O=DigiCert Inc,C=US’:\n",
+      "  Unable to locally verify the issuer's authority.\n",
+      "HTTP request sent, awaiting response... 200 OK\n",
+      "Length: 1888183 (1.8M) [application/pdf]\n",
+      "Saving to: ‘data/solar_panel_e2e_comparison/datasheet.pdf’\n",
+      "\n",
+      "data/solar_panel_e2 100%[===================>]   1.80M  7.47MB/s    in 0.2s    \n",
+      "\n",
+      "2025-04-01 14:47:56 (7.47 MB/s) - ‘data/solar_panel_e2e_comparison/datasheet.pdf’ saved [1888183/1888183]\n",
+      "\n"
+     ]
+    }
+   ],
+   "source": [
+    "!wget https://static.trinasolar.com/sites/default/files/EU_Datasheet_HoneyM_DE08M.08%28II%29_2021_A.pdf -O data/solar_panel_e2e_comparison/datasheet.pdf --no-check-certificate"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "89d2f4c9-f785-424d-a409-3381796c457c",
+   "metadata": {},
+   "source": [
+    "## Define the Structured Extraction Schema\n",
+    "\n",
+    "We define a new, rich schema called `SolarPanelSchema` to capture key technical details from the datasheet. This schema includes:\n",
+    "\n",
+    "- **PowerRange:** Structured as minimum and maximum power output (in Watts).\n",
+    "- **SolarPanelSpec:** Includes module name, power output range, maximum efficiency, certifications, and a mapping of page citations.\n",
+    "\n",
+    "This schema replaces the earlier LM317 schema and will be used when creating our extraction agent."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "bfb40d48-36e0-4b1c-97a1-32a1704c582b",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from pydantic import BaseModel, Field\n",
+    "from typing import List\n",
+    "\n",
+    "\n",
+    "class PowerRange(BaseModel):\n",
+    "    min_power: float = Field(..., description=\"Minimum power output in Watts\")\n",
+    "    max_power: float = Field(..., description=\"Maximum power output in Watts\")\n",
+    "    unit: str = Field(\"W\", description=\"Power unit\")\n",
+    "\n",
+    "\n",
+    "class SolarPanelSpec(BaseModel):\n",
+    "    module_name: str = Field(..., description=\"Name or model of the solar panel module\")\n",
+    "    power_output: PowerRange = Field(..., description=\"Power output range\")\n",
+    "    maximum_efficiency: float = Field(\n",
+    "        ..., description=\"Maximum module efficiency in percentage\"\n",
+    "    )\n",
+    "    temperature_coefficient: float = Field(\n",
+    "        ..., description=\"Temperature coefficient in %/°C\"\n",
+    "    )\n",
+    "    certifications: List[str] = Field([], description=\"List of certifications\")\n",
+    "    page_citations: dict = Field(\n",
+    "        ..., description=\"Mapping of each extracted field to its page numbers\"\n",
+    "    )\n",
+    "\n",
+    "\n",
+    "class SolarPanelSchema(BaseModel):\n",
+    "    specs: List[SolarPanelSpec] = Field(\n",
+    "        ..., description=\"List of extracted solar panel specifications\"\n",
+    "    )"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "19dc309e-7cec-43c1-8f6c-72e14df58f8f",
+   "metadata": {},
+   "source": [
+    "## Initialize Extraction Agent\n",
+    "\n",
+    "Here we initialize our extraction agent that will be responsible for extracting the schema from the solar panel datasheet."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "c9d9f4a2-2e14-493d-8a7e-d01159d38b8f",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from dotenv import load_dotenv\n",
+    "from llama_cloud_services import LlamaExtract\n",
+    "from llama_cloud.core.api_error import ApiError\n",
+    "from llama_cloud import ExtractConfig\n",
+    "\n",
+    "# Initialize the LlamaExtract client\n",
+    "llama_extract = LlamaExtract(\n",
+    "    project_id=\"2fef999e-1073-40e6-aeb3-1f3c0e64d99b\",\n",
+    "    organization_id=\"43b88c8f-e488-46f6-9013-698e3d2e374a\",\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "ec0eb2a7-6e02-45da-a6af-227e2f7c81f2",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "try:\n",
+    "    existing_agent = llama_extract.get_agent(name=\"solar-panel-datasheet\")\n",
+    "    if existing_agent:\n",
+    "        llama_extract.delete_agent(existing_agent.id)\n",
+    "except ApiError as e:\n",
+    "    if e.status_code == 404:\n",
+    "        pass\n",
+    "    else:\n",
+    "        raise\n",
+    "\n",
+    "extract_config = ExtractConfig(\n",
+    "    extraction_mode=\"BALANCED\",\n",
+    ")\n",
+    "\n",
+    "agent = llama_extract.create_agent(\n",
+    "    name=\"solar-panel-datasheet\", data_schema=SolarPanelSchema, config=extract_config\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "b4d7bb60-0456-4a2d-8d48-14f9bb3e71d2",
+   "metadata": {},
+   "source": [
+    "## Workflow Overview\n",
+    "\n",
+    "The workflow consists of four main steps:\n",
+    "\n",
+    "1. **parse_datasheet:** Reads the solar panel datasheet (PDF) and converts its content into text (with page citations).\n",
+    "2. **load_requirements:** Loads the design requirements (as a text blob) that will be injected into the prompt.\n",
+    "3. **generate_comparison_report:** Constructs a prompt using the extracted datasheet content and design requirements and triggers the LLM to generate a comparison report.\n",
+    "4. **output_result:** Logs and returns the final report as the workflow’s result.\n",
+    "\n",
+    "Each step is implemented as an asynchronous function decorated with `@step`, and the workflow is built by subclassing `Workflow`."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "7c482e3a-66b4-4e1b-8d2d-9a9c6b3967f3",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from llama_index.core.workflow import (\n",
+    "    Event,\n",
+    "    StartEvent,\n",
+    "    StopEvent,\n",
+    "    Context,\n",
+    "    Workflow,\n",
+    "    step,\n",
+    ")\n",
+    "from llama_index.llms.openai import OpenAI\n",
+    "from llama_index.core.prompts import ChatPromptTemplate\n",
+    "from llama_cloud_services import LlamaExtract\n",
+    "from llama_cloud.core.api_error import ApiError\n",
+    "from pydantic import BaseModel, Field\n",
+    "from typing import List\n",
+    "\n",
+    "\n",
+    "# Define output schema for the comparison report (for reference)\n",
+    "class ComparisonReportOutput(BaseModel):\n",
+    "    component_name: str = Field(\n",
+    "        ..., description=\"The name of the component being evaluated.\"\n",
+    "    )\n",
+    "    meets_requirements: bool = Field(\n",
+    "        ...,\n",
+    "        description=\"Overall indicator of whether the component meets the design criteria.\",\n",
+    "    )\n",
+    "    summary: str = Field(..., description=\"A brief summary of the evaluation results.\")\n",
+    "    details: dict = Field(\n",
+    "        ..., description=\"Detailed comparisons for each key parameter.\"\n",
+    "    )\n",
+    "\n",
+    "\n",
+    "# Define custom events\n",
+    "\n",
+    "\n",
+    "class DatasheetParseEvent(Event):\n",
+    "    datasheet_content: dict\n",
+    "\n",
+    "\n",
+    "class RequirementsLoadEvent(Event):\n",
+    "    requirements_text: str\n",
+    "\n",
+    "\n",
+    "class ComparisonReportEvent(Event):\n",
+    "    report: ComparisonReportOutput\n",
+    "\n",
+    "\n",
+    "class LogEvent(Event):\n",
+    "    msg: str\n",
+    "    delta: bool = False\n",
+    "\n",
+    "\n",
+    "# For our demonstration, we assume that LlamaExtract is used to parse the datasheet into text.\n",
+    "# We'll also use OpenAI (via LlamaIndex) as our LLM for generating the report.\n",
+    "\n",
+    "llm = OpenAI(model=\"gpt-4o\")  # or your preferred model"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "67a0c391-c7f5-4b93-8d6b-9e31b2d7a817",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "class SolarPanelComparisonWorkflow(Workflow):\n",
+    "    \"\"\"\n",
+    "    Workflow to extract data from a solar panel datasheet and generate a comparison report\n",
+    "    against provided design requirements.\n",
+    "    \"\"\"\n",
+    "\n",
+    "    def __init__(self, agent: LlamaExtract, requirements_path: str, **kwargs):\n",
+    "        super().__init__(**kwargs)\n",
+    "        self.agent = agent\n",
+    "        # Load design requirements from file as a text blob\n",
+    "        with open(requirements_path, \"r\") as f:\n",
+    "            self.requirements_text = f.read()\n",
+    "\n",
+    "    @step\n",
+    "    async def parse_datasheet(\n",
+    "        self, ctx: Context, ev: StartEvent\n",
+    "    ) -> DatasheetParseEvent:\n",
+    "        # datasheet_path is provided in the StartEvent\n",
+    "        datasheet_path = (\n",
+    "            ev.datasheet_path\n",
+    "        )  # e.g., \"./data/solar_panel_comparison/datasheet.pdf\"\n",
+    "        extraction_result = await self.agent.aextract(datasheet_path)\n",
+    "        datasheet_dict = (\n",
+    "            extraction_result.data\n",
+    "        )  # assumed to be a string with page citations\n",
+    "        await ctx.set(\"datasheet_content\", datasheet_dict)\n",
+    "        ctx.write_event_to_stream(LogEvent(msg=\"Datasheet parsed successfully.\"))\n",
+    "        return DatasheetParseEvent(datasheet_content=datasheet_dict)\n",
+    "\n",
+    "    @step\n",
+    "    async def load_requirements(\n",
+    "        self, ctx: Context, ev: DatasheetParseEvent\n",
+    "    ) -> RequirementsLoadEvent:\n",
+    "        # Use the pre-loaded requirements text from __init__\n",
+    "        req_text = self.requirements_text\n",
+    "        ctx.write_event_to_stream(LogEvent(msg=\"Design requirements loaded.\"))\n",
+    "        return RequirementsLoadEvent(requirements_text=req_text)\n",
+    "\n",
+    "    @step\n",
+    "    async def generate_comparison_report(\n",
+    "        self, ctx: Context, ev: RequirementsLoadEvent\n",
+    "    ) -> StopEvent:\n",
+    "        # Build a prompt that injects both the extracted datasheet content and the design requirements\n",
+    "        datasheet_content = await ctx.get(\"datasheet_content\")\n",
+    "        prompt_str = \"\"\"\n",
+    "You are an expert renewable energy engineer.\n",
+    "\n",
+    "Compare the following solar panel datasheet information with the design requirements.\n",
+    "\n",
+    "Design Requirements:\n",
+    "{requirements_text}\n",
+    "\n",
+    "Extracted Datasheet Information:\n",
+    "{datasheet_content}\n",
+    "\n",
+    "Generate a detailed comparison report in JSON format with the following schema:\n",
+    "  - component_name: string\n",
+    "  - meets_requirements: boolean\n",
+    "  - summary: string\n",
+    "  - details: dictionary of comparisons for each parameter\n",
+    "\n",
+    "For each parameter (Maximum Power, Open-Circuit Voltage, Short-Circuit Current, Efficiency, Temperature Coefficient),\n",
+    "indicate PASS or FAIL and provide brief explanations and recommendations.\n",
+    "\"\"\"\n",
+    "\n",
+    "        # extract from contract\n",
+    "        prompt = ChatPromptTemplate.from_messages([(\"user\", prompt_str)])\n",
+    "\n",
+    "        # Call the LLM to generate the report using the prompt\n",
+    "        report_output = await llm.astructured_predict(\n",
+    "            ComparisonReportOutput,\n",
+    "            prompt,\n",
+    "            requirements_text=ev.requirements_text,\n",
+    "            datasheet_content=str(datasheet_content),\n",
+    "        )\n",
+    "        ctx.write_event_to_stream(LogEvent(msg=\"Comparison report generated.\"))\n",
+    "        return StopEvent(\n",
+    "            result={\"report\": report_output, \"datasheet_content\": datasheet_content}\n",
+    "        )"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "d205f532-1a11-4a48-b5a8-87a7f85e9ce7",
+   "metadata": {},
+   "source": [
+    "## Running the Workflow\n",
+    "\n",
+    "Below, we instantiate and run the workflow. We inject the design requirements as a text blob (no custom code to load) and pass the path to the solar panel datasheet (the HoneyM datasheet from Trina).\n",
+    "\n",
+    "The design requirements are:\n",
+    "\n",
+    "```\n",
+    "Solar Panel Design Requirements:\n",
+    "- Power Output Range: ≥ 350 W\n",
+    "- Maximum Efficiency: ≥ 18%\n",
+    "- Certifications: Must include IEC61215 and UL1703\n",
+    "```\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "6b24fa61-a2f5-4ebb-84eb-1c9b48683b1b",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import nest_asyncio\n",
+    "\n",
+    "nest_asyncio.apply()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "a68bdffd-ac3c-4dcc-ba35-65939c2a6bfe",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Running step parse_datasheet\n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "Uploading files: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:01<00:00,  1.17s/it]\n",
+      "Creating extraction jobs: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  3.07it/s]\n",
+      "Extracting files: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [01:28<00:00, 88.39s/it]\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Step parse_datasheet produced event DatasheetParseEvent\n",
+      "Running step load_requirements\n",
+      "Step load_requirements produced event RequirementsLoadEvent\n",
+      "Running step generate_comparison_report\n",
+      "Step generate_comparison_report produced event StopEvent\n",
+      "\n",
+      "********Final Comparison Report:********\n",
+      " component_name='TSM-DE08M.08(II)' meets_requirements=True summary='The solar panel TSM-DE08M.08(II) meets all the specified design requirements, making it a suitable choice for the intended application.' details={'Maximum Power Output': \"PASS - The panel's power output ranges from 360 W to 385 W, exceeding the minimum requirement of 350 W.\", 'Open-Circuit Voltage': 'PASS - The datasheet does not specify Voc, but it is assumed to be within the required range based on other compliant parameters.', 'Short-Circuit Current': 'PASS - The datasheet does not specify Isc, but it is assumed to be within the required range based on other compliant parameters.', 'Efficiency': \"PASS - The panel's efficiency is 21.0%, which is above the minimum requirement of 18%.\", 'Temperature Coefficient': 'PASS - The temperature coefficient is -0.34%/°C, which is better than the maximum allowable -0.5%/°C.'}\n",
+      "\n",
+      "********Datasheet Content:********\n",
+      " {'specs': [{'module_name': 'TSM-DE08M.08(II)', 'power_output': {'min_power': 360.0, 'max_power': 385.0, 'unit': 'W'}, 'maximum_efficiency': 21.0, 'temperature_coefficient': -0.34, 'certifications': ['IEC61215/IEC61730/UL1703', 'IEC61701: Salt Mist Corrosion', 'IEC62716: Ammonia Corrosion', 'IEC60068: Blowing Sand', 'ISO9001', 'ISO14001', 'ISO45001', 'ISO14064'], 'page_citations': {}}]}\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Path to design requirements file (e.g., a text file with design criteria for solar panels)\n",
+    "requirements_path = \"./data/solar_panel_e2e_comparison/design_reqs.txt\"\n",
+    "\n",
+    "# Instantiate the workflow\n",
+    "workflow = SolarPanelComparisonWorkflow(\n",
+    "    agent=agent, requirements_path=requirements_path, verbose=True, timeout=120\n",
+    ")\n",
+    "\n",
+    "# Run the workflow; pass the datasheet path in the StartEvent\n",
+    "result = await workflow.run(\n",
+    "    datasheet_path=\"./data/solar_panel_e2e_comparison/datasheet.pdf\"\n",
+    ")\n",
+    "print(\"\\n********Final Comparison Report:********\\n\", result[\"report\"])\n",
+    "print(\"\\n********Datasheet Content:********\\n\", result[\"datasheet_content\"])"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "llama_parse",
+   "language": "python",
+   "name": "llama_parse"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

extract.md

@@ -1,8 +1,5 @@
 # LlamaExtract
 
-> **⚠️ EXPERIMENTAL**
-> This library is under active development with frequent breaking changes. APIs and functionality may change significantly between versions. If you're interested in being an early adopter, please contact us at [support@llamaindex.ai](mailto:support@llamaindex.ai) or join our [Discord](https://discord.com/invite/eN6D2HQ4aX).
-
 LlamaExtract provides a simple API for extracting structured data from unstructured documents like PDFs, text files and images (upcoming).
 
 ## Quick Start

llama_cloud_services/__init__.py

@@ -1,6 +1,7 @@
 from llama_cloud_services.parse import LlamaParse
 from llama_cloud_services.report import ReportClient, LlamaReport
 from llama_cloud_services.extract import LlamaExtract, ExtractionAgent
+from llama_cloud_services.constants import EU_BASE_URL
 
 __all__ = [
     "LlamaParse",
@@ -8,4 +9,5 @@ __all__ = [
     "LlamaReport",
     "LlamaExtract",
     "ExtractionAgent",
+    "EU_BASE_URL",
 ]

llama_cloud_services/constants.py

@@ -0,0 +1 @@
+EU_BASE_URL = "https://api.cloud.eu.llamaindex.ai"

llama_cloud_services/extract/extract.py

@@ -9,16 +9,20 @@ import httpx
 from pydantic import BaseModel
 from llama_cloud import (
     ExtractAgent as CloudExtractAgent,
+    ExtractAgentCreate,
     ExtractConfig,
     ExtractJob,
     ExtractJobCreate,
     ExtractRun,
+    ExtractSchemaValidateRequest,
+    ExtractAgentUpdate,
     File,
     ExtractMode,
     StatusEnum,
     Project,
     ExtractTarget,
     LlamaExtractSettings,
+    PaginatedExtractRunsResponse,
 )
 from llama_cloud.client import AsyncLlamaCloud
 from llama_cloud_services.extract.utils import JSONObjectType, augment_async_errors
@@ -35,7 +39,7 @@ SchemaInput = Union[JSONObjectType, Type[BaseModel]]
 
 DEFAULT_EXTRACT_CONFIG = ExtractConfig(
     extraction_target=ExtractTarget.PER_DOC,
-    extraction_mode=ExtractMode.ACCURATE,
+    extraction_mode=ExtractMode.BALANCED,
 )
 
 
@@ -53,6 +57,8 @@ class ExtractionAgent:
         num_workers: int = 4,
         show_progress: bool = True,
         verbose: bool = False,
+        verify: Optional[bool] = True,
+        httpx_timeout: Optional[float] = 60,
     ):
         self._client = client
         self._agent = agent
@@ -62,6 +68,8 @@ class ExtractionAgent:
         self.max_timeout = max_timeout
         self.num_workers = num_workers
         self.show_progress = show_progress
+        self.verify = verify
+        self.httpx_timeout = httpx_timeout
         self._verbose = verbose
         self._data_schema: Union[JSONObjectType, None] = None
         self._config: Union[ExtractConfig, None] = None
@@ -74,14 +82,20 @@ class ExtractionAgent:
 
         def run_coro() -> T:
             async def wrapped_coro() -> T:
+                # Get the original client to preserve its configuration
+                original_client = self._client._client_wrapper.httpx_client
+
+                # Create a new client with the same configuration as the original
                 async with httpx.AsyncClient(
-                    timeout=self._client._client_wrapper.httpx_client.timeout,
+                    verify=self.verify,
+                    timeout=self.httpx_timeout,
                 ) as client:
-                    original_client = self._client._client_wrapper.httpx_client
+                    # Temporarily replace the client
                     self._client._client_wrapper.httpx_client = client
                     try:
                         return await coro
                     finally:
+                        # Restore the original client
                         self._client._client_wrapper.httpx_client = original_client
 
             return asyncio.run(wrapped_coro())
@@ -114,7 +128,7 @@ class ExtractionAgent:
             )
         validated_schema = self._run_in_thread(
             self._client.llama_extract.validate_extraction_schema(
-                data_schema=processed_schema
+                request=ExtractSchemaValidateRequest(data_schema=processed_schema)
             )
         )
         self._data_schema = validated_schema.data_schema
@@ -187,8 +201,10 @@ class ExtractionAgent:
         self._agent = self._run_in_thread(
             self._client.llama_extract.update_extraction_agent(
                 extraction_agent_id=self.id,
-                data_schema=self.data_schema,
-                config=self.config,
+                request=ExtractAgentUpdate(
+                    data_schema=self.data_schema,
+                    config=self.config,
+                ),
             )
         )
 
@@ -375,15 +391,29 @@ class ExtractionAgent:
             )
         )
 
-    def list_extraction_runs(self) -> List[ExtractRun]:
+    def delete_extraction_run(self, run_id: str) -> None:
+        """Delete an extraction run by ID.
+
+        Args:
+            run_id (str): The ID of the extraction run to delete
+        """
+        self._run_in_thread(
+            self._client.llama_extract.delete_extraction_run(run_id=run_id)
+        )
+
+    def list_extraction_runs(
+        self, page: int = 0, limit: int = 100
+    ) -> PaginatedExtractRunsResponse:
         """List extraction runs for the extraction agent.
 
         Returns:
-            List[ExtractRun]: List of extraction runs
+            PaginatedExtractRunsResponse: Paginated list of extraction runs
         """
         return self._run_in_thread(
             self._client.llama_extract.list_extract_runs(
                 extraction_agent_id=self.id,
+                skip=page * limit,
+                limit=limit,
             )
         )
 
@@ -416,6 +446,12 @@ class LlamaExtract(BaseComponent):
     verbose: bool = Field(
         default=False, description="Show verbose output when extracting files."
     )
+    verify: Optional[bool] = Field(
+        default=True, description="Simple SSL verification option."
+    )
+    httpx_timeout: Optional[float] = Field(
+        default=60, description="Timeout for the httpx client."
+    )
     _async_client: AsyncLlamaCloud = PrivateAttr()
     _thread_pool: ThreadPoolExecutor = PrivateAttr()
     _project_id: Optional[str] = PrivateAttr()
@@ -431,6 +467,8 @@ class LlamaExtract(BaseComponent):
         show_progress: bool = True,
         project_id: Optional[str] = None,
         organization_id: Optional[str] = None,
+        verify: Optional[bool] = True,
+        httpx_timeout: Optional[float] = 60,
         verbose: bool = False,
     ):
         if not api_key:
@@ -448,11 +486,18 @@ class LlamaExtract(BaseComponent):
             max_timeout=max_timeout,
             num_workers=num_workers,
             show_progress=show_progress,
+            verify=verify,
+            httpx_timeout=httpx_timeout,
             verbose=verbose,
         )
+        self._httpx_client = httpx.AsyncClient(verify=verify, timeout=httpx_timeout)
+        self.verify = verify
+        self.httpx_timeout = httpx_timeout
 
         self._async_client = AsyncLlamaCloud(
-            token=self.api_key, base_url=self.base_url, timeout=None
+            token=self.api_key,
+            base_url=self.base_url,
+            httpx_client=self._httpx_client,
         )
         self._thread_pool = ThreadPoolExecutor(
             max_workers=min(10, (os.cpu_count() or 1) + 4)
@@ -481,17 +526,22 @@ class LlamaExtract(BaseComponent):
         def run_coro() -> T:
             # Create a new client for this thread
             async def wrapped_coro() -> T:
+                assert (
+                    self._httpx_client is not None
+                ), "httpx_client should be initialized"
+                # Create a new client with the same configuration as the original
                 async with httpx.AsyncClient(
-                    timeout=self._async_client._client_wrapper.httpx_client.timeout,
+                    verify=self.verify,
+                    timeout=self.httpx_timeout,
                 ) as client:
-                    # Replace the client in the coro's context
-                    original_client = self._async_client._client_wrapper.httpx_client
+                    # Temporarily replace the client
                     self._async_client._client_wrapper.httpx_client = client
                     try:
                         return await coro
                     finally:
+                        # Restore the original client
                         self._async_client._client_wrapper.httpx_client = (
-                            original_client
+                            self._httpx_client
                         )
 
             return asyncio.run(wrapped_coro())
@@ -514,6 +564,14 @@ class LlamaExtract(BaseComponent):
         Returns:
             ExtractionAgent: The created extraction agent
         """
+        if config is not None:
+            if config.extraction_mode == ExtractMode.ACCURATE:
+                warnings.warn(
+                    "ACCURATE extraction mode is deprecated. Using BALANCED instead."
+                )
+                config.extraction_mode = ExtractMode.BALANCED
+        else:
+            config = DEFAULT_EXTRACT_CONFIG
 
         if isinstance(data_schema, dict):
             data_schema = data_schema
@@ -526,11 +584,13 @@ class LlamaExtract(BaseComponent):
 
         agent = self._run_in_thread(
             self._async_client.llama_extract.create_extraction_agent(
-                name=name,
-                data_schema=data_schema,
-                config=config or DEFAULT_EXTRACT_CONFIG,
                 project_id=self._project_id,
                 organization_id=self._organization_id,
+                request=ExtractAgentCreate(
+                    name=name,
+                    data_schema=data_schema,
+                    config=config,
+                ),
             )
         )
 
@@ -592,6 +652,8 @@ class LlamaExtract(BaseComponent):
             num_workers=self.num_workers,
             show_progress=self.show_progress,
             verbose=self.verbose,
+            verify=self.verify,
+            httpx_timeout=self.httpx_timeout,
         )
 
     def list_agents(self) -> List[ExtractionAgent]:

llama_cloud_services/parse/base.py

@@ -4,6 +4,7 @@ import os
 import time
 from contextlib import asynccontextmanager
 from copy import deepcopy
+from enum import Enum
 from io import BufferedIOBase
 from pathlib import Path, PurePath, PurePosixPath
 from typing import Any, AsyncGenerator, Dict, List, Optional, Union
@@ -52,6 +53,14 @@ def build_url(
     return base_url
 
 
+class BackoffPattern(str, Enum):
+    """Backoff pattern for polling."""
+
+    CONSTANT = "constant"
+    LINEAR = "linear"
+    EXPONENTIAL = "exponential"
+
+
 class LlamaParse(BasePydanticReader):
     """A smart-parser for files."""
 
@@ -78,6 +87,15 @@ class LlamaParse(BasePydanticReader):
         description="The interval in seconds to check if the parsing is done.",
     )
 
+    backoff_pattern: BackoffPattern = Field(
+        default=BackoffPattern.LINEAR,
+        description="Controls the backoff pattern when retrying failed requests: 'constant', 'linear', or 'exponential'.",
+    )
+    max_check_interval: int = Field(
+        default=5,
+        description="Maximum interval in seconds between polling attempts when checking job status.",
+    )
+
     custom_client: Optional[httpx.AsyncClient] = Field(
         default=None, description="A custom HTTPX client to use for sending requests."
     )
@@ -111,6 +129,10 @@ class LlamaParse(BasePydanticReader):
     )
 
     # Parsing specific configurations (Alphabetical order)
+    adaptive_long_table: Optional[bool] = Field(
+        default=False,
+        description="If set to true, LlamaParse will try to detect long table and adapt the output.",
+    )
     annotate_links: Optional[bool] = Field(
         default=False,
         description="Annotate links found in the document to extract their URL.",
@@ -272,7 +294,7 @@ class LlamaParse(BasePydanticReader):
         default=None,
         description="A templated suffix to add to the beginning of each page. If it contain `{page_number}`, it will be replaced by the page number.",
     )
-    parsing_mode: Optional[str] = Field(
+    parse_mode: Optional[str] = Field(
         default=None,
         description="The parsing mode to use, see ParsingMode enum for possible values ",
     )
@@ -280,6 +302,10 @@ class LlamaParse(BasePydanticReader):
         default=False,
         description="Use our best parser mode if set to True.",
     )
+    preserve_layout_alignment_across_pages: Optional[bool] = Field(
+        default=False,
+        description="Preserve grid alignment across page in text mode.",
+    )
     skip_diagonal_text: Optional[bool] = Field(
         default=False,
         description="If set to true, the parser will ignore diagonal text (when the text rotation in degrees modulo 90 is not 0).",
@@ -520,6 +546,9 @@ class LlamaParse(BasePydanticReader):
 
         data["from_python_package"] = True
 
+        if self.adaptive_long_table:
+            data["adaptive_long_table"] = self.adaptive_long_table
+
         if self.annotate_links:
             data["annotate_links"] = self.annotate_links
 
@@ -699,9 +728,17 @@ class LlamaParse(BasePydanticReader):
             )
             data["parsing_instruction"] = self.parsing_instruction
 
+        if self.parse_mode:
+            data["parse_mode"] = self.parse_mode
+
         if self.premium_mode:
             data["premium_mode"] = self.premium_mode
 
+        if self.preserve_layout_alignment_across_pages:
+            data[
+                "preserve_layout_alignment_across_pages"
+            ] = self.preserve_layout_alignment_across_pages
+
         if self.skip_diagonal_text:
             data["skip_diagonal_text"] = self.skip_diagonal_text
 
@@ -775,54 +812,87 @@ class LlamaParse(BasePydanticReader):
             if file_handle is not None:
                 file_handle.close()
 
+    def _calculate_backoff(self, current_interval: float) -> float:
+        """Calculate the next backoff interval based on the backoff pattern.
+
+        Args:
+            current_interval: The current interval in seconds
+
+        Returns:
+            The next interval in seconds
+        """
+        if self.backoff_pattern == BackoffPattern.CONSTANT:
+            return current_interval
+        elif self.backoff_pattern == BackoffPattern.LINEAR:
+            return min(current_interval + 1, float(self.max_check_interval))
+        elif self.backoff_pattern == BackoffPattern.EXPONENTIAL:
+            return min(current_interval * 2, float(self.max_check_interval))
+        return current_interval  # Default fallback
+
     async def _get_job_result(
         self, job_id: str, result_type: str, verbose: bool = False
     ) -> Dict[str, Any]:
         start = time.time()
         tries = 0
+        error_count = 0
+        current_interval: float = float(self.check_interval)
 
         # so we're not re-setting the headers & stuff on each
         # usage... assume that there is not some other
         # coro also modifying base_url and the other client related configs.
         client = self.aclient
         while True:
-            await asyncio.sleep(self.check_interval)
-            tries += 1
-            result = await client.get(JOB_STATUS_ROUTE.format(job_id=job_id))
-            if result.status_code != 200:
+            try:
+                await asyncio.sleep(current_interval)
+                tries += 1
+                result = await client.get(JOB_STATUS_ROUTE.format(job_id=job_id))
+                result.raise_for_status()  # this raises if status is not 2xx
+                # Allowed values "PENDING", "SUCCESS", "ERROR", "CANCELED"
+                result_json = result.json()
+                status = result_json["status"]
+                if status == "SUCCESS":
+                    parsed_result = await client.get(
+                        JOB_RESULT_URL.format(job_id=job_id, result_type=result_type),
+                    )
+                    return parsed_result.json()
+                elif status == "PENDING":
+                    end = time.time()
+                    if end - start > self.max_timeout:
+                        raise Exception(f"Timeout while parsing the file: {job_id}")
+                    if verbose and tries % 10 == 0:
+                        print(".", end="", flush=True)
+                    current_interval = self._calculate_backoff(current_interval)
+                else:
+                    error_code = result_json.get("error_code", "No error code found")
+                    error_message = result_json.get(
+                        "error_message", "No error message found"
+                    )
+                    exception_str = (
+                        f"Job ID: {job_id} failed with status: {status}, "
+                        f"Error code: {error_code}, Error message: {error_message}"
+                    )
+                    raise Exception(exception_str)
+            except (
+                httpx.ConnectError,
+                httpx.ReadError,
+                httpx.WriteError,
+                httpx.ConnectTimeout,
+                httpx.ReadTimeout,
+                httpx.WriteTimeout,
+                httpx.HTTPStatusError,
+            ) as err:
+                error_count += 1
                 end = time.time()
                 if end - start > self.max_timeout:
-                    raise Exception(f"Timeout while parsing the file: {job_id}")
+                    raise Exception(
+                        f"Timeout while parsing the file: {job_id}"
+                    ) from err
                 if verbose and tries % 10 == 0:
-                    print(".", end="", flush=True)
-                await asyncio.sleep(self.check_interval)
-                continue
-
-            # Allowed values "PENDING", "SUCCESS", "ERROR", "CANCELED"
-            result_json = result.json()
-            status = result_json["status"]
-            if status == "SUCCESS":
-                parsed_result = await client.get(
-                    JOB_RESULT_URL.format(job_id=job_id, result_type=result_type),
-                )
-                return parsed_result.json()
-
-            elif status == "PENDING":
-                end = time.time()
-                if end - start > self.max_timeout:
-                    raise Exception(f"Timeout while parsing the file: {job_id}")
-                if verbose and tries % 10 == 0:
-                    print(".", end="", flush=True)
-                await asyncio.sleep(self.check_interval)
-
-            else:
-                error_code = result_json.get("error_code", "No error code found")
-                error_message = result_json.get(
-                    "error_message", "No error message found"
-                )
-
-                exception_str = f"Job ID: {job_id} failed with status: {status}, Error code: {error_code}, Error message: {error_message}"
-                raise Exception(exception_str)
+                    print(
+                        f"HTTP error: {err}...",
+                        flush=True,
+                    )
+                current_interval = self._calculate_backoff(current_interval)
 
     async def _aload_data(
         self,

llama_parse/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "poetry.core.masonry.api"
 
 [tool.poetry]
 name = "llama-parse"
-version = "0.6.2"
+version = "0.6.9"
 description = "Parse files into RAG-Optimized formats."
 authors = ["Logan Markewich <logan@llamaindex.ai>"]
 license = "MIT"
@@ -13,7 +13,7 @@ packages = [{include = "llama_parse"}]
 
 [tool.poetry.dependencies]
 python = ">=3.9,<4.0"
-llama-cloud-services = ">=0.6.2"
+llama-cloud-services = ">=0.6.9"
 
 [tool.poetry.group.dev.dependencies]
 pytest = "^8.0.0"

pyproject.toml

@@ -8,7 +8,7 @@ python_version = "3.10"
 
 [tool.poetry]
 name = "llama-cloud-services"
-version = "0.6.2"
+version = "0.6.9"
 description = "Tailored SDK clients for LlamaCloud services."
 authors = ["Logan Markewich <logan@runllama.ai>"]
 license = "MIT"
@@ -18,11 +18,12 @@ packages = [{include = "llama_cloud_services"}]
 [tool.poetry.dependencies]
 python = ">=3.9,<4.0"
 llama-index-core = ">=0.11.0"
-llama-cloud = "^0.1.11"
+llama-cloud = "^0.1.17"
 pydantic = "!=2.10"
 click = "^8.1.7"
 python-dotenv = "^1.0.1"
 eval-type-backport = {python = "<3.10", version = "^0.2.0"}
+platformdirs = "^4.3.7"
 
 [tool.poetry.group.dev.dependencies]
 pytest = "^8.0.0"

tests/extract/test_benchmark.py

@@ -60,7 +60,7 @@ def get_test_cases():
 
         settings = [
             ExtractConfig(extraction_mode=ExtractMode.FAST),
-            ExtractConfig(extraction_mode=ExtractMode.ACCURATE),
+            ExtractConfig(extraction_mode=ExtractMode.BALANCED),
         ]
 
         for input_file in sorted(input_files):

tests/extract/test_extract_api.py

@@ -182,7 +182,14 @@ class TestExtractionAgent:
         assert "new_field" in updated_agent.data_schema["properties"]
 
     def test_list_extraction_runs(self, test_agent: ExtractionAgent):
-        assert len(test_agent.list_extraction_runs()) == 0
+        assert test_agent.list_extraction_runs().total == 0
         test_agent.extract(TEST_PDF)
         runs = test_agent.list_extraction_runs()
-        assert len(runs) > 0
+        assert runs.total > 0
+
+    def test_delete_extraction_run(self, test_agent: ExtractionAgent):
+        assert test_agent.list_extraction_runs().total == 0
+        run = test_agent.extract(TEST_PDF)
+        test_agent.delete_extraction_run(run.id)
+        runs = test_agent.list_extraction_runs()
+        assert runs.total == 0

tests/extract/test_extract_e2e.py

@@ -55,7 +55,7 @@ def get_test_cases():
 
         settings = [
             ExtractConfig(extraction_mode=ExtractMode.FAST),
-            ExtractConfig(extraction_mode=ExtractMode.ACCURATE),
+            ExtractConfig(extraction_mode=ExtractMode.BALANCED),
         ]
 
         for input_file in sorted(input_files):

tests/report/test_llama_report.py

@@ -81,6 +81,11 @@ async def test_create_and_delete_report(
 
 
 @pytest.mark.asyncio
+@pytest.mark.xfail(
+    condition=lambda: os.getenv("CI"),
+    reason="Report plan sometimes times out",
+    raises=TimeoutError,
+)
 async def test_report_plan_workflow(report: ReportClient) -> None:
     """Test the report planning workflow."""
     # Wait for the plan