add compact md tables

* Fix bytes input for LlamaExtract

* backwards compatibility

* compat python 3.9

README.md

@@ -38,6 +38,25 @@ See the quickstart guides for each service for more information:
 - [LlamaReport (beta/invite-only)](./report.md)
 - [LlamaExtract (beta/invite-only)](./extract.md)
 
+## Switch to EU SaaS 🇪🇺
+
+If you are interested in using LlamaCloud services in the EU, you can adjust your base URL to `https://api.cloud.eu.llamaindex.ai`.
+
+You can also create your API key in the EU region [here](https://cloud.eu.llamaindex.ai).
+
+```python
+from llama_cloud_services import (
+    LlamaParse,
+    LlamaReport,
+    LlamaExtract,
+    EU_BASE_URL,
+)
+
+parser = LlamaParse(api_key="YOUR_API_KEY", base_url=EU_BASE_URL)
+report = LlamaReport(api_key="YOUR_API_KEY", base_url=EU_BASE_URL)
+extract = LlamaExtract(api_key="YOUR_API_KEY", base_url=EU_BASE_URL)
+```
+
 ## Documentation
 
 You can see complete SDK and API documentation for each service on [our official docs](https://docs.cloud.llamaindex.ai/).

examples/extract/data/automotive_sector_analysis/modeling_assumptions.txt

@@ -0,0 +1,10 @@
+# Financial Modeling Assumptions
+Discount Rate: 8%
+Terminal Growth Rate: 2%
+Tax Rate: 25%
+Revenue Growth (Years 1-5): 10% per annum
+Revenue Growth (Years 6-10): 5% per annum
+Capital Expenditures as % of Revenue: 7%
+Working Capital Assumption: 3% of Revenue
+Depreciation Rate: 10% per annum
+Cost of Capital Assumption: 8%

examples/extract/lm317_structured_extraction.ipynb

@@ -0,0 +1,318 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "1f6bd03d-1b8b-45a0-bc2c-5a13f1a5d8d3",
+   "metadata": {},
+   "source": [
+    "# LM317 Voltage Regulator Datasheet Structured Extraction\n",
+    "\n",
+    "<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/extract/lm317_structured_extraction.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
+    "\n",
+    "This notebook demonstrates an agentic document workflow using LlamaExtract to process an LM317 voltage regulator datasheet. In this example, we define a structured extraction schema that converts key technical fields into standardized subfields. For instance, the output voltage is split into a minimum and maximum value with a defined unit, and we capture page citations for each extracted field.\n",
+    "\n",
+    "The target user is an electronics engineer at a component manufacturing company who needs to consolidate datasheet information into a standardized specification sheet for design and quality control.\n",
+    "\n",
+    "This approach reduces manual data entry, improves extraction accuracy and standardization, and provides traceability for each technical detail."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "a3b8c8d5-ff3e-48ce-b0b8-29b6b1f517f8",
+   "metadata": {},
+   "source": [
+    "## Use Case Overview\n",
+    "\n",
+    "### Problem\n",
+    "Datasheets like that for the LM317 regulator are often distributed as PDFs containing multiple tables, charts, and complex textual descriptions. Engineers must manually extract technical details such as voltage ranges, dropout voltage, maximum current, input voltage range, and pin configurations. This process is error-prone and time-consuming.\n",
+    "\n",
+    "### Agent Workflow (Combination of Automation and Chat)\n",
+    "1. **Upload Datasheet:** The engineer uploads the LM317 datasheet PDF.  \n",
+    "2. **Structured Extraction:** An automated agent processes the PDF and extracts key technical details into structured fields (e.g., output voltage as a range with separate min/max values).\n",
+    "3. **Interactive Verification:** The engineer can query the agent (via chat) for further details or clarification (e.g., \"Show me the detailed pin configuration extraction\") and review the cited pages.\n",
+    "\n",
+    "**Value Delivered:**\n",
+    "- Up to 70% reduction in manual data extraction time.\n",
+    "- Increased accuracy and standardization with structured fields."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "a704e843-54be-4969-842b-713584cb3c35",
+   "metadata": {},
+   "source": [
+    "## Setup and Download Data\n",
+    "\n",
+    "Download the [LM317 Datasheet](https://www.ti.com/lit/ds/symlink/lm317.pdf) and setup LlamaExtract."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "6e5b1f91-8785-44d4-a710-8be1b48b76de",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!mkdir -p data/lm317_structured_extraction\n",
+    "!wget https://www.ti.com/lit/ds/symlink/lm317.pdf -O data/lm317_structured_extraction/lm317.pdf"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "f17b914a-00ed-4b63-8198-69fd7c4a7c62",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from dotenv import load_dotenv\n",
+    "from llama_cloud_services import LlamaExtract\n",
+    "from llama_cloud.core.api_error import ApiError\n",
+    "\n",
+    "# Load environment variables (ensure LLAMA_CLOUD_API_KEY is set in your .env file)\n",
+    "load_dotenv(override=True)\n",
+    "\n",
+    "# Initialize the LlamaExtract client\n",
+    "llama_extract = LlamaExtract(\n",
+    "    project_id=\"<project_id>\",\n",
+    "    organization_id=\"<organization_id>\",\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "ed9f6e9a-96c8-4ee1-8b45-0b6a4f7dbbf1",
+   "metadata": {},
+   "source": [
+    "## Defining a Structured Extraction Schema\n",
+    "\n",
+    "We now define a rich Pydantic schema to extract technical specifications from the LM317 datasheet. In this schema:\n",
+    "\n",
+    "- The **output_voltage** and **input_voltage** fields are structured as ranges with separate minimum and maximum values and a unit.\n",
+    "- The **pin_configuration** field is structured to include a pin count and a descriptive layout.\n",
+    "- Additional technical fields (e.g., dropout voltage, max current) are captured as numbers.\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "4f7e9b44-5e69-4b30-9864-cd98f1e2a7d4",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from pydantic import BaseModel, Field\n",
+    "from typing import List\n",
+    "\n",
+    "\n",
+    "class VoltageRange(BaseModel):\n",
+    "    min_voltage: float = Field(..., description=\"Minimum voltage in volts\")\n",
+    "    max_voltage: float = Field(..., description=\"Maximum voltage in volts\")\n",
+    "    unit: str = Field(\"V\", description=\"Voltage unit\")\n",
+    "\n",
+    "\n",
+    "class PinConfiguration(BaseModel):\n",
+    "    pin_count: int = Field(..., description=\"Number of pins\")\n",
+    "    layout: str = Field(..., description=\"Detailed pin layout description\")\n",
+    "\n",
+    "\n",
+    "class LM317Spec(BaseModel):\n",
+    "    component_name: str = Field(..., description=\"Name of the component\")\n",
+    "    output_voltage: VoltageRange = Field(\n",
+    "        ..., description=\"Output voltage range specification\"\n",
+    "    )\n",
+    "    dropout_voltage: float = Field(..., description=\"Dropout voltage in volts\")\n",
+    "    max_current: float = Field(..., description=\"Maximum current rating in amperes\")\n",
+    "    input_voltage: VoltageRange = Field(\n",
+    "        ..., description=\"Input voltage range specification\"\n",
+    "    )\n",
+    "    pin_configuration: PinConfiguration = Field(\n",
+    "        ..., description=\"Pin configuration details\"\n",
+    "    )\n",
+    "    features: List[str] = Field([], description=\"List of additional technical features\")\n",
+    "\n",
+    "\n",
+    "class LM317Schema(BaseModel):\n",
+    "    specs: List[LM317Spec] = Field(\n",
+    "        ..., description=\"List of extracted LM317 technical specifications\"\n",
+    "    )"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "e0508e38-35be-446c-afe7-129e39553281",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "try:\n",
+    "    existing_agent = llama_extract.get_agent(name=\"lm317-datasheet\")\n",
+    "    if existing_agent:\n",
+    "        llama_extract.delete_agent(existing_agent.id)\n",
+    "except ApiError as e:\n",
+    "    if e.status_code == 404:\n",
+    "        pass\n",
+    "    else:\n",
+    "        raise"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "bb197dfd-dd37-459e-8953-cc1b12f25bdd",
+   "metadata": {},
+   "source": [
+    "Here we use our balanced extraction mode."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "e3defc0a-c685-4fbd-bbb1-1270f1442e72",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from llama_cloud import ExtractConfig\n",
+    "\n",
+    "extract_config = ExtractConfig(\n",
+    "    extraction_mode=\"BALANCED\",\n",
+    ")\n",
+    "\n",
+    "agent = llama_extract.create_agent(\n",
+    "    name=\"lm317-datasheet\", data_schema=LM317Schema, config=extract_config\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "c0a0f9f9-2ef3-4a38-bd74-68d2c2e9e2d8",
+   "metadata": {},
+   "source": [
+    "## Extracting Information from the LM317 Datasheet\n",
+    "\n",
+    "For this demonstration, please download a publicly available LM317 voltage regulator datasheet (for example, from Texas Instruments) and save it as `lm317.pdf` in the `./data` directory. Then run the cell below to extract the structured technical specifications."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "c58e8b7a-8f9b-46f3-8f72-3c2f96b49e8f",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "Uploading files: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:01<00:00,  1.08s/it]\n",
+      "Creating extraction jobs: 100%|████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.96it/s]\n",
+      "Extracting files: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [01:27<00:00, 87.38s/it]\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Path to the LM317 datasheet PDF\n",
+    "lm317_pdf = \"./data/lm317_structured_extraction/lm317.pdf\"\n",
+    "\n",
+    "# Extract structured technical specifications from the datasheet\n",
+    "lm317_extract = agent.extract(lm317_pdf)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "1a2e2e44-6c48-4a38-a6de-5f2f3c7d4d8b",
+   "metadata": {},
+   "source": [
+    "## Assessing the Extraction Results\n",
+    "\n",
+    "The output will be a consolidated list of LM317 technical specifications. For each entry, you should see structured fields including:\n",
+    "\n",
+    "- **component_name**\n",
+    "- **output_voltage** as a range (with separate `min_voltage` and `max_voltage` plus `unit`)\n",
+    "- **dropout_voltage** and **max_current** as numbers\n",
+    "- **input_voltage** as a structured range\n",
+    "- **pin_configuration** with a `pin_count` and `layout`\n",
+    "- **features** (if available)\n",
+    "\n",
+    "This structured approach makes it easier to standardize the information for downstream integration and verification. Engineers can click on the cited page numbers (in a UI that supports it) to validate the extraction."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "fb2abc44-7c9b-4b19-958e-d0d7b390ae57",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "{'specs': [{'component_name': 'LM317',\n",
+       "   'output_voltage': {'min_voltage': 1.25, 'max_voltage': 37.0, 'unit': 'V'},\n",
+       "   'dropout_voltage': 0.0,\n",
+       "   'max_current': 1.5,\n",
+       "   'input_voltage': {'min_voltage': 4.25, 'max_voltage': 40.0, 'unit': 'V'},\n",
+       "   'pin_configuration': {'pin_count': 3,\n",
+       "    'layout': '1: ADJUST, 2: OUTPUT, 3: INPUT'},\n",
+       "   'features': ['Output voltage range adjustable from 1.25 V to 37 V',\n",
+       "    'Output current greater than 1.5 A',\n",
+       "    'Internal short-circuit current limiting',\n",
+       "    'Thermal overload protection',\n",
+       "    'Output safe-area compensation']}]}"
+      ]
+     },
+     "execution_count": null,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "# Display the extraction results\n",
+    "lm317_extract.data"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "c7a2a523-095e-40bf-b713-f509c13a7747",
+   "metadata": {},
+   "source": [
+    "You can also see the output result in the UI."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "dc22dfa5-b667-4fb0-8dbe-24e401b12389",
+   "metadata": {},
+   "source": [
+    "![](data/lm317_structured_extraction/lm317_extraction.png)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "e0e0c12a-9f89-4bb3-b40d-3e9f7c6d2fef",
+   "metadata": {},
+   "source": [
+    "## Conclusion\n",
+    "\n",
+    "This notebook demonstrated how to use LlamaExtract with a structured extraction schema for the LM317 voltage regulator datasheet. By defining detailed subfields (such as splitting voltage ranges into minimum and maximum values, and structuring the pin configuration), we ensure that the extracted data is standardized and traceable through page citations. This approach minimizes manual effort and improves accuracy, providing a robust example of an agentic document workflow for technical documentation processing.\n",
+    "\n",
+    "Feel free to modify or extend the schema to capture additional technical details or to suit your own use cases."
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "llama_parse",
+   "language": "python",
+   "name": "llama_parse"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

examples/extract/solar_panel_e2e_comparison.ipynb

@@ -0,0 +1,450 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "00f6713b-2a32-4f8f-80e5-9a7d9b6e3b90",
+   "metadata": {},
+   "source": [
+    "# Solar Panel Datasheet Comparison Workflow\n",
+    "\n",
+    "<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/extract/solar_panel_e2e_comparison.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
+    "\n",
+    "\n",
+    "This notebook demonstrates an end‑to‑end agentic workflow using LlamaExtract and the LlamaIndex event‑driven workflow framework. In this workflow, we:\n",
+    "\n",
+    "1. **Extract** structured technical specifications from a solar panel datasheet (e.g. a PDF downloaded from a vendor).\n",
+    "2. **Load** design requirements (provided as a text blob) for a lab‑grade solar panel.\n",
+    "3. **Generate** a detailed comparison report by triggering an event that injects both the extracted data and the requirements into an LLM prompt.\n",
+    "\n",
+    "The workflow is designed for renewable energy engineers who need to quickly validate that a solar panel meets specific design criteria.\n",
+    "\n",
+    "The following notebook uses the event‑driven syntax (with custom events, steps, and a workflow class) adapted from the technical datasheet and contract review examples."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "36d8e34e-ed98-46ac-b744-1642f6e253d5",
+   "metadata": {},
+   "source": [
+    "## Setup and Load Data\n",
+    "\n",
+    "We download the [Honey M TSM-DE08M.08(II) datasheet](https://static.trinasolar.com/sites/default/files/EU_Datasheet_HoneyM_DE08M.08%28II%29_2021_A.pdf) as a PDF.\n",
+    "\n",
+    "**NOTE**: The design requirements are already stored in `data/solar_panel_e2e_comparison/design_reqs.txt`."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "1de7b1b3-c285-492c-8b2e-b37974b4fc63",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "--2025-04-01 14:47:56--  https://static.trinasolar.com/sites/default/files/EU_Datasheet_HoneyM_DE08M.08%28II%29_2021_A.pdf\n",
+      "Resolving static.trinasolar.com (static.trinasolar.com)... 47.246.23.232, 47.246.23.234, 47.246.23.227, ...\n",
+      "Connecting to static.trinasolar.com (static.trinasolar.com)|47.246.23.232|:443... connected.\n",
+      "WARNING: cannot verify static.trinasolar.com's certificate, issued by ‘CN=DigiCert Global G2 TLS RSA SHA256 2020 CA1,O=DigiCert Inc,C=US’:\n",
+      "  Unable to locally verify the issuer's authority.\n",
+      "HTTP request sent, awaiting response... 200 OK\n",
+      "Length: 1888183 (1.8M) [application/pdf]\n",
+      "Saving to: ‘data/solar_panel_e2e_comparison/datasheet.pdf’\n",
+      "\n",
+      "data/solar_panel_e2 100%[===================>]   1.80M  7.47MB/s    in 0.2s    \n",
+      "\n",
+      "2025-04-01 14:47:56 (7.47 MB/s) - ‘data/solar_panel_e2e_comparison/datasheet.pdf’ saved [1888183/1888183]\n",
+      "\n"
+     ]
+    }
+   ],
+   "source": [
+    "!wget https://static.trinasolar.com/sites/default/files/EU_Datasheet_HoneyM_DE08M.08%28II%29_2021_A.pdf -O data/solar_panel_e2e_comparison/datasheet.pdf --no-check-certificate"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "89d2f4c9-f785-424d-a409-3381796c457c",
+   "metadata": {},
+   "source": [
+    "## Define the Structured Extraction Schema\n",
+    "\n",
+    "We define a new, rich schema called `SolarPanelSchema` to capture key technical details from the datasheet. This schema includes:\n",
+    "\n",
+    "- **PowerRange:** Structured as minimum and maximum power output (in Watts).\n",
+    "- **SolarPanelSpec:** Includes module name, power output range, maximum efficiency, certifications, and a mapping of page citations.\n",
+    "\n",
+    "This schema replaces the earlier LM317 schema and will be used when creating our extraction agent."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "bfb40d48-36e0-4b1c-97a1-32a1704c582b",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from pydantic import BaseModel, Field\n",
+    "from typing import List\n",
+    "\n",
+    "\n",
+    "class PowerRange(BaseModel):\n",
+    "    min_power: float = Field(..., description=\"Minimum power output in Watts\")\n",
+    "    max_power: float = Field(..., description=\"Maximum power output in Watts\")\n",
+    "    unit: str = Field(\"W\", description=\"Power unit\")\n",
+    "\n",
+    "\n",
+    "class SolarPanelSpec(BaseModel):\n",
+    "    module_name: str = Field(..., description=\"Name or model of the solar panel module\")\n",
+    "    power_output: PowerRange = Field(..., description=\"Power output range\")\n",
+    "    maximum_efficiency: float = Field(\n",
+    "        ..., description=\"Maximum module efficiency in percentage\"\n",
+    "    )\n",
+    "    temperature_coefficient: float = Field(\n",
+    "        ..., description=\"Temperature coefficient in %/°C\"\n",
+    "    )\n",
+    "    certifications: List[str] = Field([], description=\"List of certifications\")\n",
+    "    page_citations: dict = Field(\n",
+    "        ..., description=\"Mapping of each extracted field to its page numbers\"\n",
+    "    )\n",
+    "\n",
+    "\n",
+    "class SolarPanelSchema(BaseModel):\n",
+    "    specs: List[SolarPanelSpec] = Field(\n",
+    "        ..., description=\"List of extracted solar panel specifications\"\n",
+    "    )"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "19dc309e-7cec-43c1-8f6c-72e14df58f8f",
+   "metadata": {},
+   "source": [
+    "## Initialize Extraction Agent\n",
+    "\n",
+    "Here we initialize our extraction agent that will be responsible for extracting the schema from the solar panel datasheet."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "c9d9f4a2-2e14-493d-8a7e-d01159d38b8f",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from dotenv import load_dotenv\n",
+    "from llama_cloud_services import LlamaExtract\n",
+    "from llama_cloud.core.api_error import ApiError\n",
+    "from llama_cloud import ExtractConfig\n",
+    "\n",
+    "# Initialize the LlamaExtract client\n",
+    "llama_extract = LlamaExtract(\n",
+    "    project_id=\"2fef999e-1073-40e6-aeb3-1f3c0e64d99b\",\n",
+    "    organization_id=\"43b88c8f-e488-46f6-9013-698e3d2e374a\",\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "ec0eb2a7-6e02-45da-a6af-227e2f7c81f2",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "try:\n",
+    "    existing_agent = llama_extract.get_agent(name=\"solar-panel-datasheet\")\n",
+    "    if existing_agent:\n",
+    "        llama_extract.delete_agent(existing_agent.id)\n",
+    "except ApiError as e:\n",
+    "    if e.status_code == 404:\n",
+    "        pass\n",
+    "    else:\n",
+    "        raise\n",
+    "\n",
+    "extract_config = ExtractConfig(\n",
+    "    extraction_mode=\"BALANCED\",\n",
+    ")\n",
+    "\n",
+    "agent = llama_extract.create_agent(\n",
+    "    name=\"solar-panel-datasheet\", data_schema=SolarPanelSchema, config=extract_config\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "b4d7bb60-0456-4a2d-8d48-14f9bb3e71d2",
+   "metadata": {},
+   "source": [
+    "## Workflow Overview\n",
+    "\n",
+    "The workflow consists of four main steps:\n",
+    "\n",
+    "1. **parse_datasheet:** Reads the solar panel datasheet (PDF) and converts its content into text (with page citations).\n",
+    "2. **load_requirements:** Loads the design requirements (as a text blob) that will be injected into the prompt.\n",
+    "3. **generate_comparison_report:** Constructs a prompt using the extracted datasheet content and design requirements and triggers the LLM to generate a comparison report.\n",
+    "4. **output_result:** Logs and returns the final report as the workflow’s result.\n",
+    "\n",
+    "Each step is implemented as an asynchronous function decorated with `@step`, and the workflow is built by subclassing `Workflow`."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "7c482e3a-66b4-4e1b-8d2d-9a9c6b3967f3",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from llama_index.core.workflow import (\n",
+    "    Event,\n",
+    "    StartEvent,\n",
+    "    StopEvent,\n",
+    "    Context,\n",
+    "    Workflow,\n",
+    "    step,\n",
+    ")\n",
+    "from llama_index.llms.openai import OpenAI\n",
+    "from llama_index.core.prompts import ChatPromptTemplate\n",
+    "from llama_cloud_services import LlamaExtract\n",
+    "from llama_cloud.core.api_error import ApiError\n",
+    "from pydantic import BaseModel, Field\n",
+    "from typing import List\n",
+    "\n",
+    "\n",
+    "# Define output schema for the comparison report (for reference)\n",
+    "class ComparisonReportOutput(BaseModel):\n",
+    "    component_name: str = Field(\n",
+    "        ..., description=\"The name of the component being evaluated.\"\n",
+    "    )\n",
+    "    meets_requirements: bool = Field(\n",
+    "        ...,\n",
+    "        description=\"Overall indicator of whether the component meets the design criteria.\",\n",
+    "    )\n",
+    "    summary: str = Field(..., description=\"A brief summary of the evaluation results.\")\n",
+    "    details: dict = Field(\n",
+    "        ..., description=\"Detailed comparisons for each key parameter.\"\n",
+    "    )\n",
+    "\n",
+    "\n",
+    "# Define custom events\n",
+    "\n",
+    "\n",
+    "class DatasheetParseEvent(Event):\n",
+    "    datasheet_content: dict\n",
+    "\n",
+    "\n",
+    "class RequirementsLoadEvent(Event):\n",
+    "    requirements_text: str\n",
+    "\n",
+    "\n",
+    "class ComparisonReportEvent(Event):\n",
+    "    report: ComparisonReportOutput\n",
+    "\n",
+    "\n",
+    "class LogEvent(Event):\n",
+    "    msg: str\n",
+    "    delta: bool = False\n",
+    "\n",
+    "\n",
+    "# For our demonstration, we assume that LlamaExtract is used to parse the datasheet into text.\n",
+    "# We'll also use OpenAI (via LlamaIndex) as our LLM for generating the report.\n",
+    "\n",
+    "llm = OpenAI(model=\"gpt-4o\")  # or your preferred model"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "67a0c391-c7f5-4b93-8d6b-9e31b2d7a817",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "class SolarPanelComparisonWorkflow(Workflow):\n",
+    "    \"\"\"\n",
+    "    Workflow to extract data from a solar panel datasheet and generate a comparison report\n",
+    "    against provided design requirements.\n",
+    "    \"\"\"\n",
+    "\n",
+    "    def __init__(self, agent: LlamaExtract, requirements_path: str, **kwargs):\n",
+    "        super().__init__(**kwargs)\n",
+    "        self.agent = agent\n",
+    "        # Load design requirements from file as a text blob\n",
+    "        with open(requirements_path, \"r\") as f:\n",
+    "            self.requirements_text = f.read()\n",
+    "\n",
+    "    @step\n",
+    "    async def parse_datasheet(\n",
+    "        self, ctx: Context, ev: StartEvent\n",
+    "    ) -> DatasheetParseEvent:\n",
+    "        # datasheet_path is provided in the StartEvent\n",
+    "        datasheet_path = (\n",
+    "            ev.datasheet_path\n",
+    "        )  # e.g., \"./data/solar_panel_comparison/datasheet.pdf\"\n",
+    "        extraction_result = await self.agent.aextract(datasheet_path)\n",
+    "        datasheet_dict = (\n",
+    "            extraction_result.data\n",
+    "        )  # assumed to be a string with page citations\n",
+    "        await ctx.set(\"datasheet_content\", datasheet_dict)\n",
+    "        ctx.write_event_to_stream(LogEvent(msg=\"Datasheet parsed successfully.\"))\n",
+    "        return DatasheetParseEvent(datasheet_content=datasheet_dict)\n",
+    "\n",
+    "    @step\n",
+    "    async def load_requirements(\n",
+    "        self, ctx: Context, ev: DatasheetParseEvent\n",
+    "    ) -> RequirementsLoadEvent:\n",
+    "        # Use the pre-loaded requirements text from __init__\n",
+    "        req_text = self.requirements_text\n",
+    "        ctx.write_event_to_stream(LogEvent(msg=\"Design requirements loaded.\"))\n",
+    "        return RequirementsLoadEvent(requirements_text=req_text)\n",
+    "\n",
+    "    @step\n",
+    "    async def generate_comparison_report(\n",
+    "        self, ctx: Context, ev: RequirementsLoadEvent\n",
+    "    ) -> StopEvent:\n",
+    "        # Build a prompt that injects both the extracted datasheet content and the design requirements\n",
+    "        datasheet_content = await ctx.get(\"datasheet_content\")\n",
+    "        prompt_str = \"\"\"\n",
+    "You are an expert renewable energy engineer.\n",
+    "\n",
+    "Compare the following solar panel datasheet information with the design requirements.\n",
+    "\n",
+    "Design Requirements:\n",
+    "{requirements_text}\n",
+    "\n",
+    "Extracted Datasheet Information:\n",
+    "{datasheet_content}\n",
+    "\n",
+    "Generate a detailed comparison report in JSON format with the following schema:\n",
+    "  - component_name: string\n",
+    "  - meets_requirements: boolean\n",
+    "  - summary: string\n",
+    "  - details: dictionary of comparisons for each parameter\n",
+    "\n",
+    "For each parameter (Maximum Power, Open-Circuit Voltage, Short-Circuit Current, Efficiency, Temperature Coefficient),\n",
+    "indicate PASS or FAIL and provide brief explanations and recommendations.\n",
+    "\"\"\"\n",
+    "\n",
+    "        # extract from contract\n",
+    "        prompt = ChatPromptTemplate.from_messages([(\"user\", prompt_str)])\n",
+    "\n",
+    "        # Call the LLM to generate the report using the prompt\n",
+    "        report_output = await llm.astructured_predict(\n",
+    "            ComparisonReportOutput,\n",
+    "            prompt,\n",
+    "            requirements_text=ev.requirements_text,\n",
+    "            datasheet_content=str(datasheet_content),\n",
+    "        )\n",
+    "        ctx.write_event_to_stream(LogEvent(msg=\"Comparison report generated.\"))\n",
+    "        return StopEvent(\n",
+    "            result={\"report\": report_output, \"datasheet_content\": datasheet_content}\n",
+    "        )"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "d205f532-1a11-4a48-b5a8-87a7f85e9ce7",
+   "metadata": {},
+   "source": [
+    "## Running the Workflow\n",
+    "\n",
+    "Below, we instantiate and run the workflow. We inject the design requirements as a text blob (no custom code to load) and pass the path to the solar panel datasheet (the HoneyM datasheet from Trina).\n",
+    "\n",
+    "The design requirements are:\n",
+    "\n",
+    "```\n",
+    "Solar Panel Design Requirements:\n",
+    "- Power Output Range: ≥ 350 W\n",
+    "- Maximum Efficiency: ≥ 18%\n",
+    "- Certifications: Must include IEC61215 and UL1703\n",
+    "```\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "6b24fa61-a2f5-4ebb-84eb-1c9b48683b1b",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import nest_asyncio\n",
+    "\n",
+    "nest_asyncio.apply()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "be3ebad5-1f70-4671-a2ec-17bf9e4d788f",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Path to design requirements file (e.g., a text file with design criteria for solar panels)\n",
+    "requirements_path = \"./data/solar_panel_e2e_comparison/design_reqs.txt\"\n",
+    "\n",
+    "# Instantiate the workflow\n",
+    "workflow = SolarPanelComparisonWorkflow(\n",
+    "    agent=agent, requirements_path=requirements_path, verbose=True, timeout=120\n",
+    ")\n",
+    "\n",
+    "# Run the workflow; pass the datasheet path in the StartEvent\n",
+    "result = await workflow.run(\n",
+    "    datasheet_path=\"./data/solar_panel_e2e_comparison/datasheet.pdf\"\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "e1e61f1e-8701-4acc-8f99-cc89d8aae535",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "********Final Comparison Report:********\n",
+      "\n",
+      "{\n",
+      "    \"component_name\": \"TSM-DE08M.08(II)\",\n",
+      "    \"meets_requirements\": true,\n",
+      "    \"summary\": \"The solar panel TSM-DE08M.08(II) meets all the design requirements, making it a suitable choice for the intended application.\",\n",
+      "    \"details\": {\n",
+      "        \"Maximum Power Output\": \"PASS - The panel's power output ranges from 360 W to 385 W, exceeding the minimum requirement of 350 W.\",\n",
+      "        \"Open-Circuit Voltage\": \"PASS - The datasheet does not specify Voc, but the panel meets other critical requirements. Verification of Voc is recommended.\",\n",
+      "        \"Short-Circuit Current\": \"PASS - The datasheet does not specify Isc, but the panel meets other critical requirements. Verification of Isc is recommended.\",\n",
+      "        \"Efficiency\": \"PASS - The panel's efficiency is 21.0%, which is above the required 18%.\",\n",
+      "        \"Temperature Coefficient\": \"PASS - The temperature coefficient is -0.34%/°C, which is better than the maximum allowable -0.5%/°C.\"\n",
+      "    }\n",
+      "}\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(\"\\n********Final Comparison Report:********\\n\")\n",
+    "print(result[\"report\"].model_dump_json(indent=4))\n",
+    "# print(\"\\n********Datasheet Content:********\\n\", result[\"datasheet_content\"])"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "llama_parse",
+   "language": "python",
+   "name": "llama_parse"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

extract.md

@@ -1,8 +1,5 @@
 # LlamaExtract
 
-> **⚠️ EXPERIMENTAL**
-> This library is under active development with frequent breaking changes. APIs and functionality may change significantly between versions. If you're interested in being an early adopter, please contact us at [support@llamaindex.ai](mailto:support@llamaindex.ai) or join our [Discord](https://discord.com/invite/eN6D2HQ4aX).
-
 LlamaExtract provides a simple API for extracting structured data from unstructured documents like PDFs, text files and images (upcoming).
 
 ## Quick Start

llama_cloud_services/__init__.py

@@ -1,6 +1,7 @@
 from llama_cloud_services.parse import LlamaParse
 from llama_cloud_services.report import ReportClient, LlamaReport
 from llama_cloud_services.extract import LlamaExtract, ExtractionAgent
+from llama_cloud_services.constants import EU_BASE_URL
 
 __all__ = [
     "LlamaParse",
@@ -8,4 +9,5 @@ __all__ = [
     "LlamaReport",
     "LlamaExtract",
     "ExtractionAgent",
+    "EU_BASE_URL",
 ]

llama_cloud_services/constants.py

@@ -0,0 +1 @@
+EU_BASE_URL = "https://api.cloud.eu.llamaindex.ai"

llama_cloud_services/extract/__init__.py

@@ -1,3 +1,7 @@
-from llama_cloud_services.extract.extract import LlamaExtract, ExtractionAgent
+from llama_cloud_services.extract.extract import (
+    LlamaExtract,
+    ExtractionAgent,
+    SourceText,
+)
 
-__all__ = ["LlamaExtract", "ExtractionAgent"]
+__all__ = ["LlamaExtract", "ExtractionAgent", "SourceText"]

llama_cloud_services/extract/extract.py

@@ -34,15 +34,39 @@ from concurrent.futures import ThreadPoolExecutor
 
 T = TypeVar("T")
 
-FileInput = Union[str, Path, bytes, BufferedIOBase]
+
 SchemaInput = Union[JSONObjectType, Type[BaseModel]]
 
 DEFAULT_EXTRACT_CONFIG = ExtractConfig(
     extraction_target=ExtractTarget.PER_DOC,
-    extraction_mode=ExtractMode.ACCURATE,
+    extraction_mode=ExtractMode.BALANCED,
 )
 
 
+class SourceText:
+    def __init__(
+        self,
+        file: Union[bytes, BufferedIOBase, str, Path],
+        filename: Optional[str] = None,
+    ):
+        self.file = file
+        self.filename = filename
+        self._validate()
+
+    def _validate(self) -> None:
+        """Ensure filename is provided when needed."""
+        if isinstance(self.file, (bytes, BufferedIOBase)):
+            if not self.filename and hasattr(self.file, "name"):
+                self.filename = os.path.basename(str(self.file.name))
+            elif not hasattr(self.file, "name") and self.filename is None:
+                raise ValueError(
+                    "filename must be provided when file is bytes or a file-like object without a name"
+                )
+
+
+FileInput = Union[str, Path, BufferedIOBase, SourceText]
+
+
 class ExtractionAgent:
     """Class representing a single extraction agent with methods for extraction operations."""
 
@@ -141,26 +165,59 @@ class ExtractionAgent:
     def config(self, config: ExtractConfig) -> None:
         self._config = config
 
-    async def _upload_file(self, file_input: FileInput) -> File:
-        """Upload a file for extraction."""
-        if isinstance(file_input, BufferedIOBase):
-            upload_file = file_input
-        elif isinstance(file_input, bytes):
-            upload_file = BytesIO(file_input)
-        elif isinstance(file_input, (str, Path)):
-            upload_file = open(file_input, "rb")
-        else:
-            raise ValueError(
-                "file_input must be either a file path string, file bytes, or buffer object"
-            )
+    async def upload_file(self, file_input: SourceText) -> File:
+        """Upload a file for extraction.
 
+        Args:
+            file_input: The file to upload (path, bytes, or file-like object)
+
+        Raises:
+            ValueError: If filename is not provided for bytes input or for file-like objects
+                       without a name attribute.
+        """
         try:
+            file_contents: Union[BufferedIOBase, BytesIO]
+            if isinstance(file_input.file, (str, Path)):
+                file_contents = open(file_input.file, "rb")
+            elif isinstance(file_input.file, bytes):
+                file_contents = BytesIO(file_input.file)
+            else:
+                file_contents = file_input.file
+            # Add name attribute to file object if needed
+            if not hasattr(file_contents, "name"):
+                file_contents.name = file_input.filename  # type: ignore
+
             return await self._client.files.upload_file(
-                project_id=self._project_id, upload_file=upload_file
+                project_id=self._project_id, upload_file=file_contents
             )
         finally:
-            if isinstance(upload_file, BufferedReader):
-                upload_file.close()
+            if isinstance(file_contents, BufferedReader):
+                file_contents.close()
+
+    async def _upload_file(self, file_input: FileInput) -> File:
+        source_text = None
+        if isinstance(file_input, SourceText):
+            source_text = file_input
+        elif isinstance(file_input, (str, Path)):
+            path = Path(file_input)
+            source_text = SourceText(file=path, filename=path.name)
+        else:
+            # Try to get filename from the file object if not provided
+            filename = None
+            if hasattr(file_input, "name"):
+                filename = os.path.basename(str(file_input.name))
+            if filename is None:
+                raise ValueError(
+                    "Use SourceText to provide filename when uploading bytes or file-like objects."
+                )
+
+            warnings.warn(
+                "Use SourceText instead of bytes or file-like objects",
+                DeprecationWarning,
+            )
+            source_text = SourceText(file=file_input, filename=filename)
+
+        return await self.upload_file(source_text)
 
     async def _wait_for_job_result(self, job_id: str) -> Optional[ExtractRun]:
         """Wait for and return the results of an extraction job."""
@@ -208,7 +265,7 @@ class ExtractionAgent:
             )
         )
 
-    async def _queue_extraction_test(
+    async def _run_extraction_test(
         self,
         files: Union[FileInput, List[FileInput]],
         extract_settings: LlamaExtractSettings,
@@ -242,7 +299,7 @@ class ExtractionAgent:
 
         job_tasks = [run_job(file) for file in uploaded_files]
         with augment_async_errors():
-            extract_jobs = await run_jobs(
+            extract_results = await run_jobs(
                 job_tasks,
                 workers=self.num_workers,
                 desc="Running extraction jobs",
@@ -250,15 +307,13 @@ class ExtractionAgent:
             )
 
         if self._verbose:
-            for file, job in zip(files, extract_jobs):
+            for file, job in zip(files, extract_results):
                 file_repr = (
                     str(file) if isinstance(file, (str, Path)) else "<bytes/buffer>"
                 )
-                print(
-                    f"Queued file extraction for file {file_repr} under job_id {job.id}"
-                )
+                print(f"Running extraction for file {file_repr} under job_id {job.id}")
 
-        return extract_jobs[0] if single_file else extract_jobs
+        return extract_results[0] if single_file else extract_results
 
     async def queue_extraction(
         self,
@@ -564,6 +619,14 @@ class LlamaExtract(BaseComponent):
         Returns:
             ExtractionAgent: The created extraction agent
         """
+        if config is not None:
+            if config.extraction_mode == ExtractMode.ACCURATE:
+                warnings.warn(
+                    "ACCURATE extraction mode is deprecated. Using BALANCED instead."
+                )
+                config.extraction_mode = ExtractMode.BALANCED
+        else:
+            config = DEFAULT_EXTRACT_CONFIG
 
         if isinstance(data_schema, dict):
             data_schema = data_schema
@@ -581,7 +644,7 @@ class LlamaExtract(BaseComponent):
                 request=ExtractAgentCreate(
                     name=name,
                     data_schema=data_schema,
-                    config=config or DEFAULT_EXTRACT_CONFIG,
+                    config=config,
                 ),
             )
         )

llama_cloud_services/parse/base.py

@@ -4,6 +4,7 @@ import os
 import time
 from contextlib import asynccontextmanager
 from copy import deepcopy
+from enum import Enum
 from io import BufferedIOBase
 from pathlib import Path, PurePath, PurePosixPath
 from typing import Any, AsyncGenerator, Dict, List, Optional, Union
@@ -52,6 +53,14 @@ def build_url(
     return base_url
 
 
+class BackoffPattern(str, Enum):
+    """Backoff pattern for polling."""
+
+    CONSTANT = "constant"
+    LINEAR = "linear"
+    EXPONENTIAL = "exponential"
+
+
 class LlamaParse(BasePydanticReader):
     """A smart-parser for files."""
 
@@ -78,6 +87,15 @@ class LlamaParse(BasePydanticReader):
         description="The interval in seconds to check if the parsing is done.",
     )
 
+    backoff_pattern: BackoffPattern = Field(
+        default=BackoffPattern.LINEAR,
+        description="Controls the backoff pattern when retrying failed requests: 'constant', 'linear', or 'exponential'.",
+    )
+    max_check_interval: int = Field(
+        default=5,
+        description="Maximum interval in seconds between polling attempts when checking job status.",
+    )
+
     custom_client: Optional[httpx.AsyncClient] = Field(
         default=None, description="A custom HTTPX client to use for sending requests."
     )
@@ -167,7 +185,10 @@ class LlamaParse(BasePydanticReader):
         default=None,
         description="The top margin of the bounding box to use to extract text from documents expressed as a float between 0 and 1 representing the percentage of the page height.",
     )
-
+    compact_markdown_table: Optional[bool] = Field(
+        default=False,
+        description="If set to true, the parser will output compact markdown table (without trailing spaces in cells).",
+    )
     continuous_mode: Optional[bool] = Field(
         default=False,
         description="Parse documents continuously, leading to better results on documents where tables span across two pages.",
@@ -276,7 +297,7 @@ class LlamaParse(BasePydanticReader):
         default=None,
         description="A templated suffix to add to the beginning of each page. If it contain `{page_number}`, it will be replaced by the page number.",
     )
-    parsing_mode: Optional[str] = Field(
+    parse_mode: Optional[str] = Field(
         default=None,
         description="The parsing mode to use, see ParsingMode enum for possible values ",
     )
@@ -284,6 +305,10 @@ class LlamaParse(BasePydanticReader):
         default=False,
         description="Use our best parser mode if set to True.",
     )
+    preserve_layout_alignment_across_pages: Optional[bool] = Field(
+        default=False,
+        description="Preserve grid alignment across page in text mode.",
+    )
     skip_diagonal_text: Optional[bool] = Field(
         default=False,
         description="If set to true, the parser will ignore diagonal text (when the text rotation in degrees modulo 90 is not 0).",
@@ -577,6 +602,9 @@ class LlamaParse(BasePydanticReader):
         if self.bbox_top is not None:
             data["bbox_top"] = self.bbox_top
 
+        if self.compact_markdown_table:
+            data["compact_markdown_table"] = self.compact_markdown_table
+
         if self.complemental_formatting_instruction:
             print(
                 "WARNING: complemental_formatting_instruction is deprecated and may be remove in a future release. Use system_prompt, system_prompt_append or user_prompt instead."
@@ -706,9 +734,17 @@ class LlamaParse(BasePydanticReader):
             )
             data["parsing_instruction"] = self.parsing_instruction
 
+        if self.parse_mode:
+            data["parse_mode"] = self.parse_mode
+
         if self.premium_mode:
             data["premium_mode"] = self.premium_mode
 
+        if self.preserve_layout_alignment_across_pages:
+            data[
+                "preserve_layout_alignment_across_pages"
+            ] = self.preserve_layout_alignment_across_pages
+
         if self.skip_diagonal_text:
             data["skip_diagonal_text"] = self.skip_diagonal_text
 
@@ -782,54 +818,87 @@ class LlamaParse(BasePydanticReader):
             if file_handle is not None:
                 file_handle.close()
 
+    def _calculate_backoff(self, current_interval: float) -> float:
+        """Calculate the next backoff interval based on the backoff pattern.
+
+        Args:
+            current_interval: The current interval in seconds
+
+        Returns:
+            The next interval in seconds
+        """
+        if self.backoff_pattern == BackoffPattern.CONSTANT:
+            return current_interval
+        elif self.backoff_pattern == BackoffPattern.LINEAR:
+            return min(current_interval + 1, float(self.max_check_interval))
+        elif self.backoff_pattern == BackoffPattern.EXPONENTIAL:
+            return min(current_interval * 2, float(self.max_check_interval))
+        return current_interval  # Default fallback
+
     async def _get_job_result(
         self, job_id: str, result_type: str, verbose: bool = False
     ) -> Dict[str, Any]:
         start = time.time()
         tries = 0
+        error_count = 0
+        current_interval: float = float(self.check_interval)
 
         # so we're not re-setting the headers & stuff on each
         # usage... assume that there is not some other
         # coro also modifying base_url and the other client related configs.
         client = self.aclient
         while True:
-            await asyncio.sleep(self.check_interval)
-            tries += 1
-            result = await client.get(JOB_STATUS_ROUTE.format(job_id=job_id))
-            if result.status_code != 200:
+            try:
+                await asyncio.sleep(current_interval)
+                tries += 1
+                result = await client.get(JOB_STATUS_ROUTE.format(job_id=job_id))
+                result.raise_for_status()  # this raises if status is not 2xx
+                # Allowed values "PENDING", "SUCCESS", "ERROR", "CANCELED"
+                result_json = result.json()
+                status = result_json["status"]
+                if status == "SUCCESS":
+                    parsed_result = await client.get(
+                        JOB_RESULT_URL.format(job_id=job_id, result_type=result_type),
+                    )
+                    return parsed_result.json()
+                elif status == "PENDING":
+                    end = time.time()
+                    if end - start > self.max_timeout:
+                        raise Exception(f"Timeout while parsing the file: {job_id}")
+                    if verbose and tries % 10 == 0:
+                        print(".", end="", flush=True)
+                    current_interval = self._calculate_backoff(current_interval)
+                else:
+                    error_code = result_json.get("error_code", "No error code found")
+                    error_message = result_json.get(
+                        "error_message", "No error message found"
+                    )
+                    exception_str = (
+                        f"Job ID: {job_id} failed with status: {status}, "
+                        f"Error code: {error_code}, Error message: {error_message}"
+                    )
+                    raise Exception(exception_str)
+            except (
+                httpx.ConnectError,
+                httpx.ReadError,
+                httpx.WriteError,
+                httpx.ConnectTimeout,
+                httpx.ReadTimeout,
+                httpx.WriteTimeout,
+                httpx.HTTPStatusError,
+            ) as err:
+                error_count += 1
                 end = time.time()
                 if end - start > self.max_timeout:
-                    raise Exception(f"Timeout while parsing the file: {job_id}")
+                    raise Exception(
+                        f"Timeout while parsing the file: {job_id}"
+                    ) from err
                 if verbose and tries % 10 == 0:
-                    print(".", end="", flush=True)
-                await asyncio.sleep(self.check_interval)
-                continue
-
-            # Allowed values "PENDING", "SUCCESS", "ERROR", "CANCELED"
-            result_json = result.json()
-            status = result_json["status"]
-            if status == "SUCCESS":
-                parsed_result = await client.get(
-                    JOB_RESULT_URL.format(job_id=job_id, result_type=result_type),
-                )
-                return parsed_result.json()
-
-            elif status == "PENDING":
-                end = time.time()
-                if end - start > self.max_timeout:
-                    raise Exception(f"Timeout while parsing the file: {job_id}")
-                if verbose and tries % 10 == 0:
-                    print(".", end="", flush=True)
-                await asyncio.sleep(self.check_interval)
-
-            else:
-                error_code = result_json.get("error_code", "No error code found")
-                error_message = result_json.get(
-                    "error_message", "No error message found"
-                )
-
-                exception_str = f"Job ID: {job_id} failed with status: {status}, Error code: {error_code}, Error message: {error_message}"
-                raise Exception(exception_str)
+                    print(
+                        f"HTTP error: {err}...",
+                        flush=True,
+                    )
+                current_interval = self._calculate_backoff(current_interval)
 
     async def _aload_data(
         self,

llama_parse/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "poetry.core.masonry.api"
 
 [tool.poetry]
 name = "llama-parse"
-version = "0.6.6"
+version = "0.6.9"
 description = "Parse files into RAG-Optimized formats."
 authors = ["Logan Markewich <logan@llamaindex.ai>"]
 license = "MIT"
@@ -13,7 +13,7 @@ packages = [{include = "llama_parse"}]
 
 [tool.poetry.dependencies]
 python = ">=3.9,<4.0"
-llama-cloud-services = ">=0.6.5"
+llama-cloud-services = ">=0.6.9"
 
 [tool.poetry.group.dev.dependencies]
 pytest = "^8.0.0"

pyproject.toml

@@ -8,7 +8,7 @@ python_version = "3.10"
 
 [tool.poetry]
 name = "llama-cloud-services"
-version = "0.6.6"
+version = "0.6.9"
 description = "Tailored SDK clients for LlamaCloud services."
 authors = ["Logan Markewich <logan@runllama.ai>"]
 license = "MIT"
@@ -18,11 +18,12 @@ packages = [{include = "llama_cloud_services"}]
 [tool.poetry.dependencies]
 python = ">=3.9,<4.0"
 llama-index-core = ">=0.11.0"
-llama-cloud = "^0.1.15"
+llama-cloud = "^0.1.17"
 pydantic = "!=2.10"
 click = "^8.1.7"
 python-dotenv = "^1.0.1"
 eval-type-backport = {python = "<3.10", version = "^0.2.0"}
+platformdirs = "^4.3.7"
 
 [tool.poetry.group.dev.dependencies]
 pytest = "^8.0.0"

tests/extract/test_benchmark.py

@@ -60,7 +60,7 @@ def get_test_cases():
 
         settings = [
             ExtractConfig(extraction_mode=ExtractMode.FAST),
-            ExtractConfig(extraction_mode=ExtractMode.ACCURATE),
+            ExtractConfig(extraction_mode=ExtractMode.BALANCED),
         ]
 
         for input_file in sorted(input_files):
@@ -133,7 +133,7 @@ async def test_extraction(
     test_case: TestCase, extraction_agent: ExtractionAgent
 ) -> None:
     start = perf_counter()
-    result = await extraction_agent._queue_extraction_test(
+    result = await extraction_agent._run_extraction_test(
         test_case.input_file,
         extract_settings=LlamaExtractSettings(
             llama_parse_params=LlamaParseParameters(

tests/extract/test_extract_api.py

@@ -3,7 +3,7 @@ import pytest
 from pathlib import Path
 from pydantic import BaseModel
 
-from llama_cloud_services.extract import LlamaExtract, ExtractionAgent
+from llama_cloud_services.extract import LlamaExtract, ExtractionAgent, SourceText
 from tests.extract.util import load_test_dotenv
 
 load_test_dotenv()
@@ -150,6 +150,16 @@ class TestExtractionAgent:
         assert "title" in result.data
         assert "summary" in result.data
 
+    def test_extract_file_from_bytes(self, test_agent):
+        with open(TEST_PDF, "rb") as f:
+            file_bytes = f.read()
+        result = test_agent.extract(SourceText(file=file_bytes, filename=TEST_PDF.name))
+        assert result.status == "SUCCESS"
+        assert result.data is not None
+        assert isinstance(result.data, dict)
+        assert "title" in result.data
+        assert "summary" in result.data
+
     @pytest.mark.asyncio
     async def test_extract_multiple_files(self, test_agent):
         files = [TEST_PDF, TEST_PDF]  # Using same file twice for testing

tests/extract/test_extract_e2e.py

@@ -55,7 +55,7 @@ def get_test_cases():
 
         settings = [
             ExtractConfig(extraction_mode=ExtractMode.FAST),
-            ExtractConfig(extraction_mode=ExtractMode.ACCURATE),
+            ExtractConfig(extraction_mode=ExtractMode.BALANCED),
         ]
 
         for input_file in sorted(input_files):