Bump version

.github/workflows/publish_release.yml

@@ -14,7 +14,7 @@ env:
 jobs:
   build-n-publish:
     name: Build and publish to PyPI
-    if: github.repository == 'run-llama/llama_parse'
+    if: github.repository == 'run-llama/llama_cloud_services'
     runs-on: ubuntu-latest
 
     steps:
@@ -36,16 +36,12 @@ jobs:
       - name: Build and publish llama-cloud-services
         uses: JRubics/poetry-publish@v2.1
         with:
-          poetry_version: ${{ env.POETRY_VERSION }}
-          python_version: ${{ env.PYTHON_VERSION }}
           pypi_token: ${{ secrets.LLAMA_PARSE_PYPI_TOKEN }}
           poetry_install_options: "--without dev"
 
       - name: Build and publish llama-parse
         uses: JRubics/poetry-publish@v2.1
         with:
-          poetry_version: ${{ env.POETRY_VERSION }}
-          python_version: ${{ env.PYTHON_VERSION }}
           working_directory: "llama_parse"
           pypi_token: ${{ secrets.LLAMA_PARSE_PYPI_TOKEN }}
           poetry_install_options: "--without dev"

.gitignore

@@ -3,3 +3,5 @@ __pycache__/
 *.pyc
 .DS_Store
 .idea
+.env*
+.ipynb_checkpoints*

README.md

@@ -10,7 +10,7 @@ This includes:
 
 - [LlamaParse](./parse.md) - A GenAI-native document parser that can parse complex document data for any downstream LLM use case (Agents, RAG, data processing, etc.).
 - [LlamaReport (beta/invite-only)](./report.md) - A prebuilt agentic report builder that can be used to build reports from a variety of data sources.
-- [LlamaExtract (coming soon!)]() - A prebuilt agentic data extractor that can be used to transform data into a structured JSON representation.
+- [LlamaExtract (beta/invite-only)](./extract.md) - A prebuilt agentic data extractor that can be used to transform data into a structured JSON representation.
 
 ## Getting Started
 
@@ -25,17 +25,18 @@ Then, get your API key from [LlamaCloud](https://cloud.llamaindex.ai/).
 Then, you can use the services in your code:
 
 ```python
-from llama_cloud_services import LlamaParse, LlamaReport
+from llama_cloud_services import LlamaParse, LlamaReport, LlamaExtract
 
 parser = LlamaParse(api_key="YOUR_API_KEY")
 report = LlamaReport(api_key="YOUR_API_KEY")
+extract = LlamaExtract(api_key="YOUR_API_KEY")
 ```
 
 See the quickstart guides for each service for more information:
 
 - [LlamaParse](./parse.md)
 - [LlamaReport (beta/invite-only)](./report.md)
-- [LlamaExtract (coming soon!)]()
+- [LlamaExtract (beta/invite-only)](./extract.md)
 
 ## Documentation
 

examples/extract/resume_screening.ipynb

@@ -0,0 +1,834 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Extracting data from Resumes\n",
+    "\n",
+    "Let us assume that we are running a hiring process for a company and we have received a list of resumes from candidates. We want to extract structured data from the resumes so that we can run a screening process and shortlist candidates. \n",
+    "\n",
+    "Take a look at one of the resumes in the `data/resumes` directory. "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/html": [
+       "\n",
+       "        <iframe\n",
+       "            width=\"600\"\n",
+       "            height=\"400\"\n",
+       "            src=\"./data/resumes/ai_researcher.pdf\"\n",
+       "            frameborder=\"0\"\n",
+       "            allowfullscreen\n",
+       "            \n",
+       "        ></iframe>\n",
+       "        "
+      ],
+      "text/plain": [
+       "<IPython.lib.display.IFrame at 0x109a7dcd0>"
+      ]
+     },
+     "execution_count": null,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "from IPython.display import IFrame\n",
+    "\n",
+    "IFrame(src=\"./data/resumes/ai_researcher.pdf\", width=600, height=400)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "You will notice that all the resumes have different layouts but contain common information like name, email, experience, education, etc. \n",
+    "\n",
+    "With LlamaExtract, we will show you how to:\n",
+    "- *Define* a data schema to extract the information of interest. \n",
+    "- *Iterate* over the data schema to generalize the schema for multiple resumes.\n",
+    "- *Finalize* the schema and schedule extractions for multiple resumes.\n",
+    "\n",
+    "We will start by defining a `LlamaExtract` client which provides a Python interface to the LlamaExtract API. "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from dotenv import load_dotenv\n",
+    "from llama_cloud_services import LlamaExtract\n",
+    "\n",
+    "\n",
+    "# Load environment variables (put LLAMA_CLOUD_API_KEY in your .env file)\n",
+    "load_dotenv(override=True)\n",
+    "\n",
+    "# Optionally, add your project id/organization id\n",
+    "llama_extract = LlamaExtract()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Defining the data schema\n",
+    "\n",
+    "Next, let us try to extract two fields from the resume: `name` and `email`. We can either use a Python dictionary structure to define the `data_schema` as a JSON or use a Pydantic model instead, for brevity and convenience. In either case, our output is guaranteed to validate against this schema."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from pydantic import BaseModel, Field\n",
+    "\n",
+    "\n",
+    "class Resume(BaseModel):\n",
+    "    name: str = Field(description=\"The name of the candidate\")\n",
+    "    email: str = Field(description=\"The email address of the candidate\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "Uploading files: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:02<00:00,  2.20s/it]\n",
+      "Creating extraction jobs: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:02<00:00,  2.93s/it]\n",
+      "Extracting files: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:02<00:00,  2.94s/it]\n",
+      "Uploading files: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.13it/s]\n",
+      "Creating extraction jobs: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.80it/s]\n",
+      "Extracting files: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:15<00:00, 15.18s/it]\n",
+      "Uploading files: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.16it/s]\n",
+      "Creating extraction jobs: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  2.33it/s]\n",
+      "Extracting files: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:32<00:00, 32.86s/it]\n"
+     ]
+    }
+   ],
+   "source": [
+    "from llama_cloud.core.api_error import ApiError\n",
+    "\n",
+    "try:\n",
+    "    existing_agent = llama_extract.get_agent(name=\"resume-screening\")\n",
+    "    if existing_agent:\n",
+    "        llama_extract.delete_agent(existing_agent.id)\n",
+    "except ApiError as e:\n",
+    "    if e.status_code == 404:\n",
+    "        pass\n",
+    "    else:\n",
+    "        raise\n",
+    "\n",
+    "agent = llama_extract.create_agent(name=\"resume-screening\", data_schema=Resume)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[ExtractionAgent(id=1fef43b5-8230-43b4-9e80-c1cddf53889c, name=resume-screening),\n",
+       " ExtractionAgent(id=93f8508b-3570-46f0-ae62-6315b40043bd, name=receipt/noisebridge_receipt.pdf_56db3d92),\n",
+       " ExtractionAgent(id=08315f0e-7146-430b-99b8-9701cb3ace6a, name=receipt/noisebridge_receipt.pdf_5c4730a7),\n",
+       " ExtractionAgent(id=cfcd7756-015d-4dbd-b142-a3eefcb16cd3, name=resume/software_architect_resume.html_4a11cf15),\n",
+       " ExtractionAgent(id=17cb83d9-601e-4f5c-a7aa-286e3045bcb4, name=resume/software_architect_resume.html_0b7d84a8),\n",
+       " ExtractionAgent(id=adc8e88c-44d3-4613-a5aa-d666ef007494, name=slide/saas_slide.pdf_bcc627a5),\n",
+       " ExtractionAgent(id=189f14cd-6370-4476-a6ad-36eafbc62618, name=slide/saas_slide.pdf_065aa22b),\n",
+       " ExtractionAgent(id=b9938ca5-6225-43cb-89ea-b0065237792f, name=test2),\n",
+       " ExtractionAgent(id=574d37b8-59dc-41e9-bde0-5c506a8eb670, name=test)]"
+      ]
+     },
+     "execution_count": null,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "llama_extract.list_agents()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "{'name': 'Dr. Rachel Zhang', 'email': 'rachel.zhang@email.com'}"
+      ]
+     },
+     "execution_count": null,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "resume = agent.extract(\"./data/resumes/ai_researcher.pdf\")\n",
+    "resume.data"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Iterating over the data schema\n",
+    "\n",
+    "Now that we have created a data schema, let us add more fields to the schema. We will add `experience` and `education` fields to the schema. \n",
+    "- We can create a new Pydantic model for each of these fields and represent `experience` and `education` as lists of these models. Doing this will allow us to extract multiple entities from the resume without having to pre-define how many experiences or education the candidate has. \n",
+    "- We have added a `description` parameter to provide more context for extraction. We can use `description` to provide example inputs/outputs for the extraction. \n",
+    "- Note that we have annotated the `start_date` and `end_date` fields with `Optional[str]` to indicate that these fields are optional. This is *important* because the schema will be used to extract data from multiple resumes and not all resumes will have the same format. A field must only be required if it is guaranteed to be present in all the resumes. \n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from typing import List, Optional\n",
+    "\n",
+    "\n",
+    "class Education(BaseModel):\n",
+    "    institution: str = Field(description=\"The institution of the candidate\")\n",
+    "    degree: str = Field(description=\"The degree of the candidate\")\n",
+    "    start_date: Optional[str] = Field(\n",
+    "        default=None, description=\"The start date of the candidate's education\"\n",
+    "    )\n",
+    "    end_date: Optional[str] = Field(\n",
+    "        default=None, description=\"The end date of the candidate's education\"\n",
+    "    )\n",
+    "\n",
+    "\n",
+    "class Experience(BaseModel):\n",
+    "    company: str = Field(description=\"The name of the company\")\n",
+    "    title: str = Field(description=\"The title of the candidate\")\n",
+    "    description: Optional[str] = Field(\n",
+    "        default=None, description=\"The description of the candidate's experience\"\n",
+    "    )\n",
+    "    start_date: Optional[str] = Field(\n",
+    "        default=None, description=\"The start date of the candidate's experience\"\n",
+    "    )\n",
+    "    end_date: Optional[str] = Field(\n",
+    "        default=None, description=\"The end date of the candidate's experience\"\n",
+    "    )\n",
+    "\n",
+    "\n",
+    "class Resume(BaseModel):\n",
+    "    name: str = Field(description=\"The name of the candidate\")\n",
+    "    email: str = Field(description=\"The email address of the candidate\")\n",
+    "    links: List[str] = Field(\n",
+    "        description=\"The links to the candidate's social media profiles\"\n",
+    "    )\n",
+    "    experience: List[Experience] = Field(description=\"The candidate's experience\")\n",
+    "    education: List[Education] = Field(description=\"The candidate's education\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Next, we will update the `data_schema` for the `resume-screening` agent to use the new `Resume` model. "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "{'name': 'Dr. Rachel Zhang',\n",
+       " 'email': 'rachel.zhang@email.com',\n",
+       " 'links': ['linkedin.com/in/rachelzhang',\n",
+       "  'github.com/rzhang-ai',\n",
+       "  'scholar.google.com/rachelzhang'],\n",
+       " 'experience': [{'company': 'DeepMind',\n",
+       "   'title': 'Senior Research Scientist',\n",
+       "   'description': '- Lead researcher on large-scale multi-task learning systems, developing novel architectures that improve cross-task generalization by 40%\\n- Pioneered new approach to zero-shot learning using contrastive training, published in NeurIPS 2023\\n- Built and led team of 6 researchers working on foundational ML models\\n- Developed novel regularization techniques for large language models, reducing catastrophic forgetting by 35%',\n",
+       "   'start_date': '2019',\n",
+       "   'end_date': 'Present'},\n",
+       "  {'company': 'Google Research',\n",
+       "   'title': 'Research Scientist',\n",
+       "   'description': '- Developed probabilistic frameworks for robust ML, published in ICML 2018\\n- Created novel attention mechanisms for computer vision models, improving accuracy by 25%\\n- Led collaboration with Google Brain team on efficient training methods for transformer models\\n- Mentored 4 PhD interns and collaborated with academic institutions',\n",
+       "   'start_date': '2015',\n",
+       "   'end_date': '2019'},\n",
+       "  {'company': 'Columbia University',\n",
+       "   'title': 'Research Assistant Professor',\n",
+       "   'description': '- Published seminal work on Bayesian optimization methods (cited 1000+ times)\\n- Taught graduate-level courses in Machine Learning and Statistical Learning Theory\\n- Supervised 5 PhD students and 3 MSc students\\n- Secured $500K in research grants for probabilistic ML research',\n",
+       "   'start_date': '2011',\n",
+       "   'end_date': '2015'}],\n",
+       " 'education': [{'institution': 'Columbia University',\n",
+       "   'degree': 'Ph.D. in Computer Science',\n",
+       "   'start_date': '2007',\n",
+       "   'end_date': '2011'},\n",
+       "  {'institution': 'Stanford University',\n",
+       "   'degree': 'M.S. in Computer Science',\n",
+       "   'start_date': '2005',\n",
+       "   'end_date': '2007'}]}"
+      ]
+     },
+     "execution_count": null,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "agent.data_schema = Resume\n",
+    "resume = agent.extract(\"./data/resumes/ai_researcher.pdf\")\n",
+    "resume.data"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "This is a good start. Let us add a few more fields to the schema and re-run the extraction. "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "class TechnicalSkills(BaseModel):\n",
+    "    programming_languages: List[str] = Field(\n",
+    "        description=\"The programming languages the candidate is proficient in.\"\n",
+    "    )\n",
+    "    frameworks: List[str] = Field(\n",
+    "        description=\"The tools/frameworks the candidate is proficient in, e.g. React, Django, PyTorch, etc.\"\n",
+    "    )\n",
+    "    skills: List[str] = Field(\n",
+    "        description=\"Other general skills the candidate is proficient in, e.g. Data Engineering, Machine Learning, etc.\"\n",
+    "    )\n",
+    "\n",
+    "\n",
+    "class Resume(BaseModel):\n",
+    "    name: str = Field(description=\"The name of the candidate\")\n",
+    "    email: str = Field(description=\"The email address of the candidate\")\n",
+    "    links: List[str] = Field(\n",
+    "        description=\"The links to the candidate's social media profiles\"\n",
+    "    )\n",
+    "    experience: List[Experience] = Field(description=\"The candidate's experience\")\n",
+    "    education: List[Education] = Field(description=\"The candidate's education\")\n",
+    "    technical_skills: TechnicalSkills = Field(\n",
+    "        description=\"The candidate's technical skills\"\n",
+    "    )\n",
+    "    key_accomplishments: str = Field(\n",
+    "        description=\"Summarize the candidates highest achievements.\"\n",
+    "    )"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "{'name': 'Dr. Rachel Zhang, Ph.D.',\n",
+       " 'email': 'rachel.zhang@email.com',\n",
+       " 'links': ['linkedin.com/in/rachelzhang',\n",
+       "  'github.com/rzhang-ai',\n",
+       "  'scholar.google.com/rachelzhang'],\n",
+       " 'experience': [{'company': 'DeepMind',\n",
+       "   'title': 'Senior Research Scientist',\n",
+       "   'description': 'Lead researcher on large-scale multi-task learning systems, developing novel architectures that improve cross-task generalization by 40%\\nPioneered new approach to zero-shot learning using contrastive training, published in NeurIPS 2023\\nBuilt and led team of 6 researchers working on foundational ML models\\nDeveloped novel regularization techniques for large language models, reducing catastrophic forgetting by 35%',\n",
+       "   'start_date': '2019',\n",
+       "   'end_date': 'Present'},\n",
+       "  {'company': 'Google Research',\n",
+       "   'title': 'Research Scientist',\n",
+       "   'description': 'Developed probabilistic frameworks for robust ML, published in ICML 2018\\nCreated novel attention mechanisms for computer vision models, improving accuracy by 25%\\nLed collaboration with Google Brain team on efficient training methods for transformer models\\nMentored 4 PhD interns and collaborated with academic institutions',\n",
+       "   'start_date': '2015',\n",
+       "   'end_date': '2019'},\n",
+       "  {'company': 'Columbia University',\n",
+       "   'title': 'Research Assistant Professor',\n",
+       "   'description': 'Published seminal work on Bayesian optimization methods (cited 1000+ times)\\nTaught graduate-level courses in Machine Learning and Statistical Learning Theory\\nSupervised 5 PhD students and 3 MSc students\\nSecured $500K in research grants for probabilistic ML research',\n",
+       "   'start_date': '2011',\n",
+       "   'end_date': '2015'}],\n",
+       " 'education': [{'institution': 'Columbia University',\n",
+       "   'degree': 'Ph.D. in Computer Science',\n",
+       "   'start_date': '2007',\n",
+       "   'end_date': '2011'},\n",
+       "  {'institution': 'Stanford University',\n",
+       "   'degree': 'M.S. in Computer Science',\n",
+       "   'start_date': '2005',\n",
+       "   'end_date': '2007'}],\n",
+       " 'technical_skills': {'programming_languages': ['Python',\n",
+       "   'C++',\n",
+       "   'Julia',\n",
+       "   'CUDA'],\n",
+       "  'frameworks': ['PyTorch', 'TensorFlow', 'JAX', 'Ray'],\n",
+       "  'skills': ['Deep Learning',\n",
+       "   'Reinforcement Learning',\n",
+       "   'Probabilistic Models',\n",
+       "   'Multi-Task Learning',\n",
+       "   'Zero-Shot Learning',\n",
+       "   'Neural Architecture Search']},\n",
+       " 'key_accomplishments': 'AI researcher with 12+ years of experience spanning classical machine learning, deep learning, and probabilistic modeling. Led groundbreaking research in reinforcement learning, generative models, and multi-task learning. Published 25+ papers in top-tier conferences (NeurIPS, ICML, ICLR). Strong track record of transitioning theoretical advances into practical applications in both academic and industrial settings.'}"
+      ]
+     },
+     "execution_count": null,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "agent.data_schema = Resume\n",
+    "resume = agent.extract(\"./data/resumes/ai_researcher.pdf\")\n",
+    "resume.data"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Finalizing the schema\n",
+    "\n",
+    "This is great! We have extracted a lot of key information from the resume that is well-typed and can be used downstream for further processing. Until now, this data is ephemeral and will be lost if we close the session. Let us save the state of our extraction and use it to extract data from multiple resumes. "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "agent.save()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "{'type': 'object',\n",
+       " 'required': ['name',\n",
+       "  'email',\n",
+       "  'links',\n",
+       "  'experience',\n",
+       "  'education',\n",
+       "  'technical_skills',\n",
+       "  'key_accomplishments'],\n",
+       " 'properties': {'name': {'type': 'string',\n",
+       "   'description': 'The name of the candidate'},\n",
+       "  'email': {'type': 'string',\n",
+       "   'description': 'The email address of the candidate'},\n",
+       "  'links': {'type': 'array',\n",
+       "   'items': {'type': 'string'},\n",
+       "   'description': \"The links to the candidate's social media profiles\"},\n",
+       "  'education': {'type': 'array',\n",
+       "   'items': {'type': 'object',\n",
+       "    'required': ['institution', 'degree', 'start_date', 'end_date'],\n",
+       "    'properties': {'degree': {'type': 'string',\n",
+       "      'description': 'The degree of the candidate'},\n",
+       "     'end_date': {'anyOf': [{'type': 'string'}, {'type': 'null'}],\n",
+       "      'description': \"The end date of the candidate's education\"},\n",
+       "     'start_date': {'anyOf': [{'type': 'string'}, {'type': 'null'}],\n",
+       "      'description': \"The start date of the candidate's education\"},\n",
+       "     'institution': {'type': 'string',\n",
+       "      'description': 'The institution of the candidate'}},\n",
+       "    'additionalProperties': False},\n",
+       "   'description': \"The candidate's education\"},\n",
+       "  'experience': {'type': 'array',\n",
+       "   'items': {'type': 'object',\n",
+       "    'required': ['company', 'title', 'description', 'start_date', 'end_date'],\n",
+       "    'properties': {'title': {'type': 'string',\n",
+       "      'description': 'The title of the candidate'},\n",
+       "     'company': {'type': 'string', 'description': 'The name of the company'},\n",
+       "     'end_date': {'anyOf': [{'type': 'string'}, {'type': 'null'}],\n",
+       "      'description': \"The end date of the candidate's experience\"},\n",
+       "     'start_date': {'anyOf': [{'type': 'string'}, {'type': 'null'}],\n",
+       "      'description': \"The start date of the candidate's experience\"},\n",
+       "     'description': {'anyOf': [{'type': 'string'}, {'type': 'null'}],\n",
+       "      'description': \"The description of the candidate's experience\"}},\n",
+       "    'additionalProperties': False},\n",
+       "   'description': \"The candidate's experience\"},\n",
+       "  'technical_skills': {'type': 'object',\n",
+       "   'required': ['programming_languages', 'frameworks', 'skills'],\n",
+       "   'properties': {'skills': {'type': 'array',\n",
+       "     'items': {'type': 'string'},\n",
+       "     'description': 'Other general skills the candidate is proficient in, e.g. Data Engineering, Machine Learning, etc.'},\n",
+       "    'frameworks': {'type': 'array',\n",
+       "     'items': {'type': 'string'},\n",
+       "     'description': 'The tools/frameworks the candidate is proficient in, e.g. React, Django, PyTorch, etc.'},\n",
+       "    'programming_languages': {'type': 'array',\n",
+       "     'items': {'type': 'string'},\n",
+       "     'description': 'The programming languages the candidate is proficient in.'}},\n",
+       "   'description': \"The candidate's technical skills\",\n",
+       "   'additionalProperties': False},\n",
+       "  'key_accomplishments': {'type': 'string',\n",
+       "   'description': 'Summarize the candidates highest achievements.'}},\n",
+       " 'additionalProperties': False}"
+      ]
+     },
+     "execution_count": null,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "agent = llama_extract.get_agent(\"resume-screening\")\n",
+    "agent.data_schema  # Latest schema should be returned"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "#### Queueing extractions"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "For multiple resumes, we can use the `queue_extraction` method to run extractions asynchronously. This is ideal for processing batch extraction jobs."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "Uploading files: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:01<00:00,  2.13it/s]\n",
+      "Creating extraction jobs: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:00<00:00,  5.83it/s]\n"
+     ]
+    }
+   ],
+   "source": [
+    "import os\n",
+    "\n",
+    "# All resumes in the data/resumes directory\n",
+    "resumes = []\n",
+    "\n",
+    "with os.scandir(\"./data/resumes\") as entries:\n",
+    "    for entry in entries:\n",
+    "        if entry.is_file():\n",
+    "            resumes.append(entry.path)\n",
+    "\n",
+    "jobs = await agent.queue_extraction(resumes)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "To get the latest status of the extractions for any `job_id`, we can use the `get_extraction_job` method. \n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[<StatusEnum.PENDING: 'PENDING'>,\n",
+       " <StatusEnum.PENDING: 'PENDING'>,\n",
+       " <StatusEnum.PENDING: 'PENDING'>]"
+      ]
+     },
+     "execution_count": null,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "[agent.get_extraction_job(job_id=job.id).status for job in jobs]"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "We notice that all extraction runs are in a PENDING state. We can check back again to see if the extractions have completed. "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[<StatusEnum.SUCCESS: 'SUCCESS'>,\n",
+       " <StatusEnum.SUCCESS: 'SUCCESS'>,\n",
+       " <StatusEnum.SUCCESS: 'SUCCESS'>]"
+      ]
+     },
+     "execution_count": null,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "[agent.get_extraction_job(job_id=job.id).status for job in jobs]"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "#### Retrieving results\n",
+    "\n",
+    "Let us now retrieve the results of the extractions. If the status of the extraction is `SUCCESS`, we can retrieve the data from the `data` field. In case there are errors (status = `ERROR`), we can retrieve the error message from the `error` field. \n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "results = []\n",
+    "for job in jobs:\n",
+    "    extract_run = agent.get_extraction_run_for_job(job.id)\n",
+    "    if extract_run.status == \"SUCCESS\":\n",
+    "        results.append(extract_run.data)\n",
+    "    else:\n",
+    "        print(f\"Extraction status for job {job.id}: {extract_run.status}\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "{'name': 'Dr. Rachel Zhang, Ph.D.',\n",
+       " 'email': 'rachel.zhang@email.com',\n",
+       " 'links': ['linkedin.com/in/rachelzhang',\n",
+       "  'github.com/rzhang-ai',\n",
+       "  'scholar.google.com/rachelzhang'],\n",
+       " 'education': [{'degree': 'Ph.D. in Computer Science',\n",
+       "   'end_date': '2011',\n",
+       "   'start_date': '2007',\n",
+       "   'institution': 'Columbia University'},\n",
+       "  {'degree': 'M.S. in Computer Science',\n",
+       "   'end_date': '2007',\n",
+       "   'start_date': '2005',\n",
+       "   'institution': 'Stanford University'}],\n",
+       " 'experience': [{'title': 'Senior Research Scientist',\n",
+       "   'company': 'DeepMind',\n",
+       "   'end_date': None,\n",
+       "   'start_date': '2019',\n",
+       "   'description': '- Lead researcher on large-scale multi-task learning systems, developing novel architectures that improve cross-task generalization by 40%\\n- Pioneered new approach to zero-shot learning using contrastive training, published in NeurIPS 2023\\n- Built and led team of 6 researchers working on foundational ML models\\n- Developed novel regularization techniques for large language models, reducing catastrophic forgetting by 35%'},\n",
+       "  {'title': 'Research Scientist',\n",
+       "   'company': 'Google Research',\n",
+       "   'end_date': '2019',\n",
+       "   'start_date': '2015',\n",
+       "   'description': '- Developed probabilistic frameworks for robust ML, published in ICML 2018\\n- Created novel attention mechanisms for computer vision models, improving accuracy by 25%\\n- Led collaboration with Google Brain team on efficient training methods for transformer models\\n- Mentored 4 PhD interns and collaborated with academic institutions'},\n",
+       "  {'title': 'Research Assistant Professor',\n",
+       "   'company': 'Columbia University',\n",
+       "   'end_date': '2015',\n",
+       "   'start_date': '2011',\n",
+       "   'description': '- Published seminal work on Bayesian optimization methods (cited 1000+ times)\\n- Taught graduate-level courses in Machine Learning and Statistical Learning Theory\\n- Supervised 5 PhD students and 3 MSc students\\n- Secured $500K in research grants for probabilistic ML research'}],\n",
+       " 'technical_skills': {'skills': ['Deep Learning',\n",
+       "   'Reinforcement Learning',\n",
+       "   'Probabilistic Models',\n",
+       "   'Multi-Task Learning',\n",
+       "   'Zero-Shot Learning',\n",
+       "   'Neural Architecture Search'],\n",
+       "  'frameworks': ['PyTorch', 'TensorFlow', 'JAX', 'Ray'],\n",
+       "  'programming_languages': ['Python', 'C++', 'Julia', 'CUDA']},\n",
+       " 'key_accomplishments': 'AI researcher with 12+ years of experience spanning classical machine learning, deep learning, and probabilistic modeling. Led groundbreaking research in reinforcement learning, generative models, and multi-task learning. Published 25+ papers in top-tier conferences (NeurIPS, ICML, ICLR). Strong track record of transitioning theoretical advances into practical applications in both academic and industrial settings.'}"
+      ]
+     },
+     "execution_count": null,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "results[0]"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "{'name': 'Alex Park',\n",
+       " 'email': 'alex park@email.com',\n",
+       " 'links': ['linkedin.com/in/alexpark'],\n",
+       " 'education': [{'degree': 'M.S. Computer Science',\n",
+       "   'end_date': None,\n",
+       "   'start_date': None,\n",
+       "   'institution': 'University of California, Berkeley'},\n",
+       "  {'degree': 'B.S. Computer Science',\n",
+       "   'end_date': None,\n",
+       "   'start_date': None,\n",
+       "   'institution': 'University of California, Berkeley'}],\n",
+       " 'experience': [{'title': 'Senior Machine Learning Engineer',\n",
+       "   'company': 'SearchTech AI',\n",
+       "   'end_date': None,\n",
+       "   'start_date': None,\n",
+       "   'description': 'Led development of next-generation learning-to-rank system using BER\\nArchitected and deployed real-time personalization system processing 10\\nIncreasing CTR by 15%\\nImproving search relevance by 24% (NDCG@10)'},\n",
+       "  {'title': '',\n",
+       "   'company': 'Commerce Corp',\n",
+       "   'end_date': None,\n",
+       "   'start_date': None,\n",
+       "   'description': 'Developed semantic search system using transformer models and approximate nearest neighbors, reducing null search results by 35%'},\n",
+       "  {'title': 'Machine Learning Engineer',\n",
+       "   'company': 'Tech Solutions Inc',\n",
+       "   'end_date': None,\n",
+       "   'start_date': None,\n",
+       "   'description': 'Implemented query understanding pipeline'},\n",
+       "  {'title': 'Software Engineer',\n",
+       "   'company': '',\n",
+       "   'end_date': None,\n",
+       "   'start_date': None,\n",
+       "   'description': 'Built data pipelines and Flasticsearch'}],\n",
+       " 'technical_skills': {'skills': ['Elasticsearch',\n",
+       "   'Solr',\n",
+       "   'Lucene',\n",
+       "   'Python',\n",
+       "   'SQL',\n",
+       "   'Java',\n",
+       "   'Scala',\n",
+       "   'Shell Scripting'],\n",
+       "  'frameworks': ['PyTorch',\n",
+       "   'TensorFlow',\n",
+       "   'Scikit-learn',\n",
+       "   'BERT',\n",
+       "   'Word2Vec',\n",
+       "   'FastAI',\n",
+       "   'BM25',\n",
+       "   'FAISS',\n",
+       "   'Docker',\n",
+       "   'Kubernetes'],\n",
+       "  'programming_languages': []},\n",
+       " 'key_accomplishments': 'Machine Learning Engineer with 5 years of experience building and deploying large-scale search and relevance systems: Specialized in developing personalized search algorithms, learning-to-rank models; and recommendation systems. Strong track record of improving search relevance metrics and user engagement through ML-driven solutions:'}"
+      ]
+     },
+     "execution_count": null,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "results[1]"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "{'name': 'Sarah Chen',\n",
+       " 'email': 'sarah.chen@email.com',\n",
+       " 'links': [],\n",
+       " 'education': [{'degree': 'Master of Science in Computer Science',\n",
+       "   'end_date': '2013',\n",
+       "   'start_date': None,\n",
+       "   'institution': 'Stanford University'},\n",
+       "  {'degree': 'Bachelor of Science in Computer Engineering',\n",
+       "   'end_date': '2011',\n",
+       "   'start_date': None,\n",
+       "   'institution': 'University of California, Berkeley'}],\n",
+       " 'experience': [{'title': 'Senior Software Architect',\n",
+       "   'company': 'TechCorp Solutions',\n",
+       "   'end_date': None,\n",
+       "   'start_date': '2020',\n",
+       "   'description': '- Led architectural design and implementation of a cloud-native platform serving 2M+ users\\n- Established architectural guidelines and best practices adopted across 12 development teams\\n- Reduced system latency by 40% through implementation of event-driven architecture\\n- Mentored 15+ senior developers in cloud-native development practices'},\n",
+       "  {'title': 'Lead Software Engineer',\n",
+       "   'company': 'DataFlow Systems',\n",
+       "   'end_date': '2020',\n",
+       "   'start_date': '2016',\n",
+       "   'description': '- Architected and led development of distributed data processing platform handling 5TB daily\\n- Designed microservices architecture reducing deployment time by 65%\\n- Led migration of legacy monolith to cloud-native architecture\\n- Managed team of 8 engineers across 3 international locations'},\n",
+       "  {'title': 'Senior Software Engineer',\n",
+       "   'company': 'InnovateTech',\n",
+       "   'end_date': '2016',\n",
+       "   'start_date': '2013',\n",
+       "   'description': '- Developed high-performance trading platform processing 100K transactions per second\\n- Implemented real-time analytics engine reducing processing latency by 75%\\n- Led adoption of container orchestration reducing deployment costs by 35%'}],\n",
+       " 'technical_skills': {'skills': ['Architecture & Design',\n",
+       "   'Microservices',\n",
+       "   'Event-Driven Architecture',\n",
+       "   'Domain-Driven Design',\n",
+       "   'REST APIs',\n",
+       "   'Cloud Platforms'],\n",
+       "  'frameworks': ['AWS (Advanced)', 'Azure', 'Google Cloud Platform'],\n",
+       "  'programming_languages': ['Java', 'Python', 'Go', 'JavaScript/TypeScript']},\n",
+       " 'key_accomplishments': '- Co-inventor on three patents for distributed systems architecture\\n- Published paper on \"Scalable Microservices Architecture\" at IEEE Cloud Computing Conference 2022\\n- Keynote Speaker, CloudCon 2023: \"Future of Cloud-Native Architecture\"\\n- Regular presenter at local tech meetups and conferences'}"
+      ]
+     },
+     "execution_count": null,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "results[2]"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Congratulations! You now have an agent that can extract structured data from resumes. \n",
+    "- You can now use this agent to extract data from more resumes and use the extracted data for further processing. \n",
+    "- To update the schema, you can simply update the `data_schema` attribute of the agent and re-run the extraction. \n",
+    "- You can also use the `save` method to save the state of the agent and persist changes to the schema for future use. \n",
+    "\n"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}

examples/parse/advanced_rag/dynamic_section_retrieval.ipynb

@@ -7,7 +7,7 @@
    "source": [
     "# Dynamic Section Retrieval with LlamaParse\n",
     "\n",
-    "<a href=\"https://colab.research.google.com/github/run-llama/llamacloud-demo/blob/main/examples/advanced_rag/dynamic_section_retrieval.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
+    "<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services-demo/blob/main/examples/parse/advanced_rag/dynamic_section_retrieval.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
     "\n",
     "This notebook showcases a concept called \"dynamic section retrieval\".\n",
     "\n",

examples/parse/caltrain/caltrain_text_mode.ipynb

@@ -7,7 +7,7 @@
    "source": [
     "# RAG over the Caltrain Weekend Schedule \n",
     "\n",
-    "<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/caltrain/caltrain_text_mode.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
+    "<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/parse/caltrain/caltrain_text_mode.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
     "\n",
     "This example shows off LlamaParse parsing capabilities to build a functioning query pipeline over the Caltrain weekend schedule, a big timetable containing all trains northbound and southbound and their stops in various cities.\n",
     "\n",

examples/parse/demo_advanced_weaviate.ipynb

@@ -6,7 +6,7 @@
    "source": [
     "# Advanced RAG with LlamaParse + Weaviate\n",
     "\n",
-    "<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/demo_advanced_weaviate.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\\\" alt=\"Open In Colab\"/></a>\n",
+    "<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/parse/demo_advanced_weaviate.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\\\" alt=\"Open In Colab\"/></a>\n",
     "\n",
     "This notebook shows you how to use `LlamaParse` for advancd RAG applications with `LlamaIndex` and [Weaviate](https://weaviate.io/).\n",
     "\n",

examples/parse/demo_excel.ipynb

@@ -6,7 +6,7 @@
    "source": [
     "# RAG with Excel Spreadsheet using LlamaPrase\n",
     "\n",
-    "<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/demo_excel.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
+    "<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/demo_excel.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
     "\n",
     "This notebook shows you using LlamaParse with Excel Spreadsheet.\n",
     "\n",

examples/parse/demo_get_charts.ipynb

@@ -7,7 +7,7 @@
    "source": [
     "# Download Charts\n",
     "\n",
-    "<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/demo_get_charts.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
+    "<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/demo_get_charts.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
     "\n",
     "This notebook demonstrates how to download charts from a document using the JSON mode in LlamaParse.\n",
     "\n",

examples/parse/demo_insurance.ipynb

@@ -6,7 +6,7 @@
    "source": [
     "# LlamaParse - Fast checking Insurance Contract for Coverage\n",
     "\n",
-    "<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/demo_insurance.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
+    "<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/demo_insurance.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
     "\n",
     "In this notebook we will look at how LlamaParse can be used to extract structured coverage information from an insurance policy."
    ]

examples/parse/demo_json.ipynb

@@ -7,7 +7,7 @@
    "source": [
     "# LlamaParse JSON Mode + Multimodal RAG\n",
     "\n",
-    "<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/demo_json.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
+    "<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/parse/demo_json.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
     "\n",
     "This notebook shows you how to use LlamaParse JSON mode with LlamaIndex to build a simple multimodal RAG pipeline.\n",
     "\n",

examples/parse/demo_json_parsing.ipynb

@@ -7,7 +7,7 @@
    "source": [
     "# LlamaParse JSON Mode + Advanced RAG with `LlamaParseJsonNodeParser`\n",
     "\n",
-    "<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/demo_json_parsing.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
+    "<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/demo_json_parsing.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
     "\n",
     "This notebook shows you how to use LlamaParse JSON mode with LlamaIndex to build a simple recursive retrieval RAG pipeline using `LlamaParseJsonNodeParser`.\n",
     "\n",

examples/parse/demo_json_tour.ipynb

@@ -7,7 +7,7 @@
    "source": [
     "# LlamaParse JSON Mode Tour\n",
     "\n",
-    "<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/demo_json.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
+    "<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/demo_json.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
     "\n",
     "This notebook shows you how to use LlamaParse JSON mode with LlamaIndex and all the features it supports.\n",
     "\n",

examples/parse/demo_languages.ipynb

@@ -9,7 +9,7 @@
     "\n",
     "LlamaParse supports users to specify a `language` parameter before uploading documents, giving users better OCR capabilities over non-English PDFs, parsing images into more accurate representations.\n",
     "\n",
-    "You can specify 80+ different languages: see this file for a full list of supported languages: https://github.com/run-llama/llama_parse/blob/main/llama_parse/base.py.\n",
+    "You can specify 80+ different languages: see this file for a full list of supported languages: https://github.com/run-llama/llama_cloud_services/blob/main/llama_parse/base.py.\n",
     "\n",
     "This notebook shows a demo of this in action. "
    ]

examples/parse/demo_mongodb.ipynb

@@ -7,7 +7,7 @@
    "source": [
     "# LlamaParse With MongoDB\n",
     "\n",
-    "<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/demo_mongodb.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
+    "<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/parse/demo_mongodb.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
     "\n",
     "In this notebook, we provide a straightforward example of using LlamaParse with MongoDB Atlas VectorSearch.\n",
     "\n",

examples/parse/demo_starter_multimodal.ipynb

@@ -5,7 +5,7 @@
    "id": "97c79c38-38a3-40f3-ba2e-250649347d63",
    "metadata": {},
    "source": [
-    "<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/demo_starter_multimodal.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
+    "<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/parse/demo_starter_multimodal.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
    ]
   },
   {

examples/parse/demo_starter_parse_selected_pages.ipynb

@@ -4,7 +4,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/demo_starter_parse_selected_pages.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
+    "<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/parse/demo_starter_parse_selected_pages.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
    ]
   },
   {

examples/parse/demo_table_comparisons.ipynb

@@ -6,7 +6,7 @@
    "source": [
     "# RAG for Table Comparisons with LlamaParse + LlamaIndex\n",
     "\n",
-    "<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/demo_table_comparisons.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
+    "<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/parse/demo_table_comparisons.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
     "\n",
     "This notebook shows you how to do comparisons across both tabular and text data across multiple PDF documents.\n",
     "\n",

examples/parse/excel/dcf_rag.ipynb

@@ -7,7 +7,7 @@
    "source": [
     "# RAG with Excel Spreadsheet using LlamaPrase\n",
     "\n",
-    "<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/demo_excel.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
+    "<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/parse/excel/dcf_rag.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
     "\n",
     "This notebook constructs a RAG pipeline over a simple DCF template [here](https://eqvista.com/app/uploads/2020/09/Eqvista_DCF-Excel-Template.xlsx).\n",
     "\n"

examples/parse/excel/o1_excel_rag.ipynb

@@ -4,7 +4,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/excel/o1_excel_rag.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
+    "<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/excel/o1_excel_rag.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
    ]
   },
   {

examples/parse/knowledge_graphs/kg_agent.ipynb

@@ -7,7 +7,7 @@
    "source": [
     "# Knowledge Graph Agent with LlamaParse\n",
     "\n",
-    "<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/knowledge_graphs/kg_agent.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
+    "<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/parse/knowledge_graphs/kg_agent.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
     "\n",
     "Here we build a knowledge graph agent over the SF 2023 Budget Proposal. We use LlamaIndex abstractions to construct a knowledge graph, and we store the property graph in neo4j. We then build an agent that can interact with the knowledge graph as a tool."
    ]

examples/parse/multimodal/claude_parse.ipynb

@@ -7,7 +7,7 @@
    "source": [
     "# Multimodal Parsing using Anthropic Claude (Sonnet 3.5)\n",
     "\n",
-    "<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/multimodal/claude_parse.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
+    "<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/parse/multimodal/claude_parse.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
     "\n",
     "This cookbook shows you how to use LlamaParse to parse any document with the multimodal capabilities of Sonnet 3.5. \n",
     "\n",

examples/parse/multimodal/gemini2_flash.ipynb

@@ -0,0 +1,633 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "97c79c38-38a3-40f3-ba2e-250649347d63",
+   "metadata": {},
+   "source": [
+    "# Multimodal Parsing with Gemini 2.0 Flash\n",
+    "\n",
+    "<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/parse/multimodal/gemini2_flash.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
+    "\n",
+    "This cookbook shows you how to use LlamaParse to parse any document with the multimodal capabilities of Gemini 2.0 Flash.\n",
+    "\n",
+    "LlamaParse allows you to plug in external, multimodal model vendors for parsing - we handle the error correction, validation, and scalability/reliability for you.\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "15e60ecf-519c-41fc-911b-765adaf8bad4",
+   "metadata": {},
+   "source": [
+    "## Setup\n",
+    "\n",
+    "Download the data - we'll use a technical datasheet for a programmable logic device (Xilinx's XC9500 In-System Programmable CPLD)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "91a9e532-1454-40e0-bbf0-fd442c350121",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import nest_asyncio\n",
+    "\n",
+    "nest_asyncio.apply()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "0d9fb0aa-74cd-476f-8161-efd9e04248bf",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "--2025-02-06 20:24:19--  https://media.digikey.com/pdf/Data%20Sheets/AMD/XC9500_CPLD_Family.pdf\n",
+      "Resolving media.digikey.com (media.digikey.com)... 23.37.18.160\n",
+      "Connecting to media.digikey.com (media.digikey.com)|23.37.18.160|:443... connected.\n",
+      "HTTP request sent, awaiting response... 200 OK\n",
+      "Length: 201899 (197K) [application/pdf]\n",
+      "Saving to: ‘data/XC9500_CPLD_Family.pdf’\n",
+      "\n",
+      "data/XC9500_CPLD_Fa 100%[===================>] 197.17K  --.-KB/s    in 0.03s   \n",
+      "\n",
+      "2025-02-06 20:24:19 (7.67 MB/s) - ‘data/XC9500_CPLD_Family.pdf’ saved [201899/201899]\n",
+      "\n"
+     ]
+    }
+   ],
+   "source": [
+    "!wget \"https://media.digikey.com/pdf/Data%20Sheets/AMD/XC9500_CPLD_Family.pdf\" -O data/XC9500_CPLD_Family.pdf"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "4e29a9d7-5bd9-4fb8-8ec1-4c128a748662",
+   "metadata": {},
+   "source": [
+    "## Initialize LlamaParse\n",
+    "\n",
+    "Initialize LlamaParse in multimodal mode, and specify the vendor as `gemini-2.0-flash-001`.\n",
+    "\n",
+    "**NOTE**: Current pricing is 2 credits for a 1 page ($0.006 USD / page). This includes core model, infra, and algorithm costs to fully process the page. "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "dc921729-3446-42ca-8e1b-a6fd26195ed9",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from llama_index.core.schema import TextNode\n",
+    "from typing import List\n",
+    "import json\n",
+    "\n",
+    "\n",
+    "def get_text_nodes(json_list: List[dict]):\n",
+    "    text_nodes = []\n",
+    "    for idx, page in enumerate(json_list):\n",
+    "        text_node = TextNode(text=page[\"md\"], metadata={\"page\": page[\"page\"]})\n",
+    "        text_nodes.append(text_node)\n",
+    "    return text_nodes\n",
+    "\n",
+    "\n",
+    "def save_jsonl(data_list, filename):\n",
+    "    \"\"\"Save a list of dictionaries as JSON Lines.\"\"\"\n",
+    "    with open(filename, \"w\") as file:\n",
+    "        for item in data_list:\n",
+    "            json.dump(item, file)\n",
+    "            file.write(\"\\n\")\n",
+    "\n",
+    "\n",
+    "def load_jsonl(filename):\n",
+    "    \"\"\"Load a list of dictionaries from JSON Lines.\"\"\"\n",
+    "    data_list = []\n",
+    "    with open(filename, \"r\") as file:\n",
+    "        for line in file:\n",
+    "            data_list.append(json.loads(line))\n",
+    "    return data_list"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "f2e9d9cf-8189-4fcb-b34f-cde6cc0b59c8",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Started parsing the file under job_id 51538aa0-13e6-4429-a458-a492ba7eec04\n"
+     ]
+    }
+   ],
+   "source": [
+    "from llama_parse import LlamaParse\n",
+    "\n",
+    "parsing_instruction = \"\"\"\n",
+    "You are given a technical datasheet of an electronic component.\n",
+    "For any graphs, try to create a 2D table of relevant values, along with a description of the graph.\n",
+    "For any schematic diagrams, MAKE SURE to describe a list of all components and their connections to each other.\n",
+    "Make sure that you always parse out the text with the correct reading order.\n",
+    "\"\"\"\n",
+    "\n",
+    "parser = LlamaParse(\n",
+    "    result_type=\"markdown\",\n",
+    "    use_vendor_multimodal_model=True,\n",
+    "    vendor_multimodal_model_name=\"gemini-2.0-flash-001\",\n",
+    "    invalidate_cache=True,\n",
+    "    parsing_instruction=parsing_instruction,\n",
+    ")\n",
+    "json_objs = parser.get_json_result(\"./data/XC9500_CPLD_Family.pdf\")\n",
+    "json_list = json_objs[0][\"pages\"]\n",
+    "docs = get_text_nodes(json_list)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "96a81df0-1026-4e30-a930-f677dc31e344",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Optional: Save\n",
+    "save_jsonl([d.dict() for d in docs], \"docs_gemini_2.0_flash.jsonl\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "ee2e6920-8893-4b39-ae12-94d13c651406",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Optional: Load\n",
+    "from llama_index.core import Document\n",
+    "\n",
+    "docs_dicts = load_jsonl(\"docs_gemini_2.0_flash.jsonl\")\n",
+    "docs = [Document.parse_obj(d) for d in docs_dicts]"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "4f3c51b0-7878-48d7-9bc3-02b516500128",
+   "metadata": {},
+   "source": [
+    "### Setup GPT-4o baseline\n",
+    "\n",
+    "For comparison, we will also parse the document using GPT-4o ($0.03 per page)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "6fc3f258-50ae-4988-b904-c105463a498f",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Started parsing the file under job_id 23c6627c-2e3d-46c9-88a0-7945d7e65d96\n"
+     ]
+    }
+   ],
+   "source": [
+    "from llama_parse import LlamaParse\n",
+    "\n",
+    "parser_gpt4o = LlamaParse(\n",
+    "    result_type=\"markdown\",\n",
+    "    use_vendor_multimodal_model=True,\n",
+    "    vendor_multimodal_model=\"openai-gpt4o\",\n",
+    "    invalidate_cache=True,\n",
+    "    parsing_instruction=parsing_instruction,\n",
+    ")\n",
+    "json_objs_gpt4o = parser_gpt4o.get_json_result(\"./data/XC9500_CPLD_Family.pdf\")\n",
+    "json_list_gpt4o = json_objs_gpt4o[0][\"pages\"]\n",
+    "docs_gpt4o = get_text_nodes(json_list_gpt4o)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "6a47f04e-12e1-4c80-a71d-ef7721f96401",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Optional: Save\n",
+    "save_jsonl([d.dict() for d in docs_gpt4o], \"docs_gpt4o.jsonl\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "c38b5ca3-fa87-434b-b477-bf6a4962eb3d",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Optional: Load\n",
+    "from llama_index.core import Document\n",
+    "\n",
+    "docs_gpt4o_dicts = load_jsonl(\"docs_gpt4o.jsonl\")\n",
+    "docs_gpt4o = [Document.parse_obj(d) for d in docs_gpt4o_dicts]"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "44c20f7a-2901-4dd0-b635-a4b33c5664c1",
+   "metadata": {},
+   "source": [
+    "## View Results\n",
+    "\n",
+    "Let's visualize the results between GPT-4o and Gemini Flash 2.0 along with the original document page."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "bf314141-9f6d-4453-beb9-0106cdf196bf",
+   "metadata": {},
+   "source": [
+    "Check out an example page 2 below."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "c70d420d-1778-4b0d-81e2-db09276e90cf",
+   "metadata": {},
+   "source": [
+    "![xc9500_img](XC9500_CPLD_Family_p3.png)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "0950ecad-248c-4c3c-98b9-ab1a9dabd5b4",
+   "metadata": {},
+   "source": [
+    "We see that the parsed text is fairly similar between Gemini 2.0 Flash and GPT-4o. "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "778698aa-da7e-4081-b3b5-0372f228536f",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "page: 3\n",
+      "\n",
+      "The image shows the architecture of the XC9500 In-System Programmable CPLD Family, which is marked as obsolete. Here's a breakdown of the components and their connections:\n",
+      "\n",
+      "### Components and Connections:\n",
+      "\n",
+      "1. **JTAG Port:**\n",
+      "   - Connects to the JTAG Controller.\n",
+      "\n",
+      "2. **JTAG Controller:**\n",
+      "   - Interfaces with the In-System Programming Controller.\n",
+      "   - Connects to the I/O Blocks.\n",
+      "\n",
+      "3. **In-System Programming Controller:**\n",
+      "   - Interfaces with the JTAG Controller and the Fast CONNECT Switch Matrix.\n",
+      "\n",
+      "4. **I/O Blocks:**\n",
+      "   - Multiple I/O lines connect to the Fast CONNECT Switch Matrix.\n",
+      "   - Includes special I/O lines for GCK, GSR, and GTS.\n",
+      "\n",
+      "5. **Fast CONNECT Switch Matrix:**\n",
+      "   - Connects to the I/O Blocks and Function Blocks.\n",
+      "   - Provides 36 inputs and 18 outputs to each Function Block.\n",
+      "\n",
+      "6. **Function Blocks (FB):**\n",
+      "   - Each block contains 18 macrocells.\n",
+      "   - Outputs from the Function Blocks drive the I/O Blocks directly.\n",
+      "   - Multiple Function Blocks (1 to N) are shown, each with 18 macrocells.\n",
+      "\n",
+      "### Function Block Details:\n",
+      "\n",
+      "- Each Function Block consists of 18 independent macrocells.\n",
+      "- Capable of implementing combinatorial or registered functions.\n",
+      "- Receives global clock, output enable, and set/reset signals.\n",
+      "- Generates 18 outputs for the Fast CONNECT switch matrix.\n",
+      "- Logic is implemented using a sum-of-products representation.\n",
+      "- 36 inputs provide 72 true and complement signals to form 90 product terms.\n",
+      "- Product terms can be allocated to each macrocell by the product term allocator.\n",
+      "- Supports local feedback paths for fast counters and state machines.\n",
+      "\n",
+      "This architecture is designed for flexibility in implementing complex logic functions within a programmable logic device.\n"
+     ]
+    }
+   ],
+   "source": [
+    "# using Gemini 2.0 Flash\n",
+    "print(docs[2].get_content(metadata_mode=\"all\"))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "1511a30f-3efc-4142-9668-7dc056a24d0c",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "page: 3\n",
+      "\n",
+      "The diagram illustrates the architecture of the XC9500 In-System Programmable CPLD Family. Here's a breakdown of the components and their connections:\n",
+      "\n",
+      "1. **JTAG Port**: \n",
+      "   - Connects to the JTAG Controller.\n",
+      "\n",
+      "2. **JTAG Controller**: \n",
+      "   - Interfaces with the In-System Programming Controller.\n",
+      "\n",
+      "3. **In-System Programming Controller**: \n",
+      "   - Manages programming of the device.\n",
+      "\n",
+      "4. **I/O Blocks**: \n",
+      "   - Connect to external I/O pins.\n",
+      "   - Interface with the Fast CONNECT Switch Matrix.\n",
+      "\n",
+      "5. **Fast CONNECT Switch Matrix**: \n",
+      "   - Connects I/O Blocks to Function Blocks.\n",
+      "   - Provides 36 inputs and 18 outputs to each Function Block.\n",
+      "\n",
+      "6. **Function Blocks (FB)**: \n",
+      "   - Each block contains 18 macrocells.\n",
+      "   - Capable of implementing combinatorial or registered functions.\n",
+      "   - Receives global clock, output enable, and set/reset signals.\n",
+      "   - Outputs drive the Fast CONNECT Switch Matrix.\n",
+      "   - Supports local feedback paths for fast counters and state machines.\n",
+      "\n",
+      "7. **I/O/GCK, I/O/GSR, I/O/GTS**: \n",
+      "   - Special I/O pins for global clock, set/reset, and output enable signals.\n",
+      "\n",
+      "The architecture is designed for flexibility and high-speed operation, with each Function Block capable of handling complex logic functions.\n"
+     ]
+    }
+   ],
+   "source": [
+    "# using GPT-4o\n",
+    "print(docs_gpt4o[2].get_content(metadata_mode=\"all\"))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "705f7729-fa0f-4ca0-8562-c42afeaa8532",
+   "metadata": {},
+   "source": [
+    "## Setup RAG Pipeline\n",
+    "\n",
+    "Let's setup a RAG pipeline over this data.\n",
+    "\n",
+    "(we also use gpt4o-mini for the actual text synthesis step)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "5a53ee5d-cc63-421b-8896-588c83edfcf0",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from llama_index.core import Settings\n",
+    "from llama_index.llms.openai import OpenAI\n",
+    "from llama_index.embeddings.openai import OpenAIEmbedding\n",
+    "\n",
+    "Settings.llm = OpenAI(model=\"o3-mini\")\n",
+    "Settings.embed_model = OpenAIEmbedding(model=\"text-embedding-3-large\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "60972d7a-7948-4ad7-89df-57004acee917",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# from llama_index.core import SummaryIndex\n",
+    "from llama_index.core import VectorStoreIndex\n",
+    "from llama_index.llms.openai import OpenAI\n",
+    "\n",
+    "index = VectorStoreIndex(docs)\n",
+    "query_engine = index.as_query_engine(similarity_top_k=5)\n",
+    "\n",
+    "index_gpt4o = VectorStoreIndex(docs_gpt4o)\n",
+    "query_engine_gpt4o = index_gpt4o.as_query_engine(similarity_top_k=5)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "e7df7bcb-1df4-4a01-88fc-2d596b1cc74d",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "query = \"Give me the full output slew-Rate curve for (a) Rising and (b) Falling Outputs\"\n",
+    "\n",
+    "response = query_engine.query(query)\n",
+    "response_gpt4o = query_engine_gpt4o.query(query)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "b7070a31-3bb8-4134-8338-20bc2fd6f3d6",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "The full output slew-rate curve for (a) Rising and (b) Falling Outputs is represented in a graph where the output voltage starts at 1.5V and reaches the desired output level over a time period defined as T<sub>SLEW</sub>. The curve illustrates the gradual increase in voltage for rising outputs and the gradual decrease for falling outputs, effectively showing how the output edge rates can be controlled to reduce system noise.\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(response)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "7bee8167-f021-4c87-8d28-9f40a4f7b69d",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "# XC9500 In-System Programmable CPLD Family\n",
+      "\n",
+      "Each output has independent slew rate control. Output edge rates may be slowed down to reduce system noise (with an additional time delay of T<sub>SLEW</sub>) through programming. See Figure 11.\n",
+      "\n",
+      "Each IOB provides user programmable ground pin capability. This allows device I/O pins to be configured as additional ground pins. By tying strategically located programmable ground pins to the external ground connection, system noise generated from large numbers of simultaneous switching outputs may be reduced.\n",
+      "\n",
+      "A control pull-up resistor (typically 10K ohms) is attached to each device I/O pin to prevent them from floating when the device is not in normal user operation. This resistor is active during device programming mode and system power-up. It is also activated for an erased device. The resistor is deactivated during normal operation.\n",
+      "\n",
+      "The output driver is capable of supplying 24 mA output drive. All output drivers in the device may be configured for either 5V TTL levels or 3.3V levels by connecting the device output voltage supply (V<sub>CCIO</sub>) to a 5V or 3.3V voltage supply. Figure 12 shows how the XC9500 device can be used in 5V only and mixed 3.3V/5V systems.\n",
+      "\n",
+      "## Pin-Locking Capability\n",
+      "\n",
+      "The capability to lock the user defined pin assignments during design changes depends on the ability of the architecture to adapt to unexpected changes. The XC9500 devices have architectural features that enhance the ability to accept design changes while maintaining the same pinout.\n",
+      "\n",
+      "The XC9500 architecture provides maximum routing within the Fast CONNECT switch matrix, and incorporates a flexible Function Block that allows block-wide allocation of available product terms. This provides a high level of confidence of maintaining both input and output pin assignments for unexpected design changes.\n",
+      "\n",
+      "For extensive design changes requiring higher logic capacity than is available in the initially chosen device, the new design may be able to fit into a larger pin-compatible device using the same pin assignments. The same board may be used with a higher density device without the expense of board rework.\n",
+      "\n",
+      "!Output slew-Rate for (a) Rising and (b) Falling Outputs\n",
+      "\n",
+      "**Figure 11:** Output slew-Rate for (a) Rising and (b) Falling Outputs\n",
+      "\n",
+      "| Output Voltage | Time |\n",
+      "|----------------|------|\n",
+      "| 1.5V           | 0    |\n",
+      "| T<sub>SLEW</sub> |    |\n",
+      "\n",
+      "**Figure 12:** XC9500 Devices in (a) 5V Systems and (b) Mixed 5V/3.3V Systems\n",
+      "\n",
+      "| 5V CMOS or 5V TTL | 3.3V |\n",
+      "|-------------------|------|\n",
+      "| 5V                | 0V   |\n",
+      "| 3.6V              | 0V   |\n",
+      "| 3.3V              | 0V   |\n",
+      "\n",
+      "- **(a) 5V System:**\n",
+      "  - V<sub>CCINT</sub> V<sub>CCIO</sub>\n",
+      "  - XC9500 CPLD\n",
+      "  - IN OUT\n",
+      "  - GND\n",
+      "\n",
+      "- **(b) Mixed 5V/3.3V System:**\n",
+      "  - V<sub>CCINT</sub> V<sub>CCIO</sub>\n",
+      "  - XC9500 CPLD\n",
+      "  - IN OUT\n",
+      "  - GND\n",
+      "\n",
+      "www.xilinx.com\n",
+      "\n",
+      "DS063 (v6.0) May 17, 2013  \n",
+      "Product Specification\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(response.source_nodes[0].get_content())"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "5f9fef7f-510b-46a5-8716-f5616f542035",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "The output slew-rate curve for (a) Rising and (b) Falling Outputs is represented in a timing diagram where the output voltage transitions from a low state to a high state and vice versa. \n",
+      "\n",
+      "For the rising output, the curve starts at 1.5V and transitions to the desired output voltage level over a time period defined as T<sub>SLEW</sub>. \n",
+      "\n",
+      "For the falling output, the curve similarly begins at the high output voltage and decreases to a low state, also taking the time defined as T<sub>SLEW</sub> to complete the transition.\n",
+      "\n",
+      "The specific values and graphical representation would typically be illustrated in a figure, but the key takeaway is that the output slew rate can be controlled to manage system noise by programming the desired T<sub>SLEW</sub> time.\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(response_gpt4o)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "d40f9dd4-2dd4-4fa5-b636-1f901dc1601b",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "# XC9500 In-System Programmable CPLD Family\n",
+      "\n",
+      "Each output has independent slew rate control. Output edge rates may be slowed down to reduce system noise (with an additional time delay of T<sub>SLEW</sub>) through programming. See Figure 11.\n",
+      "\n",
+      "Each IOB provides user programmable ground pin capability. This allows device I/O pins to be configured as additional ground pins. By tying strategically located programmable ground pins to the external ground connection, system noise generated from large numbers of simultaneous switching outputs may be reduced.\n",
+      "\n",
+      "A control pull-up resistor (typically 10K ohms) is attached to each device I/O pin to prevent them from floating when the device is not in normal user operation. This resistor is active during device programming mode and system power-up. It is also activated for an erased device. The resistor is deactivated during normal operation.\n",
+      "\n",
+      "The output driver is capable of supplying 24 mA output drive. All output drivers in the device may be configured for either 5V TTL levels or 3.3V levels by connecting the device output voltage supply (V<sub>CCIO</sub>) to a 5V or 3.3V voltage supply. Figure 12 shows how the XC9500 device can be used in 5V only and mixed 3.3V/5V systems.\n",
+      "\n",
+      "## Pin-Locking Capability\n",
+      "\n",
+      "The capability to lock the user defined pin assignments during design changes depends on the ability of the architecture to adapt to unexpected changes. The XC9500 devices have architectural features that enhance the ability to accept design changes while maintaining the same pinout.\n",
+      "\n",
+      "The XC9500 architecture provides maximum routing within the Fast CONNECT switch matrix, and incorporates a flexible Function Block that allows block-wide allocation of available product terms. This provides a high level of confidence of maintaining both input and output pin assignments for unexpected design changes.\n",
+      "\n",
+      "For extensive design changes requiring higher logic capacity than is available in the initially chosen device, the new design may be able to fit into a larger pin-compatible device using the same pin assignments. The same board may be used with a higher density device without the expense of board rework.\n",
+      "\n",
+      "!Output slew-Rate for (a) Rising and (b) Falling Outputs\n",
+      "\n",
+      "**Figure 11:** Output slew-Rate for (a) Rising and (b) Falling Outputs\n",
+      "\n",
+      "| Output Voltage | Time |\n",
+      "|----------------|------|\n",
+      "| 1.5V           | 0    |\n",
+      "| T<sub>SLEW</sub> |    |\n",
+      "\n",
+      "**Figure 12:** XC9500 Devices in (a) 5V Systems and (b) Mixed 5V/3.3V Systems\n",
+      "\n",
+      "| 5V CMOS or 5V TTL | 3.3V |\n",
+      "|-------------------|------|\n",
+      "| 5V                | 0V   |\n",
+      "| 3.6V              | 0V   |\n",
+      "| 3.3V              | 0V   |\n",
+      "\n",
+      "- **XC9500 CPLD**  \n",
+      "  - **IN**  \n",
+      "  - **OUT**  \n",
+      "  - **GND**  \n",
+      "\n",
+      "www.xilinx.com  \n",
+      "DS063 (v6.0) May 17, 2013  \n",
+      "Product Specification\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(response_gpt4o.source_nodes[0].get_content())"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "llama_parse",
+   "language": "python",
+   "name": "llama_parse"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

examples/parse/multimodal/gpt4o_mini.ipynb

@@ -7,7 +7,7 @@
    "source": [
     "# Multimodal Parsing using GPT4o-mini\n",
     "\n",
-    "<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/multimodal/gpt4o_mini.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
+    "<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/parse/multimodal/gpt4o_mini.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
     "\n",
     "This cookbook shows you how to use LlamaParse to parse any document with the multimodal capabilities of GPT4o-mini.\n",
     "\n",

examples/parse/multimodal/insurance_rag.ipynb

@@ -6,7 +6,7 @@
    "source": [
     "# Building a Multimodal RAG Pipeline over an Auto Insurance Claim\n",
     "\n",
-    "<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/multimodal/insurance_rag.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
+    "<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/parse/multimodal/insurance_rag.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
    ]
   },
   {

examples/parse/multimodal/legal_rag.ipynb

@@ -6,7 +6,7 @@
    "source": [
     "# Building a RAG Pipeline over Legal Documents\n",
     "\n",
-    "<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/multimodal/legal_rag.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
+    "<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/parse/multimodal/legal_rag.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
     "\n",
     "This example shows how LlamaParse and LlamaIndex can be used to parse various types of legal documents, which may contain complex tabular data. The advantage of this is being able to quickly retrieve a specific answer to a legal question with comprehensive context — knowledge of precedents, statutes, and cases presented in the given documents. A user can quickly find the answer to or find out more details about a specific legal question without having to read through the often long documents by using LLMs.\n",
     "\n",

examples/parse/multimodal/multimodal_contextual_retrieval_rag.ipynb

@@ -7,7 +7,7 @@
    "source": [
     "# Contextual Retrieval for Multimodal RAG\n",
     "\n",
-    "<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/multimodal/multimodal_contextual_retrieval_rag.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
+    "<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/parse/multimodal/multimodal_contextual_retrieval_rag.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
     "\n",
     "In this cookbook we show you how to build a multimodal RAG pipeline with **contextual retrieval**.\n",
     "\n",

examples/parse/multimodal/multimodal_rag_slide_deck.ipynb

@@ -7,7 +7,7 @@
    "source": [
     "# Building a Natively Multimodal RAG Pipeline (over a Slide Deck)\n",
     "\n",
-    "<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/multimodal/multimodal_rag_slide_deck.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
+    "<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/parse/multimodal/multimodal_rag_slide_deck.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
     "\n",
     "In this cookbook we show you how to build a multimodal RAG pipeline over a slide deck, with text, tables, images, diagrams, and complex layouts.\n",
     "\n",

examples/parse/multimodal/multimodal_report_generation.ipynb

@@ -7,7 +7,7 @@
    "source": [
     "# Multimodal Report Generation (from a Slide Deck)\n",
     "\n",
-    "<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/multimodal/multimodal_report_generation.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
+    "<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/parse/multimodal/multimodal_report_generation.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
     "\n",
     "In this cookbook we show you how to build a multimodal report generator. The pipeline parses a slide deck and stores both text and image chunks. It generates a detailed response that contains interleaving text and images.\n",
     "\n",

examples/parse/multimodal/multimodal_report_generation_agent.ipynb

@@ -7,7 +7,7 @@
    "source": [
     "# Multimodal Report Generation Agent \n",
     "\n",
-    "<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/multimodal/multimodal_report_generation_agent.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
+    "<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/parse/multimodal/multimodal_report_generation_agent.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
     "\n",
     "In this cookbook we show you how to build a multimodal report generation agent from a bank of research reports. We use the a set of ICLR papers (which were also used as the dataset in our [DeepLearning.ai course](https://www.deeplearning.ai/short-courses/building-agentic-rag-with-llamaindex/?utm_campaign=llamaindexC2-launch&utm_medium=headband&utm_source=dlai-homepage).\n",
     "\n",

examples/parse/multimodal/product_manual_rag.ipynb

@@ -6,7 +6,7 @@
    "source": [
     "# Building a RAG Pipeline over IKEA Product Instruction Manuals\n",
     "\n",
-    "<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/multimodal/product_manual_rag.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
+    "<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/parse/multimodal/product_manual_rag.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
    ]
   },
   {

examples/parse/parsing_instructions/parsing_instructions.ipynb

@@ -4,7 +4,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/parsing_instructions.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
+    "<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/parse/parsing_instructions.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
     "\n",
     "# Parsing documents with Instructions\n",
     "\n",

examples/parse/parsing_modes/demo_auto_mode.ipynb

@@ -6,7 +6,7 @@
    "source": [
     "# Cost-Optimized Parsing with Auto-Mode\n",
     "\n",
-    "<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/demo_auto_mode.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
+    "<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/parse/parsing_modes/demo_auto_mode.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
     "\n",
     "![](diagram.jpg)\n",
     "\n",
@@ -735,7 +735,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "In this example, these pages aren't going to be that different when parsed, but we can verify which pages triggered auto-made by looking at the [JSON output](https://github.com/run-llama/llama_parse/blob/main/examples/demo_json_tour.ipynb) of LlamaParse:"
+    "In this example, these pages aren't going to be that different when parsed, but we can verify which pages triggered auto-made by looking at the [JSON output](https://github.com/run-llama/llama_cloud_services/blob/main/examples/demo_json_tour.ipynb) of LlamaParse:"
    ]
   },
   {

examples/parse/report_generation/rfp_response/generate_rfp.ipynb

@@ -7,7 +7,7 @@
    "source": [
     "# RFP Response Generation Workflow\n",
     "\n",
-    "<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/report_generation/rfp_response/generate_rfp.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
+    "<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/parse/report_generation/rfp_response/generate_rfp.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
     "\n",
     "This notebook shows you how to build a workflow to generate a response to an RFP. \n",
     "\n",
@@ -20,7 +20,7 @@
     "\n",
     "We use LlamaParse to parse the context documents as well as the RFP document itself.\n",
     "\n",
-    "**NOTE**: If you want to skip the indexing complexity and use LlamaCloud instead, check out the [RFP Example using LlamaCloud](https://github.com/run-llama/llamacloud-demo/blob/main/examples/report_generation/rfp_response/generate_rfp.ipynb)."
+    "**NOTE**: If you want to skip the indexing complexity and use LlamaCloud instead, check out the [RFP Example using LlamaCloud](https://github.com/run-llama/llama_cloud_services-demo/blob/main/examples/report_generation/rfp_response/generate_rfp.ipynb)."
    ]
   },
   {

examples/parse/test_tesla_impact_report/test_gpt4o.ipynb

@@ -8,7 +8,7 @@
     "# LlamaParse with GPT-4o\n",
     "\n",
     "\n",
-    "<a href=\"https://colab.research.google.com/github/run-llama/llama_parse/blob/main/examples/test_tesla_impact_report/test_gpt4o.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
+    "<a href=\"https://colab.research.google.com/github/run-llama/llama_cloud_services/blob/main/examples/parse/test_tesla_impact_report/test_gpt4o.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
     "\n",
     "GPT-4o is a [fully multimodal model by OpenAI](https://openai.com/index/hello-gpt-4o/) released in May 2024. It matches GPT-4 Turbo performance in text and code, and has significantly improved vision and audio capabilities.\n",
     "\n",

extract.md

@@ -0,0 +1,186 @@
+# LlamaExtract
+
+> **⚠️ EXPERIMENTAL**
+> This library is under active development with frequent breaking changes. APIs and functionality may change significantly between versions. If you're interested in being an early adopter, please contact us at [support@llamaindex.ai](mailto:support@llamaindex.ai) or join our [Discord](https://discord.com/invite/eN6D2HQ4aX).
+
+LlamaExtract provides a simple API for extracting structured data from unstructured documents like PDFs, text files and images (upcoming).
+
+## Quick Start
+
+```python
+from llama_cloud_services import LlamaExtract
+from pydantic import BaseModel, Field
+
+# Initialize client
+extractor = LlamaExtract()
+
+
+# Define schema using Pydantic
+class Resume(BaseModel):
+    name: str = Field(description="Full name of candidate")
+    email: str = Field(description="Email address")
+    skills: list[str] = Field(description="Technical skills and technologies")
+
+
+# Create extraction agent
+agent = extractor.create_agent(name="resume-parser", data_schema=Resume)
+
+# Extract data from document
+result = agent.extract("resume.pdf")
+print(result.data)
+```
+
+## Core Concepts
+
+- **Extraction Agents**: Reusable extractors configured with a specific schema and extraction settings.
+- **Data Schema**: Structure definition for the data you want to extract in the form of a JSON schema or a Pydantic model.
+- **Extraction Jobs**: Asynchronous extraction tasks that can be monitored.
+
+## Defining Schemas
+
+Schemas can be defined using either Pydantic models or JSON Schema:
+
+### Using Pydantic (Recommended)
+
+```python
+from pydantic import BaseModel, Field
+from typing import List, Optional
+
+
+class Experience(BaseModel):
+    company: str = Field(description="Company name")
+    title: str = Field(description="Job title")
+    start_date: Optional[str] = Field(description="Start date of employment")
+    end_date: Optional[str] = Field(description="End date of employment")
+
+
+class Resume(BaseModel):
+    name: str = Field(description="Candidate name")
+    experience: List[Experience] = Field(description="Work history")
+```
+
+### Using JSON Schema
+
+```python
+schema = {
+    "type": "object",
+    "properties": {
+        "name": {"type": "string", "description": "Candidate name"},
+        "experience": {
+            "type": "array",
+            "description": "Work history",
+            "items": {
+                "type": "object",
+                "properties": {
+                    "company": {
+                        "type": "string",
+                        "description": "Company name",
+                    },
+                    "title": {"type": "string", "description": "Job title"},
+                    "start_date": {
+                        "anyOf": [{"type": "string"}, {"type": "null"}],
+                        "description": "Start date of employment",
+                    },
+                    "end_date": {
+                        "anyOf": [{"type": "string"}, {"type": "null"}],
+                        "description": "End date of employment",
+                    },
+                },
+            },
+        },
+    },
+}
+
+agent = extractor.create_agent(name="resume-parser", data_schema=schema)
+```
+
+### Important restrictions on JSON/Pydantic Schema
+
+_LlamaExtract only supports a subset of the JSON Schema specification._ While limited, it should
+be sufficient for a wide variety of use-cases.
+
+- All fields are required by default. Nullable fields must be explicitly marked as such,
+  using `"anyOf"` with a `"null"` type. See `"start_date"` field above.
+- Root node must be of type `"object"`.
+- Schema nesting must be limited to within 5 levels.
+- The important fields are key names/titles, type and description. Fields for
+  formatting, default values, etc. are not supported.
+- There are other restrictions on number of keys, size of the schema, etc. that you may
+  hit for complex extraction use cases. In such cases, it is worth thinking how to restructure
+  your extraction workflow to fit within these constraints, e.g. by extracting subset of fields
+  and later merging them together.
+
+## Other Extraction APIs
+
+### Batch Processing
+
+Process multiple files asynchronously:
+
+```python
+# Queue multiple files for extraction
+jobs = await agent.queue_extraction(["resume1.pdf", "resume2.pdf"])
+
+# Check job status
+for job in jobs:
+    status = agent.get_extraction_job(job.id).status
+    print(f"Job {job.id}: {status}")
+
+# Get results when complete
+results = [agent.get_extraction_run_for_job(job.id) for job in jobs]
+```
+
+### Updating Schemas
+
+Schemas can be modified and updated after creation:
+
+```python
+# Update schema
+agent.data_schema = new_schema
+
+# Save changes
+agent.save()
+```
+
+### Managing Agents
+
+```python
+# List all agents
+agents = extractor.list_agents()
+
+# Get specific agent
+agent = extractor.get_agent(name="resume-parser")
+
+# Delete agent
+extractor.delete_agent(agent.id)
+```
+
+## Installation
+
+```bash
+pip install llama-extract==0.1.0
+```
+
+## Tips & Best Practices
+
+1. **Schema Design**:
+
+   - Try to limit schema nesting to 3-4 levels.
+   - Make fields optional when data might not always be present. Having required fields may force the model
+     to hallucinate when these fields are not present in the documents.
+   - When you want to extract a variable number of entities, use an `array` type. Note that you cannot use
+     an `array` type for the root node.
+   - Use descriptive field names and detailed descriptions. Use descriptions to pass formatting
+     instructions or few-shot examples.
+   - Start simple and iteratively build your schema to incorporate requirements.
+
+2. **Running Extractions**:
+   - Note that resetting `agent.schema` will not save the schema to the database,
+     until you call `agent.save`, but it will be used for running extractions.
+   - Check job status prior to accessing results. Any extraction error should be available as
+     part of `job.error` or `extraction_run.error` fields for debugging.
+   - Consider async operations (`queue_extraction`) for large-scale extraction once you have finalized your schema.
+
+## Additional Resources
+
+- [Example Notebook](examples/resume_screening.ipynb) - Detailed walkthrough of resume parsing
+- [Discord Community](https://discord.com/invite/eN6D2HQ4aX) - Get help and share feedback

llama_cloud_services/__init__.py

@@ -1,8 +1,11 @@
 from llama_cloud_services.parse import LlamaParse
 from llama_cloud_services.report import ReportClient, LlamaReport
+from llama_cloud_services.extract import LlamaExtract, ExtractionAgent
 
 __all__ = [
     "LlamaParse",
     "ReportClient",
     "LlamaReport",
+    "LlamaExtract",
+    "ExtractionAgent",
 ]

llama_cloud_services/extract/__init__.py

@@ -0,0 +1,3 @@
+from llama_cloud_services.extract.extract import LlamaExtract, ExtractionAgent
+
+__all__ = ["LlamaExtract", "ExtractionAgent"]

llama_cloud_services/extract/extract.py

@@ -0,0 +1,655 @@
+import asyncio
+import os
+import time
+from io import BufferedIOBase, BufferedReader, BytesIO
+from pathlib import Path
+from typing import List, Optional, Type, Union, Coroutine, Any, TypeVar
+import warnings
+import httpx
+from pydantic import BaseModel
+from llama_cloud import (
+    ExtractAgent as CloudExtractAgent,
+    ExtractConfig,
+    ExtractJob,
+    ExtractJobCreate,
+    ExtractRun,
+    File,
+    ExtractMode,
+    StatusEnum,
+    Project,
+    ExtractTarget,
+    LlamaExtractSettings,
+)
+from llama_cloud.client import AsyncLlamaCloud
+from llama_cloud_services.extract.utils import JSONObjectType, augment_async_errors
+from llama_index.core.schema import BaseComponent
+from llama_index.core.async_utils import run_jobs
+from llama_index.core.bridge.pydantic import Field, PrivateAttr
+from llama_index.core.constants import DEFAULT_BASE_URL
+from concurrent.futures import ThreadPoolExecutor
+
+T = TypeVar("T")
+
+FileInput = Union[str, Path, bytes, BufferedIOBase]
+SchemaInput = Union[JSONObjectType, Type[BaseModel]]
+
+DEFAULT_EXTRACT_CONFIG = ExtractConfig(
+    extraction_target=ExtractTarget.PER_DOC,
+    extraction_mode=ExtractMode.ACCURATE,
+)
+
+
+class ExtractionAgent:
+    """Class representing a single extraction agent with methods for extraction operations."""
+
+    def __init__(
+        self,
+        client: AsyncLlamaCloud,
+        agent: CloudExtractAgent,
+        project_id: Optional[str] = None,
+        organization_id: Optional[str] = None,
+        check_interval: int = 1,
+        max_timeout: int = 2000,
+        num_workers: int = 4,
+        show_progress: bool = True,
+        verbose: bool = False,
+    ):
+        self._client = client
+        self._agent = agent
+        self._project_id = project_id
+        self._organization_id = organization_id
+        self.check_interval = check_interval
+        self.max_timeout = max_timeout
+        self.num_workers = num_workers
+        self.show_progress = show_progress
+        self._verbose = verbose
+        self._data_schema: Union[JSONObjectType, None] = None
+        self._config: Union[ExtractConfig, None] = None
+        self._thread_pool = ThreadPoolExecutor(
+            max_workers=min(10, (os.cpu_count() or 1) + 4)
+        )
+
+    def _run_in_thread(self, coro: Coroutine[Any, Any, T]) -> T:
+        """Run coroutine in a separate thread to avoid event loop issues"""
+
+        def run_coro() -> T:
+            async def wrapped_coro() -> T:
+                async with httpx.AsyncClient(
+                    timeout=self._client._client_wrapper.httpx_client.timeout,
+                ) as client:
+                    original_client = self._client._client_wrapper.httpx_client
+                    self._client._client_wrapper.httpx_client = client
+                    try:
+                        return await coro
+                    finally:
+                        self._client._client_wrapper.httpx_client = original_client
+
+            return asyncio.run(wrapped_coro())
+
+        return self._thread_pool.submit(run_coro).result()
+
+    @property
+    def id(self) -> str:
+        return self._agent.id
+
+    @property
+    def name(self) -> str:
+        return self._agent.name
+
+    @property
+    def data_schema(self) -> dict:
+        return self._agent.data_schema if not self._data_schema else self._data_schema
+
+    @data_schema.setter
+    def data_schema(self, data_schema: SchemaInput) -> None:
+        processed_schema: JSONObjectType
+        if isinstance(data_schema, dict):
+            # TODO: if we expose a get_validated JSON schema method, we can use it here
+            processed_schema = data_schema  # type: ignore
+        elif isinstance(data_schema, type) and issubclass(data_schema, BaseModel):
+            processed_schema = data_schema.model_json_schema()
+        else:
+            raise ValueError(
+                "data_schema must be either a dictionary or a Pydantic model"
+            )
+        validated_schema = self._run_in_thread(
+            self._client.llama_extract.validate_extraction_schema(
+                data_schema=processed_schema
+            )
+        )
+        self._data_schema = validated_schema.data_schema
+
+    @property
+    def config(self) -> ExtractConfig:
+        return self._agent.config if not self._config else self._config
+
+    @config.setter
+    def config(self, config: ExtractConfig) -> None:
+        self._config = config
+
+    async def _upload_file(self, file_input: FileInput) -> File:
+        """Upload a file for extraction."""
+        if isinstance(file_input, BufferedIOBase):
+            upload_file = file_input
+        elif isinstance(file_input, bytes):
+            upload_file = BytesIO(file_input)
+        elif isinstance(file_input, (str, Path)):
+            upload_file = open(file_input, "rb")
+        else:
+            raise ValueError(
+                "file_input must be either a file path string, file bytes, or buffer object"
+            )
+
+        try:
+            return await self._client.files.upload_file(
+                project_id=self._project_id, upload_file=upload_file
+            )
+        finally:
+            if isinstance(upload_file, BufferedReader):
+                upload_file.close()
+
+    async def _wait_for_job_result(self, job_id: str) -> Optional[ExtractRun]:
+        """Wait for and return the results of an extraction job."""
+        start = time.perf_counter()
+        tries = 0
+        while True:
+            await asyncio.sleep(self.check_interval)
+            tries += 1
+            job = await self._client.llama_extract.get_job(
+                job_id=job_id,
+            )
+
+            if job.status == StatusEnum.SUCCESS:
+                return await self._client.llama_extract.get_run_by_job_id(
+                    job_id=job_id,
+                )
+            elif job.status == StatusEnum.PENDING:
+                end = time.perf_counter()
+                if end - start > self.max_timeout:
+                    raise Exception(f"Timeout while extracting the file: {job_id}")
+                if self._verbose and tries % 10 == 0:
+                    print(".", end="", flush=True)
+                continue
+            else:
+                warnings.warn(
+                    f"Failure in job: {job_id}, status: {job.status}, error: {job.error}"
+                )
+                return await self._client.llama_extract.get_run_by_job_id(
+                    job_id=job_id,
+                )
+
+    def save(self) -> None:
+        """Persist the extraction agent's schema and config to the database.
+
+        Returns:
+            ExtractionAgent: The updated extraction agent
+        """
+        self._agent = self._run_in_thread(
+            self._client.llama_extract.update_extraction_agent(
+                extraction_agent_id=self.id,
+                data_schema=self.data_schema,
+                config=self.config,
+            )
+        )
+
+    async def _queue_extraction_test(
+        self,
+        files: Union[FileInput, List[FileInput]],
+        extract_settings: LlamaExtractSettings,
+    ) -> Union[ExtractJob, List[ExtractJob]]:
+        if not isinstance(files, list):
+            files = [files]
+            single_file = True
+        else:
+            single_file = False
+
+        upload_tasks = [self._upload_file(file) for file in files]
+        with augment_async_errors():
+            uploaded_files = await run_jobs(
+                upload_tasks,
+                workers=self.num_workers,
+                desc="Uploading files",
+                show_progress=self.show_progress,
+            )
+
+        async def run_job(file: File) -> ExtractRun:
+            job_queued = await self._client.llama_extract.run_job_test_user(
+                job_create=ExtractJobCreate(
+                    extraction_agent_id=self.id,
+                    file_id=file.id,
+                    data_schema_override=self.data_schema,
+                    config_override=self.config,
+                ),
+                extract_settings=extract_settings,
+            )
+            return await self._wait_for_job_result(job_queued.id)
+
+        job_tasks = [run_job(file) for file in uploaded_files]
+        with augment_async_errors():
+            extract_jobs = await run_jobs(
+                job_tasks,
+                workers=self.num_workers,
+                desc="Running extraction jobs",
+                show_progress=self.show_progress,
+            )
+
+        if self._verbose:
+            for file, job in zip(files, extract_jobs):
+                file_repr = (
+                    str(file) if isinstance(file, (str, Path)) else "<bytes/buffer>"
+                )
+                print(
+                    f"Queued file extraction for file {file_repr} under job_id {job.id}"
+                )
+
+        return extract_jobs[0] if single_file else extract_jobs
+
+    async def queue_extraction(
+        self,
+        files: Union[FileInput, List[FileInput]],
+    ) -> Union[ExtractJob, List[ExtractJob]]:
+        """
+        Queue multiple files for extraction.
+
+        Args:
+            files (Union[FileInput, List[FileInput]]): The files to extract
+
+        Returns:
+            Union[ExtractJob, List[ExtractJob]]: The queued extraction jobs
+        """
+        """Queue one or more files for extraction concurrently."""
+        if not isinstance(files, list):
+            files = [files]
+            single_file = True
+        else:
+            single_file = False
+
+        upload_tasks = [self._upload_file(file) for file in files]
+        with augment_async_errors():
+            uploaded_files = await run_jobs(
+                upload_tasks,
+                workers=self.num_workers,
+                desc="Uploading files",
+                show_progress=self.show_progress,
+            )
+
+        job_tasks = [
+            self._client.llama_extract.run_job(
+                request=ExtractJobCreate(
+                    extraction_agent_id=self.id,
+                    file_id=file.id,
+                    data_schema_override=self.data_schema,
+                    config_override=self.config,
+                ),
+            )
+            for file in uploaded_files
+        ]
+        with augment_async_errors():
+            extract_jobs = await run_jobs(
+                job_tasks,
+                workers=self.num_workers,
+                desc="Creating extraction jobs",
+                show_progress=self.show_progress,
+            )
+
+        if self._verbose:
+            for file, job in zip(files, extract_jobs):
+                file_repr = (
+                    str(file) if isinstance(file, (str, Path)) else "<bytes/buffer>"
+                )
+                print(
+                    f"Queued file extraction for file {file_repr} under job_id {job.id}"
+                )
+
+        return extract_jobs[0] if single_file else extract_jobs
+
+    async def aextract(
+        self, files: Union[FileInput, List[FileInput]]
+    ) -> Union[ExtractRun, List[ExtractRun]]:
+        """Asynchronously extract data from one or more files using this agent.
+
+        Args:
+            files (Union[FileInput, List[FileInput]]): The files to extract
+
+        Returns:
+            Union[ExtractRun, List[ExtractRun]]: The extraction results
+        """
+        if not isinstance(files, list):
+            files = [files]
+            single_file = True
+        else:
+            single_file = False
+
+        # Queue all files for extraction
+        jobs = await self.queue_extraction(files)
+        # Wait for all results concurrently
+        result_tasks = [self._wait_for_job_result(job.id) for job in jobs]
+        with augment_async_errors():
+            results = await run_jobs(
+                result_tasks,
+                workers=self.num_workers,
+                desc="Extracting files",
+                show_progress=self.show_progress,
+            )
+
+        return results[0] if single_file else results
+
+    def extract(
+        self, files: Union[FileInput, List[FileInput]]
+    ) -> Union[ExtractRun, List[ExtractRun]]:
+        """Synchronously extract data from one or more files using this agent.
+
+        Args:
+            files (Union[FileInput, List[FileInput]]): The files to extract
+
+        Returns:
+            Union[ExtractRun, List[ExtractRun]]: The extraction results
+        """
+        return self._run_in_thread(self.aextract(files))
+
+    def get_extraction_job(self, job_id: str) -> ExtractJob:
+        """
+        Get the extraction job for a given job_id.
+
+        Args:
+            job_id (str): The job_id to get the extraction job for
+
+        Returns:
+            ExtractJob: The extraction job
+        """
+        return self._run_in_thread(self._client.llama_extract.get_job(job_id=job_id))
+
+    def get_extraction_run_for_job(self, job_id: str) -> ExtractRun:
+        """
+        Get the extraction run for a given job_id.
+
+        Args:
+            job_id (str): The job_id to get the extraction run for
+
+        Returns:
+            ExtractRun: The extraction run
+        """
+        return self._run_in_thread(
+            self._client.llama_extract.get_run_by_job_id(
+                job_id=job_id,
+            )
+        )
+
+    def list_extraction_runs(self) -> List[ExtractRun]:
+        """List extraction runs for the extraction agent.
+
+        Returns:
+            List[ExtractRun]: List of extraction runs
+        """
+        return self._run_in_thread(
+            self._client.llama_extract.list_extract_runs(
+                extraction_agent_id=self.id,
+            )
+        )
+
+    def __repr__(self) -> str:
+        return f"ExtractionAgent(id={self.id}, name={self.name})"
+
+
+class LlamaExtract(BaseComponent):
+    """Factory class for creating and managing extraction agents."""
+
+    api_key: str = Field(description="The API key for the LlamaExtract API.")
+    base_url: str = Field(description="The base URL of the LlamaExtract API.")
+    check_interval: int = Field(
+        default=1,
+        description="The interval in seconds to check if the extraction is done.",
+    )
+    max_timeout: int = Field(
+        default=2000,
+        description="The maximum timeout in seconds to wait for the extraction to finish.",
+    )
+    num_workers: int = Field(
+        default=4,
+        gt=0,
+        lt=10,
+        description="The number of workers to use sending API requests for extraction.",
+    )
+    show_progress: bool = Field(
+        default=True, description="Show progress when extracting multiple files."
+    )
+    verbose: bool = Field(
+        default=False, description="Show verbose output when extracting files."
+    )
+    _async_client: AsyncLlamaCloud = PrivateAttr()
+    _thread_pool: ThreadPoolExecutor = PrivateAttr()
+    _project_id: Optional[str] = PrivateAttr()
+    _organization_id: Optional[str] = PrivateAttr()
+
+    def __init__(
+        self,
+        api_key: Optional[str] = None,
+        base_url: Optional[str] = None,
+        check_interval: int = 1,
+        max_timeout: int = 2000,
+        num_workers: int = 4,
+        show_progress: bool = True,
+        project_id: Optional[str] = None,
+        organization_id: Optional[str] = None,
+        verbose: bool = False,
+    ):
+        if not api_key:
+            api_key = os.getenv("LLAMA_CLOUD_API_KEY", None)
+            if api_key is None:
+                raise ValueError("The API key is required.")
+
+        if not base_url:
+            base_url = os.getenv("LLAMA_CLOUD_BASE_URL", None) or DEFAULT_BASE_URL
+
+        super().__init__(
+            api_key=api_key,
+            base_url=base_url,
+            check_interval=check_interval,
+            max_timeout=max_timeout,
+            num_workers=num_workers,
+            show_progress=show_progress,
+            verbose=verbose,
+        )
+
+        self._async_client = AsyncLlamaCloud(
+            token=self.api_key, base_url=self.base_url, timeout=None
+        )
+        self._thread_pool = ThreadPoolExecutor(
+            max_workers=min(10, (os.cpu_count() or 1) + 4)
+        )
+        # Fetch default project id if not provided
+        if not project_id:
+            project_id = os.getenv("LLAMA_CLOUD_PROJECT_ID", None)
+            if not project_id:
+                print("No project_id provided, fetching default project.")
+                projects: List[Project] = self._run_in_thread(
+                    self._async_client.projects.list_projects()
+                )
+                default_project = [p for p in projects if p.is_default]
+                if not default_project:
+                    raise ValueError(
+                        "No default project found. Please provide a project_id."
+                    )
+                project_id = default_project[0].id
+
+        self._project_id = project_id
+        self._organization_id = organization_id
+
+    def _run_in_thread(self, coro: Coroutine[Any, Any, T]) -> T:
+        """Run coroutine in a separate thread to avoid event loop issues"""
+
+        def run_coro() -> T:
+            # Create a new client for this thread
+            async def wrapped_coro() -> T:
+                async with httpx.AsyncClient(
+                    timeout=self._async_client._client_wrapper.httpx_client.timeout,
+                ) as client:
+                    # Replace the client in the coro's context
+                    original_client = self._async_client._client_wrapper.httpx_client
+                    self._async_client._client_wrapper.httpx_client = client
+                    try:
+                        return await coro
+                    finally:
+                        self._async_client._client_wrapper.httpx_client = (
+                            original_client
+                        )
+
+            return asyncio.run(wrapped_coro())
+
+        return self._thread_pool.submit(run_coro).result()
+
+    def create_agent(
+        self,
+        name: str,
+        data_schema: SchemaInput,
+        config: Optional[ExtractConfig] = None,
+    ) -> ExtractionAgent:
+        """Create a new extraction agent.
+
+        Args:
+            name (str): The name of the extraction agent
+            data_schema (SchemaInput): The data schema for the extraction agent
+            config (Optional[ExtractConfig]): The extraction config for the agent
+
+        Returns:
+            ExtractionAgent: The created extraction agent
+        """
+
+        if isinstance(data_schema, dict):
+            data_schema = data_schema
+        elif issubclass(data_schema, BaseModel):
+            data_schema = data_schema.model_json_schema()
+        else:
+            raise ValueError(
+                "data_schema must be either a dictionary or a Pydantic model"
+            )
+
+        agent = self._run_in_thread(
+            self._async_client.llama_extract.create_extraction_agent(
+                name=name,
+                data_schema=data_schema,
+                config=config or DEFAULT_EXTRACT_CONFIG,
+                project_id=self._project_id,
+                organization_id=self._organization_id,
+            )
+        )
+
+        return ExtractionAgent(
+            client=self._async_client,
+            agent=agent,
+            project_id=self._project_id,
+            organization_id=self._organization_id,
+            check_interval=self.check_interval,
+            max_timeout=self.max_timeout,
+            num_workers=self.num_workers,
+            show_progress=self.show_progress,
+            verbose=self.verbose,
+        )
+
+    def get_agent(
+        self,
+        name: Optional[str] = None,
+        id: Optional[str] = None,
+    ) -> ExtractionAgent:
+        """Get extraction agents by name or extraction agent ID.
+
+        Args:
+            name (Optional[str]): Filter by name
+            extraction_agent_id (Optional[str]): Filter by extraction agent ID
+
+        Returns:
+            ExtractionAgent: The extraction agent
+        """
+        if id is not None and name is not None:
+            warnings.warn(
+                "Both name and extraction_agent_id are provided. Using extraction_agent_id."
+            )
+
+        if id:
+            agent = self._run_in_thread(
+                self._async_client.llama_extract.get_extraction_agent(
+                    extraction_agent_id=id,
+                )
+            )
+
+        elif name:
+            agent = self._run_in_thread(
+                self._async_client.llama_extract.get_extraction_agent_by_name(
+                    name=name,
+                    project_id=self._project_id,
+                )
+            )
+        else:
+            raise ValueError("Either name or extraction_agent_id must be provided.")
+
+        return ExtractionAgent(
+            client=self._async_client,
+            agent=agent,
+            project_id=self._project_id,
+            organization_id=self._organization_id,
+            check_interval=self.check_interval,
+            max_timeout=self.max_timeout,
+            num_workers=self.num_workers,
+            show_progress=self.show_progress,
+            verbose=self.verbose,
+        )
+
+    def list_agents(self) -> List[ExtractionAgent]:
+        """List all available extraction agents."""
+        agents = self._run_in_thread(
+            self._async_client.llama_extract.list_extraction_agents(
+                project_id=self._project_id,
+            )
+        )
+
+        return [
+            ExtractionAgent(
+                client=self._async_client,
+                agent=agent,
+                project_id=self._project_id,
+                organization_id=self._organization_id,
+                check_interval=self.check_interval,
+                max_timeout=self.max_timeout,
+                num_workers=self.num_workers,
+                show_progress=self.show_progress,
+                verbose=self.verbose,
+            )
+            for agent in agents
+        ]
+
+    def delete_agent(self, agent_id: str) -> None:
+        """Delete an extraction agent by ID.
+
+        Args:
+            agent_id (str): ID of the extraction agent to delete
+        """
+        self._run_in_thread(
+            self._async_client.llama_extract.delete_extraction_agent(
+                extraction_agent_id=agent_id
+            )
+        )
+
+
+if __name__ == "__main__":
+    from dotenv import load_dotenv
+
+    load_dotenv()
+
+    data_dir = Path(__file__).parent.parent / "tests" / "data"
+    extractor = LlamaExtract()
+    try:
+        agent = extractor.get_agent(name="test-agent")
+    except Exception:
+        agent = extractor.create_agent(
+            "test-agent",
+            {
+                "type": "object",
+                "properties": {
+                    "title": {"type": "string"},
+                    "summary": {"type": "string"},
+                },
+            },
+        )
+    results = agent.extract(data_dir / "slide" / "conocophilips.pdf")
+    extractor.delete_agent(agent.id)
+    print(results)

llama_cloud_services/extract/utils.py

@@ -0,0 +1,34 @@
+from typing import Any, Dict, List, Union, Generator
+from contextlib import contextmanager
+
+# Asyncio error messages
+nest_asyncio_err = "cannot be called from a running event loop"
+nest_asyncio_msg = (
+    "The event loop is already running. "
+    "Add `import nest_asyncio; nest_asyncio.apply()` to your code to fix this issue."
+)
+
+
+def is_jupyter() -> bool:
+    """Check if we're running in a Jupyter environment."""
+    try:
+        from IPython import get_ipython
+
+        return get_ipython().__class__.__name__ == "ZMQInteractiveShell"
+    except (ImportError, AttributeError):
+        return False
+
+
+@contextmanager
+def augment_async_errors() -> Generator[None, None, None]:
+    """Context manager to add helpful information for errors due to nested event loops."""
+    try:
+        yield
+    except RuntimeError as e:
+        if nest_asyncio_err in str(e):
+            raise RuntimeError(nest_asyncio_msg)
+        raise
+
+
+JSONType = Union[Dict[str, Any], List[Any], str, int, float, bool, None]
+JSONObjectType = Dict[str, JSONType]

llama_cloud_services/parse/base.py

@@ -37,6 +37,21 @@ JOB_STATUS_ROUTE = "/api/parsing/job/{job_id}"
 JOB_UPLOAD_ROUTE = "/api/parsing/upload"
 
 
+def build_url(
+    base_url: str, organization_id: Optional[str], project_id: Optional[str]
+) -> str:
+    query_params = {}
+    if organization_id:
+        query_params["organization_id"] = organization_id
+    if project_id:
+        query_params["project_id"] = project_id
+
+    if query_params:
+        return base_url + "?" + "&".join([f"{k}={v}" for k, v in query_params.items()])
+
+    return base_url
+
+
 class LlamaParse(BasePydanticReader):
     """A smart-parser for files."""
 
@@ -50,6 +65,14 @@ class LlamaParse(BasePydanticReader):
         default=DEFAULT_BASE_URL,
         description="The base URL of the Llama Parsing API.",
     )
+    organization_id: Optional[str] = Field(
+        default=None,
+        description="The organization ID for the LlamaParse API.",
+    )
+    project_id: Optional[str] = Field(
+        default=None,
+        description="The project ID for the LlamaParse API.",
+    )
     check_interval: int = Field(
         default=1,
         description="The interval in seconds to check if the parsing is done.",
@@ -88,6 +111,10 @@ class LlamaParse(BasePydanticReader):
     )
 
     # Parsing specific configurations (Alphabetical order)
+    adaptive_long_table: Optional[bool] = Field(
+        default=False,
+        description="If set to true, LlamaParse will try to detect long table and adapt the output.",
+    )
     annotate_links: Optional[bool] = Field(
         default=False,
         description="Annotate links found in the document to extract their URL.",
@@ -140,14 +167,7 @@ class LlamaParse(BasePydanticReader):
         default=None,
         description="The top margin of the bounding box to use to extract text from documents expressed as a float between 0 and 1 representing the percentage of the page height.",
     )
-    complemental_formatting_instruction: Optional[str] = Field(
-        default=None,
-        description="The complemental formatting instruction for the parser. Tell llamaParse how some thing should to be formatted, while retaining the markdown output.",
-    )
-    content_guideline_instruction: Optional[str] = Field(
-        default=None,
-        description="The content guideline for the parser. Tell LlamaParse how the content should be changed / transformed.",
-    )
+
     continuous_mode: Optional[bool] = Field(
         default=False,
         description="Parse documents continuously, leading to better results on documents where tables span across two pages.",
@@ -180,10 +200,7 @@ class LlamaParse(BasePydanticReader):
         default=False,
         description="Note: Non compatible with gpt-4o. If set to true, the parser will use a faster mode to extract text from documents. This mode will skip OCR of images, and table/heading reconstruction.",
     )
-    formatting_instruction: Optional[str] = Field(
-        default=None,
-        description="The Formatting instruction for the parser. Override default llamaParse behavior. In most case you want to use complemental_formatting_instruction instead.",
-    )
+
     guess_xlsx_sheet_names: Optional[bool] = Field(
         default=False,
         description="Whether to guess the sheet names of the xlsx file.",
@@ -259,6 +276,10 @@ class LlamaParse(BasePydanticReader):
         default=None,
         description="A templated suffix to add to the beginning of each page. If it contain `{page_number}`, it will be replaced by the page number.",
     )
+    parsing_mode: Optional[str] = Field(
+        default=None,
+        description="The parsing mode to use, see ParsingMode enum for possible values ",
+    )
     premium_mode: Optional[bool] = Field(
         default=False,
         description="Use our best parser mode if set to True.",
@@ -304,6 +325,14 @@ class LlamaParse(BasePydanticReader):
         default=None,
         description="The named JSON Schema to use to structure the output of the parsing job. For convenience / testing, LlamaParse provides a few named JSON Schema that can be used directly. Use 'imFeelingLucky' to let llamaParse dream the schema.",
     )
+    system_prompt: Optional[str] = Field(
+        default=None,
+        description="The system prompt. Replace llamaParse default system prompt, may impact accuracy",
+    )
+    system_prompt_append: Optional[str] = Field(
+        default=None,
+        description="String to append to default system prompt.",
+    )
     take_screenshot: Optional[bool] = Field(
         default=False,
         description="Whether to take screenshot of each page of the document.",
@@ -312,9 +341,9 @@ class LlamaParse(BasePydanticReader):
         default=None,
         description="The target pages to extract text from documents. Describe as a comma separated list of page numbers. The first page of the document is page 0",
     )
-    use_vendor_multimodal_model: Optional[bool] = Field(
-        default=False,
-        description="Whether to use the vendor multimodal API.",
+    user_prompt: Optional[str] = Field(
+        default=None,
+        description="The user prompt. Replace llamaParse default user prompt",
     )
     vendor_multimodal_api_key: Optional[str] = Field(
         default=None,
@@ -334,6 +363,18 @@ class LlamaParse(BasePydanticReader):
         default=None,
         description="The bounding box to use to extract text from documents describe as a string containing the bounding box margins",
     )
+    complemental_formatting_instruction: Optional[str] = Field(
+        default=None,
+        description="The complemental formatting instruction for the parser. Tell llamaParse how some thing should to be formatted, while retaining the markdown output.",
+    )
+    content_guideline_instruction: Optional[str] = Field(
+        default=None,
+        description="The content guideline for the parser. Tell LlamaParse how the content should be changed / transformed.",
+    )
+    formatting_instruction: Optional[str] = Field(
+        default=None,
+        description="The Formatting instruction for the parser. Override default llamaParse behavior. In most case you want to use complemental_formatting_instruction instead.",
+    )
     gpt4o_mode: Optional[bool] = Field(
         default=False,
         description="Whether to use gpt-4o extract text from documents.",
@@ -350,6 +391,11 @@ class LlamaParse(BasePydanticReader):
         default="", description="The parsing instruction for the parser."
     )
 
+    use_vendor_multimodal_model: Optional[bool] = Field(
+        default=False,
+        description="Whether to use the vendor multimodal API.",
+    )
+
     @field_validator("api_key", mode="before", check_fields=True)
     @classmethod
     def validate_api_key(cls, v: str) -> str:
@@ -478,6 +524,9 @@ class LlamaParse(BasePydanticReader):
 
         data["from_python_package"] = True
 
+        if self.adaptive_long_table:
+            data["adaptive_long_table"] = self.adaptive_long_table
+
         if self.annotate_links:
             data["annotate_links"] = self.annotate_links
 
@@ -529,11 +578,17 @@ class LlamaParse(BasePydanticReader):
             data["bbox_top"] = self.bbox_top
 
         if self.complemental_formatting_instruction:
+            print(
+                "WARNING: complemental_formatting_instruction is deprecated and may be remove in a future release. Use system_prompt, system_prompt_append or user_prompt instead."
+            )
             data[
                 "complemental_formatting_instruction"
             ] = self.complemental_formatting_instruction
 
         if self.content_guideline_instruction:
+            print(
+                "WARNING: content_guideline_instruction is deprecated and may be remove in a future release. Use system_prompt, system_prompt_append or user_prompt instead."
+            )
             data["content_guideline_instruction"] = self.content_guideline_instruction
 
         if self.continuous_mode:
@@ -561,6 +616,9 @@ class LlamaParse(BasePydanticReader):
             data["fast_mode"] = self.fast_mode
 
         if self.formatting_instruction:
+            print(
+                "WARNING: formatting_instruction is deprecated and may be remove in a future release. Use system_prompt, system_prompt_append or user_prompt instead."
+            )
             data["formatting_instruction"] = self.formatting_instruction
 
         if self.guess_xlsx_sheet_names:
@@ -600,6 +658,9 @@ class LlamaParse(BasePydanticReader):
             data["invalidate_cache"] = self.invalidate_cache
 
         if self.is_formatting_instruction:
+            print(
+                "WARNING: formatting_instruction is deprecated and may be remove in a future release. Use system_prompt, system_prompt_append or user_prompt instead."
+            )
             data["is_formatting_instruction"] = self.is_formatting_instruction
 
         if self.job_timeout_extra_time_per_page_in_seconds is not None:
@@ -639,9 +700,9 @@ class LlamaParse(BasePydanticReader):
         if self.page_suffix is not None:
             data["page_suffix"] = self.page_suffix
 
-        if self.parsing_instruction is not None:
+        if self.parsing_instruction:
             print(
-                "WARNING: parsing_instruction is deprecated. Use complemental_formatting_instruction or content_guideline_instruction instead."
+                "WARNING: parsing_instruction is deprecated. Use system_prompt, system_prompt_append or user_prompt instead."
             )
             data["parsing_instruction"] = self.parsing_instruction
 
@@ -676,13 +737,17 @@ class LlamaParse(BasePydanticReader):
             data[
                 "structured_output_json_schema_name"
             ] = self.structured_output_json_schema_name
-
+        if self.system_prompt is not None:
+            data["system_prompt"] = self.system_prompt
+        if self.system_prompt_append is not None:
+            data["system_prompt_append"] = self.system_prompt_append
         if self.take_screenshot:
             data["take_screenshot"] = self.take_screenshot
 
         if self.target_pages is not None:
             data["target_pages"] = self.target_pages
-
+        if self.user_prompt is not None:
+            data["user_prompt"] = self.user_prompt
         if self.use_vendor_multimodal_model:
             data["use_vendor_multimodal_model"] = self.use_vendor_multimodal_model
 
@@ -706,7 +771,8 @@ class LlamaParse(BasePydanticReader):
             data["gpt4o_api_key"] = self.gpt4o_api_key
 
         try:
-            resp = await self.aclient.post(JOB_UPLOAD_ROUTE, files=files, data=data)  # type: ignore
+            url = build_url(JOB_UPLOAD_ROUTE, self.organization_id, self.project_id)
+            resp = await self.aclient.post(url, files=files, data=data)  # type: ignore
             resp.raise_for_status()  # this raises if status is not 2xx
             return resp.json()["id"]
         except httpx.HTTPStatusError as err:  # this catches it

llama_cloud_services/parse/utils.py

@@ -14,6 +14,16 @@ class ResultType(str, Enum):
     STRUCTURED = "structured"
 
 
+class ParsingMode(str, Enum):
+    """The parsing mode for the parser."""
+
+    parse_page_without_llm = "parse_page_without_llm"
+    parse_page_with_llm = "parse_page_with_llm"
+    parse_page_with_lvm = "parse_page_with_lvm"
+    parse_page_with_agent = "parse_page_with_agent"
+    parse_document_with_llm = "parse_document_with_llm"
+
+
 class Language(str, Enum):
     BAZA = "abq"
     ADYGHE = "ady"

llama_parse/README.md

@@ -146,9 +146,9 @@ Full documentation for `SimpleDirectoryReader` can be found on the [LlamaIndex D
 
 Several end-to-end indexing examples can be found in the examples folder
 
-- [Getting Started](examples/demo_basic.ipynb)
-- [Advanced RAG Example](examples/demo_advanced.ipynb)
-- [Raw API Usage](examples/demo_api.ipynb)
+- [Getting Started](/examples/parse/demo_basic.ipynb)
+- [Advanced RAG Example](/examples/parse/demo_advanced.ipynb)
+- [Raw API Usage](/examples/parse/demo_api.ipynb)
 
 ## Documentation
 

llama_parse/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "poetry.core.masonry.api"
 
 [tool.poetry]
 name = "llama-parse"
-version = "0.6.0"
+version = "0.6.3"
 description = "Parse files into RAG-Optimized formats."
 authors = ["Logan Markewich <logan@llamaindex.ai>"]
 license = "MIT"
@@ -13,7 +13,7 @@ packages = [{include = "llama_parse"}]
 
 [tool.poetry.dependencies]
 python = ">=3.9,<4.0"
-llama-cloud-services = "*"
+llama-cloud-services = ">=0.6.3"
 
 [tool.poetry.group.dev.dependencies]
 pytest = "^8.0.0"

pyproject.toml

@@ -8,7 +8,7 @@ python_version = "3.10"
 
 [tool.poetry]
 name = "llama-cloud-services"
-version = "0.6.0"
+version = "0.6.4"
 description = "Tailored SDK clients for LlamaCloud services."
 authors = ["Logan Markewich <logan@runllama.ai>"]
 license = "MIT"

tests/extract/data/receipt/noisebridge_receipt.test.json

@@ -0,0 +1,37 @@
+{
+  "receiptNumber": "27215058",
+  "invoiceNumber": "87B37C90152",
+  "datePaid": "2024-07-19",
+  "paymentMethod": {
+    "type": "visa",
+    "lastFourDigits": "7267"
+  },
+  "merchant": {
+    "name": "Noisebridge",
+    "address": {
+      "street": "272 Capp St",
+      "city": "San Francisco",
+      "state": "California",
+      "postalCode": "94110",
+      "country": "United States"
+    },
+    "phone": "1 6507017829",
+    "email": "treasurer+stripe@noisebridge.net"
+  },
+  "billTo": "noisebridge@seldo.com",
+  "items": [
+    {
+      "description": "$10 / month",
+      "quantity": 1,
+      "unitPrice": 10.0,
+      "amount": 10.0,
+      "period": {
+        "start": "2024-07-19",
+        "end": "2024-08-19"
+      }
+    }
+  ],
+  "subtotal": 10.0,
+  "total": 10.0,
+  "amountPaid": 10.0
+}

tests/extract/data/receipt/schema.json

@@ -0,0 +1,124 @@
+{
+  "$schema": "http://json-schema.org/draft-07/schema#",
+  "type": "object",
+  "required": ["receiptNumber", "datePaid", "total", "items"],
+  "properties": {
+    "receiptNumber": {
+      "type": "string"
+    },
+    "invoiceNumber": {
+      "type": "string"
+    },
+    "datePaid": {
+      "type": "string",
+      "format": "date"
+    },
+    "paymentMethod": {
+      "type": "object",
+      "properties": {
+        "type": {
+          "type": "string",
+          "enum": ["visa", "mastercard", "amex", "cash", "other"]
+        },
+        "lastFourDigits": {
+          "type": "string",
+          "pattern": "^[0-9]{4}$"
+        }
+      }
+    },
+    "merchant": {
+      "type": "object",
+      "properties": {
+        "name": {
+          "type": "string"
+        },
+        "address": {
+          "type": "object",
+          "properties": {
+            "street": {
+              "type": "string"
+            },
+            "city": {
+              "type": "string"
+            },
+            "state": {
+              "type": "string"
+            },
+            "postalCode": {
+              "type": "string"
+            },
+            "country": {
+              "type": "string"
+            }
+          }
+        },
+        "phone": {
+          "type": "string"
+        },
+        "email": {
+          "type": "string",
+          "format": "email"
+        }
+      }
+    },
+    "billTo": {
+      "type": "string",
+      "format": "email"
+    },
+    "items": {
+      "type": "array",
+      "items": {
+        "type": "object",
+        "required": [
+          "description",
+          "quantity",
+          "unitPrice",
+          "amount",
+          "period"
+        ],
+        "properties": {
+          "description": {
+            "type": "string"
+          },
+          "quantity": {
+            "type": "integer",
+            "minimum": 1
+          },
+          "unitPrice": {
+            "type": "number",
+            "minimum": 0
+          },
+          "amount": {
+            "type": "number",
+            "minimum": 0
+          },
+          "period": {
+            "type": "object",
+            "properties": {
+              "start": {
+                "type": "string",
+                "format": "date"
+              },
+              "end": {
+                "type": "string",
+                "format": "date"
+              }
+            }
+          }
+        }
+      }
+    },
+    "subtotal": {
+      "type": "number",
+      "minimum": 0
+    },
+    "total": {
+      "type": "number",
+      "minimum": 0
+    },
+    "amountPaid": {
+      "type": "number",
+      "minimum": 0
+    }
+  }
+}

tests/extract/data/resume/schema.json

@@ -0,0 +1,181 @@
+{
+  "$schema": "http://json-schema.org/draft-07/schema#",
+  "title": "Resume Schema",
+  "type": "object",
+  "required": ["basics", "skills", "experience"],
+  "properties": {
+    "basics": {
+      "type": "object",
+      "required": ["name", "email"],
+      "properties": {
+        "name": {
+          "type": "string"
+        },
+        "email": {
+          "type": "string",
+          "format": "email"
+        },
+        "phone": {
+          "type": "string"
+        },
+        "location": {
+          "type": "object",
+          "properties": {
+            "city": {
+              "type": "string"
+            },
+            "region": {
+              "type": "string"
+            },
+            "country": {
+              "type": "string"
+            }
+          }
+        },
+        "profiles": {
+          "type": "array",
+          "items": {
+            "type": "object",
+            "properties": {
+              "network": {
+                "type": "string"
+              },
+              "url": {
+                "type": "string",
+                "format": "uri"
+              }
+            }
+          }
+        },
+        "summary": {
+          "type": "string"
+        }
+      }
+    },
+    "skills": {
+      "type": "array",
+      "items": {
+        "type": "object",
+        "properties": {
+          "category": {
+            "type": "string"
+          },
+          "keywords": {
+            "type": "array",
+            "items": {
+              "type": "string"
+            }
+          },
+          "level": {
+            "type": "string",
+            "enum": ["beginner", "intermediate", "advanced", "expert"]
+          }
+        }
+      }
+    },
+    "experience": {
+      "type": "array",
+      "items": {
+        "type": "object",
+        "required": ["company", "position", "startDate"],
+        "properties": {
+          "company": {
+            "type": "string"
+          },
+          "position": {
+            "type": "string"
+          },
+          "startDate": {
+            "type": "string",
+            "format": "date"
+          },
+          "endDate": {
+            "type": "string",
+            "format": "date"
+          },
+          "highlights": {
+            "type": "array",
+            "items": {
+              "type": "string"
+            }
+          },
+          "technologies": {
+            "type": "array",
+            "items": {
+              "type": "string"
+            }
+          }
+        }
+      }
+    },
+    "education": {
+      "type": "array",
+      "items": {
+        "type": "object",
+        "required": ["institution", "degree"],
+        "properties": {
+          "institution": {
+            "type": "string"
+          },
+          "degree": {
+            "type": "string"
+          },
+          "field": {
+            "type": "string"
+          },
+          "graduationDate": {
+            "type": "string",
+            "format": "date"
+          },
+          "gpa": {
+            "type": "number"
+          }
+        }
+      }
+    },
+    "certifications": {
+      "type": "array",
+      "items": {
+        "type": "object",
+        "properties": {
+          "name": {
+            "type": "string"
+          },
+          "issuer": {
+            "type": "string"
+          },
+          "date": {
+            "type": "string",
+            "format": "date"
+          },
+          "validUntil": {
+            "type": "string",
+            "format": "date"
+          }
+        }
+      }
+    },
+    "publications": {
+      "type": "array",
+      "items": {
+        "type": "object",
+        "properties": {
+          "title": {
+            "type": "string"
+          },
+          "publisher": {
+            "type": "string"
+          },
+          "date": {
+            "type": "string",
+            "format": "date"
+          },
+          "url": {
+            "type": "string",
+            "format": "uri"
+          }
+        }
+      }
+    }
+  }
+}

tests/extract/data/resume/software_architect_resume.html

@@ -0,0 +1,298 @@
+<!doctype html>
+<html>
+  <head>
+    <style>
+      body {
+        font-family: "Segoe UI", Roboto, "Helvetica Neue", Arial, sans-serif;
+        margin: 0;
+        padding: 0;
+        background: #fff;
+        color: #333;
+        line-height: 1.6;
+      }
+
+      .container {
+        display: flex;
+        max-width: 1200px;
+        margin: 0 auto;
+        box-shadow: 0 0 20px rgba(0, 0, 0, 0.1);
+        min-height: 100vh;
+      }
+
+      .sidebar {
+        background: #2c3e50;
+        color: white;
+        padding: 2rem;
+        width: 300px;
+      }
+
+      .main-content {
+        padding: 2rem;
+        flex: 1;
+      }
+
+      .profile-name {
+        font-size: 2.5rem;
+        margin: 0;
+        color: #2c3e50;
+        border-bottom: 3px solid #3498db;
+        padding-bottom: 0.5rem;
+      }
+
+      .profile-title {
+        font-size: 1.5rem;
+        color: #7f8c8d;
+        margin: 0.5rem 0 2rem 0;
+      }
+
+      .contact-info {
+        margin-bottom: 2rem;
+      }
+
+      .section-title {
+        font-size: 1.2rem;
+        text-transform: uppercase;
+        color: #3498db;
+        margin-bottom: 1rem;
+        letter-spacing: 1px;
+      }
+
+      .sidebar .section-title {
+        color: white;
+        border-bottom: 2px solid #3498db;
+        padding-bottom: 0.5rem;
+      }
+
+      .skill-category {
+        margin-bottom: 1rem;
+      }
+
+      .skill-list {
+        list-style: none;
+        padding: 0;
+        margin: 0;
+      }
+
+      .skill-list li {
+        margin-bottom: 0.5rem;
+        font-size: 0.9rem;
+      }
+
+      .experience-item {
+        margin-bottom: 2rem;
+      }
+
+      .company-name {
+        font-weight: bold;
+        color: #2c3e50;
+        font-size: 1.1rem;
+      }
+
+      .job-title {
+        color: #3498db;
+        font-weight: bold;
+      }
+
+      .date {
+        color: #7f8c8d;
+        font-size: 0.9rem;
+      }
+
+      .achievements {
+        list-style: disc;
+        padding-left: 1.2rem;
+        margin-top: 0.5rem;
+      }
+
+      .contact-info a {
+        color: white;
+        text-decoration: none;
+      }
+
+      .education-item {
+        margin-bottom: 1rem;
+      }
+    </style>
+  </head>
+
+  <body>
+    <div class="container">
+      <div class="sidebar">
+        <div class="contact-info">
+          <h2 class="section-title">Contact</h2>
+          <p>sarah.chen@email.com</p>
+          <p>(555) 123-4567</p>
+          <p>San Francisco, CA</p>
+          <p><a href="#">LinkedIn Profile</a></p>
+        </div>
+
+        <div class="skills-section">
+          <h2 class="section-title">Technical Skills</h2>
+
+          <div class="skill-category">
+            <h3>Architecture & Design</h3>
+            <ul class="skill-list">
+              <li>Microservices</li>
+              <li>Event-Driven Architecture</li>
+              <li>Domain-Driven Design</li>
+              <li>REST APIs</li>
+            </ul>
+          </div>
+
+          <div class="skill-category">
+            <h3>Cloud Platforms</h3>
+            <ul class="skill-list">
+              <li>AWS (Advanced)</li>
+              <li>Azure</li>
+              <li>Google Cloud Platform</li>
+            </ul>
+          </div>
+
+          <div class="skill-category">
+            <h3>Programming</h3>
+            <ul class="skill-list">
+              <li>Java</li>
+              <li>Python</li>
+              <li>Go</li>
+              <li>JavaScript/TypeScript</li>
+            </ul>
+          </div>
+
+          <div class="skill-category">
+            <h3>Certifications</h3>
+            <ul class="skill-list">
+              <li>AWS Solutions Architect - Professional</li>
+              <li>Google Cloud Architect</li>
+              <li>Certified Kubernetes Administrator</li>
+            </ul>
+          </div>
+        </div>
+      </div>
+
+      <div class="main-content">
+        <h1 class="profile-name">Sarah Chen</h1>
+        <div class="profile-title">Senior Software Architect</div>
+
+        <div class="section">
+          <h2 class="section-title">Professional Summary</h2>
+          <p>
+            Innovative Software Architect with over 12 years of experience
+            designing and implementing large-scale distributed systems. Proven
+            track record of leading technical teams and delivering robust
+            enterprise solutions. Expert in cloud architecture, microservices,
+            and emerging technologies with a focus on scalable, maintainable
+            systems.
+          </p>
+        </div>
+
+        <div class="section">
+          <h2 class="section-title">Professional Experience</h2>
+
+          <div class="experience-item">
+            <div class="company-name">TechCorp Solutions</div>
+            <div class="job-title">Senior Software Architect</div>
+            <div class="date">2020 - Present</div>
+            <ul class="achievements">
+              <li>
+                Led architectural design and implementation of a cloud-native
+                platform serving 2M+ users
+              </li>
+              <li>
+                Established architectural guidelines and best practices adopted
+                across 12 development teams
+              </li>
+              <li>
+                Reduced system latency by 40% through implementation of
+                event-driven architecture
+              </li>
+              <li>
+                Mentored 15+ senior developers in cloud-native development
+                practices
+              </li>
+            </ul>
+          </div>
+
+          <div class="experience-item">
+            <div class="company-name">DataFlow Systems</div>
+            <div class="job-title">Lead Software Engineer</div>
+            <div class="date">2016 - 2020</div>
+            <ul class="achievements">
+              <li>
+                Architected and led development of distributed data processing
+                platform handling 5TB daily
+              </li>
+              <li>
+                Designed microservices architecture reducing deployment time by
+                65%
+              </li>
+              <li>
+                Led migration of legacy monolith to cloud-native architecture
+              </li>
+              <li>
+                Managed team of 8 engineers across 3 international locations
+              </li>
+            </ul>
+          </div>
+
+          <div class="experience-item">
+            <div class="company-name">InnovateTech</div>
+            <div class="job-title">Senior Software Engineer</div>
+            <div class="date">2013 - 2016</div>
+            <ul class="achievements">
+              <li>
+                Developed high-performance trading platform processing 100K
+                transactions per second
+              </li>
+              <li>
+                Implemented real-time analytics engine reducing processing
+                latency by 75%
+              </li>
+              <li>
+                Led adoption of container orchestration reducing deployment
+                costs by 35%
+              </li>
+            </ul>
+          </div>
+        </div>
+
+        <div class="section">
+          <h2 class="section-title">Education</h2>
+
+          <div class="education-item">
+            <div class="company-name">Stanford University</div>
+            <div class="job-title">Master of Science in Computer Science</div>
+            <div class="date">2013</div>
+            <p>Focus: Distributed Systems and Machine Learning</p>
+          </div>
+
+          <div class="education-item">
+            <div class="company-name">University of California, Berkeley</div>
+            <div class="job-title">
+              Bachelor of Science in Computer Engineering
+            </div>
+            <div class="date">2011</div>
+            <p>Magna Cum Laude</p>
+          </div>
+        </div>
+
+        <div class="section">
+          <h2 class="section-title">Patents & Speaking</h2>
+          <ul class="achievements">
+            <li>
+              Co-inventor on three patents for distributed systems architecture
+            </li>
+            <li>
+              Published paper on "Scalable Microservices Architecture" at IEEE
+              Cloud Computing Conference 2022
+            </li>
+            <li>
+              Keynote Speaker, CloudCon 2023: "Future of Cloud-Native
+              Architecture"
+            </li>
+            <li>Regular presenter at local tech meetups and conferences</li>
+          </ul>
+        </div>
+      </div>
+    </div>
+  </body>
+</html>

tests/extract/data/resume/software_architect_resume.test.json

@@ -0,0 +1,94 @@
+{
+  "basics": {
+    "name": "Sarah Chen",
+    "email": "san.francisco@email.com",
+    "phone": "(555) 123-4567",
+    "location": {
+      "city": "San Francisco",
+      "region": "CA",
+      "country": "USA"
+    }
+  },
+  "skills": [
+    {
+      "category": "Architecture & Design",
+      "keywords": [
+        "Microservices",
+        "Event-Driven Architecture",
+        "Domain-Driven Design",
+        "REST APIs"
+      ]
+    },
+    {
+      "category": "Cloud Platforms",
+      "keywords": ["AWS", "Azure", "Google Cloud Platform"]
+    },
+    {
+      "category": "Programming Languages",
+      "keywords": ["Java", "Python", "Go", "JavaScript", "TypeScript"]
+    }
+  ],
+  "experience": [
+    {
+      "company": "TechCorp Solutions",
+      "position": "Senior Software Architect",
+      "startDate": "2020-01-01",
+      "endDate": "2024-01-10"
+    },
+    {
+      "company": "DataFlow Systems",
+      "position": "Lead Software Engineer",
+      "startDate": "2016-01-01",
+      "endDate": "2019-12-31",
+      "technologies": [
+        "Distributed Systems",
+        "Microservices",
+        "Cloud Migration"
+      ]
+    },
+    {
+      "company": "InnovateTech",
+      "position": "Senior Software Engineer",
+      "startDate": "2013-01-01",
+      "endDate": "2015-12-31",
+      "technologies": [
+        "High-performance Computing",
+        "Real-time Analytics",
+        "Container Orchestration"
+      ]
+    }
+  ],
+  "education": [
+    {
+      "institution": "Stanford University",
+      "degree": "Master of Science",
+      "field": "Computer Science",
+      "graduationDate": "2013-01-01",
+      "specialization": "Distributed Systems and Machine Learning"
+    },
+    {
+      "institution": "University of California, Berkeley",
+      "degree": "Bachelor of Science",
+      "field": "Computer Engineering",
+      "graduationDate": "2011-01-01"
+    }
+  ],
+  "certifications": [
+    {
+      "name": "AWS Solutions Architect - Professional"
+    },
+    {
+      "name": "Google Cloud Architect"
+    },
+    {
+      "name": "Certified Kubernetes Administrator"
+    }
+  ],
+  "publications": [
+    {
+      "title": "Scalable Microservices Architecture",
+      "publisher": "IEEE Cloud Computing Conference",
+      "date": "2022-01-01"
+    }
+  ]
+}

tests/extract/data/slide/saas_slide.test.json

@@ -0,0 +1,48 @@
+{
+  "companyInfo": {
+    "name": "CloudFlow Analytics",
+    "fundingStage": "Series A",
+    "foundedYear": null,
+    "industry": null,
+    "location": null
+  },
+  "financialMetrics": {
+    "mrr": {
+      "value": 580000,
+      "currency": "USD",
+      "growthRate": 27
+    },
+    "grossMargin": 88
+  },
+  "growthMetrics": {
+    "customers": {
+      "total": 1247,
+      "growth": 142,
+      "enterprisePercent": null
+    },
+    "nrr": 147
+  },
+  "marketMetrics": {
+    "tam": 50000000000,
+    "sam": null,
+    "marketShare": null,
+    "competitors": null
+  },
+  "differentiators": [
+    {
+      "claim": "Processing Speed",
+      "metric": "5x faster",
+      "comparisonTarget": "competitors"
+    },
+    {
+      "claim": "ML Accuracy",
+      "metric": "99.9%",
+      "comparisonTarget": null
+    },
+    {
+      "claim": "Market Potential",
+      "metric": "80%",
+      "comparisonTarget": "Fortune 500"
+    }
+  ]
+}

tests/extract/data/slide/schema.json

@@ -0,0 +1,122 @@
+{
+  "$schema": "http://json-schema.org/draft-07/schema#",
+  "type": "object",
+  "required": ["companyInfo", "financialMetrics", "growthMetrics"],
+  "properties": {
+    "companyInfo": {
+      "type": "object",
+      "required": ["name", "fundingStage"],
+      "properties": {
+        "name": {
+          "type": "string"
+        },
+        "fundingStage": {
+          "type": "string",
+          "enum": ["Pre-seed", "Seed", "Series A", "Series B", "Series C+"]
+        },
+        "foundedYear": {
+          "anyOf": [
+            {
+              "type": "integer"
+            },
+            {
+              "type": "null"
+            }
+          ]
+        },
+        "industry": {
+          "anyOf": [
+            {
+              "type": "string"
+            },
+            {
+              "type": "null"
+            }
+          ]
+        },
+        "location": {
+          "anyOf": [
+            {
+              "type": "string"
+            },
+            {
+              "type": "null"
+            }
+          ]
+        }
+      }
+    },
+    "financialMetrics": {
+      "type": "object",
+      "required": ["mrr", "growthRate"],
+      "properties": {
+        "mrr": {
+          "type": "object",
+          "description": "Monthly Recurring Revenue",
+          "required": ["value", "currency", "growthRate"],
+          "properties": {
+            "value": {
+              "type": "number"
+            },
+            "currency": {
+              "type": "string"
+            },
+            "growthRate": {
+              "type": "number"
+            }
+          }
+        },
+        "grossMargin": {
+          "type": "number"
+        }
+      }
+    },
+    "growthMetrics": {
+      "type": "object",
+      "required": ["customers", "nrr"],
+      "properties": {
+        "customers": {
+          "type": "object",
+          "required": ["total", "growth"],
+          "properties": {
+            "total": {
+              "type": "integer"
+            },
+            "growth": {
+              "type": "number"
+            }
+          }
+        },
+        "nrr": {
+          "description": "Net Revenue Retention",
+          "type": "number"
+        }
+      }
+    },
+    "differentiators": {
+      "type": "array",
+      "items": {
+        "type": "object",
+        "required": ["claim", "metric"],
+        "properties": {
+          "claim": {
+            "type": "string"
+          },
+          "metric": {
+            "type": "string"
+          },
+          "comparisonTarget": {
+            "anyOf": [
+              {
+                "type": "string"
+              },
+              {
+                "type": "null"
+              }
+            ]
+          }
+        }
+      }
+    }
+  }
+}

tests/extract/test_benchmark.py

@@ -0,0 +1,147 @@
+import os
+import pytest
+
+from llama_cloud_services.extract import LlamaExtract, ExtractionAgent
+from time import perf_counter
+from collections import namedtuple
+import json
+import uuid
+from llama_cloud.types import (
+    ExtractConfig,
+    ExtractMode,
+    LlamaParseParameters,
+    LlamaExtractSettings,
+)
+from tests.extract.util import load_test_dotenv
+
+
+load_test_dotenv()
+
+TEST_DIR = os.path.join(os.path.dirname(os.path.abspath(__file__)), "data")
+# Get configuration from environment
+LLAMA_CLOUD_API_KEY = os.getenv("LLAMA_CLOUD_API_KEY")
+LLAMA_CLOUD_BASE_URL = os.getenv("LLAMA_CLOUD_BASE_URL")
+LLAMA_CLOUD_PROJECT_ID = os.getenv("LLAMA_CLOUD_PROJECT_ID")
+
+TestCase = namedtuple(
+    "TestCase", ["name", "schema_path", "config", "input_file", "expected_output"]
+)
+
+
+def get_test_cases():
+    """Get all test cases from TEST_DIR.
+
+    Returns:
+        List[TestCase]: List of test cases
+    """
+    test_cases = []
+
+    for data_type in os.listdir(TEST_DIR):
+        data_type_dir = os.path.join(TEST_DIR, data_type)
+        if not os.path.isdir(data_type_dir):
+            continue
+
+        schema_path = os.path.join(data_type_dir, "schema.json")
+        if not os.path.exists(schema_path):
+            continue
+
+        input_files = []
+
+        for file in os.listdir(data_type_dir):
+            file_path = os.path.join(data_type_dir, file)
+            if (
+                not os.path.isfile(file_path)
+                or file == "schema.json"
+                or file.endswith(".test.json")
+            ):
+                continue
+
+            input_files.append(file_path)
+
+        settings = [
+            ExtractConfig(extraction_mode=ExtractMode.FAST),
+            ExtractConfig(extraction_mode=ExtractMode.ACCURATE),
+        ]
+
+        for input_file in sorted(input_files):
+            base_name = os.path.splitext(os.path.basename(input_file))[0]
+            expected_output = os.path.join(data_type_dir, f"{base_name}.test.json")
+
+            if not os.path.exists(expected_output):
+                continue
+
+            test_name = f"{data_type}/{os.path.basename(input_file)}"
+            for setting in settings:
+                test_cases.append(
+                    TestCase(
+                        name=test_name,
+                        schema_path=schema_path,
+                        input_file=input_file,
+                        config=setting,
+                        expected_output=expected_output,
+                    )
+                )
+
+    return test_cases
+
+
+@pytest.fixture(scope="session")
+def extractor():
+    """Create a single LlamaExtract instance for all tests."""
+    extract = LlamaExtract(
+        api_key=LLAMA_CLOUD_API_KEY,
+        base_url=LLAMA_CLOUD_BASE_URL,
+        project_id=LLAMA_CLOUD_PROJECT_ID,
+        verbose=True,
+    )
+    yield extract
+    # Cleanup thread pool at end of session
+    extract._thread_pool.shutdown()
+
+
+@pytest.fixture
+def extraction_agent(test_case: TestCase, extractor: LlamaExtract):
+    """Fixture to create and cleanup extraction agent for each test."""
+    # Create unique name with random UUID (important for CI to avoid conflicts)
+    unique_id = uuid.uuid4().hex[:8]
+    agent_name = f"{test_case.name}_{unique_id}"
+
+    with open(test_case.schema_path, "r") as f:
+        schema = json.load(f)
+
+    # Clean up any existing agents with this name
+    try:
+        agents = extractor.list_agents()
+        for agent in agents:
+            if agent.name == agent_name:
+                extractor.delete_agent(agent.id)
+    except Exception as e:
+        print(f"Warning: Failed to cleanup existing agent: {str(e)}")
+
+    # Create new agent
+    agent = extractor.create_agent(agent_name, schema, config=test_case.config)
+    yield agent
+
+
+@pytest.mark.skipif(
+    "CI" in os.environ,
+    reason="CI environment is not suitable for benchmarking",
+)
+@pytest.mark.parametrize("test_case", get_test_cases(), ids=lambda x: x.name)
+@pytest.mark.asyncio(loop_scope="session")
+async def test_extraction(
+    test_case: TestCase, extraction_agent: ExtractionAgent
+) -> None:
+    start = perf_counter()
+    result = await extraction_agent._queue_extraction_test(
+        test_case.input_file,
+        extract_settings=LlamaExtractSettings(
+            llama_parse_params=LlamaParseParameters(
+                invalidate_cache=True,
+                do_not_cache=True,
+            )
+        ),
+    )
+    end = perf_counter()
+    print(f"Time taken: {end - start} seconds")
+    print(result)

tests/extract/test_extract_api.py

@@ -0,0 +1,188 @@
+import os
+import pytest
+from pathlib import Path
+from pydantic import BaseModel
+
+from llama_cloud_services.extract import LlamaExtract, ExtractionAgent
+from tests.extract.util import load_test_dotenv
+
+load_test_dotenv()
+
+# Get configuration from environment
+LLAMA_CLOUD_API_KEY = os.getenv("LLAMA_CLOUD_API_KEY")
+LLAMA_CLOUD_BASE_URL = os.getenv("LLAMA_CLOUD_BASE_URL")
+LLAMA_CLOUD_PROJECT_ID = os.getenv("LLAMA_CLOUD_PROJECT_ID")
+
+# Skip all tests if API key is not set
+pytestmark = pytest.mark.skipif(
+    not LLAMA_CLOUD_API_KEY, reason="LLAMA_CLOUD_API_KEY not set"
+)
+
+
+# Test data
+class TestSchema(BaseModel):
+    title: str
+    summary: str
+
+
+# Test data paths
+TEST_DIR = Path(__file__).parent / "data"
+TEST_PDF = TEST_DIR / "slide" / "saas_slide.pdf"
+
+
+@pytest.fixture
+def llama_extract():
+    return LlamaExtract(
+        api_key=LLAMA_CLOUD_API_KEY,
+        base_url=LLAMA_CLOUD_BASE_URL,
+        project_id=LLAMA_CLOUD_PROJECT_ID,
+        verbose=True,
+    )
+
+
+@pytest.fixture
+def test_agent_name():
+    return "test-api-agent"
+
+
+@pytest.fixture
+def test_schema_dict():
+    return {
+        "type": "object",
+        "properties": {
+            "title": {"type": "string"},
+            "summary": {"type": "string"},
+        },
+    }
+
+
+@pytest.fixture
+def test_agent(llama_extract, test_agent_name, test_schema_dict, request):
+    """Creates a test agent and cleans it up after the test"""
+    test_id = request.node.nodeid
+    test_hash = hex(hash(test_id))[-8:]
+    base_name = test_agent_name
+
+    base_name = next(
+        (marker.args[0] for marker in request.node.iter_markers("agent_name")),
+        base_name,
+    )
+    name = f"{base_name}_{test_hash}"
+
+    schema = next(
+        (
+            marker.args[0][0] if isinstance(marker.args[0], tuple) else marker.args[0]
+            for marker in request.node.iter_markers("agent_schema")
+        ),
+        test_schema_dict,
+    )
+
+    # Cleanup existing agent
+    try:
+        for agent in llama_extract.list_agents():
+            if agent.name == name:
+                llama_extract.delete_agent(agent.id)
+    except Exception as e:
+        print(f"Warning: Failed to cleanup existing agent: {e}")
+
+    agent = llama_extract.create_agent(name=name, data_schema=schema)
+    yield agent
+
+    # Cleanup after test
+    try:
+        llama_extract.delete_agent(agent.id)
+    except Exception as e:
+        print(f"Warning: Failed to delete agent {agent.id}: {e}")
+
+
+class TestLlamaExtract:
+    def test_init_without_api_key(self):
+        env_backup = os.getenv("LLAMA_CLOUD_API_KEY")
+        del os.environ["LLAMA_CLOUD_API_KEY"]
+        with pytest.raises(ValueError, match="The API key is required"):
+            LlamaExtract(api_key=None, base_url=LLAMA_CLOUD_BASE_URL)
+        os.environ["LLAMA_CLOUD_API_KEY"] = env_backup
+
+    @pytest.mark.agent_name("test-dict-schema-agent")
+    def test_create_agent_with_dict_schema(self, test_agent):
+        assert isinstance(test_agent, ExtractionAgent)
+
+    @pytest.mark.agent_name("test-pydantic-schema-agent")
+    @pytest.mark.agent_schema((TestSchema,))
+    def test_create_agent_with_pydantic_schema(self, test_agent):
+        assert isinstance(test_agent, ExtractionAgent)
+
+    def test_get_agent_by_name(self, llama_extract, test_agent):
+        agent = llama_extract.get_agent(name=test_agent.name)
+        assert isinstance(agent, ExtractionAgent)
+        assert agent.name == test_agent.name
+        assert agent.id == test_agent.id
+        assert agent.data_schema == test_agent.data_schema
+
+    def test_get_agent_by_id(self, llama_extract, test_agent):
+        agent = llama_extract.get_agent(id=test_agent.id)
+        assert isinstance(agent, ExtractionAgent)
+        assert agent.id == test_agent.id
+        assert agent.name == test_agent.name
+        assert agent.data_schema == test_agent.data_schema
+
+    def test_list_agents(self, llama_extract, test_agent):
+        agents = llama_extract.list_agents()
+        assert isinstance(agents, list)
+        assert any(a.id == test_agent.id for a in agents)
+
+
+class TestExtractionAgent:
+    @pytest.mark.asyncio
+    async def test_extract_single_file(self, test_agent):
+        result = await test_agent.aextract(TEST_PDF)
+        assert result.status == "SUCCESS"
+        assert result.data is not None
+        assert isinstance(result.data, dict)
+        assert "title" in result.data
+        assert "summary" in result.data
+
+    def test_sync_extract_single_file(self, test_agent):
+        result = test_agent.extract(TEST_PDF)
+        assert result.status == "SUCCESS"
+        assert result.data is not None
+        assert isinstance(result.data, dict)
+        assert "title" in result.data
+        assert "summary" in result.data
+
+    @pytest.mark.asyncio
+    async def test_extract_multiple_files(self, test_agent):
+        files = [TEST_PDF, TEST_PDF]  # Using same file twice for testing
+        response = await test_agent.aextract(files)
+
+        assert len(response) == 2
+        for result in response:
+            assert result.status == "SUCCESS"
+            assert result.data is not None
+            assert isinstance(result.data, dict)
+            assert "title" in result.data
+            assert "summary" in result.data
+
+    def test_save_agent_updates(
+        self, test_agent: ExtractionAgent, llama_extract: LlamaExtract
+    ):
+        new_schema = {
+            "type": "object",
+            "properties": {
+                "new_field": {"type": "string"},
+                "title": {"type": "string"},
+                "summary": {"type": "string"},
+            },
+        }
+        test_agent.data_schema = new_schema
+        test_agent.save()
+
+        # Verify the update by getting a fresh instance
+        updated_agent = llama_extract.get_agent(name=test_agent.name)
+        assert "new_field" in updated_agent.data_schema["properties"]
+
+    def test_list_extraction_runs(self, test_agent: ExtractionAgent):
+        assert len(test_agent.list_extraction_runs()) == 0
+        test_agent.extract(TEST_PDF)
+        runs = test_agent.list_extraction_runs()
+        assert len(runs) > 0

tests/extract/test_extract_e2e.py

@@ -0,0 +1,139 @@
+import os
+import pytest
+
+from llama_cloud_services.extract import LlamaExtract, ExtractionAgent
+from collections import namedtuple
+import json
+import uuid
+from llama_cloud.types import ExtractConfig, ExtractMode
+from deepdiff import DeepDiff
+from tests.extract.util import json_subset_match_score, load_test_dotenv
+
+load_test_dotenv()
+
+
+TEST_DIR = os.path.join(os.path.dirname(os.path.abspath(__file__)), "data")
+# Get configuration from environment
+LLAMA_CLOUD_API_KEY = os.getenv("LLAMA_CLOUD_API_KEY")
+LLAMA_CLOUD_BASE_URL = os.getenv("LLAMA_CLOUD_BASE_URL")
+LLAMA_CLOUD_PROJECT_ID = os.getenv("LLAMA_CLOUD_PROJECT_ID")
+
+TestCase = namedtuple(
+    "TestCase", ["name", "schema_path", "config", "input_file", "expected_output"]
+)
+
+
+def get_test_cases():
+    """Get all test cases from TEST_DIR.
+
+    Returns:
+        List[TestCase]: List of test cases
+    """
+    test_cases = []
+
+    for data_type in os.listdir(TEST_DIR):
+        data_type_dir = os.path.join(TEST_DIR, data_type)
+        if not os.path.isdir(data_type_dir):
+            continue
+
+        schema_path = os.path.join(data_type_dir, "schema.json")
+        if not os.path.exists(schema_path):
+            continue
+
+        input_files = []
+
+        for file in os.listdir(data_type_dir):
+            file_path = os.path.join(data_type_dir, file)
+            if (
+                not os.path.isfile(file_path)
+                or file == "schema.json"
+                or file.endswith(".test.json")
+            ):
+                continue
+
+            input_files.append(file_path)
+
+        settings = [
+            ExtractConfig(extraction_mode=ExtractMode.FAST),
+            ExtractConfig(extraction_mode=ExtractMode.ACCURATE),
+        ]
+
+        for input_file in sorted(input_files):
+            base_name = os.path.splitext(os.path.basename(input_file))[0]
+            expected_output = os.path.join(data_type_dir, f"{base_name}.test.json")
+
+            if not os.path.exists(expected_output):
+                continue
+
+            test_name = f"{data_type}/{os.path.basename(input_file)}"
+            for setting in settings:
+                test_cases.append(
+                    TestCase(
+                        name=test_name,
+                        schema_path=schema_path,
+                        input_file=input_file,
+                        config=setting,
+                        expected_output=expected_output,
+                    )
+                )
+
+    return test_cases
+
+
+@pytest.fixture(scope="session")
+def extractor():
+    """Create a single LlamaExtract instance for all tests."""
+    extract = LlamaExtract(
+        api_key=LLAMA_CLOUD_API_KEY,
+        base_url=LLAMA_CLOUD_BASE_URL,
+        project_id=LLAMA_CLOUD_PROJECT_ID,
+        verbose=True,
+    )
+    yield extract
+    # Cleanup thread pool at end of session
+    extract._thread_pool.shutdown()
+
+
+@pytest.fixture
+def extraction_agent(test_case: TestCase, extractor: LlamaExtract):
+    """Fixture to create and cleanup extraction agent for each test."""
+    # Create unique name with random UUID (important for CI to avoid conflicts)
+    unique_id = uuid.uuid4().hex[:8]
+    agent_name = f"{test_case.name}_{unique_id}"
+
+    with open(test_case.schema_path, "r") as f:
+        schema = json.load(f)
+
+    # Clean up any existing agents with this name
+    try:
+        agents = extractor.list_agents()
+        for agent in agents:
+            if agent.name == agent_name:
+                extractor.delete_agent(agent.id)
+    except Exception as e:
+        print(f"Warning: Failed to cleanup existing agent: {str(e)}")
+
+    # Create new agent
+    agent = extractor.create_agent(agent_name, schema, config=test_case.config)
+    yield agent
+
+    # Cleanup after test
+    try:
+        extractor.delete_agent(agent.id)
+    except Exception as e:
+        print(f"Warning: Failed to delete agent {agent.id}: {str(e)}")
+
+
+@pytest.mark.skipif(
+    os.environ.get("LLAMA_CLOUD_API_KEY", "") == "",
+    reason="LLAMA_CLOUD_API_KEY not set",
+)
+@pytest.mark.parametrize("test_case", get_test_cases(), ids=lambda x: x.name)
+def test_extraction(test_case: TestCase, extraction_agent: ExtractionAgent) -> None:
+    result = extraction_agent.extract(test_case.input_file).data
+    with open(test_case.expected_output, "r") as f:
+        expected = json.load(f)
+    # TODO: fix the saas_slide test
+    assert json_subset_match_score(expected, result) > 0.3, DeepDiff(
+        expected, result, ignore_order=True
+    )

tests/extract/util.py

@@ -0,0 +1,43 @@
+from typing import Any
+
+from autoevals.string import Levenshtein
+from autoevals.number import NumericDiff
+from dotenv import load_dotenv
+from pathlib import Path
+
+
+def load_test_dotenv():
+    load_dotenv(Path(__file__).parent.parent.parent / ".env.dev", override=True)
+
+
+def json_subset_match_score(expected: Any, actual: Any) -> float:
+    """
+    Adapted from autoevals.JsonDiff to only test on the subset of keys within the expected json.
+    """
+    string_scorer = Levenshtein()
+    number_scorer = NumericDiff()
+    if isinstance(expected, dict) and isinstance(actual, dict):
+        if len(expected) == 0 and len(actual) == 0:
+            return 1
+        keys = set(expected.keys())
+        scores = [json_subset_match_score(expected.get(k), actual.get(k)) for k in keys]
+        scores = [s for s in scores if s is not None]
+        return sum(scores) / len(scores)
+    elif isinstance(expected, list) and isinstance(actual, list):
+        if len(expected) == 0 and len(actual) == 0:
+            return 1
+        scores = [json_subset_match_score(e1, e2) for (e1, e2) in zip(expected, actual)]
+        scores = [s for s in scores if s is not None]
+        return sum(scores) / max(len(expected), len(actual))
+    elif isinstance(expected, str) and isinstance(actual, str):
+        return string_scorer.eval(expected, actual).score
+    elif (isinstance(expected, int) or isinstance(expected, float)) and (
+        isinstance(actual, int) or isinstance(actual, float)
+    ):
+        return number_scorer.eval(expected, actual).score
+    elif expected is None and actual is None:
+        return 1
+    elif expected is None or actual is None:
+        return 0
+    else:
+        return 0