run-llama/llama_cloud_services
1
0
Fork 0
mirror of https://github.com/run-llama/llama_cloud_services.git synced 2026-08-27 09:21:26 -04:00
Code Issues 352 Packages Projects Releases 10 Wiki Activity

Unexpected LlamaParse Behavior in v0.6.1 – Extracting Raw OCR Instead of Analyzing Page Content #433

New Issue
Open
opened 2026-02-16 00:17:49 -05:00 by yindo · 11 comments
yindo commented 2026-02-16 00:17:49 -05:00
Owner
Copy Link
Copy Source

Originally created by @hitoshyamamoto on GitHub (Feb 15, 2025).

Describe the bug
I frequently use the LlamaParse method to obtain responses based on the content of the page image I am working on. However, after upgrading to v0.6.1, which introduced new parameters (content_guideline_instruction, formatting_instruction, complemental_formatting_instruction), I noticed an issue.
The responses are no longer as expected. Instead of analyzing the page and generating an appropriate response, LlamaParse is behaving like a simple OCR, extracting raw text from the image without considering the intended formatting or content structure.
LlamaParse should analyze the page image and generate a structured response based on the provided instructions, rather than just extracting plain text.

Job ID
7f9033bf-723b-475b-bcb8-7100074eebc2

Client:
Please remove untested options:

  • Python Library
  • API
  • Frontend (cloud.llamaindex.ai)

Additional context


import os
import asyncio
from dotenv import load_dotenv
from llama_parse import LlamaParse
from llama_parse.utils import ResultType

load_dotenv()

class LlamaParserError(Exception):
    def __init__(self, message: str, error_code: str = "LLAMA_ERROR"):
        self.message = message
        self.error_code = error_code
        super().__init__(self.message)

    def __str__(self):
        return f"[{self.error_code}] {self.message}"

async def process_image():
    parser = LlamaParse(
        api_key=[API_KEY],
        result_type=ResultType.MD,
        use_vendor_multimodal_model=True,
        vendor_multimodal_model_name="gemini-2.0-flash-001",
        language="pt",
        do_not_cache=True,
        is_formatting_instruction=True,
        content_guideline_instruction=[TEXT],
        formatting_instruction=[TEXT],
        complemental_formatting_instruction=[TEXT]
    )

    file_path = [FILE_PATH]

    try:
        documents = await parser.aload_data(file_path)

        output_file_path = "llama_result.txt"
        with open(output_file_path, "w", encoding="utf-8") as output_file:
            output_file.write("LlamaCloud Response: \n")
            output_file.write("=" * 50 + "\n\n")
            output_file.write("\n".join([doc.text.strip() for doc in documents]))
            output_file.write("\n" + "=" * 50 + "\n")

        print(f"saved: {output_file_path}")

    except Exception as e:
        raise LlamaParserError(f"Error: {str(e)}", "PROCESSING_ERROR")

asyncio.run(process_image())

Image

Originally created by @hitoshyamamoto on GitHub (Feb 15, 2025). **Describe the bug** I frequently use the LlamaParse method to obtain responses based on the content of the page image I am working on. However, after upgrading to v0.6.1, which introduced new parameters (content_guideline_instruction, formatting_instruction, complemental_formatting_instruction), I noticed an issue. The responses are no longer as expected. Instead of analyzing the page and generating an appropriate response, LlamaParse is behaving like a simple OCR, extracting raw text from the image without considering the intended formatting or content structure. LlamaParse should analyze the page image and generate a structured response based on the provided instructions, rather than just extracting plain text. **Job ID** 7f9033bf-723b-475b-bcb8-7100074eebc2 **Client:** Please remove untested options: - Python Library - API - Frontend (cloud.llamaindex.ai) **Additional context** ```python import os import asyncio from dotenv import load_dotenv from llama_parse import LlamaParse from llama_parse.utils import ResultType load_dotenv() class LlamaParserError(Exception): def __init__(self, message: str, error_code: str = "LLAMA_ERROR"): self.message = message self.error_code = error_code super().__init__(self.message) def __str__(self): return f"[{self.error_code}] {self.message}" async def process_image(): parser = LlamaParse( api_key=[API_KEY], result_type=ResultType.MD, use_vendor_multimodal_model=True, vendor_multimodal_model_name="gemini-2.0-flash-001", language="pt", do_not_cache=True, is_formatting_instruction=True, content_guideline_instruction=[TEXT], formatting_instruction=[TEXT], complemental_formatting_instruction=[TEXT] ) file_path = [FILE_PATH] try: documents = await parser.aload_data(file_path) output_file_path = "llama_result.txt" with open(output_file_path, "w", encoding="utf-8") as output_file: output_file.write("LlamaCloud Response: \n") output_file.write("=" * 50 + "\n\n") output_file.write("\n".join([doc.text.strip() for doc in documents])) output_file.write("\n" + "=" * 50 + "\n") print(f"saved: {output_file_path}") except Exception as e: raise LlamaParserError(f"Error: {str(e)}", "PROCESSING_ERROR") asyncio.run(process_image()) ``` ![Image](https://github.com/user-attachments/assets/5ab21181-7bd6-44ba-a9db-13c6dc0eaace)
yindo added the bug label 2026-02-16 00:17:49 -05:00
yindo commented 2026-02-16 00:17:50 -05:00
Author
Owner
Copy Link
Copy Source

@logan-markewich commented on GitHub (Feb 15, 2025):

Hmmm. The old params should still work, so you can still set parsing_instruction and is_formatting_instruction

I do see the same issue when trying the new params myself though

@logan-markewich commented on GitHub (Feb 15, 2025): Hmmm. The old params should still work, so you can still set `parsing_instruction` and `is_formatting_instruction` I do see the same issue when trying the new params myself though
yindo commented 2026-02-16 00:17:50 -05:00
Author
Owner
Copy Link
Copy Source

@hitoshyamamoto commented on GitHub (Feb 16, 2025):

Thank you for your response! I appreciate your time and effort in investigating this issue.

The issue still persists. I just tested it again, and the bug is still present in versions 0.6.0 and 0.6.1.

Job ID:
❌626a66c9-2f07-4ac8-a0bb-7ce7fd6a285f - Testing parameters complemental_formatting_instruction, content_guideline_instruction, and formatting_instruction
✅0035578b-6f4c-43d5-a5b5-0ff36736c112 - Only using the parameter parsing_instruction

For now, I will continue using the parsing_instruction parameter as a workaround. However, I am still looking forward to an update regarding this issue, as I would prefer to use complemental_formatting_instruction, content_guideline_instruction, and formatting_instruction.

If these parameters start working correctly again, would it be advisable to lock my project to version 0.6.0 or 0.6.1 to continue using them (complemental_formatting_instruction, content_guideline_instruction, and formatting_instruction)? Or would migrating to the newly introduced parameters be the better approach?

However, I also noticed that the documentation and GUI now mention additional parameters (user_prompt, system_prompt_append, and system_prompt).

Would it be advisable to migrate my code to these newer parameters instead? Or will the previously introduced parameters be fully supported again in future versions?

Thanks again for your assistance! Looking forward to your insights.

@hitoshyamamoto commented on GitHub (Feb 16, 2025): Thank you for your response! I appreciate your time and effort in investigating this issue. The issue still persists. I just tested it again, and the bug is still present in versions 0.6.0 and 0.6.1. **Job ID:** ❌626a66c9-2f07-4ac8-a0bb-7ce7fd6a285f - Testing parameters **complemental_formatting_instruction**, **content_guideline_instruction**, and **formatting_instruction** ✅0035578b-6f4c-43d5-a5b5-0ff36736c112 - Only using the parameter **parsing_instruction** For now, I will continue using the parsing_instruction parameter as a workaround. However, I am still looking forward to an update regarding this issue, as I would prefer to use complemental_formatting_instruction, content_guideline_instruction, and formatting_instruction. If these parameters start working correctly again, would it be advisable to lock my project to version 0.6.0 or 0.6.1 to continue using them (complemental_formatting_instruction, content_guideline_instruction, and formatting_instruction)? Or would migrating to the newly introduced parameters be the better approach? However, I also noticed that the [documentation](https://docs.cloud.llamaindex.ai/llamaparse/features/prompts) and GUI now mention additional parameters (user_prompt, system_prompt_append, and system_prompt). Would it be advisable to migrate my code to these newer parameters instead? Or will the previously introduced parameters be fully supported again in future versions? Thanks again for your assistance! Looking forward to your insights.
yindo commented 2026-02-16 00:17:50 -05:00
Author
Owner
Copy Link
Copy Source

@judithnat commented on GitHub (Feb 18, 2025):

Tested this again today in the sandbox, and although the prompt is visible in the job details, the output still seems like raw OCR and has not followed the instruction. As you say parseing_instruction still working via the API at the moment

@judithnat commented on GitHub (Feb 18, 2025): Tested this again today in the sandbox, and although the prompt is visible in the job details, the output still seems like raw OCR and has not followed the instruction. As you say parseing_instruction still working via the API at the moment
yindo commented 2026-02-16 00:17:50 -05:00
Author
Owner
Copy Link
Copy Source

@mehwishh247 commented on GitHub (Feb 19, 2025):

It's day 4 of this issue and even though parsing instruction may work with http API call, it isn't working with the python library. I tried the GUI today, and instructions are not working there either. If anyone has any suggestions for an alternative API, do suggest as my code for a client is breaking in production

@mehwishh247 commented on GitHub (Feb 19, 2025): It's day 4 of this issue and even though parsing instruction may work with http API call, it isn't working with the python library. I tried the GUI today, and instructions are not working there either. If anyone has any suggestions for an alternative API, do suggest as my code for a client is breaking in production
yindo commented 2026-02-16 00:17:50 -05:00
Author
Owner
Copy Link
Copy Source

@salahuddinfa commented on GitHub (Feb 20, 2025):

Facing the same issues here, any workaround for this, would help unblock me in my task

@salahuddinfa commented on GitHub (Feb 20, 2025): Facing the same issues here, any workaround for this, would help unblock me in my task
yindo commented 2026-02-16 00:17:50 -05:00
Author
Owner
Copy Link
Copy Source

@hitoshyamamoto commented on GitHub (Feb 20, 2025):

I've checked the current status of LlamaCloud on llamaindex.statuspage.io, and there is an ongoing issue reported as "Degraded performance on LlamaCloud". The latest update from the team indicates that they are still investigating the issue.

Image

Currently, LlamaCloud Public API and LlamaParse are experiencing a Major Outage, which has been ongoing for about 55 minutes. Before this full outage, LlamaCloud Public API was already in Partial Outage for approximately 2 hours and 40 minutes.

Given this, it seems best to wait for the service to stabilize before testing further. Once LlamaCloud is fully operational again, it will be important to verify whether the issues with LlamaParse's structured output persist.

The discussions and shared test results have been useful in understanding how this issue is impacting different use cases. The ongoing updates help provide clarity on the situation.

I appreciate the team's efforts in investigating this, and I’ll keep an eye on the status updates.

If anyone has observed differences in service behavior across different regions, it could be helpful to share those findings.

Looking forward to updates.

@hitoshyamamoto commented on GitHub (Feb 20, 2025): I've checked the current status of LlamaCloud on [llamaindex.statuspage.io](https://llamaindex.statuspage.io/), and there is an ongoing issue reported as "**Degraded performance on LlamaCloud**". The latest update from the team indicates that they are still investigating the issue. ![Image](https://github.com/user-attachments/assets/77fb8948-347a-466b-86f1-449ace4a3973) Currently, LlamaCloud Public API and LlamaParse are experiencing a Major Outage, which has been ongoing for about 55 minutes. Before this full outage, LlamaCloud Public API was already in Partial Outage for approximately 2 hours and 40 minutes. Given this, it seems best to wait for the service to stabilize before testing further. Once LlamaCloud is fully operational again, it will be important to verify whether the issues with LlamaParse's structured output persist. The discussions and shared test results have been useful in understanding how this issue is impacting different use cases. The ongoing updates help provide clarity on the situation. I appreciate the team's efforts in investigating this, and I’ll keep an eye on the status updates. If anyone has observed differences in service behavior across different regions, it could be helpful to share those findings. Looking forward to updates.
yindo commented 2026-02-16 00:17:50 -05:00
Author
Owner
Copy Link
Copy Source

@mehwishh247 commented on GitHub (Feb 20, 2025):

I have tried using the premium mode just in case and it is working better obviously. Even the parsing time isn't delayed on this mode. I also saw a new PR related to user_prompt and related fields, and I think these changes and merges definitely have something to do with the outage in the first place. I am really hopeful that the problem may resolve soon now that it is getting attention from the providers.

@mehwishh247 commented on GitHub (Feb 20, 2025): I have tried using the premium mode just in case and it is working better obviously. Even the parsing time isn't delayed on this mode. I also saw a new PR related to **user_prompt** and related fields, and I think these changes and merges definitely have something to do with the outage in the first place. I am really hopeful that the problem may resolve soon now that it is getting attention from the providers.
yindo commented 2026-02-16 00:17:50 -05:00
Author
Owner
Copy Link
Copy Source

@hitoshyamamoto commented on GitHub (Feb 20, 2025):

It seems that approximately 1.5 hours ago, the team provided an update stating that they applied a patch to restore some functionality to their services. However, there are still noticeable performance issues affecting the system.

Image

From my recent testing, the GUI interface appears to be functioning properly on my end. However, when using the API, I encountered two specific issues:

  1. Significant Processing Delays: A document page that would typically be processed within a few seconds took more than 30 minutes to complete. This suggests a major degradation in API response time.

  2. Unexpected Response Messages: In multiple instances, the API returned:

  • "NO CONTENT HERE"
  • "NOT PROCESSED" (This is a fallback response from my own code when the API fails to process the request properly.)

From my observations, "NO CONTENT HERE" seems to occur specifically when calling third-party LLMs through the API. It looks like the API is unable to interact with the selected LLM properly, leading to this response.

I was just running some troubleshooting tests to check the current state of the system and wanted to report my findings based on the latest conditions. My team is currently waiting for a resolution, but in the meantime, I’ve had to resort to alternative solutions to keep production running.

We’re hopeful that the service will return to its expected stability as soon as possible, not just for me and my team, but also for the other users who have reported similar issues. Looking forward to seeing everything back up and running smoothly soon.

@hitoshyamamoto commented on GitHub (Feb 20, 2025): It seems that approximately 1.5 hours ago, the team provided an update stating that they applied a patch to restore some functionality to their services. However, there are still noticeable performance issues affecting the system. ![Image](https://github.com/user-attachments/assets/a8a34e02-2cca-48d0-aacc-735111964e57) From my recent testing, the GUI interface appears to be functioning properly on my end. However, when using the API, I encountered two specific issues: 1) Significant Processing Delays: A document page that would typically be processed within a few seconds took more than 30 minutes to complete. This suggests a major degradation in API response time. 2) Unexpected Response Messages: In multiple instances, the API returned: - "NO CONTENT HERE" - "NOT PROCESSED" (This is a fallback response from my own code when the API fails to process the request properly.) From my observations, "NO CONTENT HERE" seems to occur specifically when calling third-party LLMs through the API. It looks like the API is unable to interact with the selected LLM properly, leading to this response. I was just running some troubleshooting tests to check the current state of the system and wanted to report my findings based on the latest conditions. My team is currently waiting for a resolution, but in the meantime, I’ve had to resort to alternative solutions to keep production running. We’re hopeful that the service will return to its expected stability as soon as possible, not just for me and my team, but also for the other users who have reported similar issues. Looking forward to seeing everything back up and running smoothly soon.
yindo commented 2026-02-16 00:17:50 -05:00
Author
Owner
Copy Link
Copy Source

@hitoshyamamoto commented on GitHub (Feb 25, 2025):

@mehwishh247 . The PR#622, which introduces the new parameters (user_prompt, system_prompt, and system_prompt_append), is still open and has not yet been merged. This means these parameters are not available for use in the API at the moment.

Image

We'll need to wait for further updates. I also commented on PR#622 asking if there's any update or an estimated timeline for when these parameters will be available.

Let's stay tuned for any progress.

@hitoshyamamoto commented on GitHub (Feb 25, 2025): @mehwishh247 . The [PR#622](https://github.com/run-llama/llama_cloud_services/pull/622), which introduces the new parameters (user_prompt, system_prompt, and system_prompt_append), is still open and has not yet been merged. This means these parameters are not available for use in the API at the moment. ![Image](https://github.com/user-attachments/assets/9cbe7bee-96c0-4ef1-8a9e-359b1f6268da) We'll need to wait for further updates. I also commented on PR#622 asking if there's any update or an estimated timeline for when these parameters will be available. Let's stay tuned for any progress.
yindo commented 2026-02-16 00:17:50 -05:00
Author
Owner
Copy Link
Copy Source

@hitoshyamamoto commented on GitHub (Feb 26, 2025):

Hi everyone,

Following my last comment, PR#622 has been merged yesterday, making the new parameters (user_prompt, system_prompt, and system_prompt_append) available for the API.

Image

I’d like to thank the maintainer responsible for the merge—this update is greatly appreciated, as many users were eagerly waiting for these parameters.

Today, I have updated all my projects codes to version 0.6.2, ensuring full compatibility with the new changes. I’m now waiting for my team to validate that everything is working as expected with LlamaParse v0.6.2.

Once my team confirms that the functionality is stable, I will proceed with closing this issue.

Thanks again for the support.

@hitoshyamamoto commented on GitHub (Feb 26, 2025): Hi everyone, Following my last comment, [PR#622](https://github.com/run-llama/llama_cloud_services/pull/622) has been merged yesterday, **making the new parameters (user_prompt, system_prompt, and system_prompt_append) available for the API**. ![Image](https://github.com/user-attachments/assets/3705c5e9-195a-47fc-9c83-0127bc921e8e) I’d like to thank the maintainer responsible for the merge—this update is greatly appreciated, as many users were eagerly waiting for these parameters. Today, I have updated all my projects codes to version 0.6.2, ensuring full compatibility with the new changes. I’m now waiting for my team to validate that everything is working as expected with LlamaParse v0.6.2. Once my team confirms that the functionality is stable, I will proceed with closing this issue. Thanks again for the support.
yindo commented 2026-02-16 00:17:50 -05:00
Author
Owner
Copy Link
Copy Source

@BinaryBrain commented on GitHub (Mar 21, 2025):

Hey @hitoshyamamoto,
Thanks for your kind message!
Can we close this issue?

@BinaryBrain commented on GitHub (Mar 21, 2025): Hey @hitoshyamamoto, Thanks for your kind message! Can we close this issue?
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
LlamaCloud
LlamaExtract
LlamaParse
bug
client
complex layout
documentation
duplicate
enhancement
figure
font
header footer
help wanted
ocr
parser
pdf_debug
pdf_debug
pull-request
Mirrored from GitHub Pull Request
 python-lib
question
server_issue
server_issue
ui
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: run-llama/llama_cloud_services#433
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?