run-llama/llama_cloud_services
1
0
Fork 0
mirror of https://github.com/run-llama/llama_cloud_services.git synced 2026-08-27 01:15:06 -04:00
Code Issues 352 Packages Projects Releases 10 Wiki Activity

Integration of llama_parse in nodejs #58

New Issue
Closed
opened 2026-02-16 00:16:45 -05:00 by yindo · 1 comment
yindo commented 2026-02-16 00:16:45 -05:00
Owner
Copy Link
Copy Source

Originally created by @MuhammadIshaq-AI on GitHub (Mar 20, 2024).

I want to integrate the llama parse below code

import nest_asyncio
nest_asyncio.apply()

from llama_parse import LlamaParse
from llama_index.core import SimpleDirectoryReader

parser = LlamaParse(
api_key="llx-...", # can also be set in your env as LLAMA_CLOUD_API_KEY
result_type="markdown", # "markdown" and "text" are available
verbose=True
)

file_extractor = {".pdf": parser}
documents = SimpleDirectoryReader("./data", file_extractor=file_extractor).load_data()

with my nodejs code, how can i do that?

this is my nodejs code

import { RecursiveCharacterTextSplitter } from 'langchain/text_splitter';
import { OpenAIEmbeddings } from 'langchain/embeddings/openai';
import { PineconeStore } from 'langchain/vectorstores/pinecone';
import { pinecone } from '@/utils/pinecone-client';
import { CustomPDFLoader } from '@/utils/customPDFLoader';
import { PINECONE_INDEX_NAME, PINECONE_NAME_SPACE } from '@/config/pinecone';
import { DirectoryLoader } from 'langchain/document_loaders/fs/directory';

/* Name of directory to retrieve your files from */
const filePath = 'new docs';

export const run = async () => {
try {
/* Load raw docs from all files in the directory */
const directoryLoader = new DirectoryLoader(filePath, {
'.pdf': (path) => new CustomPDFLoader(path),
});

const rawDocs = await directoryLoader.load();

// Extracting the file name using regular expressions and updating metadata
const processedDocs = rawDocs.map(doc => {
  const fileName = doc.metadata.source.match(/[^\\\/]+$/)?.[0] || doc.metadata.source;
  const modifiedMetadata = { ...doc.metadata, source: fileName };
  return { ...doc, metadata: modifiedMetadata };
});

/* Split text into chunks */
const textSplitter = new RecursiveCharacterTextSplitter({
  chunkSize: 1000,
  chunkOverlap: 200,
});

const docs = await textSplitter.splitDocuments(processedDocs);
console.log('split docs', docs);

console.log('creating vector store...');
/* Create and store the embeddings in the vectorStore */
const embeddings = new OpenAIEmbeddings();
const index = pinecone.Index(PINECONE_INDEX_NAME); // Change to your own index name

// Embed the PDF documents
await PineconeStore.fromDocuments(docs, embeddings, {
  pineconeIndex: index,
  namespace: PINECONE_NAME_SPACE,
  textKey: 'text',
});

} catch (error) {
console.log('error', error);
throw new Error('Failed to ingest your data');
}
};

(async () => {
await run();
console.log('ingestion complete');
})();

Originally created by @MuhammadIshaq-AI on GitHub (Mar 20, 2024). I want to integrate the llama parse below code import nest_asyncio nest_asyncio.apply() from llama_parse import LlamaParse from llama_index.core import SimpleDirectoryReader parser = LlamaParse( api_key="llx-...", # can also be set in your env as LLAMA_CLOUD_API_KEY result_type="markdown", # "markdown" and "text" are available verbose=True ) file_extractor = {".pdf": parser} documents = SimpleDirectoryReader("./data", file_extractor=file_extractor).load_data() with my nodejs code, how can i do that? this is my nodejs code import { RecursiveCharacterTextSplitter } from 'langchain/text_splitter'; import { OpenAIEmbeddings } from 'langchain/embeddings/openai'; import { PineconeStore } from 'langchain/vectorstores/pinecone'; import { pinecone } from '@/utils/pinecone-client'; import { CustomPDFLoader } from '@/utils/customPDFLoader'; import { PINECONE_INDEX_NAME, PINECONE_NAME_SPACE } from '@/config/pinecone'; import { DirectoryLoader } from 'langchain/document_loaders/fs/directory'; /* Name of directory to retrieve your files from */ const filePath = 'new docs'; export const run = async () => { try { /* Load raw docs from all files in the directory */ const directoryLoader = new DirectoryLoader(filePath, { '.pdf': (path) => new CustomPDFLoader(path), }); const rawDocs = await directoryLoader.load(); // Extracting the file name using regular expressions and updating metadata const processedDocs = rawDocs.map(doc => { const fileName = doc.metadata.source.match(/[^\\\/]+$/)?.[0] || doc.metadata.source; const modifiedMetadata = { ...doc.metadata, source: fileName }; return { ...doc, metadata: modifiedMetadata }; }); /* Split text into chunks */ const textSplitter = new RecursiveCharacterTextSplitter({ chunkSize: 1000, chunkOverlap: 200, }); const docs = await textSplitter.splitDocuments(processedDocs); console.log('split docs', docs); console.log('creating vector store...'); /* Create and store the embeddings in the vectorStore */ const embeddings = new OpenAIEmbeddings(); const index = pinecone.Index(PINECONE_INDEX_NAME); // Change to your own index name // Embed the PDF documents await PineconeStore.fromDocuments(docs, embeddings, { pineconeIndex: index, namespace: PINECONE_NAME_SPACE, textKey: 'text', }); } catch (error) { console.log('error', error); throw new Error('Failed to ingest your data'); } }; (async () => { await run(); console.log('ingestion complete'); })();
yindo closed this issue 2026-02-16 00:16:45 -05:00
yindo commented 2026-02-16 00:16:45 -05:00
Author
Owner
Copy Link
Copy Source

@hexapode commented on GitHub (Jul 9, 2024):

To integrate llamaParse node js you can use our nodejs library instead of the python package.
The doc is here: https://ts.llamaindex.ai/guides/agents/llamaparse

@hexapode commented on GitHub (Jul 9, 2024): To integrate llamaParse node js you can use our nodejs library instead of the python package. The doc is here: https://ts.llamaindex.ai/guides/agents/llamaparse
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
LlamaCloud
LlamaExtract
LlamaParse
bug
client
complex layout
documentation
duplicate
enhancement
figure
font
header footer
help wanted
ocr
parser
pdf_debug
pdf_debug
pull-request
Mirrored from GitHub Pull Request
 python-lib
question
server_issue
server_issue
ui
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: run-llama/llama_cloud_services#58
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?