Merge branch 'main' of github.com:run-llama/LlamaIndexTS into seldo/deploy-fixes

fix: generated frontend is sending text/plain

.changeset/little-boxes-allow.md

@@ -1,5 +0,0 @@
----
-"create-llama": patch
----
-
-Label bug fix

packages/core/CHANGELOG.md

@@ -4,7 +4,7 @@
 
 ### Patch Changes
 
-- 63f2108: Add multimodal support (thanks Marcus)
+- 63f2108: Add multimodal support (thanks @marcusschiesser)
 
 ## 0.0.34
 

packages/create-llama/CHANGELOG.md

@@ -1,5 +1,23 @@
 # create-llama
 
+## 0.0.8
+
+### Patch Changes
+
+- 8cdb07f: Fix Next deployment (thanks @seldo and @marcusschiesser)
+
+## 0.0.7
+
+### Patch Changes
+
+- 9f9f293: Added more to README and made it easier to switch models (thanks @seldo)
+
+## 0.0.6
+
+### Patch Changes
+
+- 4431ec7: Label bug fix (thanks @marcusschiesser)
+
 ## 0.0.5
 
 ### Patch Changes
@@ -10,16 +28,16 @@
 
 ### Patch Changes
 
-- 031e926: Update create-llama readme (thanks Logan)
+- 031e926: Update create-llama readme (thanks @logan-markewich)
 
 ## 0.0.3
 
 ### Patch Changes
 
-- 91b42a3: change version (thanks Marcus)
+- 91b42a3: change version (thanks @marcusschiesser)
 
 ## 0.0.2
 
 ### Patch Changes
 
-- e2a6805: Hello Create Llama (thanks Marcus!)
+- e2a6805: Hello Create Llama (thanks @marcusschiesser)

packages/create-llama/README.md

@@ -2,19 +2,69 @@
 
 The easiest way to get started with [LlamaIndex](https://www.llamaindex.ai/) is by using `create-llama`. This CLI tool enables you to quickly start building a new LlamaIndex application, with everything set up for you.
 
-## Features
+Just run
 
-- NextJS, ExpressJS, or FastAPI (python) stateless backend generation 💻
-- Streaming or non-streaming backend ⚡
-- Optional `shadcn` frontend generation 🎨
+```bash
+npx create-llama@latest
+```
 
-## Get Started
+to get started, or see below for more options. Once your app is generated, run
 
-You can run `create-llama` in interactive or non-interactive mode.
+```bash
+npm run dev
+```
 
-### Interactive
+to start the development server. You can then visit [http://localhost:3000](http://localhost:3000) to see your app.
 
-You can create a new project interactively by running:
+## What you'll get
+
+- A Next.js-powered front-end. The app is set up as a chat interface that can answer questions about your data (see below)
+  - You can style it with HTML and CSS, or you can optionally use components from [shadcn/ui](https://ui.shadcn.com/)
+- Your choice of 3 back-ends:
+  - **Next.js**: if you select this option, you’ll have a full stack Next.js application that you can deploy to a host like [Vercel](https://vercel.com/) in just a few clicks. This uses [LlamaIndex.TS](https://www.npmjs.com/package/llamaindex), our TypeScript library.
+  - **Express**: if you want a more traditional Node.js application you can generate an Express backend. This also uses LlamaIndex.TS.
+  - **Python FastAPI**: if you select this option you’ll get a backend powered by the [llama-index python package](https://pypi.org/project/llama-index/), which you can deploy to a service like Render or fly.io.
+- The back-end has a single endpoint that allows you to send the state of your chat and receive additional responses
+- You can choose whether you want a streaming or non-streaming back-end (if you're not sure, we recommend streaming)
+- You can choose whether you want to use `ContextChatEngine` or `SimpleChatEngine`
+  - `SimpleChatEngine` will just talk to the LLM directly without using your data
+  - `ContextChatEngine` will use your data to answer questions (see below).
+- The app uses OpenAI by default, so you'll need an OpenAI API key, or you can customize it to use any of the dozens of LLMs we support.
+
+## Using your data
+
+If you've enabled `ContextChatEngine`, you can supply your own data and the app will index it and answer questions. Your generated app will have a folder called `data`:
+
+- With the Next.js backend this is `./data`
+- With the Express or Python backend this is in `./backend/data`
+
+The app will ingest any supported files you put in this directory. Your Next.js and Express apps use LlamaIndex.TS so they will be able to ingest any PDF, text, CSV, Markdown, Word and HTML files. The Python backend can read even more types, including video and audio files.
+
+Before you can use your data, you need to index it. If you're using the Next.js or Express apps, run:
+
+```bash
+npm run generate
+```
+
+Then re-start your app. Remember you'll need to re-run `generate` if you add new files to your `data` folder. If you're using the Python backend, you can trigger indexing of your data by deleting the `./storage` folder and re-starting the app.
+
+## Don't want a front-end?
+
+It's optional! If you've selected the Python or Express back-ends, just delete the `frontend` folder and you'll get an API without any front-end code.
+
+## Customizing the LLM
+
+By default the app will use OpenAI's gpt-3.5-turbo model. If you want to use GPT-4, you can modify this by editing a file:
+
+- In the Next.js backend, edit `./app/api/chat/route.ts` and replace `gpt-3.5-turbo` with `gpt-4`
+- In the Express backend, edit `./backend/src/controllers/chat.controller.ts` and likewise replace `gpt-3.5-turbo` with `gpt-4`
+- In the Python backend, edit `./backend/app/utils/index.py` and once again replace `gpt-3.5-turbo` with `gpt-4`
+
+You can also replace OpenAI with one of our [dozens of other supported LLMs](https://docs.llamaindex.ai/en/stable/module_guides/models/llms/modules.html).
+
+## Example
+
+The simplest thing to do is run `create-llama` in interactive mode:
 
 ```bash
 npx create-llama@latest
@@ -26,9 +76,7 @@ yarn create llama
 pnpm create llama@latest
 ```
 
-You will be asked for the name of your project, along with other configuration options.
-
-Here is an example:
+You will be asked for the name of your project, along with other configuration options, something like this:
 
 ```bash
 >> npm create llama@latest
@@ -45,7 +93,7 @@ Ok to proceed? (y) y
 Creating a new LlamaIndex app in /home/my-app.
 ```
 
-### Non-interactive
+### Running non-interactively
 
 You can also pass command line arguments to set up a new project
 non-interactively. See `create-llama --help`:

packages/create-llama/create-app.ts

@@ -88,7 +88,7 @@ export async function createApp({
       path.join(root, "README.md"),
     );
   } else {
-    await installTemplate({ ...args, backend: true });
+    await installTemplate({ ...args, backend: true, forBackend: framework });
   }
 
   process.chdir(root);

packages/create-llama/package.json

@@ -1,6 +1,6 @@
 {
   "name": "create-llama",
-  "version": "0.0.5",
+  "version": "0.0.8",
   "keywords": [
     "rag",
     "llamaindex",

packages/create-llama/templates/index.ts

@@ -103,6 +103,7 @@ const installTSTemplate = async ({
   ui,
   eslint,
   customApiPath,
+  forBackend,
 }: InstallTemplateArgs) => {
   console.log(bold(`Using ${packageManager}.`));
 
@@ -120,6 +121,26 @@ const installTSTemplate = async ({
     rename,
   });
 
+  /**
+   * If the backend is next.js, rename next.config.app.js to next.config.js
+   * If not, rename next.config.static.js to next.config.js
+   */
+  if (framework == "nextjs" && forBackend === "nextjs") {
+    const nextConfigAppPath = path.join(root, "next.config.app.js");
+    const nextConfigPath = path.join(root, "next.config.js");
+    await fs.rename(nextConfigAppPath, nextConfigPath);
+    // delete next.config.static.js
+    const nextConfigStaticPath = path.join(root, "next.config.static.js");
+    await fs.rm(nextConfigStaticPath);
+  } else if (framework == "nextjs" && typeof forBackend === "undefined") {
+    const nextConfigStaticPath = path.join(root, "next.config.static.js");
+    const nextConfigPath = path.join(root, "next.config.js");
+    await fs.rename(nextConfigStaticPath, nextConfigPath);
+    // delete next.config.app.js
+    const nextConfigAppPath = path.join(root, "next.config.app.js");
+    await fs.rm(nextConfigAppPath);
+  }
+
   /**
    * Copy the selected chat engine files to the target directory and reference it.
    */

packages/create-llama/templates/types.ts

@@ -17,4 +17,5 @@ export interface InstallTemplateArgs {
   eslint: boolean;
   customApiPath?: string;
   openAIKey?: string;
+  forBackend?: string;
 }

packages/create-llama/templates/types/simple/express/gitignore

@@ -0,0 +1,2 @@
+# local env files
+.env

packages/create-llama/templates/types/simple/express/index.ts

@@ -8,12 +8,24 @@ const port = 8000;
 
 const env = process.env["NODE_ENV"];
 const isDevelopment = !env || env === "development";
+const prodCorsOrigin = process.env["PROD_CORS_ORIGIN"];
+
 if (isDevelopment) {
   console.warn("Running in development mode - allowing CORS for all origins");
   app.use(cors());
+} else if (prodCorsOrigin) {
+  console.log(
+    `Running in production mode - allowing CORS for domain: ${prodCorsOrigin}`,
+  );
+  const corsOptions = {
+    origin: prodCorsOrigin, // Restrict to production domain
+  };
+  app.use(cors(corsOptions));
+} else {
+  console.warn("Production CORS origin not set, defaulting to no CORS.");
 }
 
-app.use(express.json());
+app.use(express.text());
 
 app.get("/", (req: Request, res: Response) => {
   res.send("LlamaIndex Express Server");

packages/create-llama/templates/types/simple/express/src/controllers/chat.controller.ts

@@ -4,7 +4,7 @@ import { createChatEngine } from "./engine";
 
 export const chat = async (req: Request, res: Response, next: NextFunction) => {
   try {
-    const { messages }: { messages: ChatMessage[] } = req.body;
+    const { messages }: { messages: ChatMessage[] } = JSON.parse(req.body);
     const lastMessage = messages.pop();
     if (!messages || !lastMessage || lastMessage.role !== "user") {
       return res.status(400).json({

packages/create-llama/templates/types/simple/fastapi/app/utils/index.py

@@ -6,12 +6,18 @@ from llama_index import (
     StorageContext,
     VectorStoreIndex,
     load_index_from_storage,
+    ServiceContext,
 )
+from llama_index.llms import OpenAI
 
 
 STORAGE_DIR = "./storage"  # directory to cache the generated index
 DATA_DIR = "./data"  # directory containing the documents to index
 
+service_context = ServiceContext.from_defaults(
+    llm=OpenAI(model="gpt-3.5-turbo")
+)
+
 
 def get_index():
     logger = logging.getLogger("uvicorn")
@@ -20,7 +26,7 @@ def get_index():
         logger.info("Creating new index")
         # load the documents and create the index
         documents = SimpleDirectoryReader(DATA_DIR).load_data()
-        index = VectorStoreIndex.from_documents(documents)
+        index = VectorStoreIndex.from_documents(documents,service_context=service_context)
         # store it for later
         index.storage_context.persist(STORAGE_DIR)
         logger.info(f"Finished creating new index. Stored in {STORAGE_DIR}")
@@ -28,6 +34,6 @@ def get_index():
         # load the existing index
         logger.info(f"Loading index from {STORAGE_DIR}...")
         storage_context = StorageContext.from_defaults(persist_dir=STORAGE_DIR)
-        index = load_index_from_storage(storage_context)
+        index = load_index_from_storage(storage_context,service_context=service_context)
         logger.info(f"Finished loading index from {STORAGE_DIR}")
     return index

packages/create-llama/templates/types/simple/fastapi/main.py

@@ -1,12 +1,12 @@
+from dotenv import load_dotenv
+load_dotenv()
+
 import logging
 import os
 import uvicorn
 from app.api.routers.chat import chat_router
 from fastapi import FastAPI
 from fastapi.middleware.cors import CORSMiddleware
-from dotenv import load_dotenv
-
-load_dotenv()
 
 app = FastAPI()
 

packages/create-llama/templates/types/simple/nextjs/next.config.app.js

@@ -2,6 +2,9 @@
 const nextConfig = {
   experimental: {
     serverComponentsExternalPackages: ["llamaindex"],
+    outputFileTracingIncludes: {
+      "/*": ["./cache/**/*"],
+    },
   },
 };
 

packages/create-llama/templates/types/simple/nextjs/next.config.static.js

@@ -0,0 +1,13 @@
+/** @type {import('next').NextConfig} */
+const nextConfig = {
+  output: "export",
+  images: { unoptimized: true },
+  experimental: {
+    serverComponentsExternalPackages: ["llamaindex"],
+    outputFileTracingIncludes: {
+      "/*": ["./cache/**/*"],
+    },
+  },
+};
+
+module.exports = nextConfig;

packages/create-llama/templates/types/streaming/express/gitignore

@@ -0,0 +1,2 @@
+# local env files
+.env

packages/create-llama/templates/types/streaming/express/index.ts

@@ -8,12 +8,24 @@ const port = 8000;
 
 const env = process.env["NODE_ENV"];
 const isDevelopment = !env || env === "development";
+const prodCorsOrigin = process.env["PROD_CORS_ORIGIN"];
+
 if (isDevelopment) {
   console.warn("Running in development mode - allowing CORS for all origins");
   app.use(cors());
+} else if (prodCorsOrigin) {
+  console.log(
+    `Running in production mode - allowing CORS for domain: ${prodCorsOrigin}`,
+  );
+  const corsOptions = {
+    origin: prodCorsOrigin, // Restrict to production domain
+  };
+  app.use(cors(corsOptions));
+} else {
+  console.warn("Production CORS origin not set, defaulting to no CORS.");
 }
 
-app.use(express.json());
+app.use(express.text());
 
 app.get("/", (req: Request, res: Response) => {
   res.send("LlamaIndex Express Server");

packages/create-llama/templates/types/streaming/express/src/controllers/chat.controller.ts

@@ -6,7 +6,7 @@ import { LlamaIndexStream } from "./llamaindex-stream";
 
 export const chat = async (req: Request, res: Response, next: NextFunction) => {
   try {
-    const { messages }: { messages: ChatMessage[] } = req.body;
+    const { messages }: { messages: ChatMessage[] } = JSON.parse(req.body);
     const lastMessage = messages.pop();
     if (!messages || !lastMessage || lastMessage.role !== "user") {
       return res.status(400).json({

packages/create-llama/templates/types/streaming/fastapi/app/utils/index.py

@@ -6,12 +6,17 @@ from llama_index import (
     StorageContext,
     VectorStoreIndex,
     load_index_from_storage,
+    ServiceContext,
 )
+from llama_index.llms import OpenAI
 
 
 STORAGE_DIR = "./storage"  # directory to cache the generated index
 DATA_DIR = "./data"  # directory containing the documents to index
 
+service_context = ServiceContext.from_defaults(
+    llm=OpenAI(model="gpt-3.5-turbo")
+)
 
 def get_index():
     logger = logging.getLogger("uvicorn")
@@ -20,7 +25,7 @@ def get_index():
         logger.info("Creating new index")
         # load the documents and create the index
         documents = SimpleDirectoryReader(DATA_DIR).load_data()
-        index = VectorStoreIndex.from_documents(documents)
+        index = VectorStoreIndex.from_documents(documents,service_context=service_context)
         # store it for later
         index.storage_context.persist(STORAGE_DIR)
         logger.info(f"Finished creating new index. Stored in {STORAGE_DIR}")
@@ -28,6 +33,6 @@ def get_index():
         # load the existing index
         logger.info(f"Loading index from {STORAGE_DIR}...")
         storage_context = StorageContext.from_defaults(persist_dir=STORAGE_DIR)
-        index = load_index_from_storage(storage_context)
+        index = load_index_from_storage(storage_context,service_context=service_context)
         logger.info(f"Finished loading index from {STORAGE_DIR}")
     return index

packages/create-llama/templates/types/streaming/fastapi/main.py

@@ -1,12 +1,12 @@
+from dotenv import load_dotenv
+load_dotenv()
+
 import logging
 import os
 import uvicorn
 from app.api.routers.chat import chat_router
 from fastapi import FastAPI
 from fastapi.middleware.cors import CORSMiddleware
-from dotenv import load_dotenv
-
-load_dotenv()
 
 app = FastAPI()
 

packages/create-llama/templates/types/streaming/nextjs/next.config.app.js

@@ -2,6 +2,9 @@
 const nextConfig = {
   experimental: {
     serverComponentsExternalPackages: ["llamaindex"],
+    outputFileTracingIncludes: {
+      "/*": ["./cache/**/*"],
+    },
   },
 };
 

packages/create-llama/templates/types/streaming/nextjs/next.config.static.js

@@ -0,0 +1,13 @@
+/** @type {import('next').NextConfig} */
+const nextConfig = {
+  output: "export",
+  images: { unoptimized: true },
+  experimental: {
+    serverComponentsExternalPackages: ["llamaindex"],
+    outputFileTracingIncludes: {
+      "/*": ["./cache/**/*"],
+    },
+  },
+};
+
+module.exports = nextConfig;