Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #2112] Prisma migration error when attempting to use postgresql #1375

New Issue
Closed
opened 2026-02-22 18:24:30 -05:00 by yindo · 9 comments
yindo commented 2026-02-22 18:24:30 -05:00
Owner
Copy Link
Copy Source

Originally created by @SeaDude on GitHub (Aug 14, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/2112

Hello,

  • I've uncommented the postgresql option in server/prisma/schema.prisma and commented-out the sqlite option.
  • I've set the DATABASE_URL env variable and I can see in the logs that the app is able to auth to the postgresql database.
  • The issue I'm having is that the prisma migrations are failing.
    • Are these sqlite-specific? or is there something else I should focus on to get this working?

ERROR:

2024-08-14T04:41:56.4527110Z Environment variables loaded from .env
2024-08-14T04:41:56.4709076Z Prisma schema loaded from prisma/schema.prisma
2024-08-14T04:42:07.0389528Z
2024-08-14T04:42:07.0390667Z ✔ Generated Prisma Client (v5.3.1) to ./node_modules/@prisma/client in 5.37s
2024-08-14T04:42:07.0390836Z
2024-08-14T04:42:07.0390880Z Start using Prisma Client in Node.js (See: https://pris.ly/d/client)
2024-08-14T04:42:07.0390919Z ```
2024-08-14T04:42:07.0390957Z import { PrismaClient } from '@prisma/client'
2024-08-14T04:42:07.0390996Z const prisma = new PrismaClient()
2024-08-14T04:42:07.0391032Z ```
2024-08-14T04:42:07.0391101Z or start using Prisma Client at the edge (See: https://pris.ly/d/accelerate)
2024-08-14T04:42:07.0391137Z ```
2024-08-14T04:42:07.0391174Z import { PrismaClient } from '@prisma/client/edge'
2024-08-14T04:42:07.0391212Z const prisma = new PrismaClient()
2024-08-14T04:42:07.0391245Z ```
2024-08-14T04:42:07.0391307Z
2024-08-14T04:42:07.0391346Z See other ways of importing Prisma Client: http://pris.ly/d/importing-client
2024-08-14T04:42:07.0391379Z
2024-08-14T04:42:22.3401693Z Environment variables loaded from .env
2024-08-14T04:42:22.3483526Z Prisma schema loaded from prisma/schema.prisma
2024-08-14T04:42:22.4544716Z Datasource "db": PostgreSQL database "redacted", schema "public" at "redacted.com:5432"
2024-08-14T04:42:24.0652199Z
2024-08-14T04:42:24.0664917Z 21 migrations found in prisma/migrations
2024-08-14T04:42:24.0723843Z
2024-08-14T04:42:24.9086284Z Error: P3009
2024-08-14T04:42:24.9090144Z
2024-08-14T04:42:24.9090438Z migrate found failed migrations in the target database, new migrations will not be applied. Read more about how to resolve migration issues in a production database: https://pris.ly/d/migrate-resolve
2024-08-14T04:42:24.9090488Z The `20230921191814_init` migration started at 2024-08-14 03:53:15.193764 UTC failed

Thank you

Originally created by @SeaDude on GitHub (Aug 14, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/2112 Hello, - I've uncommented the postgresql option in `server/prisma/schema.prisma` and commented-out the sqlite option. - I've set the DATABASE_URL env variable and I can see in the logs that the app is able to auth to the postgresql database. - The issue I'm having is that the prisma migrations are failing. - Are these sqlite-specific? or is there something else I should focus on to get this working? **ERROR**: ``` 2024-08-14T04:41:56.4527110Z Environment variables loaded from .env 2024-08-14T04:41:56.4709076Z Prisma schema loaded from prisma/schema.prisma 2024-08-14T04:42:07.0389528Z 2024-08-14T04:42:07.0390667Z ✔ Generated Prisma Client (v5.3.1) to ./node_modules/@prisma/client in 5.37s 2024-08-14T04:42:07.0390836Z 2024-08-14T04:42:07.0390880Z Start using Prisma Client in Node.js (See: https://pris.ly/d/client) 2024-08-14T04:42:07.0390919Z ``` 2024-08-14T04:42:07.0390957Z import { PrismaClient } from '@prisma/client' 2024-08-14T04:42:07.0390996Z const prisma = new PrismaClient() 2024-08-14T04:42:07.0391032Z ``` 2024-08-14T04:42:07.0391101Z or start using Prisma Client at the edge (See: https://pris.ly/d/accelerate) 2024-08-14T04:42:07.0391137Z ``` 2024-08-14T04:42:07.0391174Z import { PrismaClient } from '@prisma/client/edge' 2024-08-14T04:42:07.0391212Z const prisma = new PrismaClient() 2024-08-14T04:42:07.0391245Z ``` 2024-08-14T04:42:07.0391307Z 2024-08-14T04:42:07.0391346Z See other ways of importing Prisma Client: http://pris.ly/d/importing-client 2024-08-14T04:42:07.0391379Z 2024-08-14T04:42:22.3401693Z Environment variables loaded from .env 2024-08-14T04:42:22.3483526Z Prisma schema loaded from prisma/schema.prisma 2024-08-14T04:42:22.4544716Z Datasource "db": PostgreSQL database "redacted", schema "public" at "redacted.com:5432" 2024-08-14T04:42:24.0652199Z 2024-08-14T04:42:24.0664917Z 21 migrations found in prisma/migrations 2024-08-14T04:42:24.0723843Z 2024-08-14T04:42:24.9086284Z Error: P3009 2024-08-14T04:42:24.9090144Z 2024-08-14T04:42:24.9090438Z migrate found failed migrations in the target database, new migrations will not be applied. Read more about how to resolve migration issues in a production database: https://pris.ly/d/migrate-resolve 2024-08-14T04:42:24.9090488Z The `20230921191814_init` migration started at 2024-08-14 03:53:15.193764 UTC failed ``` Thank you
yindo closed this issue 2026-02-22 18:24:30 -05:00
yindo commented 2026-02-22 18:24:31 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Aug 14, 2024):

Keep in mind we dont support using PG as the backend because of these issues

So it seems to fail on https://github.com/Mintplex-Labs/anything-llm/blob/master/server/prisma/migrations/20230921191814_init/migration.sql

This can lead to several issues because we use SQLITE AUTOINCREMENT is not valid in PG, you need to use SERIAL. Basically, you need to generate a custom PG schema migration set if you want to swap to PG because of these subtle differences.

If you swap the connector in schema.prisma and in the server delete the entire migration folder and then run yarn primsa:migrate it should regen the entire DB schema migration set to be valid in PG.

Again, this is not something we explicitly support so this ultimately falls on you for bug fixes, but that is where i would start.

@timothycarambat commented on GitHub (Aug 14, 2024): **Keep in mind we dont support using PG as the backend because of these issues** So it seems to fail on https://github.com/Mintplex-Labs/anything-llm/blob/master/server/prisma/migrations/20230921191814_init/migration.sql This can lead to several issues because we use SQLITE `AUTOINCREMENT` is not valid in PG, you need to use `SERIAL`. Basically, you need to generate a custom PG schema migration set if you want to swap to PG because of these subtle differences. If you swap the connector in `schema.prisma` and in the server delete the entire migration folder and then run `yarn primsa:migrate` it _should_ regen the entire DB schema migration set to be valid in PG. Again, this is not something we explicitly support so this ultimately falls on you for bug fixes, but that is where i would start.
yindo commented 2026-02-22 18:24:31 -05:00
Author
Owner
Copy Link
Copy Source

@SeaDude commented on GitHub (Aug 14, 2024):

Thank you.

Testing the suggestion now:

  • Remove migrations directory
  • Add yard prisma:migrate to the Dockerfile
  • Build Docker from forked repo
    🤞
@SeaDude commented on GitHub (Aug 14, 2024): Thank you. **Testing the suggestion now**: - Remove `migrations` directory - Add `yard prisma:migrate` to the Dockerfile - Build Docker from forked repo 🤞
yindo commented 2026-02-22 18:24:31 -05:00
Author
Owner
Copy Link
Copy Source

@SeaDude commented on GitHub (Aug 14, 2024):

Hm...I'm not quite sure which "root directory" to place the yarn primsa:migrate command in the Dockerfile.

Is it /app or /app/server (or another directory)?

...

# Install server layer & build node-llama-cpp
# Also pull and build collector deps (chromium issues prevent bad bindings)
FROM build AS backend-build
COPY ./server /app/server/
WORKDIR /app/server
RUN yarn install --production --network-timeout 100000 && yarn cache clean

# Is `/app` the "root directory" or is `/app/server`?
WORKDIR /app
RUN yarn prisma:migrate

# Continue with the rest of the Dockerfile
WORKDIR /app

...
@SeaDude commented on GitHub (Aug 14, 2024): Hm...I'm not quite sure which "root directory" to place the `yarn primsa:migrate` command in the Dockerfile. Is it `/app` or `/app/server` (or another directory)? ``` ... # Install server layer & build node-llama-cpp # Also pull and build collector deps (chromium issues prevent bad bindings) FROM build AS backend-build COPY ./server /app/server/ WORKDIR /app/server RUN yarn install --production --network-timeout 100000 && yarn cache clean # Is `/app` the "root directory" or is `/app/server`? WORKDIR /app RUN yarn prisma:migrate # Continue with the rest of the Dockerfile WORKDIR /app ... ```
yindo commented 2026-02-22 18:24:31 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Aug 15, 2024):

For this dockerfile

WORKDIR /app/server
RUN yarn prisma:migrate
@timothycarambat commented on GitHub (Aug 15, 2024): For this dockerfile ``` WORKDIR /app/server RUN yarn prisma:migrate ```
yindo commented 2026-02-22 18:24:31 -05:00
Author
Owner
Copy Link
Copy Source

@SeaDude commented on GitHub (Aug 17, 2024):

Whew! Was able to get the GitHub Workflow --> Dockerfile --> Azure Container Registry --> Azure Web App deployment working, but the actual application doesn't start (504 timeout).

  • The Dockerfile connects to the (Azure Database for) PostgreSQL database (I can see the _prisma_migrations table it creates)
    • But the app tables are not built.
  • The LanceDB vector database is not created in Azure Blob Storage Container even though I changed the server/utils/vectorDbProviders/lance/index.js accordingly.
  • The app comkey is created in the Azure Blob Fileshare, but not the assets and other storage contents.

Looking at the Azure Web App logs, doesn't seem like the app tables are being created...

2024-08-17T21:07:44.280Z INFO  - docker run -d --expose=8181 <site-specific-details> 
2024-08-17T20:06:08.446862043Z The table `public.system_settings` does not exist in the current database.
2024-08-17T20:06:08.446867144Z [backend] error: 
2024-08-17T20:06:08.446872444Z Invalid `prisma.system_settings.findFirst()` invocation:
2024-08-17T20:06:08.447216648Z The table `public.system_settings` does not exist in the current database.
2024-08-17T20:06:08.447221648Z [backend] info: [BackgroundWorkerService] Feature is not enabled and will not be started.

2024-08-17T20:08:44.744Z INFO  - Initiating warmup request to container 
2024-08-17T20:08:45.934Z INFO  - Container for site <my-site> initialized successfully and is ready to serve requests.
2024-08-17T20:08:45.997Z ERROR - Container didn't respond to HTTP pings on port: 80, failing site start. See container logs for debugging.
2024-08-17T20:08:46.066Z INFO  - Stopping site <my-site> because it failed during startup.

I know using PostgreSQL as app database is not supported, but Azure is one of the largest enterprise cloud providers. This is a useful (for me) endeavor and hopefully I can make this reproducable for others...challenging!

@SeaDude commented on GitHub (Aug 17, 2024): Whew! Was able to get the GitHub Workflow --> Dockerfile --> Azure Container Registry --> Azure Web App *deployment* working, but the actual application doesn't start (504 timeout). - The Dockerfile connects to the (Azure Database for) PostgreSQL database (I can see the `_prisma_migrations` table it creates) - But the app tables are not built. - The LanceDB vector database is not created in Azure Blob Storage Container even though I changed the `server/utils/vectorDbProviders/lance/index.js` accordingly. - The app comkey *is* created in the Azure Blob Fileshare, but not the `assets` and other storage contents. Looking at the Azure Web App logs, doesn't seem like the app tables are being created... ``` 2024-08-17T21:07:44.280Z INFO - docker run -d --expose=8181 <site-specific-details> 2024-08-17T20:06:08.446862043Z The table `public.system_settings` does not exist in the current database. 2024-08-17T20:06:08.446867144Z [backend] error: 2024-08-17T20:06:08.446872444Z Invalid `prisma.system_settings.findFirst()` invocation: 2024-08-17T20:06:08.447216648Z The table `public.system_settings` does not exist in the current database. 2024-08-17T20:06:08.447221648Z [backend] info: [BackgroundWorkerService] Feature is not enabled and will not be started. 2024-08-17T20:08:44.744Z INFO - Initiating warmup request to container 2024-08-17T20:08:45.934Z INFO - Container for site <my-site> initialized successfully and is ready to serve requests. 2024-08-17T20:08:45.997Z ERROR - Container didn't respond to HTTP pings on port: 80, failing site start. See container logs for debugging. 2024-08-17T20:08:46.066Z INFO - Stopping site <my-site> because it failed during startup. ``` I know using PostgreSQL as app database is not supported, but Azure is one of the largest enterprise cloud providers. This is a useful (for me) endeavor and hopefully I can make this reproducable for others...challenging!
yindo commented 2026-02-22 18:24:31 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Aug 18, 2024):

Just to align on what is setup here:

  • You have a PG database running on Azure cloud connected via some connection string
  • You also are using Azure Blob Storage Container as basically persistent storage instance of storing or doing I/O on the container ephemeral storage?
@timothycarambat commented on GitHub (Aug 18, 2024): Just to align on what is setup here: - You have a PG database running on Azure cloud connected via some connection string - You _also_ are using `Azure Blob Storage Container` as basically persistent storage instance of storing or doing I/O on the container ephemeral storage?
yindo commented 2026-02-22 18:24:31 -05:00
Author
Owner
Copy Link
Copy Source

@SeaDude commented on GitHub (Aug 18, 2024):

  1. PostgreSQL database:

    • Yes. Running Azure Database for PostgreSQL.
    • Passing a DATABASE_URL (connection string) as an environment variable during the GitHub Action build.
    • This env var is consumed by server/prisma/schema.prisma. 
      // Uncomment the following lines and comment out the SQLite datasource block above to use PostgreSQL
// Make sure to set the correct DATABASE_URL in your .env file

datasource db {
  provider = "postgresql"
  url      = env("DATABASE_URL")
}
    • This is partially working (the _prisma_migrations table was autocreated, but thats the only table created)

  2. App Storage (assets, comkey, etc.):

    • This is done by mounting /app/server/storage to a SMB Azure Blob File Share in the Web App itself
    • This is currently working (the comkeys have been created)

  3. LanceDB vector store:

    • Stored in an Azure Blob Container by editing the server/utils/vectorDbProviders/lance/index.js as described here.
    • This is currently not working.
@SeaDude commented on GitHub (Aug 18, 2024): 1. **PostgreSQL database**: - Yes. Running [Azure Database for PostgreSQL](https://learn.microsoft.com/en-us/azure/postgresql/flexible-server/overview). - Passing a `DATABASE_URL` (connection string) as an environment variable during the GitHub Action build. - This env var is consumed by `server/prisma/schema.prisma`. ``` // Uncomment the following lines and comment out the SQLite datasource block above to use PostgreSQL // Make sure to set the correct DATABASE_URL in your .env file datasource db { provider = "postgresql" url = env("DATABASE_URL") } ``` - This is partially working (the `_prisma_migrations` table was autocreated, but thats the only table created) 2. **App Storage (`assets, comkey, etc.`)**: - This is done by mounting `/app/server/storage` to a SMB Azure Blob File Share in the Web App itself - This is currently working (the `comkey`s have been created) 3. **LanceDB vector store**: - Stored in an Azure Blob Container by editing the `server/utils/vectorDbProviders/lance/index.js` as described [here](https://github.com/Mintplex-Labs/anything-llm/issues/1825#issuecomment-2276732312). - This is currently **not** working.
yindo commented 2026-02-22 18:24:31 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Aug 19, 2024):

LanceDB vector store:

For what its worth, the reason that probably does not work is the the az:// protocol is not recognized by LanceDB itself, not that the system does not recognize it. Since you already have /app/server/storage mounted you can actually just keep LanceDB's file the exact same.

https://github.com/Mintplex-Labs/anything-llm/blob/a8d25c7dd31e90b06c9ea8575ada2321928dcfa4/server/utils/vectorDbProviders/lance/index.js#L15

Will save to /app/server/storage/lancedb since STORAGE_DIR is likely set as an env as well. If not, it should also be /app/server/storage

@timothycarambat commented on GitHub (Aug 19, 2024): > LanceDB vector store: For what its worth, the reason that probably does not work is the the `az://` protocol is not recognized by LanceDB itself, not that the system does not recognize it. Since you already have `/app/server/storage` mounted you can actually just keep LanceDB's file the exact same. https://github.com/Mintplex-Labs/anything-llm/blob/a8d25c7dd31e90b06c9ea8575ada2321928dcfa4/server/utils/vectorDbProviders/lance/index.js#L15 Will save to `/app/server/storage/lancedb` since STORAGE_DIR is likely set as an env as well. If not, it should also be `/app/server/storage`
yindo commented 2026-02-22 18:24:31 -05:00
Author
Owner
Copy Link
Copy Source

@ccr-fan-3000 commented on GitHub (Sep 16, 2024):

Whew! Was able to get the GitHub Workflow --> Dockerfile --> Azure Container Registry --> Azure Web App deployment working, but the actual application doesn't start (504 timeout).

  • The Dockerfile connects to the (Azure Database for) PostgreSQL database (I can see the _prisma_migrations table it creates)

    • But the app tables are not built.

  • The LanceDB vector database is not created in Azure Blob Storage Container even though I changed the server/utils/vectorDbProviders/lance/index.js accordingly.

  • The app comkey is created in the Azure Blob Fileshare, but not the assets and other storage contents.

Looking at the Azure Web App logs, doesn't seem like the app tables are being created...

2024-08-17T21:07:44.280Z INFO  - docker run -d --expose=8181 <site-specific-details> 
2024-08-17T20:06:08.446862043Z The table `public.system_settings` does not exist in the current database.
2024-08-17T20:06:08.446867144Z �[36m[backend]�[0m �[31merror�[39m: 
2024-08-17T20:06:08.446872444Z Invalid `prisma.system_settings.findFirst()` invocation:
2024-08-17T20:06:08.447216648Z The table `public.system_settings` does not exist in the current database.
2024-08-17T20:06:08.447221648Z �[36m[backend]�[0m �[32minfo�[39m: �[36m[BackgroundWorkerService]�[0m Feature is not enabled and will not be started.

2024-08-17T20:08:44.744Z INFO  - Initiating warmup request to container 
2024-08-17T20:08:45.934Z INFO  - Container for site <my-site> initialized successfully and is ready to serve requests.
2024-08-17T20:08:45.997Z ERROR - Container didn't respond to HTTP pings on port: 80, failing site start. See container logs for debugging.
2024-08-17T20:08:46.066Z INFO  - Stopping site <my-site> because it failed during startup.

I know using PostgreSQL as app database is not supported, but Azure is one of the largest enterprise cloud providers. This is a useful (for me) endeavor and hopefully I can make this reproducable for others...challenging!

I am getting this same error in my setup. So far following this issue has been very useful for resolving errors. Any update on correctly setting up PostGreSQL in Azure to avoid this timeout error?

@ccr-fan-3000 commented on GitHub (Sep 16, 2024): > Whew! Was able to get the GitHub Workflow --> Dockerfile --> Azure Container Registry --> Azure Web App _deployment_ working, but the actual application doesn't start (504 timeout). > > * The Dockerfile connects to the (Azure Database for) PostgreSQL database (I can see the `_prisma_migrations` table it creates) > > * But the app tables are not built. > * The LanceDB vector database is not created in Azure Blob Storage Container even though I changed the `server/utils/vectorDbProviders/lance/index.js` accordingly. > * The app comkey _is_ created in the Azure Blob Fileshare, but not the `assets` and other storage contents. > > Looking at the Azure Web App logs, doesn't seem like the app tables are being created... > > ``` > 2024-08-17T21:07:44.280Z INFO - docker run -d --expose=8181 <site-specific-details> > 2024-08-17T20:06:08.446862043Z The table `public.system_settings` does not exist in the current database. > 2024-08-17T20:06:08.446867144Z �[36m[backend]�[0m �[31merror�[39m: > 2024-08-17T20:06:08.446872444Z Invalid `prisma.system_settings.findFirst()` invocation: > 2024-08-17T20:06:08.447216648Z The table `public.system_settings` does not exist in the current database. > 2024-08-17T20:06:08.447221648Z �[36m[backend]�[0m �[32minfo�[39m: �[36m[BackgroundWorkerService]�[0m Feature is not enabled and will not be started. > > 2024-08-17T20:08:44.744Z INFO - Initiating warmup request to container > 2024-08-17T20:08:45.934Z INFO - Container for site <my-site> initialized successfully and is ready to serve requests. > 2024-08-17T20:08:45.997Z ERROR - Container didn't respond to HTTP pings on port: 80, failing site start. See container logs for debugging. > 2024-08-17T20:08:46.066Z INFO - Stopping site <my-site> because it failed during startup. > ``` > > I know using PostgreSQL as app database is not supported, but Azure is one of the largest enterprise cloud providers. This is a useful (for me) endeavor and hopefully I can make this reproducable for others...challenging! I am getting this same error in my setup. So far following this issue has been very useful for resolving errors. Any update on correctly setting up PostGreSQL in Azure to avoid this timeout error?
yindo changed title from Prisma migration error when attempting to use postgresql to [GH-ISSUE #2112] Prisma migration error when attempting to use postgresql 2026-06-05 14:40:26 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#1375
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?