Merge pull request #77 from Mintplex-Labs/docker-builds

Automated Docker builds

.github/workflows/build-and-push-image.yaml

@@ -0,0 +1,46 @@
+name: Publish Docker image and Github Registry
+
+on:
+  push:
+    branches: ['master']
+
+jobs:
+  push_to_registries:
+    name: Push Docker image to multiple registries
+    runs-on: ubuntu-latest
+    permissions:
+      packages: write
+      contents: read
+    steps:
+      - name: Check out the repo
+        uses: actions/checkout@v4
+      
+      - name: Log in to Docker Hub
+        uses: docker/login-action@f4ef78c080cd8ba55a85445d5b36e214a81df20a
+        with:
+          username: ${{ secrets.DOCKER_USERNAME }}
+          password: ${{ secrets.DOCKER_PASSWORD }}
+      
+      - name: Log in to the Container registry
+        uses: docker/login-action@65b78e6e13532edd9afa3aa52ac7964289d1a9c1
+        with:
+          registry: ghcr.io
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+      
+      - name: Extract metadata (tags, labels) for Docker
+        id: meta
+        uses: docker/metadata-action@9ec57ed1fcdbf14dcef7dfbe97b2010124a938b7
+        with:
+          images: |
+            mintplexlabs/vectoradmin
+            ghcr.io/${{ github.repository }}
+      
+      - name: Build and push Docker images
+        uses: docker/build-push-action@3b5e8027fcad23fda98b2e3ac259d8d67585f671
+        with:
+          context: .
+          file: ./docker/Dockerfile
+          push: true
+          tags: ${{ steps.meta.outputs.tags }}
+          labels: ${{ steps.meta.outputs.labels }}

docker/DOCKER.md

@@ -2,35 +2,51 @@
 
 Running VectorAdmin in Docker is the easily way to get a locally hosted option running as quickly as possible.
 
-
 ## Requirements
 - Install [Docker](https://www.docker.com/) on your computer or machine.
+- A running Postgres DB (RDS, remote, docker, local host machine)
 
-## How to install
-- `git clone` this repo and `cd vector-admin` to get to the root directory.
-- `yarn dev:setup`
+**Note**  Failure to use a valid DB connection string prior to building or starting vector-admin will result in a failure.
+On boot the `vdbms` database will be created using the connection string.
+
+**Run containerized Postgres DB**
+Run this command first to get a dockerized Postgres container running:
+`docker-compose up -d --build postgres`
+
+## Run from Docker pre-built image
+- `docker pull mintplexlabs/vectoradmin:master` to pull in latest image
+- Run the command with env variables and image defined.
+`docker run -d -p 3001:3001 \
+-e SERVER_PORT="3001" \
+-e JWT_SECRET="your-random-string-here" \
+-e SYS_EMAIL="root@vectoradmin.com" \
+-e SYS_PASSWORD="password" \
+-e INNGEST_EVENT_KEY="background_workers" \
+-e INNGEST_SIGNING_KEY="random-string-goes-here" \
+-e INNGEST_LANDING_PAGE="true" \
+-e DATABASE_CONNECTION_STRING="postgresql://vectoradmin:password@xxxxxxx:5432/vdbms" \
+mintplexlabs/vectoradmin:master`
+
+
+## Build docker image from source
+- `git clone git@github.com:Mintplex-Labs/vector-admin.git`
+- `cd vector-admin`
 - `cd docker/`
-- `cp .env.example .env` to create the `.env` file.
-- Edit `.env` file and update the variables. **please** update `JWT_SECRET`,`SYS_PASSWORD`, and `INNGEST_SIGNING_KEY`
-
-**Note** You must have a postgres DB running - the default ENV assumes a containerized PG instance running, but it can
-be located anywhere (RDS, remote, local host machine). Failure to use a valid DB connection string prior to building
-vector-admin will result in a build failure.
-- `DATABASE_CONNECTION_STRING` should be a valid connection string. On boot the `vdbms` database will be created for the connection string.
-
-**If running the included containerized Postgres DB**
-Run this command first: `docker-compose up -d --build postgres`
-
-**Boot up vector-admin**
-- `docker-compose up -d --build vector-admin` to build the image - this will take a few moments.
-
-Your docker host will show the image as online once the build process is completed. This will build the app to `http://localhost:3001`.
+- `cp .env.example .env`.
+- Edit `.env` file and update the variables. **please** update all of the following:
+```shell
+JWT_SECRET="some-random-string"
+SYS_EMAIL="root@vectoradmin.com"
+SYS_PASSWORD="password"
+DATABASE_CONNECTION_STRING="postgresql://vectoradmin:password@host.docker.internal:5433/vdbms" # Valid PG Connection string.
+INNGEST_SIGNING_KEY="some-random-string"
+```
+- `docker-compose up -d --build vector-admin`
 
 
-## How to use the user interface
+## How to use the user interface and login for the first time.
 - To access the full application, visit `http://localhost:3001` in your browser.
-- You first login will require you to use the `SYS_EMAIL` and `SYS_PASSWORD` set in the `.env` file. After onboarding this login will be permanently disabled.
-
+- You first login will require you to use the `SYS_EMAIL` and `SYS_PASSWORD` set via ENV during build or run. After onboarding this login will be permanently disabled.
 
 # Connecting to a Vector Database
 

docker/Dockerfile

@@ -1,19 +1,20 @@
 # Setup base image
 FROM ubuntu:jammy-20230522 AS base
 
-# Build arguments
-ARG DATABASE_CONNECTION_STRING
-
 # Install system dependencies
 RUN DEBIAN_FRONTEND=noninteractive apt-get update && \
     DEBIAN_FRONTEND=noninteractive apt-get install -yq --no-install-recommends \
-        curl libgfortran5 python3 python3-pip tzdata netcat \
+        curl gnupg libgfortran5 python3 python3-pip tzdata netcat \
         libasound2 libatk1.0-0 libc6 libcairo2 libcups2 libdbus-1-3 libexpat1 libfontconfig1 \
         libgcc1 libglib2.0-0 libgtk-3-0 libnspr4 libpango-1.0-0 libx11-6 libx11-xcb1 libxcb1 \
         libxcomposite1 libxcursor1 libxdamage1 libxext6 libxfixes3 libxi6 libxrandr2 libxrender1 \
         libxss1 libxtst6 ca-certificates fonts-liberation libappindicator1 libnss3 lsb-release \
         xdg-utils && \
-    curl -fsSL https://deb.nodesource.com/setup_18.x | bash - && \
+
+    mkdir -p /etc/apt/keyrings && \
+    curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key | gpg --dearmor -o /etc/apt/keyrings/nodesource.gpg && \
+    echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_18.x nodistro main" | tee /etc/apt/sources.list.d/nodesource.list && \
+    apt-get update && \
     apt-get install -yq --no-install-recommends nodejs && \
     curl -LO https://github.com/yarnpkg/yarn/releases/download/v1.22.19/yarn_1.22.19_all.deb \
         && dpkg -i yarn_1.22.19_all.deb \
@@ -79,10 +80,6 @@ RUN cd /app/document-processor && \
     . v-env/bin/activate && \
     pip install --no-cache-dir -r requirements.txt
 
-# Reown files and init Prisma Client
-RUN cd ./backend && DATABASE_CONNECTION_STRING=$DATABASE_CONNECTION_STRING npx prisma generate --schema=./prisma/schema.prisma
-RUN cd ./backend && DATABASE_CONNECTION_STRING=$DATABASE_CONNECTION_STRING npx prisma migrate deploy --schema=./prisma/schema.prisma
-
 # Setup the environment
 ENV NODE_ENV=production
 ENV PATH=/app/document-processor/v-env/bin:$PATH

docker/docker-compose.yml

@@ -26,8 +26,10 @@ services:
     build:
       context: ../.
       dockerfile: ./docker/Dockerfile
-      args:
-        DATABASE_CONNECTION_STRING: ${DATABASE_CONNECTION_STRING}
+    volumes:
+      - "./.env:/app/backend/.env"
+      - "../backend/storage:/app/backend/storage"
+      - "../document-processor/hotdir/:/app/document-processor/hotdir"
     ports:
       - "3001:3001"
       - "3355:3355"

docker/docker-entrypoint.sh

@@ -1,9 +1,18 @@
 #!/bin/bash
+
+# Migrate Postgres using and prisma client
+cd /app/backend
+DATABASE_CONNECTION_STRING=$DATABASE_CONNECTION_STRING npx prisma migrate deploy --schema=./prisma/schema.prisma
+DATABASE_CONNECTION_STRING=$DATABASE_CONNECTION_STRING npx prisma generate --schema=./prisma/schema.prisma
+
+# Start backend server & workers
 node /app/backend/index.js &
 NODE_ENV=development node /app/workers/index.js &
-{ cd /app/workers/ && NODE_ENV=development yarn run inngest-cli dev -u http://0.0.0.0:3355/background-workers; } &
-{ FLASK_ENV=production FLASK_APP=wsgi.py cd document-processor && gunicorn --timeout 300 --workers 4 --bind 0.0.0.0:8888 wsgi:api; } &
-wait -n
-exit $?
 
-yarn run inngest-cli dev -u http://127.0.0.1:3355/background-workers
+cd /app/workers/
+NODE_ENV=development yarn run inngest-cli dev -u http://0.0.0.0:3355/background-workers &
+
+cd /app/document-processor
+FLASK_ENV=production FLASK_APP=wsgi.py gunicorn --timeout 300 --workers 4 --bind 0.0.0.0:8888 wsgi:api &
+wait -n
+exit $?