incorporating variety of feedback on improving dev experience (#41)

* incorporating variety of feedback on impproving dev experience when setting things up locally

* reword FAQ.md

FAQ.md

@@ -37,3 +37,17 @@ You will also find that some of the prompts used in the application are specific
 * [Vector Index tool descriptions](https://github.com/run-llama/sec-insights/blob/e81c83958a428e2aa02e8cb1280c3a17c55c4aa9/backend/app/chat/engine.py#L295-L296)
 * System Message ([template](https://github.com/run-llama/sec-insights/blob/e81c83958a428e2aa02e8cb1280c3a17c55c4aa9/backend/app/chat/constants.py#L3-L17) and [construction](https://github.com/run-llama/sec-insights/blob/e81c83958a428e2aa02e8cb1280c3a17c55c4aa9/backend/app/chat/engine.py#L336))
 * [User Message Prefix](https://github.com/run-llama/sec-insights/blob/e81c83958a428e2aa02e8cb1280c3a17c55c4aa9/backend/app/chat/messaging.py#L143-L145)
+
+## How do I completely refresh my database?
+During development, you may find it useful or necessary to completely wipe out your database and start fresh with empty tables.
+
+To make this process simple, we have included a `make refresh_db` command in `backend/Makefile`. To use it, just do the following:
+- `cd` into the `backend/` folder if you're not already in it
+- Run `set -a` then `source .env`
+   - See instructions in `README.md` for more information on what this step does
+- Run `make refresh_db`
+   - This will ask for confirmation first and run as soon as you type either `y` or `N`.
+
+**What is this script doing?**
+
+When you run the database in the `db` container using `docker compose` and the various `make` commands, the container shares a data volume with your local machine. This ensures that the data in this local database is persisted even as the `db` container is started and stopped. As such, to completely refresh this database, you would first need to stop your DB container, delete these volumes, re-create the DB container, and re-apply the alembic migrations. That's what `make refresh_db` does.

README.md

@@ -62,7 +62,7 @@ Use this repository as a reference when building out your own RAG application or
 ## Usage 💻
 See `README.md` files in `frontend/` & `backend/` folders for individual setup instructions for each. As mentioned above, we also have a YouTube tutorial [here](https://youtu.be/2O52Tfj79T4?si=1Tm3zvuqna5ei4Cu&t=677) that covers how to setup this project's development environment.
 
-We've also included a config for a [GitHub Codespace](https://github.com/features/codespaces) in [`.devcontainer/devcontainer.json`](https://github.com/run-llama/sec-insights/blob/main/.devcontainer/devcontainer.json). If you choose to use GitHub Codespaces, your codespace will come pre-configured with a lot of the libraries and system dependencies that are needed to run this project. This is probably the fastest way to get this project up and running!
+We've also included a config for a [GitHub Codespace](https://github.com/features/codespaces) in [`.devcontainer/devcontainer.json`](https://github.com/run-llama/sec-insights/blob/main/.devcontainer/devcontainer.json). If you choose to use GitHub Codespaces, your codespace will come pre-configured with a lot of the libraries and system dependencies that are needed to run this project. This is probably the fastest way to get this project up and running! Having said that, developers have successfully set-up this project in Linux, macOS, and Windows environments!
 
 If you have any questions when trying to run this project, you may find your answer quickly by reviewing our [FAQ](./FAQ.md) or by searching through our [GitHub issues](https://github.com/run-llama/sec-insights/issues)! If you don't see a satisfactory answer to your question, feel free to [open a GitHub issue](https://github.com/run-llama/sec-insights/issues/new) so we may assist you!
 

backend/Makefile

@@ -14,6 +14,22 @@ migrate:
 	docker compose start db
 	poetry run python -m alembic upgrade head
 
+refresh_db:
+	# First ask for confirmation.
+	read -p "Are you sure you want to refresh the local database? This will delete all data in your local db. [y/N] " -n 1 -r; \
+	echo ""; \
+	if [[ $$REPLY =~ ^[Yy]$$ ]]; then \
+		make confirmed_refresh_db; \
+	else \
+		echo "Aborting."; \
+	fi
+
+confirmed_refresh_db:
+	echo "Refreshing database."
+	docker compose down db
+	docker volume rm backend_postgres_data
+	make migrate
+
 test:
 	poetry run python -m pytest tests/
 

backend/alembic/versions/477cee72edc4_init_tables.py

@@ -106,4 +106,8 @@ def downgrade() -> None:
     op.drop_table("message")
     op.drop_index(op.f("ix_conversation_id"), table_name="conversation")
     op.drop_table("conversation")
+    # remove enum types
+    op.execute('DROP TYPE "MessageRoleEnum"')
+    op.execute('DROP TYPE "MessageStatusEnum"')
+    op.execute('DROP TYPE "MessageSubProcessSourceEnum"')
     # ### end Alembic commands ###

backend/app/chat/engine.py

@@ -157,7 +157,8 @@ async def build_doc_id_to_index_map(
         logger.debug("Loaded indices from storage.")
     except ValueError:
         logger.error(
-            "Failed to load indices from storage. Creating new indices.", exc_info=True
+            "Failed to load indices from storage. Creating new indices. "
+            "If you're running the seed_db script, this is normal and expected."
         )
         storage_context = StorageContext.from_defaults(
             persist_dir=persist_dir, vector_store=vector_store, fs=fs