To prevent frustrated tester, we should let buggy code to be generated instead of failing the project generation (if it failed on the linter) by appending a # TODO line on the code.
---------
Co-authored-by: majdyz <[email protected]>
* Introduce event logging, captured event: All non-Get HTTP operations.
* Refactor routes.py on each API: removed redundant try-catch for unknown exception, use JsonResponse
* Fix broken api_test.py due to DB seeding being removed.
---------
Co-authored-by: majdyz <[email protected]>
* Add debug commands to codex main module
* Refactor debug command and add WhatToDebug model
* Refactor debug command and add explore_database function
* Add Interview field to Specification model
* Removed need for resume points table
* Formatting od debug messages
* poetry lock update
* Add migrations
- Add `create_pyproject` and `poetry_lock` subroutines in packager.py
- Replace step to create requirements.txt with step to create pyproject.toml and poetry.lock
- Add default value `None` to `python_path` parameter on `execute_command` utility function
- Update generated Dockerfile and README.md for switch to Poetry
Apparently the generated DB schema is duplicated across every `CompiledRoute`.
Combining the schemas of N routes with M tables will result in N-fold duplication and `prisma generate` failure.
In turn, this breaks code fixing for server.py, resulting in broken deployments.
Tempfix, applied in this commit: use the schema from only one of the `CompiledRoutes`.
Long-term fix: clean up Codex' DB structure.
The hardcoded dependencies `fastapi`, `pydantic`, `uvicorn` and `prisma` need special treatment,
because if they are included in the `packages` of a generated function,
those requirements are not scrutinized during code validation, allowing invalid versions.
* fix(deploy): Deprecate `pipreqs` as a secondary package guesser
The develop stage already outputs a list of packages required by the application,
and `pipreqs` is unreliable.
- Remove usage of `pipreqs` from deploy.packager
- Remove `pipreqs` dependency
* tweak incantations regarding imports & requirements
* Add costs command to retrieve and display cost analytics
* Update imports in compile.py
* Fix import issue in compile.py
* Remove unnecessary code in compile.py
* Refactor compile.py: Format code and import statements
* Refactor code to use async/await for CodeValidator
Subprocess execution implementation (for Ruff & PyRight) is currently a blocking call. This doesn't work well with the event-loop used by the fastapi.
Scope:
Make AiBlock.validate async
Make subprocess execution async.
---------
Co-authored-by: majdyz <[email protected]>
Integrating PyRight project-based code-linting. This will allow us to detect:
* Misuse of async functions.
* Misuse of optional type and other annotated types (currently disabled)
* Wrong usage of pydantic and prisma model.
* Wrong usage of third-party library.
* Use of undeclared field
* Use of uninstalled third-party library (which will prompt LLM to add the requirement list)
Known trade-off:
Setting-up PyRight requires building a full-project (generate the Prisma model code, installing 3rd-party library, compiling the function with the object dependency) this slows down the code-validation during the code-validation step.
---------
Co-authored-by: majdyz <[email protected]>
Co-authored-by: SwiftyOS <[email protected]>
* Update Interview and LLMCallAttempt models
* Refactor LLMCallAttempt model
* Refactor code and update logging IDs
* Refactor AIBlock class and handle exception in start_interview
* Add template analytics
* Refactor analytics.py to use get_template_performance() instead of load_data()
* Add migration lock file and analytics functionality
* Pr coments
* Add model field to AIBlock and LLMCallTemplate
This is a large but a much needed change to allow more sophisticated static code analysis to be added.
PyRight (used by pylance) will allow us to detect:
Misuse of async functions.
Misuse of optional type and other annotated types.
Wrong usage of pydantic and prisma model.
Wrong usage of third-party library.
The overhaul is required to allow us retrying while fixing the code as a project (not only as a function) and allow us to keep adding non-LLM fixes to it so that we can avoid wasting the retry.
The scope of this PR is still a refactoring and the adding of the PyRight linter, so this PR try to keep the existing behaviour.
The enablement and more tuning on the linter will be done in the followup PR.
---------
Co-authored-by: majdyz <[email protected]>
Lookup on the existing database table is already implemented, but this bug sometimes can get in the way.
---------
Co-authored-by: majdyz <[email protected]>
* Re-apply improvements from bb43387, with additional fixes:
- Fix Prisma client generation
- Fix `DATABASE_URL` in docker-compose.yml
* Simplify mechanism to wait for database availability before starting app container (introduced in a1963c9 through #133)
- Remove deploy/wait-for-it.sh
* Add instructions to run app to generated README
- Add `generate_readme` function to packager.py
- Fix usage/formatting of block strings in packager.py
* Add docstring to generate_readme
* Fix instructions regarding .env
* Change author in generated README to "AutoGPT" (remove mention of Codex)
* Update README generated by packager.py
Remove `cd` steps to go into and out of `project` folder. This is no longer needed since requirements.txt is now at the root level of the package.
* Update instructions in packager.py
---------
Co-authored-by: SwiftyOS <[email protected]>
* Add Git user name and email to .env.example
* Add Git repository initialization and commit code in deploy step
* Extract git repository init to function.
* Update Git user configuration to only apply locally in newly initialized repository.
* Unset local Git user configuration after commiting
* Lint
- Remove `ffmpeg` from `apt-get install` step
- Fix and deduplicate `COPY` statements: exclude irrelevant files and optimize for caching
- Add `DATABASE_URL` to `app` service in `docker-compose.yml`
* Fix error handling in exec_external_on_contents function
* Add new models and update existing model in database schema
* Delete User-related models and enums
* Update code block delimiter in DatabaseGenerationBlock
* Refactor code for DatabaseEnums and DatabaseTable classes, and exec_external_on_contents function
* fix test
* Fix exec_external_on_contents function calls
* formatting
* Add database schema implementation requirements
Swifty