* everything except plugin server and sync_available_features
* sync_available_features_done, some plugin_server done?
* and a tiny bit more
* linting
* try to fix some tests
* more test fixes/
* clean up typos
* weed wacking bugs
* more test shenanigans
* fix plugin server
* actually fix plugin server test?
* still fixing tests
* another attempt
* some pr feedback
* small fix
* fix database query accessor
* fix functional tests
* fix tests
* Update query snapshots
* Update query snapshots
* Update query snapshots
* update some comments and fxn names
* fix plugin server test
* Update query snapshots
* Update query snapshots
* Update query snapshots
---------
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
* split out InternalPerson and Person types
* disable old backfill code
* add lazy person env var setting
* lazily create person/distinct_id rows
* remove backfill.ts
* drop properties after person lookup
* remove leftover backfill bit
* fix: for empty group props
We were failing if the props were empty.
* refactor: make property definitions db dependencies explicit
We were relying on implicit details setup via `resetTestDatabase` which
makes it hard to reason about what's going on. It also uses hard coded
ideas which limits our ability to isolate tests properly, which would
help both for writing tests with confidence that they are correct, and
for adding for example parallelism.
* add test
This removes the timekeeper library and uses jest fake timers instead.
This also creates the hub once and reuses it for all tests, which is
faster than creating a new hub for each test.
* add support for token field in kafka message
* formPipelineEvent
* rename pipeline files according to new order
* wip team_id and anonymize ips
* conditional handlers and tests
* some plugin server fixes
* fix capture bug
* fix
* more fixes
* fix capture tests
* pipeline update
* fix + investigate database resets
* fix import order
* testing and typing updates
* add test for capture endpoint
* testing
* python typing
* plugin server test
* functional test
* fix test
* another fix
* make sure no team ids clash in tests
* fix
* add more metrics and logs
* cache nulls
* updates
* add more metrics
* chore(plugins-server): use Kafka to buffer app jobs requests
To remove the dependency on the Graphile Worker database on things that
may be requesting app job runs we push the jobs to a Kafka topic.
* chore: use KAFKA_JOBS instead of string literal `'jobs'`
* chore: rename startJobsBufferConsumer -> startJobsConsumer
* avoid checking eventId
* fix lint
* fix producer wrapper tests
* fix retries test
* handle offset sync
* wip
* wip
* remove exports
* do better
* use Producer not wrapper
* reset db
* mock once
* Add test for raising to the consumer
* Update plugin-server/tests/main/ingestion-queues/run-async-handlers-event-pipeline.test.ts
Co-authored-by: Yakko Majuri <38760734+yakkomajuri@users.noreply.github.com>
* and in the darkness bind them
* fix tests
* don't forget the name update!
* rename DependencyError to DependencyUnavailable
* separate dlq
* update comment
Co-authored-by: Yakko Majuri <38760734+yakkomajuri@users.noreply.github.com>
* chore(plugin-server): use DELETE instead of TRUNCATE
Truncate seems a little slow. Other options to consider:
1. PostgreSQL fsync settings in tests
2. using tmpfs for "persistence"
3. use transaction/rollback: not totally sure we'd be able to do this
in our tests but may be worth a try.
* wip
* wip
* set fsync=off
* Delete all tables in current schema
* don't bother with fsync=off
