Commit Graph

116 Commits

Author SHA1 Message Date
Ben White
1b308916e8 feat(cdp): Refactored logger (#29933) 2025-03-17 12:55:25 +01:00
Ben White
81117d1d61 feat: Added simd json parsing option (#30005) 2025-03-17 09:59:17 +00:00
Ben White
5821ba14ec feat: Improve plugin server startup (#29425) 2025-03-07 11:43:02 +01:00
Ben White
d234167f80 feat(cdp): Mini purge of old ingesters (#29341) 2025-03-03 15:28:31 +01:00
Ben White
dc77ad8b01 chore(cdp): Removed job and schedules (#29352) 2025-03-03 13:31:54 +01:00
Ben White
e5c82e7927 feat: Producer refactor (#27672) 2025-01-21 11:12:51 +01:00
Michael Matloka
12ff477551 refactor(group-analytics): Add project field to group type (#25600)
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2024-11-01 00:12:28 +01:00
Oliver Browne
24d497ee7b feat: get property definitions out of plugin-server (#24843) 2024-09-09 11:15:49 +03:00
Oliver Browne
7207da93e9 feat: Inline plugins (plugins without running VM2) (#23443)
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Ben White <ben@posthog.com>
2024-07-24 14:49:31 +03:00
Tiina Turban
54875e188a fix: Webhooks groups properties (#23822) 2024-07-22 14:21:46 -06:00
Tiina Turban
60ba76d9c4 feat: Part 1 of ability to spin up different pipelines (#23601) 2024-07-17 14:11:49 +02:00
ted kaemming
1ed9426eb7 refactor: Remove legacy overrides handling from plugin-server (#23616) 2024-07-11 08:24:54 -07:00
Tiina Turban
9734894bc2 feat: Remove groups properties and created from each event (#22962) 2024-07-10 13:07:58 +02:00
Bianca Yang
6acfaf55f5 feat: Final piece of deprecating available features, DB references (#22402)
* remove db ref

* update the migration

* remove referencess from test fixture jsons

* some more removals

* Update UI snapshots for `chromium` (1)

* Update UI snapshots for `chromium` (2)

* Update query snapshots

* Update query snapshots

* Update query snapshots

* Update query snapshots

* Update UI snapshots for `chromium` (1)

* Update query snapshots

* fix migration

* add reverse sql

* fix the migration

* Update UI snapshots for `chromium` (2)

* Update query snapshots

* Update UI snapshots for `chromium` (2)

* Update UI snapshots for `chromium` (2)

* Update UI snapshots for `chromium` (1)

* Update UI snapshots for `chromium` (2)

* Update query snapshots

* Update UI snapshots for `chromium` (1)

* Update UI snapshots for `chromium` (2)

* Update UI snapshots for `chromium` (2)

* Update query snapshots

* Update UI snapshots for `chromium` (2)

* Update UI snapshots for `chromium` (2)

* Update UI snapshots for `chromium` (2)

* Update query snapshots

* Update UI snapshots for `chromium` (2)

* Update query snapshots

* Update UI snapshots for `chromium` (2)

* Update query snapshots

* Update query snapshots

* i think i finally fixed the migration issue

* Update UI snapshots for `chromium` (2)

* Update query snapshots

* Update UI snapshots for `chromium` (2)

* Update UI snapshots for `chromium` (2)

* update migration post merge

---------

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2024-05-30 16:41:22 +00:00
Bianca Yang
9dd22a3757 feat: Deprecate available_features pt 2 (#22373)
* everything except plugin server and sync_available_features

* sync_available_features_done, some plugin_server done?

* and a tiny bit more

* linting

* try to fix some tests

* more test fixes/

* clean up typos

* weed wacking bugs

* more test shenanigans

* fix plugin server

* actually fix plugin server test?

* still fixing tests

* another attempt

* some pr feedback

* small fix

* fix database query accessor

* fix functional tests

* fix tests

* Update query snapshots

* Update query snapshots

* Update query snapshots

* update some comments and fxn names

* fix plugin server test

* Update query snapshots

* Update query snapshots

* Update query snapshots

---------

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2024-05-28 17:29:24 -07:00
Bianca Yang
008698a785 feat: remove most frontend references to available_features (#22337)
* remove most frontend references to available_features

* some more

* Update UI snapshots for `chromium` (1)

* Update UI snapshots for `chromium` (2)

* adding back some items, fix tests

* Update UI snapshots for `chromium` (1)

* Update UI snapshots for `chromium` (2)

* fix test

* Update UI snapshots for `chromium` (1)

* Update UI snapshots for `chromium` (1)

---------

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2024-05-20 14:24:32 -07:00
Ben White
224a5d5d0c feat: Action steps refactor (part 2) (#22092) 2024-05-08 09:46:52 +02:00
Ben White
ee7ac6489f feat: Refactor processing to allow lazy loading of elementsList (#21983) 2024-05-06 10:47:24 +02:00
Brett Hoerner
634cd8308d chore(plugin-server): $process_person -> $process_person_profile (#21575) 2024-04-16 08:44:55 -06:00
Brett Hoerner
f4595d4225 chore(plugin-server): warn about invalid $process_person, add normali… (#21413)
chore(plugin-server): warn about invalid $process_person, add normalized field to properties
2024-04-09 07:28:38 -06:00
Brett Hoerner
feb6d65269 fix(plugin-server): minor personless changes (#21370) 2024-04-05 09:01:24 -06:00
Brett Hoerner
ea05d8e276 feat(plugin-server): handle process_person=false (#21262)
* chore(plugin-server): extract normalizeEvent method, since this has nothing to do with persons

* chore(plugin-server): cleanup PersonState args

* chore(plugin-server): lazily json serialize some timeoutguard context

* feat(plugin-server): handle process_person=false
2024-04-05 08:34:08 -06:00
Brett Hoerner
30bafdd382 chore(plugin-server): kafka ack cleanup and metric (#21111)
* cleanup: remove unused team arg from registerLastStep

* cleanup: rename promises to ackPromises to make it more clear thats what they are

* cleanup(plugin-server): make waitForAck explicit/required

* add Kafka produce/ack metrics

* Clarify Kafka produce metric/labels
2024-03-25 13:01:15 +00:00
Michael Matloka
f02d045bf9 chore(environments): Add migration to backfill projects (#20887)
* chore(environments): Add migration to backfill projects

* Fix `noop`

* Add `project_id` to plugin server test setup

* Fix `project_id`

* Also add `posthog_project` to plugin server tests

* Update `createTeam`

* Fix func tests
2024-03-14 13:45:28 +01:00
Tiina Turban
9c0802a541 chore: skip falky autocapture functional test (#20638)
chore: skip falky autocapture test

We're not really touching that code atm, this test flakes quite a bit, let's skip it.
2024-02-29 14:24:33 +01:00
Xavier Vello
6117206ea2 feat(ingestion): pass PLUGIN_SERVER_MODE as pg app name (#20613) 2024-02-29 10:34:28 +01:00
Marius Andra
4f50326aec fix(taxonomy): don't convert numeric strings to numbers (#19774) 2024-01-22 17:58:43 +01:00
Tiina Turban
e391387794 feat: Remove exportEvents (#18682) 2023-12-05 14:40:05 +01:00
ted kaemming
9299aa09e5 fix(plugin-server): Remove Postgres-based plugin error logging in favor of existing ClickHouse-based approaches (#18764) 2023-11-27 10:41:36 -08:00
ted kaemming
f342f35f33 test(plugin-server): Reduce flakiness of plugin teardown functional tests (#18896) 2023-11-27 07:51:39 -08:00
Xavier Vello
538a1be24f chore(CI): remove functional_tests/exports-v2.test.ts (#18549) 2023-11-10 15:59:23 +01:00
Tiina Turban
e3298f897f feat: p-s to support composeWebhook (#18465) 2023-11-09 18:19:18 +01:00
ted kaemming
9ade506258 feat: Log ingest warning on messages that are too large (#18318) 2023-11-03 07:56:46 -07:00
Xavier Vello
68fd523c35 fix(ci): fix flaky 'plugins can use attachements' test (#18341) 2023-11-02 15:40:33 +01:00
Brett Hoerner
d16408784f fix(plugin-server): fix unicode null byte blowing up the pipeline (#18282) 2023-10-30 16:39:10 -06:00
Brett Hoerner
286b689998 chore(plugin-server): remove INGESTION_DELAY_WRITE_ACKS and workerMethods (#17932)
* chore: stop using piscina worker methods for runEventPipeline

* chore(plugin-server): remove INGESTION_DELAY_WRITE_ACKS

---------

Co-authored-by: Tiina Turban <tiina303@gmail.com>
2023-10-16 09:19:49 -06:00
Paul D'Ambra
31c1cdf301 chore: yeet CH recordings ingestion (#17572)
Removing ClickHouse based recordings

One big yeet for a man, a great yeet for humanity
2023-10-11 14:23:41 +01:00
Brett Hoerner
d68fa14f10 chore(plugin-server): run node-rdkafka with cooperative rebalancing patched in (#17747)
* Remove node-rdkafka-acosom

* Add node-rdkafka

* Replace node-rdkafka-acosom imports with node-rdkafka

* Patch node-rdkafka with changes from https://github.com/PostHog/node-rdkafka/

* Add patch directions to README
2023-10-10 08:13:05 -06:00
Ben White
05079fa1a7 feat: Save app properties and others to Person from events (#17393) 2023-09-14 12:57:54 +02:00
Xavier Vello
1b6628055d feat(plugin-server): allow to use several PG connection pools (#17001)
Co-authored-by: Tiina Turban <tiina303@gmail.com>
2023-08-24 11:09:10 +02:00
Ben White
d8df34f4ab feat: Replay events consumer (#16642) 2023-07-20 14:41:25 +00:00
Harry Waye
f901665bfa chore: make sure dlqs exist in function tests before consuming (#16550)
In CI it's often the case that we get an error saying the
topic-partition pair doesn't exist. This creates the topic explicitly.
2023-07-13 10:49:01 +01:00
Harry Waye
06bd75ee1e chore(plugin-server): we split onevent and webhooks consumers (#16511)
* Revert "Revert "chore(plugin-server): we split onevent and webhooks consumers" (#16510)"

This reverts commit 59af5b904d.

* remove capabiliity

* fix load actions

* fix typing

* fix tests

* wip
2023-07-12 15:44:21 +00:00
Harry Waye
5b883cbda4 fix: historical test was missing a topic for one capture (#16493)
* fix: historical test was missing a topic for one capture

As a result we were processing out of order and the test was failing.

* remove comment
2023-07-11 14:37:35 +00:00
Harry Waye
9f0cf9f40f chore: make attachement test less flakey
We do this by making sure the plugin config and the attachement are
committed together.
2023-07-11 14:08:29 +00:00
Ben White
5a636f6bd1 feat: Optimise resource usage for blob ingester (#16478) 2023-07-11 15:11:36 +02:00
Tiina Turban
34f4f12d99 feat: backfill consumer (#16460)
* feat: backfill consumer

* Add functional test

* stop consumer

* fix test flake

---------

Co-authored-by: Harry Waye <harry@posthog.com>
2023-07-11 11:40:47 +00:00
Harry Waye
47fcd871bd chore: disable slack on zapier test (#16483)
There's a race condition in the test in that if we check the request
after slack has fired but before zapier has, we'll get a false negative.
2023-07-11 10:55:33 +00:00
Harry Waye
c85d94266c chore: disconnect consumer on error handling functional_tests (#16480)
If we don't we end up with a bunch of errors in the logs about imports
happening after jest tests have finished.
2023-07-11 10:40:40 +00:00
Harry Waye
c6a2449d3a chore(plugin-server): just remove jobs-worker test, it's flaky (#16319)
It's also an optimization as we also already do not schedule the job if
the plugin is disabled. It just means that if loads of jobs are
scheduled to graphile before the plugin was disabled then it will take a
while to get through.
2023-06-30 11:18:02 +01:00