Commit Graph

70 Commits

Author SHA1 Message Date
Eli Reisman
a19e147c6a chore(dev): extract Rust services from Docker Compose into bin/start (#28681)
Co-authored-by: Eli Reisman <elireisman@github.com>
2025-02-18 00:57:03 +00:00
Daniel Escribano
7561539421 chore: split coordinator node setup for local development (#28293) 2025-02-04 17:37:40 +00:00
Daniel Escribano
bd296736bf refactor: rename workers to data nodes (#28277) 2025-02-04 13:45:18 +00:00
Daesgar
9b30f8dac2 feat: add support for coordinator schemas (#28031)
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2025-01-31 09:00:40 +00:00
Dylan Martin
c5762a3b34 feat(flags): more flags production-readiness (#26998) 2025-01-26 17:51:18 -05:00
Meikel Ratz
d4dbac3663 feat(cdp): remove legacy kafka based consumer (#27760)
Co-authored-by: Ben White <ben@posthog.com>
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2025-01-23 14:09:32 +01:00
Daesgar
75dea84d46 chore: kill query on initiator host when possible (#27161) 2024-12-30 15:34:47 +01:00
James Greenhill
1e4d4eca92 feat: start of clickhouse server version bump to 24.8 (#25189)
Co-authored-by: Robbie Coomber <robbie.coomber@gmail.com>
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Alexander Spicer <aspicer@gmail.com>
Co-authored-by: Daniel Escribano <daniel.e@posthog.com>
2024-11-25 19:09:49 -08:00
Oliver Browne
3f640c02c4 chore(propdefs): add propdefs to docker stack (#25153) 2024-09-24 21:32:07 +03:00
Sandy Spicer
7fa73a86af feat(insights): launch funnels as a Clickhouse UDF behind a feature flag (#23587)
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2024-09-05 11:40:33 -07:00
Ben White
524ebfffb6 feat(cdp): Masking destinations (#24266) 2024-08-14 13:36:57 +02:00
Frank Hamand
d32b035c49 feat: use capture-rs and vector replay capture in local compose (#24295) 2024-08-13 13:38:35 +01:00
Frank Hamand
09700cc16f feat: Add replay capture vector config+docker image and deployment (#24264) 2024-08-08 16:57:13 +01:00
Michael Matloka
edd76ca31b chore(dev): Live events locally (#22837)
* chore(dev): Live events locally

* Fix JWT secret var

* Also add Postgres URL

* Update UI snapshots for `chromium` (2)

---------

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2024-06-10 21:13:30 +02:00
Ben White
95a0b85848 feat: DeliveryHog (#22706) 2024-06-10 15:12:04 +02:00
Tom Owers
71905736f0 feat(data-warehouse): Added warehouse schema validation (#22142)
* WIP

* Added UI for showing invalid columns

* Added tests

* Updated object_storage references

* Corrected logic

* mypy updates

* Update query snapshots

* Added missing modifier to schema

* Fixed tests

* Updated mypy

* Update UI snapshots for `chromium` (2)

* Updated mypy

* typo fix

* Update query snapshots

* Update UI snapshots for `chromium` (1)

* Update UI snapshots for `chromium` (1)

---------

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2024-05-09 11:44:03 +02:00
Tomás Farías Santana
5bb8d098c8 fix: Wait for healthy db when starting temporal (#21610) 2024-04-17 14:55:01 +02:00
danielxnj
8b722bfef2 chore: Fix docker-compose circular dependecies (#20905)
Fix docker-compose circular dependecies

Co-authored-by: xneyder <xneyder@gmail.com>
2024-03-13 12:38:46 -04:00
Zach Waterfield
b841c438bc fix: remove the duplicate port config for temporal containers (#20836)
Remove the dup port config for temporal containers
2024-03-11 19:38:22 +00:00
Eric Duong
a1c21f9b2f feat(data-warehouse): integrating data warehouse with trends insight (#20320)
* wip with new datawarehousenode type in trendsquerybuilder

* split out query builder

* add test

* add property support

* add test for entity property and non entity property filter

* basic breakdown working

* typing

* typing

* more typing

* use default args

* more typing

* resolved mypy

* Update query snapshots

* remove config

* generate schema properly

* Update UI snapshots for `chromium` (2)

* add breakdown type

* Update UI snapshots for `chromium` (2)

* trim data warehouse query builder

* Update UI snapshots for `chromium` (2)

* Update UI snapshots for `chromium` (2)

* update schema.json

* add ci config

* try localhost

* mapping for linux

* Update query snapshots

* more typing

* types

* frontend typing

* backend typing

* add data warehouse logic path to funnel

* typo

* typing

* more typo

* more types

* sync

* Update UI snapshots for `chromium` (2)

* Update UI snapshots for `chromium` (2)

* Update UI snapshots for `webkit` (2)

* Update UI snapshots for `chromium` (2)

* finally

* Update UI snapshots for `webkit` (2)

* Update UI snapshots for `chromium` (2)

* Update query snapshots

* Update query snapshots

* add breakdown prop test

* update tests

* add test

* add type check

* Update query snapshots

* Update UI snapshots for `chromium` (2)

* Update UI snapshots for `chromium` (2)

* fix tests

* Update UI snapshots for `chromium` (2)

* abstract class

* Update UI snapshots for `chromium` (2)

* Update UI snapshots for `chromium` (2)

* Update query snapshots

* more tpying..

* Update query snapshots

* Update UI snapshots for `chromium` (2)

* Update UI snapshots for `chromium` (2)

* match signature

* match signature

* update test

* Update query snapshots

* add missing fields

* add schema

* update typing

* Update UI snapshots for `chromium` (2)

* Update query snapshots

---------

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2024-02-29 16:36:46 -05:00
Ben White
47b2882412 chore: Debugging options for V3 ingester (#20569) 2024-02-27 11:44:08 +00:00
Ben White
45c4443d05 fix: Add timeout to subscription reports (#19811) 2024-01-17 17:38:14 +01:00
Xavier Vello
886c80454a chore(compose): add capture-rs profile with containerised capture for testing (#18140) 2023-10-25 17:58:12 +02:00
Brett Hoerner
3089d276d0 fix: use prebuilt kafka 2.8.2 container that works on amd64 AND arm64 (#17366)
* Pull in bitnami Kafka 2.8 files

* Make Kafka container multi-arch and upgrade to 2.8.2

* Use local Docker container for Kafka

* Expose Zookeeper locally

* Use ghcr prebuilt images
2023-09-18 09:20:35 -06:00
Eric Duong
123470c445 chore(data-warehouse): Make sure alias is accounted for on s3 table (#16526)
* make sure alias is accounted for

* adjust test

* Update query snapshots

* Update query snapshots

* Update query snapshots

* Update query snapshots

* update tests

* add comment

---------

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2023-07-25 10:02:28 -04:00
Joe Martin
d1899c6005 fix: Removing some self-hosted upgrade paths and references (#16711)
* Remove self-hosted references

* Update messages in tests too

* Update query snapshots

* Update UI snapshots for `chromium` (2)

---------

Co-authored-by: Michael Matloka <michal@matloka.com>
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2023-07-24 19:44:44 +02:00
Tomás Farías Santana
8f77da2e18 fix: Temporal service waits for db to be healthy (#16079)
fix: Temporal dev service waits for db to be healthy
2023-06-16 07:43:14 +01:00
James Greenhill
0223d7ab86 feat: Add Temporal to the dev and hobby stacks (#14658)
* feat: Add Temporal to the dev and hobby stacks

* disable elastic for hobby because of resources

* checkpoint

* update requirements

* worker is up, but without the sandbox

* ensure temporal does not depend on elastic

* Feedbacked

* pip-compile dev

* mypy fixes

* add a bit of colorful logging

* add django temporal worker to the mix

* checkpoint for dev-full docker

* Working on docker-full, but checkpointing for now

* add migration bits for full
2023-03-20 16:56:14 -06:00
Paul D'Ambra
22f17540d3 chore: move dev postgres settings out of base dockerfile (#13294) 2022-12-13 13:25:18 +00:00
James Greenhill
696028e800 feat: simplify the docker-compose setup so we do less version coordinations (#12998)
* feat: remove version from docker compose to support new spec

* feat: simplify the docker-compose setup so we do less version coordinations

* update hobby bin

* bump docker-compose version for hobby for extends compat

* move ci to ubuntu-latest

* Revert "move ci to ubuntu-latest"

This reverts commit a0462adfecf182ca7398d809ebb49fac36110d63.

* use docker compose for github ci

* correct comments on base
2022-11-29 20:50:42 +00:00
James Greenhill
e0c887d3d7 feat: remove version from docker compose to support new spec (#12997) 2022-11-29 03:29:17 +00:00
Marius Andra
ddd15799da chore: increase dev db connection count (#12473) 2022-11-03 20:02:38 +01:00
Karl-Aksel Puulmann
87b5eee160 chore(ci): Set stream_poll_timeout_ms in dev (#12122)
This speeds up tests that rely on clickhouse ingestion (plugin-server) significantly. The previous default was 7500, setting this shaves off seconds off of every single test

We don't set this in production due to the extra load it causes.
2022-10-07 10:23:38 +03:00
Harry Waye
d7998cef30 Revert "chore(dev): use network mode host for docker-compose services (#10917)" (#10926)
This reverts commit 225a41db72.
2022-07-22 10:25:59 +01:00
Harry Waye
225a41db72 chore(dev): use network mode host for docker-compose services (#10917)
* chore(dev): use network mode host for docker-compose services

This removes the need to add kafka to /etc/hosts.

As far as I can tell this should be fine for peoples local dev except
they will be required to reset and re-migrate ClickHouse tables as they
will be trying to pull from `kafka` instead of `localhost`.

* remove ports from redis

* Update a few more references
2022-07-21 15:29:31 +01:00
Guido Iaquinti
cda6b83148 chore(Dockerfile): remove dev.Dockerfile (#10529) 2022-06-28 13:50:57 +02:00
Guido Iaquinti
451951d77d fix(dockercompose): pin service version (#10530) 2022-06-28 11:45:04 +00:00
Eric Duong
46faa8fab2 refactor(FOSS): foss remove all ee dependencies from /posthog (#10319)
* refactor: foss move properties to /posthog

* refactor: move replication and materializtaion

* refactor: move file

* refactor: move test and journeys

* refactor: move breakdown props

* refactor: move query imports

* refactor: move more ee dependencies

* refactor: restore groupsjoinquery

* fix: errors

* refactor: no ee.clickhouse dependncies

* try import tasks

* refactor: move materialization

* refactor: change foss split for column optimizer

* run black

* fix: imports

* remove comment

* Update snapshots

* run black

* skip isort

* Update snapshots

* format

* more fixes

* refactor(FOSS): split out paths query code (#10378)

* refactor: move migrations

* refactor: move idl

* fix: move more imports

* import adjustments

* fix: test import

* fix: test import

* fix: remove restriction

* refactor: split out paths query code

* refactor: more code splitting

* fix: types

* refactor(FOSS): Setup such that FOSS is deployable (#10352)

* refactor: move migrations

* refactor: move idl

* fix: move more imports

* import adjustments

* fix: test import

* fix: test import

* fix: remove restriction

* fix import

* refactor: add snapshost

* subscription-import

* fix: safe import

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-06-22 17:24:03 -04:00
Ben White
dd0695a24c feat(subscriptions): Insight Subscriptions (#10197)
* Added modal for subscriptions
* Added unsubscribe endpoint
* Added cronjob for sending subscriptions
2022-06-14 15:14:44 +00:00
Michael Matloka
9faad16402 ci: Fix crashing E2E tests (#10129) 2022-06-03 11:25:24 +02:00
Michael Matloka
64317238e6 refactor: Eliminate the KAFKA_ENABLED setting (#10059)
* refactor: Eliminate the `KAFKA_ENABLED` setting

* Remove dead code

* Consolidate plugin server test scripts and CI

* Fix CI command

* Remove Celery queues

* Rearrange test directories

* Update import paths
2022-05-30 18:39:33 +00:00
Ben White
57874f9db2 feat(exports): Dashboard / Insight exporting (#9830)
* Adds chromium / selenium for image exporting
* Added uploading of downloads folder to artefacts
* Adds ExportButton to generate desired asset
2022-05-27 14:31:17 +02:00
Guido Iaquinti
6e0feccdfd chore(docker-compose): remove what is not needed (#9945) 2022-05-26 11:36:44 +02:00
James Greenhill
2e69e0d4bb chore: bump defaults for clickhouse to 22.3 across the board (#9847)
* chore: bump defaults for clickhouse to 22.3 across the board

* remove docker build instructions for clickhouse

* remove 21.11.11.1 from matrix

* prettier it
2022-05-20 20:32:22 -07:00
Paul D'Ambra
49e3ceef5c feat(object storage): add unused object storage (#9846)
* feat(object_storage): add unused object storage with health checks

* only prompt debug users if object storage not available at preflight

* safe plugin server health check for unused object storage

* explicit object storage settings

* explicit object storage settings

* explicit object storage settings

* downgrade pip tools

* without spaces?

* like this?

* without updating pip?

* remove object_storage from dev volumes

* named volume on hobby

* lazily init object storage

* simplify conditional check

* reproduced error locally

* reproduced error locally

* object_storage_endpoint not host and port

* log more when checking kafka and clickhouse

* don't filter docker output

* add kafka to hosts before starting stack?

* silly cloud tests (not my brain)
2022-05-20 09:56:50 +01:00
James Greenhill
49c56fabe3 chore: Add ClickHouse version 22.3 to CI and set 21.11.11.1 to be default except for ARM64 compose (#9743)
* chore: Remove ClickHouse version 21.6 and add 22.3 for testing and docker-compose (21.6 is not in clickhouse dockerhub)

* update service requirements

* update arm64 compose as well

* Default to 21.11.11.1 and support multiple dockerhub repositories

* prettier .github/workflows/ci-backend.yml
2022-05-16 14:00:43 -07:00
Tiina Turban
35a5dd636e feat: Set redis maxmemory and memory policy for docker compose (#9573) 2022-05-05 14:17:32 +02:00
Guido Iaquinti
ac10b93348 ci(zookeeper version/image): use zookeeper:3.7.0 (#9638) 2022-05-04 16:07:54 +02:00
Guido Iaquinti
90ef8e2373 ci(redis version/image): use redis:6.2.7-alpine (#9620) 2022-05-03 15:35:26 +02:00
James Greenhill
bff0fb863c fix: Pin redis verison to 6.2 for docker-compose.dev.yml (#9561) 2022-04-27 21:52:00 -07:00