* chore(clickhouse): Capture final SQL in Sentry errors
* Upgrade Sentry and use the `clickhouse-driver` integration
* Turn `before_send_transaction` into `error_sampler`
* Update `escape_param_for_clickhouse`
* Fix Sentry init
* Revert "Turn `before_send_transaction` into `error_sampler`"
This reverts commit 8810b49416bf2cd8433a3ccd67a7684ea40d0f31.
* Fix order of pip installations in migration checks
* Revert "chore(CI): revert "chore: syphar is deprecated. fangle python actions…"
This reverts commit 1f2ff4ac32.
* not on depot?
* was missing a syphar removal :/
* was missing a syphar removal :/
* Update query snapshots
* back to depot
---------
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
* chore: catch bad indexes for CH migrations
* loggin instead of print
* update exit states
* include changes to ci
* add more details to logger
* git it
* checkpoint
* nicer logs
* don't use branch on ci
* handle git states more gracefully
* looking pretty good
* remove test migration
* fix: cannot read body twice
* only read it once no matter what
* fix tests but yucky
* nicer
* does this mean backend checks won't incorrectly run
* comment
* Upgrade pydantic and all related
* Upgrade mypy
* Add mypy-baseline
To update baseline when you fix something (only then!) use:
[mypy cmd] | mypy-baseline sync
* feat: Bump dev clickhouse version to 23.11
* add 23.11.2.11-alpine to ci matrix as well
* async migrations count of queries 4->5
* Update query snapshots
* Update query snapshots
* debug
* matrix async migrations
* update tests
* improve test assert to actually be able to debug it when failes
* update async migration tests
* sequential tests of operations
* new partition determination is different maybe?
* tweak
* omit inconsistent partition
* remove index 2 test (inconsitent and not important)
* Update query snapshots
* Update query snapshots
* remove comment
---------
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
* fix: Checkout master before checking for hogql changes
* fix: But go back to branch after done
* fix: Instead just diff with origin/master
* fix: Install libantlr, what's the worst that can happen?
* fix: Install antlr the hogql-way
* fix: Let's just checkout everything
* feat(batch-exports): Add backfill model and service support
* test(batch-exports-backfills): Add Workflow test
* chore(batch-exports-backfill): Bump migration
* feat(batch-exports): Add RedshiftBatchExportWorkflow
* feat(batch-exports): Add Redshift to BatchExport destinations
* feat(batch-exports): Support properties_data_type Redshift plugin parameter
* refactor(batch-exports): Insert rows instead of using COPY
* fix: Remove unused migration
* chore: Require aiokafka
* feat: Implement new structlog batch exports logger
* refactor: Use new structlog logger everywhere
* test: Add tests, fix things
* fix: Remove old tests
* chore: Change typing of return logger
* chore: Bump structlog
* chore: Extend docstrings
* fix: Don't use async logging as it's unsupported by temporal runtime
* test: Add logger tests
* fix: Mix pytestmark lists
* fix: Remove unused imports
* fix: Cleanup pytest warnings
* fix: Create and drop dataset for bigquery tests
* fix: Typing issue?
* fix: Let's just checkout everything
* fix: Use global event loop in tests
* fix: Blow-up cache
* fix: Truncate only if table exists
* revert-me: Skip all postgres tests
* fix: Connect to kafka in localhost
* fix: Lazily connect to Kafka
* fix: Resolve conflicts
* fix: Capture temporal context once and bind it to the logger
* fix: Make configure logger sync
* fix: Keep strong reference to background tasks
* fix: Continue consuming from log queue even if we fail to produce
* fix: Also catch the producer not starting
* fix: Remove unused await
* fix: Log kafka producer error after logger is configured
* feat(3000): Show the 3000 index-less navbar with labels
* Use new icons and move "Event explorer"
* Update UI snapshots for `chromium` (1)
* Update UI snapshots for `chromium` (2)
* Update UI snapshots for `chromium` (1)
* Update UI snapshots for `chromium` (1)
* feat: Allow new nav to be collapsed (#18343)
* Add alpha/beta tags, move "Toolbar", rename "Persons"
* Fix tooltips not showing when navbar collapsed
* Align "Session replay" in using sentence case
* Update UI snapshots for `chromium` (1)
---------
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Ben White <ben@posthog.com>
* Use ruff formatter
Ruff is now also a formatter! And it runs in a fraction of a second
across our whole codebase vs "I waited more than 30s and cancelled it
because I got bored".
* Config and command -> ruff
* Run 'ruff format .'
* Update query snapshots
* Run 'ruff format .'
* Fix format after commit hook
* Update query snapshots
* Fix type error - ignore comment moved
* Exclude hogql grammar from formatting and reverted grammar
* Run format
---------
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Tom Owers <tom@paace.co>
* Allow placeholders in place of ratio expressions in hogql
* Added C++ changes for RatioExpr
* Bump `hogql_parser` version
* Upgrade ANTLR version to align with brew latest
* Update query snapshots
* Fix parser change message body
The quotes weren't right, so the step failed instead of posting a comment.
* Revert "Update query snapshots"
This reverts commit 1302d3b24f138e36590cf5ff45769323379a4f95.
* Update MD5 sum
* Bump version again
* Use new hogql-parser version
---------
Co-authored-by: Michael Matloka <michal@matloka.com>
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Michael Matloka <dev@twixes.com>
* Add partial C++ parser of HogQL
* Support all the rules and add proper error handling
* Use `AlignAfterOpenBracket: BlockIndent`
* Reuse all the parser tests for the C++ backend
* Fix relationship between headers and implementations
* Add more error handling and fix minor issues
* Build both Python and C++ parsers in the package.json script
* Update ARRAY JOIN error assertion
* Improve timeit
* Move the C extension to the top level
* Refactor `vector_to_list_string`
* Build the parser on Linux
* Build wheels for the parser
* Simplify Linux build and fix macOS
* Fix Homebrew paths on x86 and don't fail fast in CI
* Set MACOSX_DEPLOYMENT_TARGET for C++20
* Set up QEMU for Linux ARM builds
* Publish the wheels on PyPI
* Avoiding Linux ARM emulation in CI for now
* Build sdist too
* Revert Dockerfile changes
* Fix PyPI publish
* Add README and optimize sdist build
* Use setup.py directly instead of build
* Use PyPI hogql-parser instead of local
* Also revert production-unit.Dockerfile
* Fix sdist upload and add Linux ARM back
* No Linux ARM build in the end
* Fix artifact uploading
* Do try building Linux ARM
We need this for prod.
* Use `npm` in `grammar:build`
`pnpm` is not available in that job.
* Fix formatting of hogql_parser
* Build everything on macOS
* Revert "Build everything on macOS"
Not so fast actually.
* Use hogql-parser=0.1.1
* Fix dylib in macOS wheel
* Bump hogql-parser version
* Fix missing module error
* Delete timeit.py
* Make error handling robust
* Format the C++
* Use `hogql-parser==0.1.1`
* Fix reserved keyword error assertions
* Use HEAD hogql_paresr in CI
* Fix `apt` usage
* Add some sudo in CI
* Ensure package will be releasable before build
* Bump version to 0.1.3
* Cover C++ `unquote_string` with tests
* Use BuildJet ARM runners for ARM builds
* Add some instructions
* Add HogQL version check to backend CI
* Update requirements.txt
* Use `setuptools` instead of the deprecated `distutils`
* Fix working dir in backend CI
* Align ANTLR versions
* Add test for "mismatched input"
This is thrown differently than other HogQLSyntaxExceptions in C++, so might help reveal what's going on with tests failing only on Linux CI and not macOS dev
* Add types and bump version
* Comment instead of failing version check
* Automate hogql-release version bump
* Fix checkout token
* Don't build hogql-parser if there were no changes
* Update query snapshots
* Update query snapshots
* Update query snapshots
* Update query snapshots
* Improve documentation
* Use new hogql-parser version
* Fix error start and end initialization
* Note `antlr4-cpp-runtime`
Co-authored-by: Marius Andra <marius.andra@gmail.com>
* Also remove NUL chars in C++
* Check ANTLR4 runtime archive checksum for security
* Note more decrefs to add
* Add vector size checks
* Use new hogql-parser version
* Don't support the `start` arg in C++ `parse_expr`
* Use new hogql-parser version
---------
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Marius Andra <marius.andra@gmail.com>
* ci-backend runs-on -> buildkite
* Use buildjet runners
* Go away old cache
* Fix deps
* Ensure to update the package registry
* Use bigger runners for django tests
* Use ARM for jobs with less concurrency
* Revert "Use ARM for jobs with less concurrency"
This reverts commit bcbf0d752412d4231b7b86c3413022b1d12facb4.
* Fix typo
* ci: fix waiting for temporal to be up in backend tests
I think it got merged before because we weren't running the backend
tests on these script changes so I've also added them to the list of
paths to watch for changes.
* increase timeout to 180 seconds
* chore: Remove ClickHouse 22.8 and settle on 23.4.2.11
* snuck a ci change in there - removed
* Update query snapshots
* Update query snapshots
* Update query snapshots
* Update query snapshots
* adjust service version requirements for clickhouse
---------
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
* feat(batch_exports): add backend API and S3 temporal workflow
This adds the backend API for batch exports, which will handle reverse
ETL exports to e.g. S3, Snowflake etc.
---------
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
* feature: bump clickhouse version to 23.4 for really exciting features
* remove default for kafka table
* Update query snapshots
* don't use defaults on kafka tables
* Update query snapshots
* clickhouse formatDateTime string has changed
* run both 22.8 and 23.4 for now
---------
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
