steampipe/CHANGELOG.md

v0.12.3 [tbd]

Bug fixes

• Ensure SetConnectionConfig is only called once. (#1368)
• Fix plugin validation error sometimes causing Steampipe to crash. (#1387, #146)
• Fix plugin validation errors not being displayed as warnings on startup. (#1413)

v0.12.2 [2022-01-27]

Bug fixes

• Fix occasional Unrecognized remote plugin message errors on startup when running update checks. (#1354)

v0.12.1 [2022-01-22]

Bug fixes

• When running queries with csv output, "loading results..." remains on screen after displaying results. (#1340)

v0.12.0 [2022-01-20]

What's new?

• Update check to support template based export and output formats. (#1289)
• Add new check output format: asff (AWS Security Finding Format). (#1305)
• Add new check output format: nunit3. (#1196)

Bug fixes

• Fixes issue where plugins, FDW and Postgres were logging using a different timestamp formats. Now all timestamps use UTC (#927)

v0.11.2 [2022-01-10]

Bug fixes

• Fix issue where steampipe check table output only displays the summary. (#1300)

v0.11.1 [2022-01-06]

Bug fixes

• Plugin instantiation failures should be reported as warnings not errors. (#1283)
• Fix issue where database name is not printed in output of steampipe service start. (#1270)
• Fix issue where service is not shutdown if interrupted while interactive prompt is initialising. (#1004)
• Add support for installer to detect running service when upgrading. (#1269)

v0.11.0 [2021-12-21]

What's new?

• Add support for mod management commands: mod install, mod update, mod uninstall, mod list, mod init. (#442, #443)
• Startup optimizations.
  • When retrieving plugin schema, identify the minimum set of schemas we need to fetch - to allow for multiple connections with the same schema. (#1183)
  • Avoid retrieving schema from database for check and non-interactive query execution.
  • Update plugin manager to instantiate plugins in parallel.
  • Only create prepared statements if the query has parameters. (#1231)
  • Update Postgres driver to pgx. (This removes the need to query the database for the db connection Pid every time we execute a query.) (#1179)
  • Update connection management to use file modified time instead of filehash to detect connection changes. (#1186)
• Show query timing at the end of the query results. (#1177)
• Update workspace-database argument to handle connection strings starting with both postgres and postgresql. (#1199)
• Enables the tablefunc extension for the Steampipe database. (#1154)
• Improve plugin uninstall output when connections remain. (#1158)
• Disable progress when running in a non-tty environment. (#1210)
• Bump Go to 1.17
• Add support for protoc-gen-go-grpc 1.1.0_2

Changed Behaviour

• Only load pseudo-resources if there is a modfile in the workspace folder. (Note - a modfile can be created by running steampipe mod init). (#1238)

Bug fixes

• Update database planning code give required key columns a lower cost than than optional key columns. Fixes some complex queries with in clauses. (#116, #117, #124)
• Fix issue where local plugins are not evaluated as local as given in docs. (#1176)
• Fix nil reference exception during refresh connections when using dynamic plugins. (#1223)
• Fix issue where running service had to be stopped to install in a new install-dir. (#1216)
• Fix warning not being shown when running 'steampipe check'. (#1229)

v0.10.0 [2021-11-24]

What's new?

• Add support for parallel control execution. (#1001)
  • Only spawn a single plugin per steampipe connection, no matter how many db connections use it.
  • Share a single query result cache between multiple database connections.
• Add support for connecting to a remote database, including a Steampipe Cloud workspace database. (#1175)
• When cli displays error messages from plugins, they are now be prefixed with plugin name. (#1071)
• Do not show plugin error messages in JSON/CSV output. (#1110)
• Provider more responsive feedback for control runs. (#1101)
• Create prepared statements one by one to allow accurate error reporting and reduce memory burden. (#1148)
• Improve display of asyncronous error in interactive prompt. (#1085)
• Deprecate workspace argument, replace with workspace-chdir

Bug fixes

• Table names with special characters are now escaped correctly in auto-complete and .inspect. (#1109)
• Fix reflection error when loading a workspace from a hidden folder. (#1157)
• Fix intermittent crash when using boolean quals on jsonb columns. (#122)

v0.9.1 [2021-11-11]

Bug fixes

• Escape schema names when dropping connection schema. (#1074)
• Add support for quoted arguments with whitespace in query meta-commands (e.g. .inspect). (#1067)
• Fix issue where Postgres usernames weren't getting escaped properly when setting search path. (#1094).
• Add support to fall back to more (if available) where less is not available in the environment. (#1072)
• Non-turbot plugin installs now show link to documentation. (#1075)
• Constrain check table-output rendering to a minimum width to avoid rendering crashes. (#1062)
• steampipe check --dry-run should not display control summary. (#1053)

v0.9.0 [2021-10-24]

What's new?

• Update check command to support markdown and HTML output. (#480, #1011)
• Add support for plugins with dynamic schema - reload plugin schema on startup. (#1012)
• Add steampipe_reference introspection table. (#972)
• Add steampipe_variable reflection table. (#859)
• Add check summary in table output. (#710)
• Update DateTime and Timestamp columns to use "timestamp with time zone", not "timestamp". (#94)
• Add support for setting a custom database name when installing. (#936)
• Support JSON and YAML connection config. (#969)
• Allow plugin uninstall even if there are active connections. (#852)
• Control results are now ordered by status. (465)
• Add support for SSL certificate validation and rotation. (#1020)
• Remove deprecated flags --db-listen and --db-port from service start. (#582)

Bug fixes

• Plugin commands now exit with a non-zero code on error. (#980)
• Fix for incorrect message from service status when service is not running. (#975)
• Update introspection tables to ensure naming consistency - fix mods and pseudo resources to remove type prefix. (#959)
• Fix for plugin list failing with 'invalid memory address'. (#984)

v0.8.5 [2021-10-07]

Bug fixes

• Fix handling of null unicode chars in JSON fields. (#102)
• Fix issue where queries with alimit clause not always listing all results. Only pass the limit to the plugin if all quals are supported by plugin key columns. #103)

v0.8.4 [2021-09-29]

Bug fixes

• Update client error handling to only refresh session data for a 'context deadline exceeded' error. This avoids recursion in the error handling. (#970)

v0.8.3 [2021-09-28]

What's new?

• Update service start command to support database-password arg and STEAMPIPE_DATABASE_PASSWORD environment variable, to allow a custom password to be used when running in service mode. (#725)
• Small updates to output of steampipe service commands. (#812)
• Add support for piping stdout and stderr from service start to the TRACE log. (#810)

Bug fixes

• Update Docker image to remove password file. (#957)
• Fix filewatching to ensure prepared statements are correctly created and updated to reflect SQL file changes. (#901)
• Ensure session data is restored after a SQL client error. Reset SQL client after a failure to create a transaction. (#939)
• Fix service lifecycle management issues when state file is deleted while service is running. (#872)
• Fix issue where service stop shuts down service even if non-Steampipe clients are connected. (#887)
• Fix connection config not being passed when instantiating plugins to retrieve their schema. This resulted in descriptions not being shown for dynamic tables dynamic tables. (#932)
• Fix issue where install.sh fails for IPv6 enabled system. (#861)

v0.8.2 [2021-09-14]

Bug fixes

• Fix nil pointer error when running a fully qualified query (i.e. including mod name). (#902)

v0.8.1 [2021-09-12]

Bug fixes

• Disable database log polling, which was causing high CPU usage.
• Fix null reference exception for certain is null queries. (#97)
• Add support for CIDROID type when converting Postgres datums to qual values. (#54)
• Fix autocomplete casing for .cache metacommands. (#875)

v0.8.0 [2021-09-09]

What's new?

• Add HCL support for variables. (#754)
• Add HCL support for passing parameters to queries. (#802)
• Add completion command providing completion support for bash, zshell and fish. (#481)
• Add .cache metacommand to control the FDW cache from the interactive prompt. (#688)
• Remove hardcoded Postgres runtime flags by adding defaults to postgresql.conf (#767)
• Add support for syntax highlighting in interactive prompt. (#64)
• Update interactive prompt to use adaptive suggestion window instead of giving console window is too small error. (#712)
• Log Postgres output if database initialisation fails. (#800)
• Various minor UI tweaks. (#786)

Bug fixes

• Fix issue where the > prompt disappears when messages are shown from file watcher or asyncronous initialisation. (#713)
• Fix errors during async interactive startup leaving the prompt in a bad state. (#728)
• Fix for delay in loading results spinner showing, caused by asyncronous initialisation. (#671)
• Fix for missing control_description, control_title in csv output of check command. (#739)
• Fix for 0 exit code even if service start fails. (#762)
• Fix issue where configs referring to unavailable plugin will display incorrect error message. (#796)
• Mod parsing now raises an error if duplicate locals are found. (#846)
• Fix JSON data with '\u0000' resulting in Postgres error "unsupported Unicode escape sequence". (#93)

v0.7.3 [2021-08-18]

Bug fixes

• Retry a control run if the plugin crashes. (#757)
• Restart a plugin if it exits unexpectedly. (#89)

v0.7.2 [2021-08-06]

Bug fixes

• Fix issue where interactive prompt hangs with a ; input. (#700)
• Fix cancellation not working when database client becomes unresponsive. (#733)
• Prevent update checks from getting triggered for service stop. (#745)
• Add initializing spinner while waiting for asynchronous initialization to finish. (#671)
• Prevent interactive prompt from disappearing after asynchronous messages are shown. (#713)

v0.7.1 [2021-07-29]

What's new?

• Add open_graph property to steampipe_mod reflection table. (#692)

Bug fixes

• When an aggregator connection is evaluating a wildcard, only include connections with compatible plugin type. (#687)
• Fix search path not being honored by steampipe check. (#708)
• Fix interactive console becoming unresponsive after ";" query. (#700)
• Fix nil pointer exception in steampipe plugin. (#678)

v0.7.0 [2021-07-22]

What's new?

• Add support for aggregator connections. (#610)
• Service management improvements:
  • Remove locking from service code to allow multiple query and check sessions in parallel without requiring a service start.(#579)
  • Update service start to 'claim' a service started by query or check session, instead of failing. (#580)
  • Update service status - add --all flag to list status for all running services.(#580)
  • Update service start to add --foreground flag. (#535)
• Improvements for Docker:
  • Run initdb if database is installed but data directory is empty. (#575)
  • Split versions.json into 2 files, one in the plugins dir, one in the database dir. (#576)
  • Update plugin install to put temp files underneath the plugin directory. (#600)
  • Steampipe service startup now validates that the data-dir is writable. (#659)
• Optimise interactive startup by initializing asynchronously. (#627)
• Optimise query caching - construct key based on the columns returned by the plugin, not the columns requested.(#82)
• Update Steampipe service to support SSL. (#602)
• Show timer result before query output, so it is visible even if results require paging. (#655)
• Increase length of history file to 500 entries. (#664)

Bug fixes

• Do not disable pager when errors are displayed in interactive mode. (#606)
• Fixes issue where STEAMPIPE_INSTALL_DIR was not being respected. (#613)
• Fix multiple ctrl+C presses causing a crash on control runs. (#630)
• Ensure multiline control errors are rendered in full (#672)
• Fix crash when benchmark has duplicate children. Instead, raise a validaiton failure. (#667)
• Fixes issue where service stop does not work on Linux systems. (#653)
• Plugin schema validation errors should be displayed as warning, and not cause Steampipe to exit. (#644)

v0.6.2 [2021-07-08]

Bug fixes

• Revert prototype code inadvertently included in 0.6.1

v0.6.1 [2021-07-08]

What's new?

• Support executing control queries using the query command. (#470)
• Update steampipe-plugin-sdk reference version to support ProtocolVersion 20210701

Bug fixes

• Fix issue where dimension values were not rendered in generated CSV for check. (#587)
• Fix Linux Installer script showing verification error for Amazon Linux. (#479)
• Fix issue where using --timing with check was not showing duration. (#571)
• Fix problem where milliseconds of timestamps were not being displayed (#76)
• Fix freezing issues with 'limit' and cancellation. (#74)
• Fix incorrect caching of 'get' query results for plugins build with sdk >= 0.3.0. (#60)

v0.6.0 [2021-06-17]

What's new?

• Add csv output format to check command. (#479)
• Add --export flag to check command. (#511)
• Add --dry-run flag to check command to show which controls would be run. (#468)
• Add --tag and --where arguments to check command to provide filtering of the controls which are run. (#539)
• Update service status to make messaging more helpful when the service is running for a query session. (#531)
• Update query to add support for reading from STDIN. (#499)
• Validate that plugin versions required by the workspace mod are installed. (#557)

Bug fixes

• Update check exit code to be the number of alerts. (#498)
• Update check output formatting is now consistent when there is both a plugin and steampipe update. (#423)
• Fix failure to load SQL files from workspace folder if they include $$ escape characters. (#554)

v0.5.3 [2021-06-14]

Bug fixes

• Fixes Steampipe failing to run when too many benchmarks use the same controls. (#528)

v0.5.2 [2021-06-10]

Bug fixes

• Ensure consistent ordering of query result cache key when more than one qual is used. (#53)
• Fixes check command json output. (#525)

v0.5.1 [2021-05-27]

What's new?

• Update the check output to show the tree structure of the benchmarks and controls. (#500)

Bug fixes

• Fix issue where interactive prompt sometimes hangs on cancellation. (#507)
• Fix stack overflow error when allocating colors for large number of dimension property values. (#509)
• Fix query result cache key being built incorrectly when more than one qual is used. (#453)

v0.5.0 [2021-05-20]

What's new?

• New check command, to run controls and benchmarks. (#410, #413)
• Add resource reflection tables steampipe_mod, steampipe_query, steampipe_control and steampipe_benchmark. (#406)
• Parsing of variable references, functions and locals. (#405)
• Support for cancellation of queries and control runs. (#475)

v0.4.3 [2021-05-13]

Bug fixes

• Fix cache check code incorrectly identifying a cache hit after a count(*) query. (#44)
• Fix spinner displaying multiple newlines if spinner text is wider than the terminal. (#450)

v0.4.2 [2021-05-06]

Bug fixes

• Make .inspect column headers lowercase. (#439)
• Fix edge case where update notification may be displayed once when running in query batch mode, instead if being suppressed. This occurred the very first time an update check was performed. (#428)
• When checking for SDK compatibility of loaded plugins, use the protocol version, not the SDK version. (#453)

v0.4.1 [2021-04-22]

Bug fixes

• Ensure we report an error and do not start database service if port is already in use. (#399)
• Update check should not run when executing query command non-interactively. (#301)

v0.4.0 [2021-04-15]

What's new?

• Named query support - all SQL file in current folder (or the folder specified by the workspace argument) will be loaded and available to run as named queries. (#369)
• When running in interactive mode, a file watcher is enabled for the current workspace (can be disabled using the watch argument or terminal config property). When enabled, any new or updated SQL files in the workspace will be reflected in the available named queries. (#380)
• The query command now accepts multiple unnamed arguments, each of which may be either a filepath to a SQL file, a named query or the raw SQL of the query. (#388)
• The search path for the steampipe database service may be specified using the database config. (#353)
• The search path and search path prefix terminal sessions may be specified using terminal config, command line argument or meta-commands. (#353, #357, #358)

v0.3.6 [2021-04-08]

Bug fixes

• Fix log trimming, which was broken by the change of log location. (#344)
• Plugin updates should be listed alphabetically. (#339)

v0.3.5 [2021-04-02]

Bug fixes

• Fix .inspect not working with unqualified table names. (#346)

v0.3.4 [2021-04-01]

Bug fixes

• Ensure that after adding a connection, search path changes are reflected in the current query session. (#340)
• Fix extra trailing white-space issue in line output. (#332)
• Remove HTML escaping from JSON output. (#336)
• Fix issue where service is always listening on network listener. (#330)
• Fix incorrect error message when trying to update a non-installed plugin (#343)
• Fix the search path not being updated when removing the last connection. (#345)

v0.3.3 [2021-03-22]

Bug fixes

• Verify the steampipe foreign server exists when starting the database service and if it does not, re-initialise the FDW and create the server. (#324)

v0.3.2 [2021-03-20]

Bug fixes

• Remove Postgres synchronous_commit=off setting, which could cause FDW setup in Postgres to not be committed during setup (on Linux). (#319)
• .header terminal setting should also affect table output. (#312)

v0.3.1 [2021-03-19]

Bug fixes

• Fix crash when doing "is (not) null" checks on JSON fields. (#38)

v0.3.0 [2021-03-18]

What's new?

• Support setting Steampipe options using a config file. (#230)
• Add install-dir argument to specify location of the installation folder. (#241)
• Improve the handling of database quals. Query restrictions are now passed the plugin for a much wider ranger of queries including joins and nested queries. (#3)
• Improve handling and reporting of config parsing failures. (#307)
• Move the log location to ~/.steampipe/logs (#278)
• Change postgres log prefix to database- (#310)
• Deprecate db-port and listener arguments, replace with database-port and database-listener. (#302)

v0.2.5 [2021-03-15]

Bug fixes

• Fix crash when installing a plugin after a fresh install. (#283)
• Fix .inspect meta-command failure if no arguments are provided. (#282)

v0.2.4 [2021-03-11]

What's new?

• Autocomplete now includes public schema. (#123)
• Add bug report and feature request issue templates. (#266)
• Add SECURITY.md. (#266)
• Update spacing for plugin update and install messages. (#264)

Bug fixes

• Remove invalid update notifications for plugins which cannot be found in the registry. (#265)
• Fix typo in install.sh.

v0.2.3 [2021-03-03]

What's new?

• Increase timeout for plugin update HTTP call. (#216)
• plugin update now checks installed version of a plugin is out of date before updating. (#234)
• Improve the error messages for sql errors. (#118)
• Wrap plugin list output to window width. (#235)

Bug fixes

• Fix timestamp quals not being passed to plugin. (#247)
• Fix steampipe server not found error after failed connection validation. (#220)
• Ensure all panics are recovered. (#246)

v0.2.2 [2021-02-25]

What's new?

• Set Inspect column width to no larger than required to display data. (#155)
• Plugin SDK version check should ignore patch and prerelease version. (#217)
• Enforce reserved connection name ('public', 'internal'). (#168)
• Do not allow Steampipe to run from Root. (#167)
• plugin update, plugin install and plugin uninstall commands display error if no plugins specified in args. (#199)
• Remove global --config flag. (#215)

Bug fixes

• Fix cache retrieving incorrect data for multi-connection queries.(#223)
• Ensure search path is set for clients other than Steampipe. (#218)
• Spinner should not be displayed in non-interactive query mode. (#227)

v0.2.1 [2021-02-20]

Bug fixes

• Ensure all hydrate errors are reported. (#206)
• Change plugin update URL to hub.steampipe.io. (#201)
• Steampipe version string should include 'prerelease' suffix if it is set. (#200)
• Column headers in table output should respect casing of the column name. (#181)

v0.2.0 [2021-02-18]

What's new?

• Add support for multiregion queries. (#197)
• Add support for connection config. (#173)
• Add plugin update command. (#176)
• Add automatic checking of plugin versions. (#164)
• Add caching of query results. This is disabled by default but may be enabled by setting STEAMPIPE_CACHE=true NOTE: It is expected this will be updated to default to true in the next patch release. (#11)
• Log whether Steampipe is running in Windows subsystem for Linux. (#171)
• All env vars should have STEAMPIPE_ prefix. (#172)
• Display null column values as instead of an empty string. (#186)
• Validate that plugins do not have an sdk version greater than the version steampipe is built against. (#183)

Bug fixes

• Fix hitting a space after a meta-command causing runtime error. (#182)

v0.1.3 [2021-02-11]

What's new?

• Add 'line' output format. (#114)
• Log files older than 7 days are deleted. (#121)

Bug fixes

• Fix multi line editing issues. (#103)
• Fix command-Right breaking for unicode chars (#9)
• Fix 'no unpinned buffers available' error. (#122)
• Fix database installation failure for certain Linux configurations. (#133)

v0.1.2 [2021-02-04]

What's new?

• The .inspect command no longer requires the fully qualified name for tables. (#21)
• The helper function glob has been added. (#134)
• The output of the plugin install command now shows the installed version. (#93)
• The .help command now displays a link to the inline help docs. (#92)
• The wait spinner is now only shown in interactive mode. (#106)

Bug fixes

• Fix JSON and bool columns displaying as strings. (#95)
• Fix column headings displaying in upper case. (#94)

v0.1.1 [2021-01-28]

What's new?

• A new meta-command .help has been added. (#54)
• After steampipe plugin install, a link to the plugin docs is displayed.
• A spinner is now displayed for slow queries. (#77)
• A maximum column width of 1024 is now enforced - content longer than this will wrap. (#12)
• The description column of the .inspect command now fills the available horizontal screen space. (#11)
• The Linux installation package now uses tar instead of zip. (#63)

Bug fixes

• Fix results paging failure for very long rows (> 64k chars). (#75)
• Fix invalid query resulting in the database session remaining open. (#60)
• Fix data formatting in json output. (#14)
• Fix incorrect plugin hub link.
• Fix steampipe query panic when exiting after service stopped --force has been run. (#38)
• Fix runtime error: slice bounds out of range [1:0]. (#40)
• Fix boolean meta-command showing wrong status when no parameter is passed. (#48)