tinybird-cli 5.9.0

Tinybird Command Line Tool

The Tinybird command-line tool allows you to use all the Tinybird functionality directly from the command line. Additionally, it includes several functions to create and manage data projects easily.

Changelog

5.9.0

• Added tb pull has a new –fmt flag to apply the same formatting as tb fmt when pulling a datafile.

• Changed tb auth info and tb auth ls now show the region API host properly.

• Removed misleading output message when tb auth –token with TB_TOKEN env var also defined.

• Fixed error messages when performing operations without being authenticated or using a token.

• Fixed tb auth uses explicitly the –host if provided.

5.8.1

• Improved error message of tb push when pushing .datasource files.

5.8.0

• Added tb datasource scheduling commands to manage the scheduling of a Data Source

• Added tb tag commands to manage tags from the CLI.

• Added support to TAGS in tb fmt.

• Added support to TAGS in tb pull and tb push. Allows tagging resources for filtering in the UI.

5.7.0

• Changed support prod as main git branch name when running tb init –git

• Fixed issue with tb auth ignoring the token value when using –region and –token options together.

• Fixed tb auth not getting the proper value when using the TB_TOKEN env var.

5.6.0

• Added Add IMPORT_FROM_TIMESTAMP to the list of supported directives in the CLI for S3 Data Sources, files added to the S3 bucket before that time will be ignored. [Read our docs for more information.](https://www.tinybird.co/docs/ingest/s3)

• Fixed region list for AWS and custom regions when usign tb auth –interactive

5.5.0

• Added Support gte and lte in sql_and directive. See https://www.tinybird.co/docs/cli/advanced-templates#sql-and

• Changed info and error messages when using “user token” vs “admin [e-mail]” token

• Fixed regions list when usign tb auth –interactive, tb auth ls and tb auth info

5.4.0

• Added support to Python 3.12

• Added support to Stream Pipes in tb push and tb pull commands. (Feature in development. It requires access)

• Added json and jsonl as supported formats for S3 Data Sources

• Changed tb init –git now supports run from a develop branch in git

• Fixed Do not try to deploy shared Datafiles from the vendor folder on tb deploy

5.3.0

• Added tb push now supports dynamodb as service type

• Added tb connection create now supports dynamodb as service type

• Added error when trying to push a data source with SETTINGS instead of ENGINE_SETTINGS

• Added support for buckets with gzip files when creating S3 Data Sources

5.2.1

• Added tb push prevents pushing pipes that have nodes with the same name as the pipe.

• Changed helper messages to rename Token to Static Token

• Fixed error messages when using tb push with already taken names for other resources.

5.2.0

• Added internal changes

• Added tb pipe unlink command

• Changed No need for –node param in tb pipe populate

5.1.0

• Added COPY pipes now support COPY_MODE replace.

• Added improve checks for dependencies in pipes when altering Data Sources.

5.0.0

• Breaking change Make insertion_date column explicit. This column is no longer inferred, it must be present in the Data Source file.

  Detailed info at https://www.tinybird.co/docs/changelog/2024-06-17-insertion_date-deprecation

• Added parameter to tb pipe regression-test –relative-change

• Added tb push displays warnings when using a reserved parameter in a pipe.

• Added –role parameter to tb workspace members add

• Changed Point region hosts to new app.tinybird.co domain

• Improved help message of tb workspace members set-role now displays the 3 valid roles: viewer|guest|admin.

• Improved syntax error messages when parsing endpoints.

4.1.1

• Fixed workspace info gathering when switching branches. We were prioritizing the general workspace properties over the user ones.

4.1.0

• Added tb token create command to be able to create static and JWT tokens from the CLI. You can check more information at https://www.tinybird.co/blog-posts/jwt-api-endpoints-public-beta

• Fixed tb init –git to pin tinybird-cli>=4,<5 in requirements.txt to avoid issues with the latest version of the CLI.

4.0.0

This is a major release, please read the commands affected below and consider updating your scripts and workflow before upgrading to this version.

• Deprecated –semver flag and tb release commands are now deprecated. You can keep using tb deploy to integrate and deploy from git. Changes are deployed to the main Workspace instead of to a Release.

• Removed –cicd flag and CI/CD templates generation from tb init. You can still use the git integration, just create your own pipelines. You can use the ones in this repo as an example https://github.com/tinybirdco/ci

• Removed tb env command is removed, use tb branch instead.

• Deprecated .datasource files with ENGINE “Join” is deprecated, use Engine “MergeeTree” instead.

• Deprecated tb materialize

• Removed Drop the –timeout flag from tb push which made the populate job to timeout. You can use now –wait to wait for the job to finish or nothing to just create the job and return.

• Removed Support for KEY directive is removed. The KEY was used to create a Data Source with Join engine by the given KEY column name. Join engines are also deprecated, you can use a regular MergeTree Data Source instead and adapt the pipes SQL accordingly.

3.12.0

• Added new “us-west-2” region to tb auth ls

• Fixed “Appended 0 new rows” message when appending NDJSON

3.11.0

• Added confirmation step to tb datasource sync command

• Fixed support for KAFKA_STORE_BINARY_HEADERS for new Data Sources. Only relevant when KAFKA_STORE_HEADERS is also enabled, used to store Kafka headers as binary data (default option from now on).

• Fixed Report error when tb push –force –yes cannot apply a change in a .datasource schema

• Fixed Skip error when using tb push –only-changes and no local .git directory is present

3.10.0

• Added support for unlink materialized pipes doing tb workspace clear

• Added support for adding multiple tokens when pushing a .datasource file

• Fixed s3 iamrole connection creation will not fail when pbcopy dependency is not available

• Fixed Fixed a bug on data branching

• Fixed Support ngram index

3.9.0

• Added Support for connection names when doing tb connection rm

• Added New –policy option for create s3_iamrole command that will generate different hints depending on the case

• Fixed Avoid system vars evaluation when doing tb fmt

• Fixed environment variables substitution for Data Source engine parameters.

3.8.0

• Added Support for S3IAM role for ingest

• Changed Upgrade clickhouse-toolset to 0.30dev0

• Changed Decouple cli from connector_settings module

3.7.0

• Changed fixed major version of tinybird-cli to lower than 4 when using tb init –git

• Changed behavior when running tb deploy on a branch to push the connection settings to the backend. This change is the backend that decides what to do.

• Changed tb pipe sink commands are now available

• Fixed regression tests query when filtering by specific parameter

• Fixed Avoid tb fmt to error if there’s a CASE in the sql

• Fixed Some issues when using –fmt: off with CASE in the sql

• Fixed tb deploy supports deployment when a .incl file is removed

• Fixed Report better error message on tb diff when two Datafiles have the same name

• Fixed Added correct URL for checking available regions

• Fixed Support .incl files with environment variables in tb fmt

3.6.0

• Added Add new envvar TB_INCL_RELATIVE_PATH in .tinyenv to look for Datafiles out of the Data Project directory

• Changed On-demand Sink run checks the pipe type instead of checking the workspace connectors

3.5.0

• Added Token management in Copy pipes datafiles

• Added Better error report on tb push when trying to push a pipe with bad templating

• Added credential validation for S3 connections

• Added tb connection create now supports s3_iamrole as service type

• Changed error message when trying to modify a datasource without bumping the major semver version to add a link to the documentation.

• Changed GH workflow files and secret includes the workspace name when generated from the UI and CLI

• Changed rename INDICES to INDEXES

• Changed Save git commit to workspace on tb push –only-changes

• Fixed Fix error message when deleting an .incl file with the git workflow

• Fixed support tb deploy –dry-run

3.4.0

• Added Support tb deploy .datasource with INDEXES

• Added Support tb push .datasource with INDEXES

• Added Support tb deploy in a post release of a .datasource with INDEXES

• Added Check backfill on preview is required when tb deploy modified datasources. Disable check with TB_CHECK_BACKFILL_REQUIRED=0.

• Changed Allow spaces when managing multiple Workspace members

• Changed tb deploy to not allow pushing materialized views with datasource definition in the same datafile. We are blocking this as has unsupported behavior and it misleading and less flexible than having 2 files one for the datasource and one for the materialized view.

• Changed tb deploy to deploy the datasources from top to bottom to make sure that if we modified the column in some datasource in the top. It will be deployed before the datasources that depend on it.

• Changed Post-release required when deploying changes that include changing a Copy Pipe.

• Changed script exec_test.sh from tb init –git to run retries to avoid flakiness in tests

• Changed Update tb release rm –oldest-rollback to use new API

• Fixed Post-release recommended instead of required when deploying changes that include changing a Copy Pipe. Displays a warning when other number of the semver is increased.

• Fixed Fixed bug when a Data Source was created using an INDEX

• Fixed Avoid None git diff change type

• Fixed tb deploy to deploy the endpoints in the correct order when there are endpoints depending on other endpoints.

• Fixed tb push pipe with multiple tokens

• Fixed IMPORT and ENGINE changes were not detected tb deploy

• Removed part of the code supporting python 3.6 as we stopped giving support 18 month ago

3.3.0

• Added Show warning when doing data or pipe operations directly against Live.

• Added bi_stats, bi_stats_rt and endpoint_errors to the list of service data sources that the CLI ignore when looking for dependencies of pipes being pushed.

• Added S3 Iam Role connection creation

• Added Connector credential validation

• Added Data quality tests inside a skip_in_branch folder inside the tests folder are skipped when running on a branch. Use this to skip tests in the CI piipeline that should only run with production data.

• Changed Stop creating endpoints folder on tb init

• Changed Small refactoring of the connector credentials

• Fixed Progress bar in tb init –git.

• Fixed Don’t do diff when there are no modified resources on tb deploy.

• Fixed Upgrade GitPython dependency to 3.1.32. That version fixes the security vulnerability CVE-2022-24439 <https://github.com/advisories/GHSA-hcpj-qp55-gfph>

• Fixed Homepage URL in PyPI

3.2.0

• Changed tb init –git uses v3.1.0 of the CI templates (See release notes: https://github.com/tinybirdco/ci/releases/tag/v3.1.0)

• Changed Allow alter data source with JSON paths to add DEFAULTS if the data source already has any (See release notes: https://github.com/tinybirdco/ci/releases/tag/v3.1.0)

3.1.0

• Added tb init –cicd to generate CI/CD files not only when using –git. Combine with –force to overwrite

• Added Ignore format changes on tb deploy by default

• Changed tb init –git and tb release generate include set -euxo pipefail in the generated bash scripts

• Changed tb init –git uses github.com/tinybirdco/ci => v3.0.0

• Changed Internal changes to allow parametrize fmt

• Changed tb release promote automatically deletes de oldest rollback Release if export TB_FORCE_REMOVE_OLDEST_ROLLBACK=1

• Changed option –node in tb pipe populate to required as it is already mandatory to specify the node to populate. Now, it will provide an error message if the –node option is not provided.

• Changed Better error report on tb fmt –diff. Use it in CI to standardize Datafile formatting

• Changed tb init –git generates a requirements.txt file to install the latest tinybird-cli release from PyPI

• Changed tb deploy shows an example when failing because of a missing –semver argument

• Fixed When parsing the response of a CSV tb sql command, the CLI was interpreting the result as an error in some cases.

3.0.0

Major version updated to 3.0.0 due to the change in the logic of tb deploy. From this version on, tb deploy promotes Releases to Live or Preview based on the semver version.

See our Version Control guides for more details about our semver deployment behaviour.

• Added syncing status to git remote settings

• Added tb release rm –oldest-rollback –force –yes deletes the oldest rollback Release by creation date

• Changed Updated logic in tb deploy

• Changed Updated scripts/exec_test.sh generated from tb init –git

• Changed Pointing to versions.tinybird.co for docs pages related to Versions

• Changed Initial git files to include mandatory directories

• Changed Unify generation of ci/cd script resources

• Changed Update clickhouse-toolset to 0.29.dev0

• Changed When removing a Release, the CLI will now inform if Share Data Sources are still being used in a Materialized View in other Workspaces.

• Changed Update CLI to generate ci/cd resources: new append_fixtures.sh and updated exec_tests.sh

• Changed Update tb release generate to generate deploy.sh and postdeploy.sh with instructions on how they work

• Changed .tinyenv default template includes all possible vars

• Changed behaviour of tb env regression-test when no requests are found. Now, it will show a warning message and will not fail the command.

• Changed Add usage validation on tb release rm and allow –force to avoid the validation

• Changed Support raw content on datafile parses

• Fixed tb deploy now fails with exit 1 in case a release limit has been reached.

• Fixed tb deploy validates post –semver

• Fixed append_fixtures.sh script to exit with 1 when there’s an error

• Fixed Error while pushing Sink Pipes using IAM Role Connector for S3

• Fixed Deploy pipes with shared Data Sources using tb deploy

• Fixed Bug on tb deploy which did not preserve a proper order when deploying resources

• Fixed Small error message misspell. The argument to use the User token in certain operations like sharing a Data Source is –user_token and was written as –user-token.

• Removed –all argument from tb branch

2.2.0

• Changed tb init –git update to get ready to use cicd with releases

• Changed Use tb branch instead of tb env. tb env is still usable but deprecated and will be removed in the next version.

• Fixed command tb pipe regression-tests to handle the edge case when all the requests to the endpoint that we want to check fails as the value returned is no what we expect.

• Fixed tb deploy fixes to work with –dry-run and push changes of new and modified files

• Fixed tb check doesn’t exit with error when checking a .incl file.

• Fixed tb fmt adds empty quoted strings in .datasource connection section.

• Fixed Be smarter in tb fmt when detecting if .incl file contains Data Source or Pipe definitions.

• Fixed tb env use when using a token directly, without a .tinyb file.

2.1.0

• Changed exec_test.sh now runs with parallel -j 4 to speed up CI cycles

• Changed tb release rollback now requires double confirmation

• Fixed Ignore .diff_tmp/ on tb deploy

• Changed Regression tests select faster the requests used for the checks.

• Improve peformance when using the option –only-change in the tb push command. This was a regression introduced in the version 2.0.0

• Added Add option c or –concurrency to tb run test to run tests in parallel

• Removed hidden option –only-changes from tb deploy as it’s no longer required.

• Changed Speed-up CLI startup by lazy-importing some dependencies.

• Added tb auth shows and uses the new AWS regions

• Added Add –auto-promote to tb deploy to automatically promote the Release created. It automatically applies to Environments created by CI.

• Added Support TAGS “backfil_column=<column_name>” in .datasource definition to do backfill operations when running a populate job

• Changed Regression tests now also use Pipe requests done with the POST HTTP method.

• Changed tb env commands to use the new CLIConfig class to manage the authentication and the configuration of the CLI. This change allows us to have a better control of the configuration and the authentication of the CLI. This should be transparent for the users.

• Changed Internal changes in tb deploy

• Added tb deploy removes resources from datafiles deleted

2.0.0

Released new version 2.0.0 with all these changes:

• Changed All commands exit with 1 when there’s an exception

• Removed Prefixes are not supported anymore ([context](https://www.tinybird.co/changelog/2023-10-09-versions), and [how to migrate](https://www.tinybird.co/docs/guides/staging-and-production-workspaces.html#migrating-from-prefixes))

• Removed tb pipe create

1.3.0

Released new version 1.3.0 with all these changes:

• Added Support for Copy pipes in tb fmt

• Added Support for .datasource.incl files in tb fmt

• Fixed tb auth use to move between regions saved in the config file

• Fixed Skip regression.yaml in tb test run since it’s reserved for regression tests

• Fixed Skip replacing .incl files in tb fmt

1.2.0

Released new version 1.2.0 with all these changes:

• Changed Upgrade CICD version templates for tb init –git

• Changed Support tb release commands in Environments

• Fixed Correct paths for triggering CICD generated by tb init –git

• Fixed rolled back the version of the snowflake-connectors-python package.

• Fixed Do not print data branching summary in tb env create <env_name>

1.1.0

Released new version 1.1.0 with all these changes:

• Added Support skip individual regression tests through the regression.yaml configuration file. CI/CD guide at https://www.tinybird.co/docs/guides/continuous-integration.html

• Added Fail tb env regression-tests if no tests are run for any pipe, use skip to avoid the error

• Added Skip POST requests by default in tb env regression-tests until it’s supported

• Added Support append –fixtures to shared Data Sources in Environments

• Added support –assert-max-time in tb env regression-tests so that serves as threshold for response time and avoid flakiness on fast endpoints.

• Added support tb env regression-tests –main -f regression.yaml so you can run regression tests agains the main Environment together with a configuration file.

• Added tb env regression-tests coverage –main runs coverage regression tests in the main Environment. This command is experimental and undocumented.

• Changed tb init –git –folder <folder> includes the <folder> in the CI/CD templates as data_project_dir and in the paths to track changes so CI and CD only run when there’s a change on that folder. If <folder> is not provided then it is set to the current directory .

• Changed Force tb init –git to run in the main or master git branch

• Changed the creation of the CD/CI workflow when running tb init –git to use the admin workspace token instead of the token of your admin user.

• Changed Expose resources and ids when removing releases tb release rm

• Changed –override-commit option to not require needing to pass the –git option as well. Now, running tb init –override-commit should work as expected.

• Changed Copy and Sink on-demand jobs now return the job URL.

• Changed allow Workspace admin tokens for CI/CD workflows. Previously, you were asked for user admin token. You can update your TB_ADMIN_TOKEN to be the Workspace admin token instead the admin token from a specific user. CI/CD guide at https://www.tinybird.co/docs/guides/continuous-integration.html

• Changed Improve CI/CD output. Added warning hint in custom deployments.

• Fixed tb deploy raises an error if it detects resources renamed in git

• Fixed On tb push –force –yes a BigQuery Data Source, it updates both the SCHEMA and connector settings.

• Fixed Connection settings of BigQuery Data Sources are preserved in Environments, although the connection does not sync by design, to ingest data use –fixtures

• Removed warning message that recommended not using the admin token for CI/CD workflows as we introduced a fix in the version 1.0.1 to avoid the problems that the admin token was causing.

• Removed fallback when running tb pipe regression-test does was doing requests to /v0/pipes/XXX/requests as it was mostly not used. This endpoint was being called when there weren’t requests in tinybird.pipe_stats_rt in the last 7 days. This endpoint was really heavy and most of the time wouldn’t return any actual data as endpoint not being called after 7 days most likely are not in production.

• Removed async parameter in the /v0/environments requests as it’s not being used.

1.0.1

Released new version 1.0.1 with all these changes:

• Added Google Cloud Storage as a Data Source connector (BETA)

• Added support for fixtures using Parquet format. Also, we have modified the message when no fixture is found. Before it would show something like ** Warning: datasources/fixtures/current_events.ndjson file not found as it was the last format was looking for, now it will display ** Warning: No fixture found for the datasource current_events

• Added tb deploy output now exposes what is detecting for deploying

• Added tb env regression-tests commands now support flag –skip-regression-tests or export TB_SKIP_REGRESSION=1 to skip execution of regression tests. Handy for CI environments were regression tests might be flaky

• Changed tb init –git uses v2.3.0 of the CI/CD templates. See [release notes](https://github.com/tinybirdco/ci/releases/tag/v2.3.0)

• Changed tb auth info to give better information in case of authentication problems

• Changed When running tb init –git for the first time the exec_test.sh script to run fixtures tests now includes an environment variable export TB_VERSION_WARNING=0 to skip the CLI version warning

• Changed Support OBFUSCATE_REGEX_PATTERN and OBFUSCATE_PATTERN_SEPARATOR environment variables to define a regex pattern and a separator (in case of a single string with multiple regex) to obfuscate secrets in the CLI output

• Changed Update clickhouse-toolset to 0.28.dev0

• Fixed Skip error when tb deploy –populate –wait and a copy pipe has been deployed

• Fixed Typo when there is an error with a connection

• Fixed that all the HTTP requests follow the environment variable TB_DISABLE_SSL_CHECKS. This environment variable allows users to disable the SSL verification if needed. By default is not disabled. Before this version, some HTTP request were not disabling the SSL validation

• Fixed tb env rm and tb env use when using the admin workspace token. Before when using these command and using the admin token, this command would return an error saying that the workspace was not found. Now, these commands should work as expected

• Fixed tb connection ls now lists BigQuery connections

• Fixed tb pull and tb diff now respect multiline in IMPORT_QUERY

1.0.0

• Released Version 1.0.0, from now on the tinybird-cli package uses the standard semver convention for stable versions. Development versions will be tagged as with the .devX suffix where X is an integer number.

1.0.0b448

• Changed Better error feedback when Environment is not ready for tb deploy

1.0.0b447

• Added tb datasource delete new option dry-run to check number of maching rows before execute

1.0.0b446

• Added tb token ls to list all the available tokens

• Added tb token copy to copy a token to the clipboard

• Added tb token rm to remove a token

• Added tb token refresh to refresh a token

• Added tb token scopes to list the scopes of a token

1.0.0b445

• Changed tb env create waits job by default. Use –no-wait in case you wan’t to avoid waiting.

• Changed Better error feedback for tb deploy when Environment creation is not completed

1.0.0b444

• Fixed: Data Sources definition: Default column modifier not allowed when there’s json paths

1.0.0b443

• Changed Bump CI/CD release to v2.2.0 => https://github.com/tinybirdco/ci/releases/tag/v2.2.0

1.0.0b442

• Fixed Check if not main when creating an Environment

1.0.0b441

• Changed tb token –git unhidden

1.0.0b440

• Changed tb token rm needs to be confirmed or run with –yes flag to be removed

1.0.0b439

• Added References to the latest release of our CI/CD actions

1.0.0b438

• Fixed tb diff just using progressbar in tb init –git

• Fixed tb env rm returns proper error when trying to remove from another Environment

• Changed tb auth better output suggesting to use region host

1.0.0b437

• Fixed tb check compatility with tokens Datafiles

• Fixed tb token rm returns proper error when token does not exists

1.0.0b436

• Changed Internal changes

1.0.0b435

• Fixed tb init –git fails if diffs detected and not executing tb pull –auto –force

1.0.0b434

• Changed Versions Gitlab ci/cd pipeline to ensure temporal envs are removed

1.0.0b433

• Addded tb deploy exposes git diff output before pushing resources.

1.0.0b432

• Changed multiple dependencies to be less strict:

  • click: From ==8.1.6 to >=8.1.6,<8.2

  • cryptography: From ==3.4.8 to >=3.4.8

  • pyyaml: From ==6.0 to >=6.0,<6.1

  • requests: From ==2.28.1 to >=2.28.1,<3

  • urllib3: From ==1.26.14 to >=1.26.14,<2

1.0.0b431

• Changed tb pull defaults now to –auto True, that means that it preserves the Data Project default folder structure. You can still pull all files to the current folder by using the –no-auto flag.

1.0.0b430

• Deprecated tb pipe new command is deprecated

1.0.0b429

• Changed –prefix deprecation message to indicate next EOL on 20th Dec.

1.0.0b428

• Added support for the new Versions feature.

1.0.0b427

• Changed tb pull omits insertion_date meta-column

1.0.0b426

• Added Commands for the new Versions feature (https://www.tinybird.co/blog-posts/git-for-real-time-data-projects).

1.0.0b425

• Changed Don’t run regression checks on tb deploy

1.0.0b424

• Fixed Bug in tb env regression-tests when –no-assert-result-rows-count provided

1.0.0b423

• Tokens are hidden in the output by default

• Added option to show token values in the output using the –show-tokens flag.

• Deprecated to hide token values in the output using the –hide-tokens flag.

1.0.0b422

• Fixed Errors in the last release.

1.0.0b421

• Fixed Provide feedback error message for erroneous commit on ‘tb deploy’

1.0.0b420

• Changed The way the command tb workspace connect handles connections.

1.0.0b419

• Fixed tb init Avoid to create example tests when tests folder already exists

1.0.0b418

• Added Option ‘–commit’ to provide and force git commit reference to ‘tb init –git’

1.0.0b417

• Fixed ‘tb push’ and ‘tb deploy’ just pushes pipe endpoints at the end when no forward deps

1.0.0b416

• Added tb init –git Checks that .diff_tmp is added to .gitignore. Throws an error if it’s not.

1.0.0b415

• Changed the behaviour of tb pull, tb datasource ls and tb pipe ls to not detect the prefixes in the resources by default. In case, you want to use tb pull, tb datasource ls or tb pipe ls in the same way as before, you will need to specify which prefix you want to detect using –prefix.

1.0.0b414

• Fixed tb init –git unexpected error when using token not associated to user

1.0.0b413

• Changed Improved error outputs working with environments

1.0.0b412

• Changed tb init –git cicd files are generated after release is finished

1.0.0b411

• Added tb test pipe endpoint test example on ‘tb init’ data project generation

1.0.0b410

• Changed tb init –git order of execution to be more reliable

1.0.0b409

• Changed tb deploy –fixtures to append the fixtures instead of replacing the data.

• Fixed tb deploy –fxitures to append the fixtures even if there is no other change

1.0.0b408

• Deprecated Hide and deprecate warning for the –prefix flag in all commands. The prefix feature is superseded by Workspaces, use tb workspace create instead.

1.0.0b407

• Fixed tb token copy not working properly in Linux

1.0.0b406

• Fixed Fail tb init –git if there are untracked files in project paths

1.0.0b405

• Added Extra information about the current token and host to the tb auth info command.

1.0.0b404

• Fixed Command tb workspace create when running in a CI environment.

1.0.0b403

• Changed behaviour of tb push to only execute the SHARED_WITH logic when passing the –user_token option

1.0.0b402

• Fixed Pushing a token with SQL condition

1.0.0b401

• Changed Shows instructions snippet when creating a Snowflake connection and password not provided.

1.0.0b400

• Added Support SETTINGS clause in query definition. Only a few of them are enabled in the backend.

1.0.0b399

• Added Allow to copy data between Data Sources releases

1.0.0b398

• Fixed Not shared resources now go to the correct folder when doing tb pull –auto

1.0.0b397

• Added Experimental token management from the CLI (Feature in development. It requires access)

1.0.0b396

• Changed Remove S3 feature flags

1.0.0b395

• Added support to sink Pipes in tb push command (Feature in development. It requires access)

1.0.0b393

• Changed Upgrade version of generated cicd templates

1.0.0b393

• Fixed tb diff now shows the correct message when one of its arguments is deprecated

1.0.0b392

• Changed Internal changes

1.0.0b391

• Changed Internal changes

1.0.0b390

• Improve Feedback messages on tb init

1.0.0b389

• Changed Python 3.7 is no longer supported. The minimum supported version is now Python 3.8.

1.0.0b388

• Fixed tb fmt when used with include files that contains nodes that are not being used in the parent pipe

1.0.0b387

• Changed creation of gcs connection when creating data connector for sinks to gcs_hmac from gcs

1.0.0b386

• Fixed Adding a column to Kafka Data Sources with tb push -f

1.0.0b385

• Changed Added an ‘End of support for Python 3.7 starting on 2023-08-01’ notice.

1.0.0b384

• Fixed integration and stage are not prompted anymore when creating a Snowflake connection

1.0.0b383

• Fixed bug introduced in version 1.0.0b375 causing the regression-test to fail when the size of the response length was different

1.0.0b382

• Fix Environments minor fixes

1.0.0b381

• Improve feedback messages in different commands

1.0.0b380

• Fix small bug when trying to unshare a datasource from all the workspaces removing the SHARED_WITH entry

1.0.0b379

• Added S3 connector creation

1.0.0b378

• Added support of SHARED_WITH in tb pull and tb diff

1.0.0b377

• Update click package dependency from 8.1.3 to 8.1.6

1.0.0b376

• Changed tb init now creates additional folders

1.0.0b375

• Changed Internal changes

1.0.0b374

• Fixed Overriding a pipe when changing nodes from endpoint to copy and vice versa when running tb push with the –force flag

1.0.0b373

• Changed Internal changes

1.0.0b372

• Changed Internal changes

1.0.0b371

• Changed Internal changes

1.0.0b370

• Changed Internal improvements.

1.0.0b369

• Changed Filter requests with environment parameters in regression tests when running tb push or tb pipe regression-test

1.0.0b368

• Fixed PLAIN default –sasl-mechanism value in tb connection create kafka command

1.0.0b367

• Changed Internal changes

1.0.0b366

• Fixed tb append now returns the correct message when the append is successful

1.0.0b365

• Fixed tb fmt removes empty space after empty comment

1.0.0b364

• Changed Internal changes

1.0.0b363

• Changed Internal changes

1.0.0b362

• When passing –token parameter, refresh credentials to be able to rely on them

1.0.0b361

• Changed Internal changes

1.0.0b360

• Changed Internal changes

1.0.0b359

• Changed Internal changes

1.0.0b358

• Added Option –ignore-datasources not to attach data in environments for specified datasources

1.0.0b357

• Changed Internal changes

1.0.0b356

• Changed Internal changes

1.0.0b355

• Changed Filter requests with errors in regression tests when running tb push or tb pipe regression-test

1.0.0b354

• Fixed Allow not having git executable

1.0.0b353

• Changed Improve tb env regression-tests output

1.0.0b352

• Added tb fmt –diff which exits 1 if local and formatted files are different. Useful for CI/CD pipelines.

• Fixed Some fixes in tb fmt regarding empty newlines.

1.0.0b351

• Internal enhancements

1.0.0b350

• Fixed tb fmt now formats datafiles properly when attributes are present in include

1.0.0b349

• Fixed Report shared Data Sources as shared on tb diff

1.0.0b348

• Added Copy Pipe commands

1.0.0b347

• Fixed Node renaming when pushing a copy pipe

1.0.0b346

• Fixed Message after reaching limits of a copy pipe

1.0.0b345

• Fixed tb pull omits the __value column in Kafka Data Sources if they are created with NDJSON format

• Fixed tb fmt finishes with an end line break

• Fixed Node renaming when pushing a copy pipe

1.0.0b344

• Added compatibility with ClickHouse 23.5 functions and syntax

1.0.0b343

• Fixed Message after pushing a copy pipe without schedule

1.0.0b342

• Fixed Copy schedule is deleted when cron is not present in .pipe file

1.0.0b341

• Fixed Error handling when copy pipes reach the maximum number of executions

1.0.0b340

• Added tb pipe rm <localfile.pipe> now supports a local file to a pipe file to remove it from the remote server

1.0.0b339

• Added Versioning support for Copy Pipes

1.0.0b338

• Fixed tb test run handles properly executions with errors

1.0.0b337

• Added support for single line comments in schema definitions.

1.0.0b336

• Fixed tb pull didn’t work properly when pulling the latest version of a remote resource

1.0.0b335

• Added integration and stage fields to Snowflake connection configuration

1.0.0b334

• Added tb test not supports a new pipe directive to perform data quality tests over pipes published as endpoints

1.0.0b333

• Added Resume/pause actions to scheduled copy pipes

1.0.0b332

• Fixed format ENGINE_TTL expression on tb diff to match the one in the remote server

1.0.0b331

• Fixed Requests to v0/tokens now send data in the body when the URI is too long

1.0.0b330

• Fixed tb push –push-deps to push endpoints in the correct order when there are multiple dependent endpoints

1.0.0b329

• Fixed A bug when pushing pipes with tokens that affected versions from 1.0.0b326 to 1.0.0b328. If you are using one of these versions, please upgrade to 1.0.0b329 and review your tokens scopes.

1.0.0b328

• Added Support to tb pull when the pipe has type copy

1.0.0b327

• Changed Requests to tb pipe data now send data in the body when the URI is too long

1.0.0b326

• Changed Requests to v0/tokens now send data in the body

1.0.0b325

• Rename tb branch to tb env

1.0.0b324

• tb diff improvements. It does not diff meta columns in Kafka Data Sources or automatically added ENGINE_PARTITION_KEY.

• tb diff just diffs mutable attributes in a .datasource file: ENGINE_TTL, SCHEMA, TOKEN, VERSION.

• tb pull now downloads TOKEN with scope APPEND if the Data Source had it in the Workspace.

1.0.0b323

• Validate Kafka connection settings when pushing a Kafka data source

1.0.0b322

• Fix to create and use branches with the workspace admin token

1.0.0b321

• Fix on tb pull and tb diff when a workspace has prefixed resources

1.0.0b320

• Internal enhancements

1.0.0b319

• Fix python3.7 compatibility

1.0.0b318

• Add croniter dependency to the CLI

1.0.0b315

• Changed default cron expression for connectors.

1.0.0b314

• Fix a bug that was impeding some users to push BigQuery-connected datasources.

1.0.0b313

• Internal enhancements

1.0.0b312

• Branching experimental feature commands

1.0.0b311

• Allow to change Engine of a Kafka Data Source

1.0.0b310

• Internal enhancements.

1.0.0b309

• Internal enhancements.

1.0.0b308

• Removing tabulate dependency in the CLI

1.0.0b307

• Upgrade toposort dependency from 1.5 to 1.10

1.0.0b306

• Internal enhancements.

1.0.0b305

• Remove requests-toolbelt dependency

1.0.0b304

• Internal improvements in regression-tests

1.0.0b303

• Fix a bug in tb diff that made indentation not work properly

1.0.0b302

• Break execution when tb workspace create` fails

1.0.0b301

• Minor internal fixes

1.0.0b300

• Remove code incompatible with python 3.7

1.0.0b299

• Fix an issue when sending certain telemetry data to Tinybird.

1.0.0b298

• Improve error messages when pushing pipe files when the file is missing a node name, data source name or a sql query

1.0.0b297

• When pushing a datasource with SHARED_WITH, we support the option –workspace_map. This is useful if you have multiple workspaces where you replicate the same project like CI or staging environment

1.0.0b296

• Internal improvements.

1.0.0b295

• Internal regression-tests improvements.

1.0.0b294

• Upgrade click package from 8.0.0 to 8.1.3

1.0.0b293

• Internal regression-tests improvements.

1.0.0b292

• Send params to the Alter Data Source API in the body to avoid a 414 error when the URL is too long

1.0.0b291

• Improve tb push –push-deps to always push pipe endpoints the last when they are not a dependency of a Materialized Node. Fixed a bug.

1.0.0b290

• Improve tb push –push-deps to always push pipe endpoints the last when they are not a dependency of a Materialized Node

1.0.0b289

• Add retry to tb pipe regression-test to retry 3 times to avoid false alerts when comparing endpoints that have continuos ingestion

1.0.0b288

• Internal telemetry improvements.

1.0.0b287

• Fixed adding new columns to a Data Source if the TTL is set to False.

1.0.0b286

• Internal fixes.

1.0.0b285

• You can share a datasource directly when doing tb push –user_token <USER_TOKEN> and the datafile contains the entry SHARED_WITH

1.0.0b284

• Improved error message when tb auth command fails.

1.0.0b283

• Internal improvements on regression-tests.

1.0.0b282

• Adds tb diff –no-verbose to be able to run tb diff without showing the diff output.

1.0.0b281

• Internal improvements on regression-tests.

1.0.0b280

• Fix error raised when pushing copy pipes

1.0.0b279

• Internal fixes.

1.0.0b278

• List BigQuery connections in tb connection ls command

1.0.0b277

• Added BigQuery option in tb connection create –help command

1.0.0b276

• Internal improvements in regression-tests.

1.0.0b275

• Internal improvements in regression-tests.

1.0.0b274

• Added option –confirm_hard_delete to be able to remove a workspace without soft delete and directly remove workspace

1.0.0b273

• Disabling the Telemetry allowing to raise the exceptions properly

1.0.0b272

• The CLI sends telemetry data to Tinybird. Telemetry data helps out team understand how the commands are used so we can improve your experience. For more information and how to opt-out for this feature, please refer to our [CLI docs](https://www.tinybird.co/docs/cli.html#cli-telemetry).

1.0.0b271

• Internal change to use a new version of an internal endpoint

1.0.0b270

• tb test run outputs the information following a table

• When a test from tb test run fails, the CLI will raise an exception to exit 1

1.0.0b269

• Internal improvements in regression-tests.

1.0.0b268

• Added tb datasource unshare to be able to unshare a datasource.

1.0.0b267

• Internal fixes.

1.0.0b266

• Update clickhouse-toolset to 0.25.dev0: newer CH compatibilty and better error messages

1.0.0b265

• Fix compatibility with python 3.7

1.0.0b264

• Fix a bug in tb fmt that replaced single quotes by double quotes breaking some queries

1.0.0b263

• Fix issue when parsing the IMPORT_QUERY parameter.

1.0.0b262

• Internal improvements in regression-tests.

1.0.0b261

• Fix issue when pushing resources with tb push –fixtures

1.0.0b260

• Enable -f option along –force option in tb pull/init to match tb push.

1.0.0b259

• Docker image updated with no Python dependencies defined on it.

• Pin urllib3 and cryptography dependencies for more reproducible installations.

1.0.0b258

• Improved error messages in the tb connection create bigquery command.

1.0.0b257

• Improve error message when trying to share a datasource that does not exists.

1.0.0b256

• Fix authentication problems when using the CLI from the Docker images.

1.0.0b255

• Allow using the CLI with Python 3.11.

1.0.0b254

• Improve error message when trying to push a pipe with unclosed {% if %}

1.0.0b253

• Internal improvements in tb push command.

1.0.0b252

• Minor internal improvements.

1.0.0b251

• tb pipe stats support formatting the result in JSON

1.0.0b250

• Replace cydifflib by difflib

1.0.0b249

• Fixing error when trying to run a huge sql doing tb sql

1.0.0b248

• Minor changes to tb diff. Now it shows a message when downloading remote resources, change the remote file description and remote resources are downloaded at once.

1.0.0b247

• tb check supports running validation pointing to directories

1.0.0b246

• Minor internal improvements.

1.0.0b245

• Minor internal improvements.

1.0.0b244

• Fixing retry mechanism when uploading file using multipart that was raising a HTTP timeout error

1.0.0b243

• Fix handle exception on tb diff

1.0.0b242

• Update clickhouse-toolset to 0.24.dev0: better performance, smaller size.

1.0.0b241

• Add –pipeline to tb sql to run a SQL query that refers a node in the current –pipeline passed as an argument. See [API docs](https://www.tinybird.co/docs/api-reference/query-api.html#id5).

• Add –pipe and –node to tb sql to run the SQL query of a node of a pipe.

1.0.0b240

• Minor internal improvements.

1.0.0b239

• Minor internal improvements.

1.0.0b238

• Minor internal improvements.

1.0.0b237

• Fix –ignore-order on tb push or tb pipe regression-test to support sorting with mix of columns with null and actual values

1.0.0b236

• Minor changes

1.0.0b235

• Change tb diff to support VERSION, INCLUDE and diff folders, for instance, tb diff pipes

1.0.0b234

• Fixes a bug where the CLI would raise an exception if it tries to show an empty table when the terminal is small.

1.0.0b233

• Fix tb fmt Data Sources DESCRIPTION is now included

1.0.0b232

• Add tb prompt with instructions on how to include info about the CLI in the shell PROMPT

1.0.0b231

• Fix bug for tb workspace current and tb workspace clear when working with admin workspace token.

1.0.0b230

• Minor changes

1.0.0b229

• Minor changes

1.0.0b228

• Minor changes

1.0.0b227

• Make tb diff format files by default, so only changes are diffed

1.0.0b226

• Add –unlink-on-populate-error to automatically unlink a Materialized View if a populate job fails. By default Materialized Views are unlinked the first time a populate job fails. See [docs](https://www.tinybird.co/docs/api-reference/pipe-api.html#post–v0-pipes-(.+)-nodes-(.+)-population) for more details.

1.0.0b225

• Adds tb diff command to compare local with remote files

1.0.0b224

• Minor improvements

1.0.0b223

• Improve support for remote disconnect exception

1.0.0b222

• Improve feedback message for when using tb push –fixtures

1.0.0b221

• Minor changes

1.0.0b220

• When pushing a resource, skip it if the version of the resource exist even thought is not the lastest instead of raising an exception like we currently do.

1.0.0b219

• Minor changes

1.0.0b218

• Upgrade of Python dependencies: requests and requests-toolbelt

1.0.0b217

• Minor changes

1.0.0b216

• Minor changes

1.0.0b215

• Minor changes

1.0.0b214

• Better tb create workspace help, it explains what –user_token does

• Display the name prompt when needed, and not requesting a non-valid option

1.0.0b213

• Fix tb fmt with files that have VERSION 0

1.0.0b212

• Fixed JSON output format for tb pipe data –format json …

1.0.0b211

• Added automatic retries in timeout errors from server.

• Updated the error message handling for unsuccesful requests for populate status: this separates confirmed errors and cancellations from other possible errors where the state of the job is not known.

1.0.0b210

• Added warning messages when pushing a materialized pipe.

1.0.0b209

• Fixed erroneous populate when running push pipe force if the source datasource file is not found locally.

1.0.0b208

• Skipping .incl files when running tb push as they are included files that shouldn’t need to be pushed

1.0.0b207

• Fixed URL to API host in auth success reminder message.

• Fixed URL to UI host with user tokens in auth interactive prompt.

1.0.0b206

• Added -h alias for commands help

1.0.0b205

• Improved handling of some SQL queries.

1.0.0b204

• Fix tb test run <file> –fail to show only failed tests.

1.0.0b203

• Append a new node to a Pipe fixed: tb pipe append.

1.0.0b202

• When running tb auth –host XXXX, we strip the host parameter to keep just the domain (https://XXXX.tinybird.co)

1.0.0b201

• Improved how we deal with workspace_map replacements, making it more flexible, and less prone to errors.

1.0.0b200

• Revisited the tb test suite, now reduced to tb test [init,run,parse], simplified the tests, and based in the YAML format.

1.0.0b199

• Better error message on missing arguments to tb datasource append.

1.0.0b198

• When running the pipe checker, we were checking that the processed bytes of the new version not to increase more than 25%. Now, this validation will be optional.

1.0.0b197

• When running the pipe checker, we were checking that the processed bytes of the new version not to increase more than 25%. Now, this validation will be optional.

1.0.0b196

• Improve delete Data Source prompt message when they are the target of materialized views.

1.0.0b195

• Minor internal fixes and improvements.

1.0.0b194

• Support for role management using the subcommand tb workspace members set-role.

1.0.0b193

• Improve tb fmt, now Python code is better formatted

1.0.0b192

• Allow to hide token values in the output using the –hide-tokens flag.

1.0.0b191

• Add Data Source dependency information to tb datasource rm, also allowing Data Source forced deletion if it is the target of a materialized view.

1.0.0b190

• Pipe checker to support non-json endpoints (.ndjson, .csv)

1.0.0b189

• Minor internal fixes and improvements.

1.0.0b188

• Improve error reporting when tb push pipe_file and there’s a timeout

1.0.0b187

• Minor fixes in tb fmt

1.0.0b186

• Added a workspace creation wizard, triggered by calling tb workspace create without additional arguments.

• Added the new option –starter-kit to the tb workspace create command to allow applying an starter kit during workspace creation.

1.0.0b185

• Minor fix in tb fmt to better report usage

1.0.0b184

• Add tb fmt datafile_path to format a local datafile

1.0.0b183

• Fix tb pipe data to actually respect the query pass as parameter

1.0.0b182

• Fix when running regression-test

1.0.0b181

• Add option to truncate all depending Data Sources in cascade using a new flag tb truncate –cascade DS_name

1.0.0b180

• Remove when running tb push or tb regression-test the validation of response time not increasing > 25% as the response time can variate according the the status of the cluster. Instead we have added a validation of the bytes read as it remains constant

1.0.0b179

• Improve error reporting

1.0.0b178

• Fix issue when dealing with server gateway errors

1.0.0b177

• Internal improvements when dealing with server gateway errors

1.0.0b176

• Added –override-datasource to tb push to override an existing Data Source by a new one with different settings when creating a Materialized View

1.0.0b175

• Avoid requesting used_by field when doing calls to /v0/datasources to avoid unncessary work

1.0.0b174

• Added –ignore-empty to tb datasource replace to not throw error if the data is empty

1.0.0b173

• Fix: avoid removing a Pipe when running tb push –force and the push fails because there’s an error in any Node. If the Pipe can’t be overridden, the current Pipe remains untouched.

• Fix: avoid removing the Data Source of a Materialized View when running tb push –force. The Materialized View gets unlinked, but the Data Source won’t be removed.

1.0.0b172

• Added support for CODECS when using json schemas in .datasource files

1.0.0b171

• When doing tb push or tb regression-test, the pipe checker will verify if new versions of the endpoint does not have a degradation of respose time.

1.0.0b170

• Update clickhouse-toolset to 0.21.dev0 to provide better dependency detection when pushing projects.

1.0.0b169

• Added api column to tb auth ls output.

1.0.0b168

• Fix encoding when pushing a pipe.

1.0.0b167

• Added new workspace members management commands.

1.0.0b166

• Fix tb datasource append command by making URL not mandatory.

1.0.0b165

• Add mandatory target Data Source argument for most tb datasource X commands.

1.0.0b164

• Fix bug tb workspace delete should ask for a user token if not provided.

1.0.0b163

• Add –token as a valid flag for tb auth.

1.0.0b162

• Internal fix

1.0.0b161

• Add support for modifying the Data Source TTL

1.0.0b160

• Fix default array values in Data Source Schemas: arr Array(String) DEFAULT [‘-‘]

1.0.0b159

• Added tb datasource share to be able to share a datasource with another workspace

• Added –fork option to tb workspace create to share all the datasources when the workspace gets created

1.0.0b158

• Change tb workspace clear to skip Shared Data Sources correctly.

1.0.0b157

• Remove –populate-sql-condition alias since it was misleading, it’s enough with tb push pipe –populate –sql-condition “the_condition”`

1.0.0b156

• Improved tb auth. Now it automatically selects the proper region based on the provided token.

1.0.0b155

• Partially fix a bug that makes comments and SQL formatting to change before pushing a datafile

1.0.0b154

• Fixed problem with update version warning being displayed when using –token. Now the update version warning is never displayed with this command.

1.0.0b153

• New tb pipe populate command

1.0.0b152

• Deprecate ENGINE_FULL option.

1.0.0b151

• Update command tb workspace clear. Now deletes all files in the workspace, not only the ones you have in your local folder.

1.0.0b150

• Added option –sample-by-params when running tb push -f or tb pipe regression-test to specify the number of sample calls to validate against. For each combination of query parameters, we will sample the data and validate the results.

1.0.0b149

• Update clickhouse-toolset to 0.20.dev0

1.0.0b148

• Add tb pipe publish and tb pipe unpublish commands

1.0.0b147

• Add –recursive flag to tb dependencies to get recursive dependencies

• Add –populate-sql-condition as an alias for tb push pipe –populate –sql-condition

1.0.0b146

• Improved documentation on tb test commands

• Improved readibility of tb test

• tb test run return non-zero exit code on failure

• –fixtures option NDJSON fixtures (only CSV was supported before)

1.0.0b145

• Add –sql-condition to send an arbitrary SQL condition together with the –populate flag when pushing a materialized pipe. See tb push –help for usage information.

1.0.0b144

• Report errors 500

1.0.0b143

• Add support to create/update tokens from datasource definition

1.0.0b142

• When running the pipe checker, checker endpoints will added to the TOKEN scope if token is specified

1.0.0b141

• Publishing Docker images for linux/arm64 arch next to the existing linux/amd64 arch

1.0.0b140

• Fix CLI for OS without clickhouse-toolset available

1.0.0b139

• Add force format option (json) to datasource / pipe ls

1.0.0b138

• Fix CLI for OS without clickhouse-toolset available

1.0.0b137

• Drop support for –skip-table-checks

1.0.0b136

• Deprecate –skip-table-checks

1.0.0b135

• Update clickhouse-toolset to 0.19.dev2

1.0.0b134

• Update clickhouse-toolset to 0.19.dev0: Improved performance and better checks around ClickHouse functions

1.0.0b133

• Now tb check verifies also query syntax

1.0.0b132

• Upgraded click package from version 7.0.0 to 8.0.0

1.0.0b131

• New tb test feature.

1.0.0b130

• Fix tinybird-cli releases for python 3.10

1.0.0b129

• Connectors performing multiple appends at once but honoring GCP limitations

1.0.0b128

• Changed the HTTP Client for the CLI

1.0.0b127

• Added new metrics about endpoint response times (max,min,mean,median an p90) on pipe command regression-test

1.0.0b126

• Connectors fixes and improvements

1.0.0b125

• Connectors performing multiple appends at once to improve performance

1.0.0b124

• Fixes in tb materialize

• Now tb materialize properly supports –prefix and datafiles with VERSION

• Removed unneeded prompts, the command does not ask you to override resources when it’s not necessary

• Changed command prompts so they explain better the action to be performed

• Added tb materialize documentation

1.0.0b123

• Revert to 1.0.0b121

1.0.0b122

• Update clickhouse-toolset to 0.17.dev2: Improved performance and better checks around ClickHouse functions

1.0.0b121

• Connectors fix and improvements

1.0.0b120

• Added –failfast/-ff and –ignore-order to tb pipe regression-test and tb push

1.0.0b119

• Add support for creating/deleting workspaces

1.0.0b118

• Improved syntax error reporting

1.0.0b117

• Fix dependencies check on tb push

1.0.0b116

• Return better error when accessing wrong resources

1.0.0b115

• tb materialize command now is interactive

1.0.0b114

• Add support for description in datasources

1.0.0b113

• Fixed problem with tb push datasource

1.0.0b112

• Fixed problem with web requests on Windows

1.0.0b111

• Requests send by the checker will be tag to avoid running always, so we can ignore them if run again the checker

• Now checker will run a sample of requests by each group of params. This way, we improve the coverage of the endpoints

• Improve checker diff to provide much more feedback when something is different between original result and checker one

1.0.0b110

• Nodes with TYPE materialized won’t be published as endpoints anymore. This was a historic design decision we are changing due to multiple side effects.

• This change adds a breaking change: if you had a pipe using the name of a materialized node, and the materialized node is not published as an endpoint the pipe won’t work. To fix the issue you have to make your pipes depend on the Materialized Node target Data Source.

1.0.0b109

• Improve feedback if server returns something that the HTTP Client can not parse

1.0.0b108

• Replace –populate –sample by –populate –subset, since sample was misleading.

1.0.0b107

• Add option –skip-incompatible-partition-key when replacing a data source with condition, i.e: tb datasource replace <name> –skip-incompatible-partition-key

1.0.0b106

• Fixed push Kafka DS with custom columns (!3079)

1.0.0b105

• Change tb pipe ls, now it returns when a pipe was modified and not just when it was created

1.0.0b104

• Fix CLI format auto detection with URLs (!3054)

1.0.0b103

• Fix tb push path/ allowing to push the content of a directory.

1.0.0b102

• Add option to clear the workspace by running tb workspace clear

1.0.0b101

• Add support for Parquet files

1.0.0b100

• Add tb materialize path/to/pipe.pipe node_name. It analyzes the node_name SQL query to generate the .datasource and .pipe files needed to push a new materialize view.

1.0.0b99

• Fix broken release 1.0.0b98. The published package missed one dependency required for some commands.

1.0.0b98

• tb auth –interactive now supports custom regions

1.0.0b97

• Add –sample option for –populate

1.0.0b96

• Use ‘replace’ instead of ‘append’ when pushing fixtures.

1.0.0b95

• Fix broken release 1.0.0b94.

1.0.0b94

• Changed the name for workspace plans.

1.0.0b93

• Improved error message format pushing a pipe.

1.0.0b92

• Added a new command to perform regression tests. See tb pipe regression-test –help for more information about the new command.

1.0.0b91

• Update clickhouse-toolset to 0.15dev1: Performance improvements in query parsing and replacement and fixes an issue with the M1 build depending on system libraries.

1.0.0b90

• Rollback to 1.0.0b88

1.0.0b89

• Update clickhouse-toolset to 0.15dev0: Performance improvements in query parsing and replacement and fixes an issue with the M1 build depending on system libraries.

1.0.0b88

• Increased timeout for tb dependencies for situations where a Data Source contains a large number of dependent Data Sources.

1.0.0b87

• Added a way to check the dependencies of a Data Source that would be affected by a partial replace. Example: tb dependencies –datasource my_datasource –check-for-partial-replace

1.0.0b86

• Return 1 as an exit code (not zero) in case of a problem appending to a datasource

1.0.0b85

• Checker should raise an error when fails retrieving most common requests while performing regression tests

1.0.0b84

• Improved error message suggesting –skip-table-checks

• Return file path for push errors

1.0.0b83

• Fixed a bug when pushing a pipe with the TOKEN command

1.0.0b82

• Forcing checker to be a standard pipe even when pushing with force a materialized view pipe

1.0.0b81

• Improve message showed in imports/replaces when the operation is not correctly configured.

1.0.0b80

• When pulling a resource with versioned dependencies, dependecy names are correctly replaced by the original names

1.0.0b78

• When pushing a new resource with versioning, sucessful create message displays the name with the version

1.0.0b77

• Changed dry-run prompts to include resources versions

1.0.0b76

• Fix authentication using localhost without http/https prefix

1.0.0b75

• Add delete subcommand to datasource to delete rows matching a SQL condition

1.0.0b74

• Fixed problem with new version warning not being displayed

1.0.0b73

• Added shortcut -i to tb auth –interactive

• Removed explorations folder as it is not used and cause confusion

1.0.0b72

• Update clickhouse-toolset to 0.14dev1.

1.0.0b71

• Fix TABLE macro to work with shared data sources

1.0.0b70

• Update clickhouse-toolset to 0.14dev0.

1.0.0b69

• Removed –skip_update_validation in favor of –version_warning

1.0.0b68

• Fix installation when the toolset binary dependency isn’t available. Improve OSX 11 support.

1.0.0b67

• Add support to validate and replace queries locally (on selected architectures and OSs).

1.0.0b66

• Added support for multi-region auth

1.0.0b65

• Fixes a problem when pushing a Kafka Data Source with the new KAFKA_STORE_RAW_VALUE key.

• Improved the message shown when a user tries to push –force a Kafka Data Source.

• When using Snowflake connector, the CSV exported to TB will be represented by and empty space between delimeters ( ex: ,,)

1.0.0b64

• Fixes tb auth when already ran a sucessful auth

1.0.0b63

• Fix Auth error when trying to remove datasource/pipe

1.0.0b62

• Added –timeout option when running tb push <PIPE>

1.0.0b61

• CLI client will retry when reach request limit (HTTP code 429)

1.0.0b60

• push, pull and append now work with NDJSON

1.0.0b59

• Improve CLI message when GCS compose does not have data because SQL query did not return any

1.0.0b58

• Added option –skip-update-validation to skip the update validation that the CLI does to check for a new version

1.0.0b57

• Support –token and –host param for auth and workspace commands. This is useful when you want to automate commands via shell scripts or similar and you want to pass those parameters via environment variables without having to do tb auth with the token prompt. For instance tb –token <token> workspace ls will list the workspaces for the token passed as a parameter.

1.0.0b56

• Fixed push when shared datasources are used in a SQL. Shared datasources names are pushed as-is.

1.0.0b55

• Removed the following ‘tb auth’ commands: ‘tb auth use’, ‘tb auth ls’, ‘tb auth add’, and ‘tb auth rm’.

• Fixed ‘tb workspace ls’ command to list correctly the workspaces a user has access to.

• Added ‘tb workspace use’ command to switch between the workspaces a user has access to.

1.0.0b54

• Fixed push pipes with escaped quotes

• Bump snowflake client to 2.6.1

1.0.0b53

• Changed tb datasource generate and tb datasource analyze to use the new v0/analyze API. It works with local and remote CSV and NDJSON files and Snowflake tables.

1.0.0b52

• Added the posibility of using env variables in a template. For example, you can have a statements like this

` INCLUDE "includes/${env}_secrests.incl" `

and call tb like this

` env=test tb push file.datasource ` - Added –no-versions to push so dependencies versions are not used. Useful for data migrations.

1.0.0b51

• Changed internal user authentication.

• Added the possibility to use a workspace as a dependecy. See tb push –workspace option.

1.0.0b50

• Fixed Fixed duplicated rows in snowflake connector import

1.0.0b49

• Fixed snowflake secrets file generation when running tb auth –connector snowflake wizard

1.0.0b48

• When new version available, the message will return the pip command to execute

1.0.0b47

• Fix error given when trying to delete a non-existent data source

• Add flag to save data sources and pipes into their own directories, it respects the default behaviour

• Instead of avoid push an already defined data source from a materialized node, only raise an error if the schema and the engine don’t match

• Capture error properly during a populate job that fails

1.0.0b46

• Enable connection commands

• Support push and pull Kafka Data Sources

1.0.0b45

• Add shared from to the datasource ls command to show Data Source’s original Workspace.

1.0.0b44

• Support SQL in nodes with no indentation

• Standardize ls command output

• Add –only-response-times flag when pushing a pipe. It’ll just run regression tests related to check response times

1.0.0b43

• Improve pull command to make it able to interact with shared Data Sources.

1.0.0b42

• Improve parser’s error handling of files: add line number

1.0.0b41

• Fix URL escaping with tb generate

1.0.0b40

• Automatically prettify .tinyb files

1.0.0b39

• Improved host option. You can use URLs ended with a backslash or not. Example: tb –host https://my-host/ auth

1.0.0b38

• Fixed misleading warnings when pushing a pipe, related to join tables.

• Fixed a bug parsing the data sources statistics when running tb datasource ls

• Changed error message in case the datasource pushed has incorrect syntax, now it gives hints on the part of the schema that is wrong.

1.0.0b37

• Gives better error messages in case the SQL in a node exceeds the current size limits (8KB).

1.0.0b36

• The option –ignore-sql-errors in tb push is no longer needed as we have fixed one of the underlying issues that was forcing its use on certain occasions.

1.0.0b35

• Added tb connection' commands. Supported connectors: Kafka

• Added tb datasource connect <id> command. Supported connectors: Kafka

1.0.0b34

• Improved the message showed in case a Pipe being pushed gets removed in the middle of the operation.

• Improved the way the CLI detect changes when working with resources with and without prefixes.

1.0.0b33

• Enables the possibility to add new columns to an existing datasource.

1.0.0b32

• Fix drop-prefix command, now it drops first pipes and its dependencies and then source data sources

1.0.0b31

• Do no report negative lines on append

• Auth improvements and changes. Credentials precendence set to –token => TB_TOKEN envvar => .tinyb file.

1.0.0b30

• Improve server error handling and sort datasource’s dependencies.

• Fix the Job tracking in cli operations when the Job enters cancelled status.

• Fix returning an error if there are wrong options when creating a data source

1.0.0b29

• Added support for Python 3.9

1.0.0b28

• Changed --skip-table-check error message to indicate how to use the flag

• Fixed checking Content-Type header

1.0.0b27

• Added flag --skip-table-check to skip materialized view and table checks

1.0.0b26

• Fix regression tests execution.

1.0.0b25

• Added support for job cancellation tb job cancel <job_id>

1.0.0b24

• Added support for ingesting data from Snowflake

1.0.0b23

• Added support for truncate operations tb datasource truncate <datasource_name>

• Added support for replace operations tb datasource replace <datasource_name> <URL or file>

• Added support for replace with condition operations tb datasource replace <datasource_name> <URL or file> –sql-condition=”country=’ES’”

• Added support for ingesting data from BigQuery

• Fix –wait option on populate

• Report progress bar when waiting for a populate job to finish

• Pull also resources with prefixes

• Disable detecting TOKEN envvar

• Fix auth from other commands

1.0.0b22

• Add support for ENGINE_SETTINGS and ENGINE_TTL

1.0.0b21

• Added population time to populate pipe option. It returns the actual population time from the population job when debug is used along with populate and wait options.

1.0.0b20

• Add missing dependency. It fixes version 1.0.0b19

1.0.0b19

• Remove red color on pull (#847) (!1243)

• Change color palette feedback (!1243)

• When doing a query against a datasource that doesn’t exist, the CLI returns a proper error (#846) (!1243)

• Fix ‘blog_log’ key error when pulling (#656) (!1243)

• Make tb commands async (!1243)

• Display a warning if there is a more recent version (#950) (!1243)

1.0.0b18

• Fix sql command (!1264)

1.0.0b17

• Fix auth CLI workspace commands (!1256)

1.0.0b16

• Add CLI workspace commands (!1246)

1.0.0b15

• Support for CSV and JSON output to “sql” CLI command

• Fixed datasource rm command not working when there is a MV pointing to the datasource being removed

1.0.0b14

• Added missed dependency. It fixes version 1.0.0b13.

1.0.0b13

• Added wait option to push command for waiting the populate job to finish.

• Fixed appending large files (more than 2GB). Now, files are uploaded using multipart and without loading the whole file in memory. (!833)(!1221)

• Added feedback about appended rows, total rows and errors to append command output (!1205)

1.0.0b12

• Added support for ENGINE options to match the API parameters

• Deprecating ENGINE_FULL, SORTING_KEY, PRIMARY_KEY, SAMPLING_KEY, and TTL options.

• Added job commands to list jobs and see job details

• Added pipe_stats command

• New set of tokens are created per prefix

• Fixed drop-prefix to remove resources in the right order

1.0.0b11

• Better error reporting when a table for a data source does not exist (!1020)

1.0.0b10

• Better error reporting when pushing a not supported file extension (!966)

1.0.0b9

• Fix parsing schemas in datasource files when using aggregate function (!956)

1.0.0b8

• Fix CLI error when pushing fixtures (!938)

2020-08-28 1.0.0b7

• Support for Python versions 3.6, 3.7 and 3.8 for both MacOSX and Linux

2020-08-26: 1.0.0b6

• Fixes saving pipe descriptions on pull and push

2020-08-24: 1.0.0b5

• Fixes appending a datasource from a URL

2020-08-10: 1.0.0b4

• Fixes the version parser.

2020-08-10: 1.0.0b3

• Limited the amount of rows fetched when running tb sql to 100. Added --row-limit.

• Fixed warning when pipes use internal tables like datasources_ops_log

• Raise an error when INSERT sql statement is used.