Any feature in BigQuery that can migrate a whole dataset in another project w/o executing copy data?

Question

While our project grows, at some point we realized that we need to create new projects and reorganize our dataset. One case is that we need to isolate one dataset from others in

Answer 1

There's no built-in feature but I helped write a tool that we've open-sourced that will do this for you: https://github.com/uswitch/big-replicate.

It will let you synchronise/copy tables between projects or datasets (within the same project). Most of the details are in the project's README but for reference it looks a little like:

java -cp big-replicate-standalone.jar \
  uswitch.big_replicate.sync \
  --source-project source-project-id \
  --source-dataset 98909919 \
  --destination-project destination-project-id \
  --destination-dataset 98909919

You can set options that will control how many tables to copy, how many jobs run concurrently and where to store the intermediate data in Cloud Storage. The destination dataset must already exist but this means you'll be able to copy data between locations too (US, EU, Asia etc.).

Binaries are built on CircleCI and published to GitHub releases.