Documentation

BigQuery

Connect a Google BigQuery warehouse from Equals

To connect BigQuery to Equals you need to create a service account with the right permissions and then upload the resulting key to Equals. The easiest way to handle permissions is to grant Equals the BigQuery Data Viewer role and the BigQuery Job User role.

If you need more discrete control over permissions, you can also choose to create a custom role and set the required permissions yourself.

You can follow the instructions below or watch the 2-minute video walkthrough below.

Using predefined roles

Predefined roles cover all required roles for Equals to work with BigQuery.

Create a service account

  1. In the top nav search type Service accounts and choose Service accounts under Products & Pages.
  2. Click Create Service Account give your account a name like Equals and click Create and Continue.
  3. Add role BigQuery Job User.
  4. If you would like to grant access to all datasets, add role BigQuery Data Viewer. Otherwise, you can grant this role to the individual datasets you want to grant access to later.
  5. Once you’ve added the appropriate roles, click Continue.

Create Private Key

  1. Select your service account you just created from the Service accounts list page.
  2. Click Keys in the tab bar at the top, click Add key and then Create new key.
  3. Select JSON in the modal and then click Create
  4. A JSON file will be downloaded to your computer. Upload this file as the Private Key in Equals. Equals should now be able to connect to your BigQuery instance.

Using custom roles

Custom roles give you more control over what Equals has access to.

Create a custom role

You need to create a custom role to grant a service account permissions to view your schema tables. We need access to these schema tables for querying in Equals to work.

  1. In the top nav search type roles and choose Roles under Products & Pages.
  2. Click Create role in the top nav
  3. Name the custom role
  4. Click on Add permissions
  5. In the modal, search and add the following permissions:
    1. If you plan to restrict access to specific datasets, add this permission.
      1. bigquery.jobs.create Enables Equals to execute SQL queries against a BigQuery dataset or table. This is required at the service account level.
      2. Follow instructions under the restricting access section to create a separate role that you can apply to specific tables and datasets in your project.
        ii. If you plan to give access to all datasets in your project, add the rest of these permissions at the service account level.
      3. bigquery.datasets.get Enables Equals to see a dataset
      4. bigquery.tables.get Enables Equals to see a table
      5. bigquery.tables.list Enables Equals to get the tables on a dataset
  6. Click Create

Create a service account

  1. In the top nav search type Service accounts and choose Service accounts under Products & Pages.
  2. Click Create Service Account give your account a name like Equals and click Create and Continue.
  3. You need to grant the service account access to the custom role you just created.
  4. Once you’ve added the custom role, click Continue.

Create a private key

  1. Select your service account you just created from the Service accounts list page.
  2. Click Keys in the tab bar at the top, click Add key and then Create new key.
  3. Select JSON in the modal and then click Create
  4. A JSON file will be downloaded to your computer. Upload this file as the Private Key in Equals. Equals should now be able to connect to your BigQuery instance.

Restricting access

Specific datasets

If you need to restrict access to specific datasets in your project, you can either:

  1. Assign the pre-defined role you created to each dataset you want Equals to have access to.
  2. Create a new custom role with these permissions and then assign it to each dataset you want Equals to have access to.
    1. bigquery.datasets.get
    2. bigquery.tables.get
    3. bigquery.tables.list

Tables in an allowed dataset

If you need to restrict access to specific tables within a dataset, you’ll need to create two custom roles.

The first role should include these permissions:

  1. bigquery.datasets.get
  2. bigquery.tables.get
  3. bigquery.tables.list

Assign this role to the dataset your table is in.

The second role should include these permissions:

  1. bigquery.tables.getData

Assign this role to the tables you want to give Equals access to.

External tables

If your project is using external tables, you’ll need to add these permissions.

  1. If you’re using Pre-defined roles, add the BigQuery Connection User role to your service account.
  2. If you’re using custom roles, add the bigquery.connections.user permission to your role.

Permissions errors

Equals needs the bigquery.jobs.create permission

The service account must have the bigquery.jobs.create permission to work. You have to apply this to the service account level.

If you’re using Pre-defined roles the easiest way to do this is to give the service account the BigQuery Job User role. You’ll find the guide under Using Pre-defined Roles.

If you’re using Custom Roles, make sure this permission is included in the Custom Role you’re using for the service account. You’ll find the guide under Using Custom Roles.

Equals needs the bigquery.connections.user permission

The service account must have the bigquery.connections.user permission to work. You have to apply this to the service account level.

If you’re using Pre-defined roles the easiest way to do this is to give the service account the BigQuery Connection User role. You’ll find the guide under Using Pre-defined Roles.

If you’re using Custom Roles, make sure this permission is included in the Custom Role you’re using for the service account. You’ll find the guide under Using Custom Roles.

Equals doesn’t have access to any dataset in BigQuery

This is likely due to an improper set up of permissions in BigQuery. Check the following:

  1. Make sure your role has the bigquery.jobs.create permission at the service account level.
  2. Make sure the other required permissions are set at either the service account level, or for specific datasets and tables if you’re restricting access.

Missing dataset permission

We have access to 1 or more datasets, but we don’t have adequate permissions on some or all datasets to generate schema. Check the following:

  1. Make sure your role has the bigquery.jobs.create permissions at the service account level.
  2. Make sure the other required permissions are set for all datasets you want us to have access to.