chore(deps): upgrade google-cloud-bigquery

[miketheman]

Remove version constraint and include the new dependencies of numpy and pyarrow.

Corrects current broken state where the combination of dependencies prevents resolution, thanks to google-cloud-bigquery pinning to protobf under 4.

$ make deps
...
+ pip-compile --upgrade --allow-unsafe -o /tmp/tmp.eRh6xSE0Bn/main.txt requirements/main.in
Could not find a version that matches protobuf<4.0.0dev,<5.0.0dev,>=3.12.0,>=3.15.0,>=3.19.0,>=3.20.1,>=4.21.3 (from google-cloud-bigquery==2.34.4->-r requirements/main.in (line 18))
Tried: 2.0.3, 2.3.0, 2.4.1, 2.5.0, 2.6.0, 2.6.1, 3.0.0, 3.0.0, 3.1.0, 3.1.0.post1, 3.1.0.post1, 3.2.0, 3.2.0, 3.3.0, 3.4.0, 3.4.0, 3.5.0.post1, 3.5.0.post1, 3.5.1, 3.5.1, 3.5.2, 3.5.2, 3.5.2.post1, 3.5.2.post1, 3.6.0, 3.6.0, 3.6.1, 3.6.1, 3.7.0, 3.7.0, 3.7.1, 3.7.1, 3.8.0, 3.8.0, 3.9.0, 3.9.0, 3.9.1, 3.9.1, 3.9.2, 3.9.2, 3.10.0, 3.10.0, 3.11.0, 3.11.0, 3.11.1, 3.11.1, 3.11.2, 3.11.2, 3.11.3, 3.11.3, 3.12.2, 3.12.2, 3.12.4, 3.12.4, 3.13.0, 3.13.0, 3.14.0, 3.14.0, 3.15.0, 3.15.0, 3.15.1, 3.15.1, 3.15.2, 3.15.2, 3.15.3, 3.15.3, 3.15.4, 3.15.4, 3.15.5, 3.15.5, 3.15.6, 3.15.6, 3.15.7, 3.15.7, 3.15.8, 3.15.8, 3.16.0, 3.16.0, 3.17.0, 3.17.0, 3.17.1, 3.17.1, 3.17.2, 3.17.2, 3.17.3, 3.17.3, 3.18.0, 3.18.0, 3.18.1, 3.18.1, 3.18.3, 3.18.3, 3.19.0, 3.19.0, 3.19.0, 3.19.1, 3.19.1, 3.19.1, 3.19.2, 3.19.2, 3.19.2, 3.19.3, 3.19.3, 3.19.3, 3.19.4, 3.19.4, 3.19.4, 3.19.5, 3.19.5, 3.19.5, 3.20.0, 3.20.0, 3.20.0, 3.20.1, 3.20.1, 3.20.1, 3.20.2, 3.20.2, 3.20.2, 4.21.0, 4.21.0, 4.21.0, 4.21.0, 4.21.1, 4.21.1, 4.21.1, 4.21.1, 4.21.2, 4.21.2, 4.21.2, 4.21.2, 4.21.3, 4.21.3, 4.21.3, 4.21.3, 4.21.4, 4.21.4, 4.21.4, 4.21.4, 4.21.5, 4.21.5, 4.21.5, 4.21.5, 4.21.6, 4.21.6, 4.21.6, 4.21.6
Skipped pre-versions: 2.0.0b0, 3.0.0a2, 3.0.0a3, 3.0.0b1, 3.0.0b1.post1, 3.0.0b1.post2, 3.0.0b2, 3.0.0b2, 3.0.0b2.post1, 3.0.0b2.post1, 3.0.0b2.post2, 3.0.0b2.post2, 3.0.0b3, 3.0.0b4, 3.0.0b4, 3.2.0rc1, 3.2.0rc1, 3.2.0rc1.post1, 3.2.0rc1.post1, 3.2.0rc2, 3.2.0rc2, 3.7.0rc2, 3.7.0rc2, 3.7.0rc3, 3.7.0rc3, 3.8.0rc1, 3.8.0rc1, 3.9.0rc1, 3.9.0rc1, 3.10.0rc1, 3.10.0rc1, 3.11.0rc1, 3.11.0rc1, 3.11.0rc2, 3.11.0rc2, 3.13.0rc3, 3.13.0rc3, 3.14.0rc1, 3.14.0rc1, 3.14.0rc2, 3.14.0rc2, 3.14.0rc3, 3.14.0rc3, 3.15.0rc1, 3.15.0rc1, 3.15.0rc2, 3.15.0rc2, 3.16.0rc1, 3.16.0rc1, 3.16.0rc2, 3.16.0rc2, 3.17.0rc1, 3.17.0rc1, 3.17.0rc2, 3.17.0rc2, 3.18.0rc1, 3.18.0rc1, 3.18.0rc2, 3.18.0rc2, 3.19.0rc1, 3.19.0rc1, 3.19.0rc1, 3.19.0rc2, 3.19.0rc2, 3.19.0rc2, 3.20.0rc1, 3.20.0rc1, 3.20.0rc1, 3.20.0rc2, 3.20.0rc2, 3.20.0rc2, 3.20.1rc1, 3.20.1rc1, 3.20.1rc1, 4.0.0rc1, 4.0.0rc1, 4.0.0rc2, 4.0.0rc2, 4.21.0rc1, 4.21.0rc1, 4.21.0rc1, 4.21.0rc1, 4.21.0rc2, 4.21.0rc2, 4.21.0rc2, 4.21.0rc2
There are incompatible versions in the resolved dependencies:
  protobuf<5.0.0dev,>=3.15.0 (from googleapis-common-protos==1.56.4->google-api-core[grpc]==2.10.1->google-cloud-storage==2.5.0->-r requirements/main.in (line 19))
  protobuf<5.0.0dev,>=3.20.1 (from google-api-core[grpc]==2.10.1->google-cloud-bigquery==2.34.4->-r requirements/main.in (line 18))
  protobuf<5.0.0dev,>=3.19.0 (from proto-plus==1.22.1->google-cloud-bigquery==2.34.4->-r requirements/main.in (line 18))
  protobuf>=4.21.3 (from grpcio-status==1.49.0->google-api-core[grpc]==2.10.1->google-cloud-storage==2.5.0->-r requirements/main.in (line 19))
  protobuf<4.0.0dev,>=3.12.0 (from google-cloud-bigquery==2.34.4->-r requirements/main.in (line 18))
ERROR: 2
make: *** [deps] Error 2

Increases overall image size by ~100MB to include these dependencies.

If we can accept this tradeoff until googleapis/python-bigquery#1196 is resolved, then we can have fewer dependency conflicts during updates.

Closes #12234.

[miketheman]

I also tried pinning back google-cloud-storage to 2.4.0 or earlier, but got the same issue - due to google-cloud-bigquery pinning an upper boundary, preventing us from using protobuf 4.x needed by grpcio-status, which isn't a direct dependecy.

[miketheman]

This feels like a moving target, since now the deps check fails on types-cryptography 😖

[alex]

:-/ types-cryptography is just going to be a minefield and should absolutely not be required. Since 3.4 cryptography has had its own type hints.

[miketheman]

@alex appreciate the commiseration 😀

What's confusing at this juncture is that the depchecker result is showing:
- types-cryptography - indicating that the package is removed as a result of the changeset, which is 😕 confusing to me, since it's not included already, nor added as part of this changes, so we're out of sync somehow.

[di]

The depchecker takes into account the current latest state of all dependencies, so if there's a new version of some existing dependency on PyPI that introduces a new subdependency, it will cause a failure here.

In this case, it looks like the latest pyjwt introduces types-cryptograph as a subdependency: #12234, so we should probably merge that PR first as it will block all other deps PRs.

[di]

(or, cherry-pick it here because it seems like we need both)

[miketheman]

Thanks for solving, @di!