Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DRAFT] feat: improve error group tracebacks on < py11 #823

Closed
wants to merge 227 commits into from
Closed
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
227 commits
Select commit Hold shift + click to select a range
748310b
chore(deps): Update nox in .kokoro/requirements.in [autoapprove] (#750)
gcf-owl-bot[bot] Mar 16, 2023
45d3e43
docs: Fix formatting of request arg in docstring (#756)
gcf-owl-bot[bot] Mar 23, 2023
505273b
fix: Pass the "retry" when calling read_rows. (#759)
Mariatta Mar 30, 2023
efe332c
chore(samples): Fixed a typo in the readme (#760)
shweta345 Apr 5, 2023
18b8f18
chore(deps): update all dependencies (#746)
renovate-bot Apr 6, 2023
128b4e1
docs: fix delete from column family example (#764)
billyjacobson Apr 14, 2023
b5d10d6
chore(deps): update all dependencies (#763)
renovate-bot Apr 18, 2023
1d02154
added initial implementation of mutate_rows
daniel-sanche Apr 24, 2023
ab63cba
implemented mutation models
daniel-sanche Apr 24, 2023
cf9daa5
added retries to mutate_row
daniel-sanche Apr 24, 2023
1247da4
return exception group if possible
daniel-sanche Apr 24, 2023
3b3ed8c
check for idempotence
daniel-sanche Apr 24, 2023
5d20037
initial implementation for bulk_mutations
daniel-sanche Apr 24, 2023
3d322a1
include successes in bulk mutation error message
daniel-sanche Apr 24, 2023
a31232b
fixed style checks
daniel-sanche Apr 24, 2023
8da2d65
added basic system tests
daniel-sanche Apr 24, 2023
2b89d9c
added unit tests for mutate_row
daniel-sanche Apr 25, 2023
47c5985
ran blacken
daniel-sanche Apr 25, 2023
38fdcd7
improved exceptions
daniel-sanche Apr 25, 2023
504d2d8
added bulk_mutate_rows unit tests
daniel-sanche Apr 25, 2023
b16067f
ran blacken
daniel-sanche Apr 25, 2023
3ab1405
support __new___ for exceptions for python3.11+
daniel-sanche Apr 25, 2023
0a6c0c6
added exception unit tests
daniel-sanche Apr 25, 2023
ec043cf
makde exceptions tuple
daniel-sanche Apr 26, 2023
518530e
got exceptions to print consistently across versions
daniel-sanche Apr 26, 2023
9624729
added test for 311 rich traceback
daniel-sanche Apr 27, 2023
3087081
moved retryable row mutations to new file
daniel-sanche Apr 27, 2023
9df588f
use index map
daniel-sanche Apr 27, 2023
7ed8be3
added docstring
daniel-sanche Apr 27, 2023
2536cc4
added predicate check to failed mutations
daniel-sanche Apr 27, 2023
1f6875c
added _mutate_rows tests
daniel-sanche Apr 27, 2023
1ea24e6
improved client tests
daniel-sanche Apr 27, 2023
25ca2d2
refactored to loop by raising exception
daniel-sanche Apr 28, 2023
c0787db
refactored retry deadline logic into shared wrapper
daniel-sanche Apr 28, 2023
3ed5c3d
ran black
daniel-sanche Apr 28, 2023
a91fbcb
pulled in table default timeouts
daniel-sanche Apr 28, 2023
df8a058
added tests for shared deadline parsing function
daniel-sanche Apr 28, 2023
b866b57
added tests for mutation models
daniel-sanche Apr 28, 2023
54a4d43
fixed linter errors
daniel-sanche Apr 28, 2023
bd51dc4
added tests for BulkMutationsEntry
daniel-sanche Apr 28, 2023
921b05a
improved mutations documentation
daniel-sanche Apr 28, 2023
53f7ad2
added basic implementation for batcher
daniel-sanche May 1, 2023
f65b2e5
added flush timer
daniel-sanche May 1, 2023
7051cd2
added flush on size and count
daniel-sanche May 1, 2023
9731f4c
improved queue
daniel-sanche May 1, 2023
34bf655
raise exceptions on close
daniel-sanche May 1, 2023
582457a
added args for flush
daniel-sanche May 1, 2023
eeb24a5
improved closing logic
daniel-sanche May 1, 2023
110cd96
use custom queue type to track size and count
daniel-sanche May 1, 2023
375a77d
added flow control
daniel-sanche May 1, 2023
5f9fa3f
added comments and todos
daniel-sanche May 1, 2023
e63cfc2
allow None for batcher limits
daniel-sanche May 2, 2023
d5e15aa
strip indices from exceptions
daniel-sanche May 2, 2023
82ea61f
refactored mutate_rows logic into helper function
daniel-sanche May 2, 2023
fa42b86
implemented callbacks for mutate_rows
daniel-sanche May 2, 2023
a47aeda
Merge branch 'mutate_rows' into mutate_rows_batch
daniel-sanche May 2, 2023
ee23721
release locks as mutations are completed, instead of waiting until end
daniel-sanche May 2, 2023
b69cc96
refacotr row processing
daniel-sanche May 2, 2023
9e1850c
FlowControl returns exceptions rather than raises them
daniel-sanche May 2, 2023
9e4d013
raise previous exceptions on flush with flag set
daniel-sanche May 2, 2023
2c7ec9c
disallow empty bulk mutation entries
daniel-sanche May 3, 2023
5c4f739
simplified staged mutation store
daniel-sanche May 3, 2023
fa4947f
ran blacken
daniel-sanche May 3, 2023
914b24b
replaced semaphores with condition
daniel-sanche May 3, 2023
8bda651
fixed mypy issues
daniel-sanche May 3, 2023
742194b
added raise_exceptions fn
daniel-sanche May 3, 2023
45fe1f5
fixed failing tests
daniel-sanche May 3, 2023
d8f3a4d
added skeleton for tests
daniel-sanche May 3, 2023
171fea6
feat: publish RateLimitInfo and FeatureFlag protos (#768)
gcf-owl-bot[bot] May 4, 2023
7faebc0
improved and tested flow control
daniel-sanche May 4, 2023
10da89a
refactored flushes to run sequentially
daniel-sanche May 5, 2023
0eca65e
added batcher tests
daniel-sanche May 5, 2023
2532b63
ran blacken
daniel-sanche May 5, 2023
8313b57
added additional tests
daniel-sanche May 5, 2023
01a16f3
made exceptions into a tuple
daniel-sanche May 5, 2023
da9b6cf
Merge branch 'mutate_rows' into mutate_rows_batch
daniel-sanche May 5, 2023
b7d3fcf
added warning for unflushed mutation entries
daniel-sanche May 5, 2023
7521a61
Feat: Threaded MutationsBatcher (#722)
Mariatta May 10, 2023
8f9de8d
chore(main): release 2.18.0 (#757)
release-please[bot] May 10, 2023
a767cff
fix: Revert "Feat: Threaded MutationsBatcher" (#773)
mutianf May 11, 2023
a4e7325
Revert "fix: Revert "Feat: Threaded MutationsBatcher" (#773)" (#775)
mutianf May 11, 2023
9e35ad2
chore(main): release 2.18.1 (#774)
release-please[bot] May 16, 2023
6140acb
remove aborted from retryable errors
daniel-sanche May 22, 2023
36ba2b6
improved SetCell mutation
daniel-sanche May 22, 2023
b3c9017
fixed mutations tests
daniel-sanche May 22, 2023
cac9e2d
SetCell timestamps use millisecond precision
daniel-sanche May 22, 2023
34b051f
renamed BulkMutationsEntry to RowMutationEntry
daniel-sanche May 22, 2023
63ac35c
Merge branch 'v3' into mutate_rows
daniel-sanche May 24, 2023
bb72ea8
Merge branch 'mutate_rows' into mutate_rows_batch
daniel-sanche May 24, 2023
cef70f2
feat: add ChangeStreamConfig to CreateTable and UpdateTable (#786)
gcf-owl-bot[bot] May 25, 2023
ae477b5
build(deps): bump requests from 2.28.1 to 2.31.0 in /synthtool/gcp/te…
gcf-owl-bot[bot] May 25, 2023
ecf539c
docs(samples): add region tags (#788)
Sita04 May 25, 2023
a51201c
added metadata to mutate rows and bulk mutate rows
daniel-sanche May 25, 2023
a21bebf
moved _convert_retry_deadline wrapper from exceptions into _helpers
daniel-sanche May 25, 2023
4ca89d9
fixed system tests
daniel-sanche May 25, 2023
b240ee1
only handle precision adjustment when creating timestamp
daniel-sanche May 26, 2023
cb0e951
added _from_dict for mutation models
daniel-sanche May 26, 2023
a9cf385
rpc timeouts adjust when approaching operation_timeout
daniel-sanche May 26, 2023
eddc1c9
pass table instead of request dict
daniel-sanche May 26, 2023
f8b26aa
refactoring mutate rows
daniel-sanche May 27, 2023
5b80dc5
made on_terminal_state into coroutine
daniel-sanche May 27, 2023
9e5b80a
fixed style issues
daniel-sanche May 27, 2023
f7539f6
moved callback rewriting into retryable attempt
daniel-sanche May 27, 2023
e77a4fa
fixed tests
daniel-sanche May 27, 2023
4e19ed0
pop successful mutations from error dict
daniel-sanche May 30, 2023
920e4b7
removed unneeded check
daniel-sanche May 30, 2023
725f5ff
refactoring
daniel-sanche May 30, 2023
1054bc4
pass list of exceptions in callback
daniel-sanche May 30, 2023
f39a891
raise error in unexpected state
daniel-sanche May 30, 2023
e5f65f2
Merge branch 'mutate_rows' into mutate_rows_batch
daniel-sanche May 31, 2023
8038662
added timeout to user facing flush
daniel-sanche May 31, 2023
c2ec26d
added comments
daniel-sanche May 31, 2023
1d97135
removed callback
daniel-sanche May 31, 2023
88e2bf5
refactoring mutation attempt into class
daniel-sanche May 31, 2023
a3c0166
use partial function
daniel-sanche May 31, 2023
70c35ef
renamed class
daniel-sanche May 31, 2023
e00f592
added comments
daniel-sanche Jun 1, 2023
18af78a
added tests
daniel-sanche Jun 1, 2023
23e84f5
improved helpers
daniel-sanche Jun 1, 2023
c4fd78f
build(deps): bump cryptography from 39.0.1 to 41.0.0 in /synthtool/gc…
gcf-owl-bot[bot] Jun 3, 2023
56fdf7c
refactored operation into class only
daniel-sanche Jun 5, 2023
aca31f0
restructured how remaining indices are tracked
daniel-sanche Jun 5, 2023
5a5d541
fixed tests
daniel-sanche Jun 5, 2023
afed731
added docstrings
daniel-sanche Jun 5, 2023
2396ec8
moved index deletion to end of block
daniel-sanche Jun 6, 2023
3d441a2
added comment to exception types
daniel-sanche Jun 6, 2023
0013c4d
Merge branch 'mutate_rows' into mutate_rows_batch
daniel-sanche Jun 6, 2023
8b41518
update flow control after batch, instead of after each entry
daniel-sanche Jun 6, 2023
5a8b9e9
Merge branch 'v3' into mutate_rows_batch
daniel-sanche Jun 6, 2023
bdd7f45
fixed test
daniel-sanche Jun 6, 2023
553bc4a
add kwargs docstring
daniel-sanche Jun 7, 2023
20156c6
allow mutations over flow limits
daniel-sanche Jun 7, 2023
b9c4467
added limits for mutate_rows mutation and entry counts
daniel-sanche Jun 7, 2023
8b8d8ab
refactored mutation system tests
daniel-sanche Jun 7, 2023
a1de714
added batcher system tests
daniel-sanche Jun 7, 2023
2382a04
allow batcher to append multiple entries
daniel-sanche Jun 7, 2023
1db0377
improved timeout handling
daniel-sanche Jun 7, 2023
589aa5d
fix: add a callback function on flush_rows (#796)
mutianf Jun 8, 2023
fbe6ace
chore(main): release 2.19.0 (#789)
release-please[bot] Jun 8, 2023
6a66c48
made batch rpcs concurrent
daniel-sanche Jun 14, 2023
e1718f0
allow single mutations in flow control public methods
daniel-sanche Jun 14, 2023
eeda2e9
mutation batcher only accepts single entries
daniel-sanche Jun 15, 2023
eeb483f
ensure that proper exceptions are raised for each flush
daniel-sanche Jun 15, 2023
853b5f2
surface unexpected exceptions in MutationsExceptionGroup
daniel-sanche Jun 15, 2023
2865acd
prefixed flow control attributes with underscore
daniel-sanche Jun 15, 2023
e568d92
batcher signature improvements
daniel-sanche Jun 15, 2023
bc7b850
use request mutation limit instead of entry limit
daniel-sanche Jun 15, 2023
ff0aed6
reduce repeated flush tasks when entries appended in a loop
daniel-sanche Jun 15, 2023
f71eebc
renamed variables
daniel-sanche Jun 15, 2023
c0dc760
added test; renamed variables
daniel-sanche Jun 15, 2023
a6b12dc
removed outdated comment
daniel-sanche Jun 16, 2023
d5128eb
added comments
daniel-sanche Jun 16, 2023
335391c
renamed variable
daniel-sanche Jun 16, 2023
836171b
made append async; removed _pending_entries
daniel-sanche Jun 20, 2023
06f16bc
use futures instead of tasks for type hints
daniel-sanche Jun 20, 2023
09a5d51
refactored asyncio calls out to make sync implementation easier
daniel-sanche Jun 20, 2023
0b0f30e
renamed flow_control_max_count
daniel-sanche Jun 21, 2023
283a949
removed None as a flow_control_max_* option
daniel-sanche Jun 21, 2023
834db1a
fixed type annotation
daniel-sanche Jun 21, 2023
524f2d8
fixed tests
daniel-sanche Jun 21, 2023
df8d2d5
improved docstrings
daniel-sanche Jun 21, 2023
eaed9ab
added timeouts to batcher init
daniel-sanche Jun 21, 2023
e0b5e27
fixed 3.11 test issue
daniel-sanche Jun 22, 2023
d50d89b
fixed 3.7 unit test issues
daniel-sanche Jun 22, 2023
32d39fd
fixed lint issue
daniel-sanche Jun 22, 2023
1343b34
fixed lint
daniel-sanche Jun 22, 2023
4f3aa3a
fixed up unit tests
daniel-sanche Jun 22, 2023
0b99b89
added ping and warm to system test
daniel-sanche Jun 22, 2023
6a35be4
fixed broken ping and warm
daniel-sanche Jun 22, 2023
aa35901
added table and app profile to warming params
daniel-sanche Jun 22, 2023
4e1ec6f
fixed _remove_instance_registration
daniel-sanche Jun 22, 2023
8f4b34f
improved tests
daniel-sanche Jun 22, 2023
9c8df9f
allow warming for single instance
daniel-sanche Jun 22, 2023
584f94d
updated tests
daniel-sanche Jun 22, 2023
3c605ba
fixed metadata format
daniel-sanche Jun 22, 2023
12099f5
remove index from exceptions
daniel-sanche Jun 23, 2023
6833707
added test for memory leaks
daniel-sanche Jun 23, 2023
4c40299
clear task after completion
daniel-sanche Jun 23, 2023
2931aa8
added exception list truncation
daniel-sanche Jun 23, 2023
1dc760e
removed index information
daniel-sanche Jun 23, 2023
18fd309
Merge branch 'v3' into warm_improvements
daniel-sanche Jun 23, 2023
badbb83
fixed existing tests
daniel-sanche Jun 23, 2023
97e6680
Merge branch 'v3' into mutate_rows_batch
daniel-sanche Jun 23, 2023
6da9a64
removed manual flushing
daniel-sanche Jun 23, 2023
0ecbbcd
added tests
daniel-sanche Jun 23, 2023
61b367a
pr comment improvements
daniel-sanche Jun 23, 2023
0ce16c6
fixed metadata
daniel-sanche Jun 23, 2023
697cefd
Merge branch 'mutate_rows_batch' into restructure_paths
daniel-sanche Jun 23, 2023
9a407f0
moved v3 client into /data folder
daniel-sanche Jun 23, 2023
c1587e5
moved legacy client back into root
daniel-sanche Jun 23, 2023
a6a6b7b
moved async classes into own sub-folder
daniel-sanche Jun 23, 2023
7e7a667
moved ReadRowsIterator into _read_rows
daniel-sanche Jun 23, 2023
66afd15
moved unit tests into data folder
daniel-sanche Jun 23, 2023
fe4e280
fixed v2 unit tests
daniel-sanche Jun 23, 2023
badaf65
rewrote some test paths
daniel-sanche Jun 24, 2023
48b2d2c
renamed async subpath to _async
daniel-sanche Jun 24, 2023
c00b395
fixed imports
daniel-sanche Jun 24, 2023
e4a6546
split up test files for state machine class
daniel-sanche Jun 24, 2023
1191444
got unit tests running
daniel-sanche Jun 24, 2023
2ca05d8
fixed batcher tests
daniel-sanche Jun 24, 2023
6f12cda
got unit tests passing
daniel-sanche Jun 24, 2023
67f5a3d
fixed system tests
daniel-sanche Jun 24, 2023
d7b6dca
fixed v2 system tests
daniel-sanche Jun 24, 2023
14a3d3d
pin to api_core with streaming retries
daniel-sanche Jun 24, 2023
563a133
fixed type errors
daniel-sanche Jun 24, 2023
b030fb5
updated submodule
daniel-sanche Jun 24, 2023
ac0ef64
fixed batcher comments
daniel-sanche Jun 26, 2023
0b8a651
Merge branch 'main' into restructure_paths
daniel-sanche Jun 26, 2023
d90430a
Merge branch 'v3' into restructure_paths
daniel-sanche Jun 26, 2023
4122a16
updated docs folder
daniel-sanche Jun 26, 2023
ff325ef
changed mypy to only check data client
daniel-sanche Jun 26, 2023
e996946
fixed some test issues
daniel-sanche Jun 26, 2023
d63d68c
reverted gapic tests
daniel-sanche Jun 26, 2023
5b7db3e
added verbose logging for unit tests
daniel-sanche Jun 26, 2023
9947afd
fixed atexit test that was breaking coverage reports
daniel-sanche Jun 26, 2023
9e8a0b3
removed external api-core installation for prerelease-deps
daniel-sanche Jun 26, 2023
30b9b11
improve timeouts
daniel-sanche Jun 27, 2023
b6ad653
fixed docstrings
daniel-sanche Jun 27, 2023
eb55460
chore: remove pinned Sphinx version [autoapprove] (#818)
gcf-owl-bot[bot] Jun 28, 2023
10d12b5
Merge branch 'v3' into improve-timeouts
daniel-sanche Jun 28, 2023
12c388c
Merge branch 'main' into improve-timeouts
daniel-sanche Jun 28, 2023
a723cb2
improve exception messages
daniel-sanche Jun 28, 2023
ccc9ae4
attach cause in more general way
daniel-sanche Jun 28, 2023
f0b3558
fixed tests for 3.11
daniel-sanche Jun 28, 2023
e2f263c
added improved output tests for 3.11
daniel-sanche Jun 28, 2023
6f269a8
got 3.10 tests passing
daniel-sanche Jun 28, 2023
e090689
added tests for 3.10- output
daniel-sanche Jun 28, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fixed imports
  • Loading branch information
daniel-sanche committed Jun 24, 2023
commit c00b395a8927e686944470949b300098df282212
36 changes: 18 additions & 18 deletions google/cloud/bigtable/data/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,21 +18,21 @@

from google.cloud.bigtable import gapic_version as package_version

from google.cloud.bigtable.client import BigtableDataClient
from google.cloud.bigtable.client import Table

from google.cloud.bigtable.read_rows_query import ReadRowsQuery
from google.cloud.bigtable.read_rows_query import RowRange
from google.cloud.bigtable.row import Row
from google.cloud.bigtable.row import Cell

from google.cloud.bigtable.mutations_batcher import MutationsBatcher
from google.cloud.bigtable.mutations import Mutation
from google.cloud.bigtable.mutations import RowMutationEntry
from google.cloud.bigtable.mutations import SetCell
from google.cloud.bigtable.mutations import DeleteRangeFromColumn
from google.cloud.bigtable.mutations import DeleteAllFromFamily
from google.cloud.bigtable.mutations import DeleteAllFromRow
from google.cloud.bigtable.data._async.client import BigtableDataClientAsync
from google.cloud.bigtable.data._async.client import TableAsync

from google.cloud.bigtable.data.read_rows_query import ReadRowsQuery
from google.cloud.bigtable.data.read_rows_query import RowRange
from google.cloud.bigtable.data.row import Row
from google.cloud.bigtable.data.row import Cell

from google.cloud.bigtable.data._async.mutations_batcher import MutationsBatcherAsync
from google.cloud.bigtable.data.mutations import Mutation
from google.cloud.bigtable.data.mutations import RowMutationEntry
from google.cloud.bigtable.data.mutations import SetCell
from google.cloud.bigtable.data.mutations import DeleteRangeFromColumn
from google.cloud.bigtable.data.mutations import DeleteAllFromFamily
from google.cloud.bigtable.data.mutations import DeleteAllFromRow

# Type alias for the output of sample_keys
RowKeySamples = List[Tuple[bytes, int]]
Expand All @@ -42,12 +42,12 @@
__version__: str = package_version.__version__

__all__ = (
"BigtableDataClient",
"Table",
"BigtableDataClientAsync",
"TableAsync",
"RowKeySamples",
"ReadRowsQuery",
"RowRange",
"MutationsBatcher",
"MutationsBatcherAsync",
"Mutation",
"RowMutationEntry",
"SetCell",
Expand Down
14 changes: 7 additions & 7 deletions google/cloud/bigtable/data/_async/_mutate_rows.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,20 +19,20 @@

from google.api_core import exceptions as core_exceptions
from google.api_core import retry_async as retries
import google.cloud.bigtable.exceptions as bt_exceptions
from google.cloud.bigtable._helpers import _make_metadata
from google.cloud.bigtable._helpers import _convert_retry_deadline
from google.cloud.bigtable._helpers import _attempt_timeout_generator
import google.cloud.bigtable.data.exceptions as bt_exceptions
from google.cloud.bigtable.data._helpers import _make_metadata
from google.cloud.bigtable.data._helpers import _convert_retry_deadline
from google.cloud.bigtable.data._helpers import _attempt_timeout_generator

# mutate_rows requests are limited to this number of mutations
from google.cloud.bigtable.mutations import MUTATE_ROWS_REQUEST_MUTATION_LIMIT
from google.cloud.bigtable.data.mutations import MUTATE_ROWS_REQUEST_MUTATION_LIMIT

if TYPE_CHECKING:
from google.cloud.bigtable_v2.services.bigtable.async_client import (
BigtableAsyncClient,
)
from google.cloud.bigtable.client import Table
from google.cloud.bigtable.mutations import RowMutationEntry
from google.cloud.bigtable.data.client import Table
from google.cloud.bigtable.data.mutations import RowMutationEntry


class _MutateRowsOperationAsync:
Expand Down
16 changes: 8 additions & 8 deletions google/cloud/bigtable/data/_async/_read_rows.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,16 +33,16 @@

from google.cloud.bigtable_v2.types.bigtable import ReadRowsResponse
from google.cloud.bigtable_v2.services.bigtable.async_client import BigtableAsyncClient
from google.cloud.bigtable.row import Row, _LastScannedRow
from google.cloud.bigtable.exceptions import InvalidChunk
from google.cloud.bigtable.exceptions import _RowSetComplete
from google.cloud.bigtable.exceptions import IdleTimeout
from google.cloud.bigtable._read_rows_state_machine import _StateMachine
from google.cloud.bigtable.data.row import Row, _LastScannedRow
from google.cloud.bigtable.data.exceptions import InvalidChunk
from google.cloud.bigtable.data.exceptions import _RowSetComplete
from google.cloud.bigtable.data.exceptions import IdleTimeout
from google.cloud.bigtable.data._read_rows_state_machine import _StateMachine
from google.api_core import retry_async as retries
from google.api_core import exceptions as core_exceptions
from google.cloud.bigtable._helpers import _make_metadata
from google.cloud.bigtable._helpers import _attempt_timeout_generator
from google.cloud.bigtable._helpers import _convert_retry_deadline
from google.cloud.bigtable.data._helpers import _make_metadata
from google.cloud.bigtable.data._helpers import _attempt_timeout_generator
from google.cloud.bigtable.data._helpers import _convert_retry_deadline


class _ReadRowsOperationAsync(AsyncIterable[Row]):
Expand Down
54 changes: 27 additions & 27 deletions google/cloud/bigtable/data/_async/client.py
Original file line number Diff line number Diff line change
Expand Up @@ -43,34 +43,34 @@
from google.api_core.exceptions import GoogleAPICallError
from google.api_core import retry_async as retries
from google.api_core import exceptions as core_exceptions
from google.cloud.bigtable._read_rows import _ReadRowsOperationAsync
from google.cloud.bigtable.data._async._read_rows import _ReadRowsOperationAsync
from google.cloud.bigtable.data._async._read_rows import ReadRowsIteratorAsync

import google.auth.credentials
import google.auth._default
from google.api_core import client_options as client_options_lib
from google.cloud.bigtable.row import Row
from google.cloud.bigtable.read_rows_query import ReadRowsQuery
from google.cloud.bigtable.iterators import ReadRowsIteratorAsync
from google.cloud.bigtable.exceptions import FailedQueryShardError
from google.cloud.bigtable.exceptions import ShardedReadRowsExceptionGroup

from google.cloud.bigtable.mutations import Mutation, RowMutationEntry
from google.cloud.bigtable._mutate_rows import _MutateRowsOperationAsync
from google.cloud.bigtable._helpers import _make_metadata
from google.cloud.bigtable._helpers import _convert_retry_deadline
from google.cloud.bigtable.mutations_batcher import MutationsBatcher
from google.cloud.bigtable.mutations_batcher import _MB_SIZE
from google.cloud.bigtable._helpers import _attempt_timeout_generator

from google.cloud.bigtable.read_modify_write_rules import ReadModifyWriteRule
from google.cloud.bigtable.row_filters import RowFilter
from google.cloud.bigtable.row_filters import StripValueTransformerFilter
from google.cloud.bigtable.row_filters import CellsRowLimitFilter
from google.cloud.bigtable.row_filters import RowFilterChain
from google.cloud.bigtable.data.row import Row
from google.cloud.bigtable.data.read_rows_query import ReadRowsQuery
from google.cloud.bigtable.data.exceptions import FailedQueryShardError
from google.cloud.bigtable.data.exceptions import ShardedReadRowsExceptionGroup

from google.cloud.bigtable.data.mutations import Mutation, RowMutationEntry
from google.cloud.bigtable.data._async._mutate_rows import _MutateRowsOperationAsync
from google.cloud.bigtable.data._helpers import _make_metadata
from google.cloud.bigtable.data._helpers import _convert_retry_deadline
from google.cloud.bigtable.data._async.mutations_batcher import MutationsBatcherAsync
from google.cloud.bigtable.data._async.mutations_batcher import _MB_SIZE
from google.cloud.bigtable.data._helpers import _attempt_timeout_generator

from google.cloud.bigtable.data.read_modify_write_rules import ReadModifyWriteRule
from google.cloud.bigtable.data.row_filters import RowFilter
from google.cloud.bigtable.data.row_filters import StripValueTransformerFilter
from google.cloud.bigtable.data.row_filters import CellsRowLimitFilter
from google.cloud.bigtable.data.row_filters import RowFilterChain

if TYPE_CHECKING:
from google.cloud.bigtable import RowKeySamples
from google.cloud.bigtable import ShardedQuery
from google.cloud.bigtable.data.data import RowKeySamples
from google.cloud.bigtable.data.data import ShardedQuery

# used by read_rows_sharded to limit how many requests are attempted in parallel
CONCURRENCY_LIMIT = 10
Expand Down Expand Up @@ -556,7 +556,7 @@ async def read_row(
See read_rows_stream

Raises:
- google.cloud.bigtable.exceptions.RowNotFound: if the row does not exist
- google.cloud.bigtable.data.exceptions.RowNotFound: if the row does not exist
Returns:
- the individual row requested, or None if it does not exist
"""
Expand Down Expand Up @@ -762,7 +762,7 @@ def mutations_batcher(
flow_control_max_bytes: int = 100 * _MB_SIZE,
batch_operation_timeout: float | None = None,
batch_per_request_timeout: float | None = None,
) -> MutationsBatcher:
) -> MutationsBatcherAsync:
"""
Returns a new mutations batcher instance.

Expand All @@ -782,9 +782,9 @@ def mutations_batcher(
- batch_per_request_timeout: timeout for each individual request, in seconds. If None,
table default_per_request_timeout will be used
Returns:
- a MutationsBatcher context manager that can batch requests
- a MutationsBatcherAsync context manager that can batch requests
"""
return MutationsBatcher(
return MutationsBatcherAsync(
self,
flush_interval=flush_interval,
flush_limit_mutation_count=flush_limit_mutation_count,
Expand Down Expand Up @@ -930,7 +930,7 @@ async def bulk_mutate_rows(
if per_request_timeout is not None and per_request_timeout > operation_timeout:
raise ValueError("per_request_timeout must be less than operation_timeout")

operation = _MutateRowsOperation(
operation = _MutateRowsOperationAsync(
self.client._gapic_client,
self,
mutation_entries,
Expand Down
19 changes: 8 additions & 11 deletions google/cloud/bigtable/data/_async/mutations_batcher.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,16 +20,13 @@
import warnings
from collections import deque

from google.cloud.bigtable.mutations import RowMutationEntry
from google.cloud.bigtable.exceptions import MutationsExceptionGroup
from google.cloud.bigtable.exceptions import FailedMutationEntryError
from google.cloud.bigtable.data.mutations import RowMutationEntry
from google.cloud.bigtable.data.exceptions import MutationsExceptionGroup
from google.cloud.bigtable.data.exceptions import FailedMutationEntryError

from google.cloud.bigtable._mutate_rows import _MutateRowsOperation
from google.cloud.bigtable._mutate_rows import MUTATE_ROWS_REQUEST_MUTATION_LIMIT
from google.cloud.bigtable.mutations import Mutation

if TYPE_CHECKING:
from google.cloud.bigtable.client import Table # pragma: no cover
from google.cloud.bigtable.data._async._mutate_rows import _MutateRowsOperationAsync
from google.cloud.bigtable.data._async._mutate_rows import MUTATE_ROWS_REQUEST_MUTATION_LIMIT
from google.cloud.bigtable.data.mutations import Mutation

# used to make more readable default values
_MB_SIZE = 1024 * 1024
Expand Down Expand Up @@ -179,7 +176,7 @@ class MutationsBatcherAsync:

def __init__(
self,
table: "Table",
table: TableAsync,
*,
flush_interval: float | None = 5,
flush_limit_mutation_count: int | None = 1000,
Expand Down Expand Up @@ -353,7 +350,7 @@ async def _execute_mutate_rows(
if self._table.app_profile_id:
request["app_profile_id"] = self._table.app_profile_id
try:
operation = _MutateRowsOperation(
operation = _MutateRowsOperationAsync(
self._table.client._gapic_client,
self._table,
batch,
Expand Down
2 changes: 1 addition & 1 deletion google/cloud/bigtable/data/_helpers.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
import time

from google.api_core import exceptions as core_exceptions
from google.cloud.bigtable.exceptions import RetryExceptionGroup
from google.cloud.bigtable.data.exceptions import RetryExceptionGroup

"""
Helper functions used in various places in the library.
Expand Down
4 changes: 2 additions & 2 deletions google/cloud/bigtable/data/_read_rows_state_machine.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,8 @@
from typing import Type

from google.cloud.bigtable_v2.types.bigtable import ReadRowsResponse
from google.cloud.bigtable.row import Row, Cell, _LastScannedRow
from google.cloud.bigtable.exceptions import InvalidChunk
from google.cloud.bigtable.data.row import Row, Cell, _LastScannedRow
from google.cloud.bigtable.data.exceptions import InvalidChunk

"""
This module provides classes for the read_rows state machine:
Expand Down
6 changes: 3 additions & 3 deletions google/cloud/bigtable/data/exceptions.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,13 +19,13 @@
from typing import Any, TYPE_CHECKING

from google.api_core import exceptions as core_exceptions
from google.cloud.bigtable.row import Row
from google.cloud.bigtable.data.row import Row

is_311_plus = sys.version_info >= (3, 11)

if TYPE_CHECKING:
from google.cloud.bigtable.mutations import RowMutationEntry
from google.cloud.bigtable.read_rows_query import ReadRowsQuery
from google.cloud.bigtable.data.mutations import RowMutationEntry
from google.cloud.bigtable.data.read_rows_query import ReadRowsQuery


class IdleTimeout(core_exceptions.DeadlineExceeded):
Expand Down
16 changes: 0 additions & 16 deletions google/cloud/bigtable/data/gapic_version.py

This file was deleted.

2 changes: 1 addition & 1 deletion google/cloud/bigtable/data/mutations.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
from abc import ABC, abstractmethod
from sys import getsizeof

from google.cloud.bigtable.read_modify_write_rules import MAX_INCREMENT_VALUE
from google.cloud.bigtable.data.read_modify_write_rules import MAX_INCREMENT_VALUE

# special value for SetCell mutation timestamps. If set, server will assign a timestamp
SERVER_SIDE_TIMESTAMP = -1
Expand Down
6 changes: 3 additions & 3 deletions google/cloud/bigtable/data/read_rows_query.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,11 +18,11 @@
from bisect import bisect_right
from collections import defaultdict
from dataclasses import dataclass
from google.cloud.bigtable.row_filters import RowFilter
from google.cloud.bigtable.data.row_filters import RowFilter

if TYPE_CHECKING:
from google.cloud.bigtable import RowKeySamples
from google.cloud.bigtable import ShardedQuery
from google.cloud.bigtable.data import RowKeySamples
from google.cloud.bigtable.data import ShardedQuery


@dataclass
Expand Down
22 changes: 11 additions & 11 deletions tests/unit/data/_async/test_client.py
Original file line number Diff line number Diff line change
Expand Up @@ -1294,12 +1294,12 @@ async def test_read_rows_per_request_timeout(

@pytest.mark.asyncio
async def test_read_rows_idle_timeout(self):
from google.cloud.bigtable.data._async.client import ReadRowsIterator
from google.cloud.bigtable.data._async.client import ReadRowsIteratorAsync
from google.cloud.bigtable_v2.services.bigtable.async_client import (
BigtableAsyncClient,
)
from google.cloud.bigtable.data.exceptions import IdleTimeout
from google.cloud.bigtable.data._async._read_rows import _ReadRowsOperation
from google.cloud.bigtable.data._async._read_rows import _ReadRowsOperationAsync

chunks = [
self._make_chunk(row_key=b"test_1"),
Expand All @@ -1310,15 +1310,15 @@ async def test_read_rows_idle_timeout(self):
chunks
)
with mock.patch.object(
ReadRowsIterator, "_start_idle_timer"
ReadRowsIteratorAsync, "_start_idle_timer"
) as start_idle_timer:
client = self._make_client()
table = client.get_table("instance", "table")
query = ReadRowsQuery()
gen = await table.read_rows_stream(query)
# should start idle timer on creation
start_idle_timer.assert_called_once()
with mock.patch.object(_ReadRowsOperation, "aclose", AsyncMock()) as aclose:
with mock.patch.object(_ReadRowsOperationAsync, "aclose", AsyncMock()) as aclose:
# start idle timer with our own value
await gen._start_idle_timer(0.1)
# should timeout after being abandoned
Expand Down Expand Up @@ -1397,13 +1397,13 @@ async def test_read_rows_revise_request(self):
"""
Ensure that _revise_request is called between retries
"""
from google.cloud.bigtable.data._async._read_rows import _ReadRowsOperation
from google.cloud.bigtable.data._async._read_rows import _ReadRowsOperationAsync
from google.cloud.bigtable.data.exceptions import InvalidChunk

with mock.patch.object(
_ReadRowsOperation, "_revise_request_rowset"
_ReadRowsOperationAsync, "_revise_request_rowset"
) as revise_rowset:
with mock.patch.object(_ReadRowsOperation, "aclose"):
with mock.patch.object(_ReadRowsOperationAsync, "aclose"):
revise_rowset.return_value = "modified"
async with self._make_table() as table:
read_rows = table.client._gapic_client.read_rows
Expand Down Expand Up @@ -1431,11 +1431,11 @@ async def test_read_rows_default_timeouts(self):
"""
Ensure that the default timeouts are set on the read rows operation when not overridden
"""
from google.cloud.bigtable.data._async._read_rows import _ReadRowsOperation
from google.cloud.bigtable.data._async._read_rows import _ReadRowsOperationAsync

operation_timeout = 8
per_request_timeout = 4
with mock.patch.object(_ReadRowsOperation, "__init__") as mock_op:
with mock.patch.object(_ReadRowsOperationAsync, "__init__") as mock_op:
mock_op.side_effect = RuntimeError("mock error")
async with self._make_table(
default_operation_timeout=operation_timeout,
Expand All @@ -1454,11 +1454,11 @@ async def test_read_rows_default_timeout_override(self):
"""
When timeouts are passed, they overwrite default values
"""
from google.cloud.bigtable.data._async._read_rows import _ReadRowsOperation
from google.cloud.bigtable.data._async._read_rows import _ReadRowsOperationAsync

operation_timeout = 8
per_request_timeout = 4
with mock.patch.object(_ReadRowsOperation, "__init__") as mock_op:
with mock.patch.object(_ReadRowsOperationAsync, "__init__") as mock_op:
mock_op.side_effect = RuntimeError("mock error")
async with self._make_table(
default_operation_timeout=99, default_per_request_timeout=97
Expand Down