Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

multitenant: create roachtest that uses in-process tenants #95990

Closed
msbutler opened this issue Jan 26, 2023 · 0 comments · Fixed by #95999
Closed

multitenant: create roachtest that uses in-process tenants #95990

msbutler opened this issue Jan 26, 2023 · 0 comments · Fixed by #95999
Assignees
@msbutler
Labels
C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-multitenant Issues owned by the multi-tenant virtual team

Comments

@msbutler
Copy link
Collaborator

msbutler commented Jan 26, 2023 •
edited
Loading

There does not currently exist a roachtest that runs with in-process tenants, aside from the C2C replication roachtests. We ought to build a simple, in process tenant specific, roachtest to increase test coverage and to better triage in-process tenant specific bugs from C2C specific bugs caught by the C2C roachtests.

Jira issue: CRDB-23883
@msbutler msbutler added C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-multitenant Issues owned by the multi-tenant virtual team labels Jan 26, 2023
@msbutler msbutler self-assigned this Jan 26, 2023
@msbutler msbutler mentioned this issue Jan 26, 2023
Merged
msbutler added a commit to msbutler/cockroach that referenced this issue Jan 26, 2023
@msbutler
multitenant: add multitenant/shared-process/basic roachtest
1d1512d 
This patch introduces a simple roachtest that runs in a shared-process tenant.
This test imports a 500 tpcc workload (about 30 GB of replicated data), and
runs the workload for 10 minutes. The test is run on a 4 node, 4vcpu cluster
with local ssds.

A future patch could complicate the test by running schema changes or other
bulk operations.

Fixes cockroachdb#95990

Release note: None
@msbutler msbutler changed the title multitenant: create unified architecture roachtest multitenant: create roachtest that uses in-process tenants Jan 27, 2023
msbutler added a commit to msbutler/cockroach that referenced this issue Jan 31, 2023
@msbutler
multitenant: add multitenant/shared-process/basic roachtest
26f1944 
This patch introduces a simple roachtest that runs in a shared-process tenant.
This test imports a 500 tpcc workload (about 30 GB of replicated data), and
runs the workload for 10 minutes. The test is run on a 4 node, 4vcpu cluster
with local ssds.

A future patch could complicate the test by running schema changes or other
bulk operations.

Fixes cockroachdb#95990

Release note: None
craig bot pushed a commit that referenced this issue Feb 1, 2023
@adityamaru @jbowens
@msbutler @Xiang-Gu @Shivs11 @cucaroach @DrewKimball @rail
Merge #95622 #95865 #95999 #96115 #96202 #96235 #96328 #96366
08a188a 
95622: backupccl,storage: add logs around manifest handling and ExportRequest pagination r=stevendanna a=adityamaru

backupccl: add logging to backup manifest handling

Release note: None

storage: log the ExportRequest pagination reason

Release note: None

Epic: None

95865: cmd/roachtest: adapt disk-stall detection roachtest r=nicktrav,erikgrinaker a=jbowens

Move the existing disk-stall/* roachtests under disk-stall/fuse/* (for the FUSE
filesystem approach to stalling) and skip them for now. Currently, they're not
capable of stalling the disk longer 50us (see #95886), which makes them
unreliable at exercising stalls.

Add two new roachtests, disk-stall/dmsetup and disk-stall/cgroup that use
dmsetup and cgroup bandwidth restrctions respectively to reliably induce a
write stall for an indefinite duration.

Informs #94373.
Epic: None
Release note: None

95999: multitenant: add multitenant/shared-process/basic roachtest r=stevendanna a=msbutler

This patch introduces a simple roachtest that runs in a shared-process tenant.
This test imports a 500 tpcc workload (about 30 GB of replicated data), and
runs the workload for 10 minutes. The test is run on a 4 node, 4vcpu cluster
with local ssds.

A future patch could complicate the test by running schema changes or other
bulk operations.

Fixes #95990

Release note: None

96115: schemachanger: Implement `DROP CONSTRAINT` in declarative schema changer r=Xiang-Gu a=Xiang-Gu

This PR implements `ALTER TABLE t DROP CONSTRAINT cons_name` in declarative schema changer.

Supported constraints include Checks, FK, and UniqueWithoutIndex.

Dropping PK or Unique constraints will fall back to legacy schema changer, which in turn spits out an "not supported yet" error.

Epic: None

96202: opt: inverted-index accelerate filters of the form j->0 @> '{"b": "c"} r=Shivs11 a=Shivs11

Previously, the optimizer did not plan inverted index scans for filters
having an integer as the index for the fetch value in a filter alongside
the "contains" or the "contained by" operator.

To address this, we now build JSON arrays from fetch value expressions
with integer indexes. From these JSON arrays, inverted spans are built
for constraining scans over inverted indexes. With these changes chains
of both integer and string fetch value operators are now supported
alongside the "contains" and the "contained by" operators.
(e.g., j->0 `@>` '{"b": "c"}' and j->0 <@ '{"b": "c"}').

Epic: [CRDB-3301](https://cockroachlabs.atlassian.net/browse/CRDB-3301)
Fixes: #94667

Release note (performance improvement): The optimizer now plans
inverted index scans for queries that filter by JSON fetch value
operators (->) with integer indices alongside the "contains" or
the "contained by" operators, e.g, json_col->0 `@>` '{"b": "c"}'
or json_col->0 <@ '{"b": "c"}'

96235: sem/tree: add support for producing vectorized data from strings r=cucaroach a=cucaroach

tree.ValueHandler exposes raw machine type hooks that are used by
vec_handler to build coldata.Vec's.

Epic: CRDB-18892
Informs: #91831
Release note: None


96328: udf: allow strict UDF with no arguments r=DrewKimball a=DrewKimball

This patch fixes the case when a strict UDF (returns null on null input) has no arguments. Previously, attempting to call such a function would result in `ERROR: reflect: call of reflect.Value.Pointer on zero Value`.

Fixes #96326

Release note: None

96366: release: skip nil GitHub events r=celiala a=rail

Previously, we referenced `*event.Event`, but in some cases the event objects are `nil`.

This PR skips the nil GitHub event objects.

Epic: none
Release note: None

Co-authored-by: adityamaru <[email protected]>
Co-authored-by: Jackson Owens <[email protected]>
Co-authored-by: Michael Butler <[email protected]>
Co-authored-by: Xiang Gu <[email protected]>
Co-authored-by: Shivam Saraf <[email protected]>
Co-authored-by: Tommy Reilly <[email protected]>
Co-authored-by: Drew Kimball <[email protected]>
Co-authored-by: Rail Aliiev <[email protected]>
craig bot pushed a commit that referenced this issue Feb 1, 2023
@jbowens @msbutler
@Shivs11 @DrewKimball
Merge #95865 #95999 #96202 #96328
c77104e 
95865: cmd/roachtest: adapt disk-stall detection roachtest r=nicktrav,erikgrinaker a=jbowens

Move the existing disk-stall/* roachtests under disk-stall/fuse/* (for the FUSE
filesystem approach to stalling) and skip them for now. Currently, they're not
capable of stalling the disk longer 50us (see #95886), which makes them
unreliable at exercising stalls.

Add two new roachtests, disk-stall/dmsetup and disk-stall/cgroup that use
dmsetup and cgroup bandwidth restrctions respectively to reliably induce a
write stall for an indefinite duration.

Informs #94373.
Epic: None
Release note: None

95999: multitenant: add multitenant/shared-process/basic roachtest r=stevendanna a=msbutler

This patch introduces a simple roachtest that runs in a shared-process tenant.
This test imports a 500 tpcc workload (about 30 GB of replicated data), and
runs the workload for 10 minutes. The test is run on a 4 node, 4vcpu cluster
with local ssds.

A future patch could complicate the test by running schema changes or other
bulk operations.

Fixes #95990

Release note: None

96202: opt: inverted-index accelerate filters of the form j->0 @> '{"b": "c"} r=Shivs11 a=Shivs11

Previously, the optimizer did not plan inverted index scans for filters
having an integer as the index for the fetch value in a filter alongside
the "contains" or the "contained by" operator.

To address this, we now build JSON arrays from fetch value expressions
with integer indexes. From these JSON arrays, inverted spans are built
for constraining scans over inverted indexes. With these changes chains
of both integer and string fetch value operators are now supported
alongside the "contains" and the "contained by" operators.
(e.g., j->0 `@>` '{"b": "c"}' and j->0 <@ '{"b": "c"}').

Epic: [CRDB-3301](https://cockroachlabs.atlassian.net/browse/CRDB-3301)
Fixes: #94667

Release note (performance improvement): The optimizer now plans
inverted index scans for queries that filter by JSON fetch value
operators (->) with integer indices alongside the "contains" or
the "contained by" operators, e.g, json_col->0 `@>` '{"b": "c"}'
or json_col->0 <@ '{"b": "c"}'

96328: udf: allow strict UDF with no arguments r=DrewKimball a=DrewKimball

This patch fixes the case when a strict UDF (returns null on null input) has no arguments. Previously, attempting to call such a function would result in `ERROR: reflect: call of reflect.Value.Pointer on zero Value`.

Fixes #96326

Release note: None

Co-authored-by: Jackson Owens <[email protected]>
Co-authored-by: Michael Butler <[email protected]>
Co-authored-by: Shivam Saraf <[email protected]>
Co-authored-by: Drew Kimball <[email protected]>
@craig craig bot closed this as completed in dbed9e4 Feb 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@msbutler msbutler

Labels
C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-multitenant Issues owned by the multi-tenant virtual team
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

multitenant: add multitenant/shared-process/basic roachtest msbutler/cockroach
1 participant
@msbutler