Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

storage: clean up preemptive snapshot when receiving replica ID as learner #3

Closed
wants to merge 52 commits into from
Closed

storage: clean up preemptive snapshot when receiving replica ID as learner #3

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
338b6e1
storage/reports: point to issue for skipped test
andreimatei Sep 19, 2019
3940886
sql: add syntax for arrays of oid wrapper types
jordanlewis Sep 20, 2019
3dde922
tree: add support for REGTYPE casts of the trigger type
jordanlewis Sep 20, 2019
a8b7df5
sql: Add telemetry for interleaved table join
Sep 19, 2019
5d00053
tla-plus: update ParallelCommit spec with AckImpliesCommit invariant
nvanbenschoten Sep 23, 2019
dd7134b
colflow: wrap SupportsVectorized with an error catcher
yuzefovich Sep 23, 2019
d3e456c
util/syncutil: don't unconditionally allocate in IntMap.Range
nvanbenschoten Sep 24, 2019
ece7b8b
c-deps: bump rocksdb for unique cache IDs on Windows
ajkr Sep 24, 2019
dfd09f3
Merge #41000
Sep 24, 2019
723aea6
opt: fix With and WithScan properties
RaduBerinde Sep 24, 2019
a34d705
opt: disallow mutations under union
RaduBerinde Sep 23, 2019
70be7eb
util/log: fix GC of secondary loggers
knz Sep 24, 2019
27d2fe4
roachtest: attempt to fix cancel test
jordanlewis Sep 24, 2019
ded2b6c
Merge #41032
Sep 24, 2019
66b9550
cli: fix the demo licensing code
knz Sep 24, 2019
83eb885
Merge #41018 #41020
Sep 24, 2019
9f0f622
importccl: DELIMITED should handle delimiters only at end of enclosed…
Sep 21, 2019
5296a27
roachtest: deflake bank/{node-restart,cluster-recovery}
irfansharif Sep 23, 2019
5a73aa5
Merge #40920
Sep 24, 2019
4242276
Merge #40997 #41029
Sep 24, 2019
9fd4ce1
Merge #40960
Sep 24, 2019
0d1d9da
stats: add logging to TestDefaultColumns to help debug flake
rytaft Sep 24, 2019
88f4c2f
demo: Add test to verify workload is running with --with-load
Sep 9, 2019
dcc9a15
Merge #40613
Sep 24, 2019
477d887
tree: fix LIKE suboperator with NULL LHS
rafiss Sep 24, 2019
5ae9615
roachtest: cast data points to ints in verifyNodeLiveness
nvanbenschoten Sep 24, 2019
3439b8e
roachtest: improve/fix Jepsen error whitelist
nvanbenschoten Sep 24, 2019
ec3947e
Merge #41037 #41042
Sep 24, 2019
785faa0
Merge #40975
Sep 24, 2019
0c588c8
Merge #41015 #41043
Sep 24, 2019
f6600fa
Merge #40892
Sep 24, 2019
905068e
Merge #40914
Sep 24, 2019
c11a254
Merge #41005
Sep 25, 2019
798a757
sql/sem/tree: run TestEval through the vectorized engine
asubiotto Sep 16, 2019
73cc4ea
Merge #40790
Sep 25, 2019
5df1260
cli: avoid running Example_demo_locality in the race detector
tbg Sep 24, 2019
d98e083
Merge #41044
Sep 25, 2019
df82e4e
movr: Initialize movr's faker if it hasn't been already in workload
Sep 25, 2019
ce7d287
colexec: miscellaneous cleanups
yuzefovich Sep 23, 2019
be0b1ff
Merge #40949
Sep 25, 2019
369fe76
storage: check for RaftTombstone again once synchronized with Store
nvanbenschoten Sep 24, 2019
69ef011
sqlsmith: tell sqlsmith about more things that vectorized supports
yuzefovich Sep 25, 2019
a43202f
Merge #40999 #41072
Sep 25, 2019
982aeb8
Merge #41075
Sep 25, 2019
1ecaa0f
roachtest: properly fail when uploading binaries fails
nvanbenschoten Sep 25, 2019
1311b51
Merge #41050
Sep 25, 2019
fe735c9
Merge #41083
Sep 25, 2019
2c744d4
roachtest: fix schemachange/mixed/tpcc for 19.1
lucy-zhang Sep 25, 2019
d714f55
Merge #41079
Sep 25, 2019
05c8a9e
roachtest: fix panic when tests fail using an existing cluster
ajwerner Sep 25, 2019
7df2a12
Merge #41086
Sep 25, 2019
99a031d
storage: clean up preemptive snapshot when receiving replica ID as le…
ajwerner Sep 18, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion c-deps/rocksdb
Open in desktop
Submodule rocksdb updated 1 files
+11 −54 port/win/io_win.cc
16 changes: 8 additions & 8 deletions docs/generated/sql/bnf/stmt_block.bnf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1099,7 +1099,6 @@ cast_target ::=
typename ::=
simple_typename opt_array_bounds
| simple_typename 'ARRAY'
| postgres_oid

collation_name ::=
unrestricted_name
Expand Down Expand Up @@ -1496,18 +1495,12 @@ simple_typename ::=
| bit_with_length
| character_with_length
| const_interval
| postgres_oid

opt_array_bounds ::=
'[' ']'
|

postgres_oid ::=
'REGPROC'
| 'REGPROCEDURE'
| 'REGCLASS'
| 'REGTYPE'
| 'REGNAMESPACE'

expr_tuple1_ambiguous ::=
'(' ')'
| '(' tuple1_ambiguous_values ')'
Expand Down Expand Up @@ -1862,6 +1855,13 @@ character_with_length ::=
const_interval ::=
'INTERVAL'

postgres_oid ::=
'REGPROC'
| 'REGPROCEDURE'
| 'REGCLASS'
| 'REGTYPE'
| 'REGNAMESPACE'

tuple1_ambiguous_values ::=
a_expr
| a_expr ','
Expand Down
170 changes: 93 additions & 77 deletions docs/tla-plus/ParallelCommits/ParallelCommits.tla
View file
Open in desktop

Large diffs are not rendered by default.

115 changes: 60 additions & 55 deletions docs/tla-plus/ParallelCommits/ParallelCommits.toolbox/ParallelCommits___Model_1.launch
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,58 +1,63 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<launchConfiguration type="org.lamport.tla.toolbox.tool.tlc.modelCheck">
<stringAttribute key="TLCCmdLineParameters" value=""/>
<stringAttribute key="configurationName" value="Model_1"/>
<booleanAttribute key="deferLiveness" value="false"/>
<intAttribute key="dfidDepth" value="100"/>
<booleanAttribute key="dfidMode" value="false"/>
<intAttribute key="distributedFPSetCount" value="0"/>
<stringAttribute key="distributedNetworkInterface" value="192.168.13.240"/>
<intAttribute key="distributedNodesCount" value="1"/>
<stringAttribute key="distributedTLC" value="off"/>
<stringAttribute key="distributedTLCVMArgs" value=""/>
<intAttribute key="fpBits" value="1"/>
<intAttribute key="fpIndex" value="1"/>
<intAttribute key="maxHeapSize" value="25"/>
<intAttribute key="maxSetSize" value="1000000"/>
<booleanAttribute key="mcMode" value="true"/>
<stringAttribute key="modelBehaviorInit" value=""/>
<stringAttribute key="modelBehaviorNext" value=""/>
<stringAttribute key="modelBehaviorSpec" value="Spec"/>
<intAttribute key="modelBehaviorSpecType" value="1"/>
<stringAttribute key="modelBehaviorVars" value="intent_writes, to_write, txn_epoch, txn_ts, pipelined_keys, to_check, prevent_epoch, pc, commit_ack, to_resolve, record, have_staged_record, found_writes, prevent_ts, parallel_keys, tscache, attempt"/>
<stringAttribute key="modelComments" value=""/>
<booleanAttribute key="modelCorrectnessCheckDeadlock" value="true"/>
<listAttribute key="modelCorrectnessInvariants">
<listEntry value="1TypeInvariants"/>
</listAttribute>
<listAttribute key="modelCorrectnessProperties">
<listEntry value="1TemporalTxnRecordProperties"/>
<listEntry value="1TemporalIntentProperties"/>
<listEntry value="1TemporalTSCacheProperties"/>
<listEntry value="1ImplicitCommitLeadsToExplicitCommit"/>
<listEntry value="1AckLeadsToExplicitCommit"/>
</listAttribute>
<stringAttribute key="modelExpressionEval" value=""/>
<stringAttribute key="modelParameterActionConstraint" value=""/>
<listAttribute key="modelParameterConstants">
<listEntry value="KEYS;;{k1, k2};1;1"/>
<listEntry value="PREVENTERS;;{p1, p2};1;1"/>
<listEntry value="MAX_ATTEMPTS;;3;0;0"/>
</listAttribute>
<stringAttribute key="modelParameterContraint" value=""/>
<listAttribute key="modelParameterDefinitions"/>
<stringAttribute key="modelParameterModelValues" value="{}"/>
<stringAttribute key="modelParameterNewDefinitions" value=""/>
<intAttribute key="numberOfWorkers" value="2"/>
<booleanAttribute key="recover" value="false"/>
<stringAttribute key="result.mail.address" value=""/>
<intAttribute key="simuAril" value="-1"/>
<intAttribute key="simuDepth" value="100"/>
<intAttribute key="simuSeed" value="-1"/>
<stringAttribute key="specName" value="ParallelCommits"/>
<listAttribute key="traceExploreExpressions">
<listEntry value="1ImplicitCommit"/>
</listAttribute>
<stringAttribute key="view" value=""/>
<booleanAttribute key="visualizeStateGraph" value="false"/>
<stringAttribute key="TLCCmdLineParameters" value=""/>
<intAttribute key="collectCoverage" value="1"/>
<stringAttribute key="configurationName" value="Model_1"/>
<booleanAttribute key="deferLiveness" value="false"/>
<intAttribute key="dfidDepth" value="100"/>
<booleanAttribute key="dfidMode" value="false"/>
<intAttribute key="distributedFPSetCount" value="0"/>
<stringAttribute key="distributedNetworkInterface" value="192.168.13.156"/>
<intAttribute key="distributedNodesCount" value="1"/>
<stringAttribute key="distributedTLC" value="off"/>
<stringAttribute key="distributedTLCVMArgs" value=""/>
<intAttribute key="fpBits" value="1"/>
<intAttribute key="fpIndex" value="111"/>
<booleanAttribute key="fpIndexRandom" value="true"/>
<intAttribute key="maxHeapSize" value="25"/>
<intAttribute key="maxSetSize" value="1000000"/>
<booleanAttribute key="mcMode" value="true"/>
<stringAttribute key="modelBehaviorInit" value=""/>
<stringAttribute key="modelBehaviorNext" value=""/>
<stringAttribute key="modelBehaviorSpec" value="Spec"/>
<intAttribute key="modelBehaviorSpecType" value="1"/>
<stringAttribute key="modelBehaviorVars" value="intent_writes, to_write, txn_epoch, txn_ts, pipelined_keys, to_check, prevent_epoch, pc, commit_ack, to_resolve, record, have_staged_record, found_writes, prevent_ts, parallel_keys, tscache, attempt"/>
<stringAttribute key="modelComments" value=""/>
<booleanAttribute key="modelCorrectnessCheckDeadlock" value="true"/>
<listAttribute key="modelCorrectnessInvariants">
<listEntry value="1TypeInvariants"/>
<listEntry value="1AckImpliesCommit"/>
</listAttribute>
<listAttribute key="modelCorrectnessProperties">
<listEntry value="1TemporalTxnRecordProperties"/>
<listEntry value="1TemporalIntentProperties"/>
<listEntry value="1TemporalTSCacheProperties"/>
<listEntry value="1ImplicitCommitLeadsToExplicitCommit"/>
<listEntry value="1AckLeadsToExplicitCommit"/>
</listAttribute>
<intAttribute key="modelEditorOpenTabs" value="4"/>
<stringAttribute key="modelExpressionEval" value=""/>
<stringAttribute key="modelParameterActionConstraint" value=""/>
<listAttribute key="modelParameterConstants">
<listEntry value="KEYS;;{k1, k2};1;1"/>
<listEntry value="PREVENTERS;;{p1, p2};1;1"/>
<listEntry value="MAX_ATTEMPTS;;3;0;0"/>
</listAttribute>
<stringAttribute key="modelParameterContraint" value=""/>
<listAttribute key="modelParameterDefinitions"/>
<stringAttribute key="modelParameterModelValues" value="{}"/>
<stringAttribute key="modelParameterNewDefinitions" value=""/>
<intAttribute key="numberOfWorkers" value="8"/>
<booleanAttribute key="recover" value="false"/>
<stringAttribute key="result.mail.address" value=""/>
<intAttribute key="simuAril" value="-1"/>
<intAttribute key="simuDepth" value="100"/>
<intAttribute key="simuSeed" value="-1"/>
<stringAttribute key="specName" value="ParallelCommits"/>
<stringAttribute key="tlcResourcesProfile" value="local custom"/>
<listAttribute key="traceExploreExpressions">
<listEntry value="1ImplicitCommit"/>
</listAttribute>
<stringAttribute key="view" value=""/>
<booleanAttribute key="visualizeStateGraph" value="false"/>
</launchConfiguration>
8 changes: 6 additions & 2 deletions pkg/ccl/cliccl/demo.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,13 +17,17 @@ import (

"github.com/cockroachdb/cockroach/pkg/build"
"github.com/cockroachdb/cockroach/pkg/cli"
"github.com/cockroachdb/cockroach/pkg/util/envutil"
"github.com/cockroachdb/cockroach/pkg/util/log"
"github.com/cockroachdb/cockroach/pkg/util/uuid"
"github.com/cockroachdb/errors"
)

// This URL grants a license that is valid for 24 hours.
const licenseURL = "https://register.cockroachdb.com/api/license"
const licenseDefaultURL = "https://register.cockroachdb.com/api/license"

// We make licenseURL configurable for use in tests.
var licenseURL = envutil.EnvOrDefaultString("COCKROACH_DEMO_LICENSE_URL", licenseDefaultURL)

func getLicense(clusterID uuid.UUID) (string, error) {
client := &http.Client{
Expand Down Expand Up @@ -60,7 +64,7 @@ func getLicense(clusterID uuid.UUID) (string, error) {
func getAndApplyLicense(db *gosql.DB, clusterID uuid.UUID, org string) (bool, error) {
license, err := getLicense(clusterID)
if err != nil {
fmt.Fprintf(log.OrigStderr, "error when contacting licensing server: %+v\n", err)
fmt.Fprintf(log.OrigStderr, "\nerror while contacting licensing server:\n%+v\n", err)
return false, nil
}
if _, err := db.Exec(`SET CLUSTER SETTING cluster.organization = $1`, org); err != nil {
Expand Down
87 changes: 83 additions & 4 deletions pkg/ccl/importccl/import_stmt_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -243,20 +243,88 @@ d
data: "1\t2",
err: "row 1: too many columns, expected 1",
},
{
name: "cannot parse data",
create: `i int8, j int8`,
typ: "DELIMITED",
data: "bad_int\t2",
err: "row 1: parse",
},
{
name: "unexpected number of columns",
create: `a string, b string`,
typ: "DELIMITED",
data: "1,2",
err: "row 1: unexpected number of columns, expected 2 got 1",
},
{
name: "unexpected number of columns in 1st row",
create: `a string, b string`,
typ: "DELIMITED",
data: "1,2\n3\t4",
err: "row 1: unexpected number of columns, expected 2 got 1",
},
{
name: "field enclosure",
create: `a string, b string`,
with: `WITH fields_enclosed_by = '$'`,
typ: "DELIMITED",
data: "$foo$\tnormal",
query: map[string][][]string{
`SELECT * from t`: {{"foo", "normal"}},
},
},
{
name: "field enclosure in middle of unquoted field",
create: `a string, b string`,
with: `WITH fields_enclosed_by = '$'`,
typ: "DELIMITED",
data: "fo$o\tb$a$z",
query: map[string][][]string{
`SELECT * from t`: {{"fo$o", "b$a$z"}},
},
},
{
name: "field enclosure in middle of quoted field",
create: `a string, b string`,
with: `WITH fields_enclosed_by = '$'`,
typ: "DELIMITED",
data: "$fo$o$\t$b$a$z$",
query: map[string][][]string{
`SELECT * from t`: {{"fo$o", "b$a$z"}},
},
},
{
name: "unmatched field enclosure",
create: `i int8`,
with: `WITH fields_enclosed_by = '"'`,
create: `a string, b string`,
with: `WITH fields_enclosed_by = '$'`,
typ: "DELIMITED",
data: "$foo\tnormal",
err: "row 1: unmatched field enclosure at start of field",
},
{
name: "unmatched field enclosure at end",
create: `a string, b string`,
with: `WITH fields_enclosed_by = '$'`,
typ: "DELIMITED",
data: "foo$\tnormal",
err: "row 1: unmatched field enclosure at end of field",
},
{
name: "unmatched field enclosure 2nd field",
create: `a string, b string`,
with: `WITH fields_enclosed_by = '$'`,
typ: "DELIMITED",
data: "normal\t$foo",
err: "row 1: unmatched field enclosure at start of field",
},
{
name: "unmatched field enclosure at end 2nd field",
create: `a string, b string`,
with: `WITH fields_enclosed_by = '$'`,
typ: "DELIMITED",
data: "\"1",
err: "row 1: unmatched field enclosure",
data: "normal\tfoo$",
err: "row 1: unmatched field enclosure at end of field",
},
{
name: "unmatched literal",
Expand All @@ -266,6 +334,17 @@ d
data: `\`,
err: "row 1: unmatched literal",
},
{
name: "escaped field enclosure",
create: `a string, b string`,
with: `WITH fields_enclosed_by = '$', fields_escaped_by = '\',
fields_terminated_by = ','`,
typ: "DELIMITED",
data: `\$foo\$,\$baz`,
query: map[string][][]string{
`SELECT * from t`: {{"$foo$", "$baz"}},
},
},
{
name: "weird escape char",
create: `s STRING`,
Expand Down
Loading