Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Log Events for Alter Table, Async Schema Change #7571

Merged
merged 1 commit into from
Jul 7, 2016

Conversation

mrtracy
Copy link
Contributor

@mrtracy mrtracy commented Jun 30, 2016

Commit adds three new event log types:

  • "Alter Table", logged for every ALTER TABLE DDL statement.
  • "Finish Schema Change", logged when an asynchronous schema change is
    completed.
  • "Reverse Schema Change", logged when an asynchronous schema change encounters
    an error (such as a constraint violation during backfill) and is rolled back.

This Commit encountered an interesting, perhaps unintended effect with
SystemConfigTrigger: in a "mixed" transaction which updates both system and
non-system keys, the system changes will only be gossiped if the transaction
record is located on the system range. In other words, "mixed" transactions will
only work correctly if the system keys are modified first. Issue #7570 had been
logged to track this unexpected behavior; this commit has also added a number of
comments and a logged error to address this.


This change is Reviewable

@tamird
Copy link
Contributor

tamird commented Jun 30, 2016

:lgtm:

Sorry you had to learn about this the hard way :(


Reviewed 9 of 9 files at r1.
Review status: all files reviewed at latest revision, 5 unresolved discussions, all commit checks successful.


sql/alter_table.go, line 246 [r1] (raw file):

  }

  // Log "Alter Table" event.

this comment isn't useful =/


sql/event_log.go, line 44 [r1] (raw file):

  // EventLogDropTable is recorded when a table is dropped.
  EventLogDropTable EventLogType = "drop_table"
  // EventLogAlterTable is recorded when an ALTER TABLE statement is

none of the others reference a specific syntax. In fact, I think we run through AlterTable on ADD COLUMN, so this comment is inaccurate.


sql/lease.go, line 340 [r1] (raw file):

          // More efficient batching can be used if no event log message
          // is required.

Slightly less code duplication:

txn.SetSystemConfigTrigger()
b := txn.NewBatch()
b.Put(descKey, desc)
if logEvent != nil {
  if err := txn.Run(b); err != nil {
    return err
  }
  if err := logEvent(txn); err != nil {
    return err
  }
  return txn.Commit()
}
return txn.CommitInBatch()

sql/schema_changer.go, line 520 [r1] (raw file):

              Error      string
              MutationID uint32
          }{causingError.Error(), uint32(sc.mutationID)},

you might want to fmt.Sprintf("%+v", causingError); that will give you the error's stack trace if the error came from a call to errors.Errorf or one of its friends.


sql/testdata/event_log, line 263 [r1] (raw file):

----
53 1

remove


Comments from Reviewable

@vivekmenezes
Copy link
Contributor

Review status: all files reviewed at latest revision, 7 unresolved discussions, all commit checks successful.


sql/event_log.go, line 49 [r1] (raw file):

  // EventLogReverseSchemaChange is recorded when an in-progress schema change
  // encounters a problem and is reversed.
  EventLogReverseSchemaChange EventLogType = "reverse_schema_change"

s/reverse/reversing


sql/event_log.go, line 52 [r1] (raw file):

  // EventLogFinishSchemaChange is recorded when a previously initiated schema
  // change has completed.
  EventLogFinishSchemaChange EventLogType = "finish_schema_change"

s/finish/finished


Comments from Reviewable

@tamird
Copy link
Contributor

tamird commented Jul 1, 2016

Review status: all files reviewed at latest revision, 7 unresolved discussions, all commit checks successful.


sql/event_log.go, line 49 [r1] (raw file):

Previously, vivekmenezes wrote…

s/reverse/reversing

current tense seems consistent with the others in here

sql/event_log.go, line 52 [r1] (raw file):

Previously, vivekmenezes wrote…

s/finish/finished

current tense seems consistent with the others in here

Comments from Reviewable

@vivekmenezes
Copy link
Contributor

Review status: all files reviewed at latest revision, 7 unresolved discussions, all commit checks successful.


sql/event_log.go, line 49 [r1] (raw file):

Previously, tamird (Tamir Duberstein) wrote…

current tense seems consistent with the others in here

ok

sql/event_log.go, line 52 [r1] (raw file):

Previously, tamird (Tamir Duberstein) wrote…

current tense seems consistent with the others in here

ok

Comments from Reviewable

Commit adds three new event log types:
+ "Alter Table", logged for every ALTER TABLE DDL statement.
+ "Finish Schema Change", logged when an asynchronous schema change is
completed.
+ "Reverse Schema Change", logged when an asynchronous schema change encounters
an error (such as a constraint violation during backfill) and is rolled back.

This Commit encountered an interesting, perhaps unintended effect with
`SystemConfigTrigger`: in a "mixed" transaction which updates both system and
non-system keys, the system changes will only be gossiped if the transaction
record is located on the system range. In other words, "mixed" transactions will
only work correctly if the system keys are modified first. Issue cockroachdb#7570 had been
logged to track this unexpected behavior; this commit has also added a number of
comments and a logged error to address this.
@mrtracy mrtracy force-pushed the mtracy/alter_table_events branch from b71e9d8 to bde7d24 Compare July 7, 2016 18:31
@mrtracy
Copy link
Contributor Author

mrtracy commented Jul 7, 2016

Review status: 1 of 9 files reviewed at latest revision, 7 unresolved discussions, some commit checks pending.


sql/alter_table.go, line 246 [r1] (raw file):

Previously, tamird (Tamir Duberstein) wrote…

this comment isn't useful =/

Done.

sql/event_log.go, line 44 [r1] (raw file):

Previously, tamird (Tamir Duberstein) wrote…

none of the others reference a specific syntax. In fact, I think we run through AlterTable on ADD COLUMN, so this comment is inaccurate.

Done.

sql/schema_changer.go, line 520 [r1] (raw file):

Previously, tamird (Tamir Duberstein) wrote…

you might want to fmt.Sprintf("%+v", causingError); that will give you the error's stack trace if the error came from a call to errors.Errorf or one of its friends.

Done.

sql/testdata/event_log, line 263 [r1] (raw file):

Previously, tamird (Tamir Duberstein) wrote…

remove

Done.

Comments from Reviewable

@mrtracy
Copy link
Contributor Author

mrtracy commented Jul 7, 2016

Review status: 1 of 9 files reviewed at latest revision, 7 unresolved discussions, some commit checks pending.


sql/lease.go, line 340 [r1] (raw file):

Previously, tamird (Tamir Duberstein) wrote…

Slightly less code duplication:

txn.SetSystemConfigTrigger()
b := txn.NewBatch()
b.Put(descKey, desc)
if logEvent != nil {
  if err := txn.Run(b); err != nil {
    return err
  }
  if err := logEvent(txn); err != nil {
    return err
  }
  return txn.Commit()
}
return txn.CommitInBatch()
Done.

Comments from Reviewable

@mrtracy mrtracy merged commit d19df0f into cockroachdb:master Jul 7, 2016
@mrtracy mrtracy deleted the mtracy/alter_table_events branch July 7, 2016 19:52
@tamird
Copy link
Contributor

tamird commented Jul 9, 2016

Reviewed 8 of 8 files at r2.
Review status: all files reviewed at latest revision, 1 unresolved discussion, all commit checks successful.


sql/alter_table.go, line 246 [r1] (raw file):

Previously, mrtracy wrote…

Done.

I meant that you should remove it; it's still not useful - the line it's "documenting" is already clear and succinct.

Comments from Reviewable

pav-kv pushed a commit to pav-kv/cockroach that referenced this pull request Mar 5, 2024
advance() should use rs.req.Entries[0].Data as the context instead of
req.Context for deletion. Since req.Context is never set, there won't be
any context being deleted from pendingReadIndex; results mem leak.

FIXES cockroachdb#7571
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants