Adds support for custom TTL of repaired docs and ViewQueryPageSize defaults #2902
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 2 commits
September 14, 2017 13:56
…epairBucketParams.ViewQueryPageSize when not set
|
Can you recreate this PR against 1.3.1.4? |
|
@adamcfraser done, it was coded off release/1.3.1.4 branch, but I ended up on a detached HEAD when I pushed to GitHub. |
adamcfraser
requested changes
Sep 14, 2017
db/repair_bucket.go
Outdated
| const ( | ||
| RepairRevTreeCycles = RepairJobType("RepairRevTreeCycles") | ||
| ) | ||
|
|
||
| // Params suitable for external (eg, HTTP) invocations to describe a RepairBucket operation | ||
| type RepairBucketParams struct { | ||
| DryRun bool `json:"dry_run"` | ||
| ViewQueryPageSize int `json:"view_query_page_size"` | ||
| ViewQueryPageSize *int `json:"view_query_page_size"` | ||
| RepairedFileTTL *int `json:"repaired_file_ttl_seconds"` |
There was a problem hiding this comment.
Can we rename this to "repair_output_ttl_seconds"? As written, it sounds like the repaired file itself gets a ttl, which is a bit confusing.
There was a problem hiding this comment.
Modified parameter name is pushed up
adamcfraser
changed the title
tleyden
pushed a commit
that referenced
this pull request
Sep 27, 2017
…afults (#2902) Adds support for custom TTL of repaired docs and ViewQueryPageSize defaults
tleyden
pushed a commit
that referenced
this pull request
Sep 29, 2017
…afults (#2902) Adds support for custom TTL of repaired docs and ViewQueryPageSize defaults
tleyden
pushed a commit
that referenced
this pull request
Oct 3, 2017
…afults (#2902) Adds support for custom TTL of repaired docs and ViewQueryPageSize defaults
adamcfraser
pushed a commit
that referenced
this pull request
Oct 3, 2017
…go into infinite loop + rev tree repair tool) (#2869) * Fixes #2847 Getting doc history can go into infinite loop: Backport for 1.4.1.2 (#2858) * Fixes #2847 Getting doc history can go into infinite loop: Backport for 1.3.1.2 (#2856) * Repro case for #2847 - Getting doc history can go into infinite loop #2847 * Add revTree.Validate() method * Invoke rawDoc.History.RenderGraphvizDot(), change timeout checking * Add new test * Cherry pick commit from feature/issue_2847_cycles Cherry pick commit d8feb1d from feature/issue_2847_cycles, which is based on the master branch * Fixes issue #2847 by wwitching the dangling parent check in pruneRevisions to happen after branches are deleted #2847 (comment) * Comments on test * Fixes issue #2847 by fixing marshalJSON to better handle dangling parents during marshal #2847 (comment) * Replace getHistory with getValidatedHistory TestRevsHistoryInfiniteLoop now passes * Rename getValidatedHistory -> getHistory * Gofmt * Remove parent rev from history string * Remove unneeded fmt.Sprintf() * Run sg-accel tests * Try pointing to specific commit to fix build failure * Use git url instead of ssh for sga accel repo * Revert "Use git url instead of ssh for sga accel repo" This reverts commit 1c5e061. * Revert "Try pointing to specific commit to fix build failure" This reverts commit dd3f9d9. * Revert "Run sg-accel tests" This reverts commit d5cc940. * Remove rawDoc.History.Validate(). Does not help test catch any issues. * Remove commented log * Remove unneeded import # Conflicts: # db/revtree_test.go * Fix compile errors # Conflicts: # db/crud.go # db/revtree_test.go * Fixes #2857: Revision tree repair tool (#2866) * TestRepairRevsHistoryWithCycles() * Add Repair() — unit test passes now * Add repair bucket (in progress) * Test passes * Fix dry run * TestRepairBucketRevTreeCycles() passes * Add _repair endpoint * Fix invocation of InitFrom * Refactor RepairBucket() to return repaired docs to enable more test assertions * WriteRepairedDocsToDisk by default to make diagnosis of repair tool easier * Run gofmt + goimports * PR feedback, remove redundant repair_job * DocTransformer takes raw bytes instead of the marshalled document * Return RepairBucketResult with doc and and repair job * Update _repair endpoint to marshal result to response * Add WriteRepairedDocsToBucket(), fix super nasty dev bug along the way * Fix bug in WriteRepairedDocsToBucket() * Change to 24 hours and fix up _sync: doc id * gofmt * Return DryRun and BackupOrDryRunDocId in results * Repair() -> RepairCycles() * More test docs * Handle bucket.Update err * Add TestRepairBucketDryRun() * Use bucket.GetRaw() * Gofmt + goimports * TestRepairBucket had wrong number of docs in assertion + saw error on drone that made me think there is interference between walrus buckets w/ same name. * Fix compile error * Fix null return value when no repairs done * Change from Warn -> Crud * Disable _repair endpoint * Fix test compile errors * Fixes #2892 repair tool efficiency (#2893) * Fixes #2892 - Repair tool efficiency improvements Initial first pass at iterating over the view w/ paging * Add documentation * More comments * Comment regarding min pageSIzeViewResult * pull ViewQueryPageSize out to a constant. Enhance tests to add more docs to excercise the view iteration * Address PR feedback * Update comment * Change constant to DefaultViewQueryPageSize * Run gofmt # Conflicts: # base/constants.go # db/repair_bucket_test.go * Adds support for custom TTL of repaired docs and ViewQueryPageSize deafults (#2902) Adds support for custom TTL of repaired docs and ViewQueryPageSize defaults * correctly apply _repair parameter value for params.RepairedFileTTL (#2909) * Fixes #2919 Revtree repair tool gets stuck if node points to itself a… (#2920) * Fixes #2919 Revtree repair tool gets stuck if node points to itself as parent * PR feedback
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
fixes #2892