add support for repairCursor command with test coverage #41
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| @@ -1872,6 +1881,60 @@ func (c *Collection) Pipe(pipeline interface{}) *Pipe { | |||
| } | |||
| } | |||
|
|
|||
| func (c *Collection) Repair() *Iter { | |||
There was a problem hiding this comment.
Please provide user-oriented documentation. The other methods are a good example for the style in which it should be worded.
Also, is there any documentation at the docs site for what the command does?
There was a problem hiding this comment.
OK - added some docstring info to this method.
Unfortunately there's no information about this at the docs site since it's new feature, but I can look into getting it added there and then update this section when there's something I can link to.
| @@ -1872,6 +1881,65 @@ func (c *Collection) Pipe(pipeline interface{}) *Pipe { | |||
| } | |||
| } | |||
|
|
|||
| // Repair calls the 'repairCursor' command (supported in mongo 2.7.8 and later) | |||
| // and returns an iterator to go through the results. | |||
There was a problem hiding this comment.
Thanks, but I'd prefer to avoid mentioning implementation details. We don't do that in any other case.
Here is a suggested version, based on yours:
// Repair returns an iterator that goes over all recovered documents in the collection,
// in a best-effort manner. This is most useful when there are damaged data files.
// Multiple copies of the same document may be returned by the iterator.
//
// Repair is supported in MongoDB 2.7.8 and later.
(update: included the MongoDB version)
There was a problem hiding this comment.
Sounds good. Updated to use your suggested text.
|
Hey Gustavo, sorry but just wanted to check in on this - any other changes you'd like to see here? |
|
Hey Mike, Sorry for the feedback. This is looking good, and was already in my agenda for merging today. I'm going to merge it directly to v2 since it feels independent enough to not cause any harm to people that are not depending on the feature yet. Hopefully this will give you a hand there. |
|
Thanks Gustavo! |
bachue
pushed a commit
to bachue/mgo
that referenced
this pull request
Dec 20, 2017
* add DropAllIndexes() method (go-mgo#25) Create a new method to drop all the indexes of a collection in a single call * readme: credit @feliixx for go-mgo#25 (#26) * send metadata during handshake (#28) fix [#484](https://github.com/go-mgo/mgo/issues/484) Annotate connections with metadata provided by the connecting client. informations send: { "aplication": { // optional "name": "myAppName" } "driver": { "name": "mgo", "version": "v2" }, "os": { "type": runtime.GOOS, "architecture": runtime.GOARCH } } to set "application.name", add `appname` param in options of string connection URI, for example : "mongodb://localhost:27017?appname=myAppName" * Update README to add appName (go-mgo#32) * docs: elaborate on what appName does * readme: add appName to changes * add method CreateView() (go-mgo#33) Fix go-mgo#30. Thanks to @feliixx for the time and effort. * readme: credit @feliixx in the README (go-mgo#36) * Don't panic on indexed int64 fields (#23) * Stop all db instances after tests (go-mgo#462) If all tests pass, the builds for mongo earlier than 2.6 are still failing. Running a clean up fixes the issue. * fixing int64 type failing when getting indexes and trying to type them * requested changes relating to case statement and panic * Update README.md to credit @mapete94. * tests: ensure indexed int64 fields do not cause a panic in Indexes() See: * globalsign#23 * https://github.com/go-mgo/mgo/issues/475 * go-mgo#476 * Add collation option to collection.Create() (go-mgo#37) - Allow specifying the default collation for the collection when creating it. - Add some documentation to query.Collation() method. fix go-mgo#29 * Test against MongoDB 3.4.x (go-mgo#35) * test against MongoDB 3.4.x * tests: use listIndexes to assert index state for 3.4+ * make test pass against v3.4.x - skip `TestViewWithCollation` because of SERVER-31049, cf: https://jira.mongodb.org/browse/SERVER-31049 - add versionAtLeast() method in init.js script to better detect server version fixes go-mgo#31 * Introduce constants for BSON element types (go-mgo#41) * bson.Unmarshal returns time in UTC (go-mgo#42) * readme: add missing features / credit * Adds missing collation feature description (by @feliixx). * Adds missing 3.4 tests description (by @feliixx). * Adds BSON constants description (by @bozaro). * Adds UTC time.Time unmarshalling (by @gazoon). * fix golint, go vet and gofmt warnings (#44) Fixes #43 * readme: credit @feliixx (#46) * Fix GetBSON() method usage (go-mgo#40) * Fix GetBSON() method usage Original issue --- You can't use type with custom GetBSON() method mixed with structure field type and structure field reference type. For example, you can't create custom GetBSON() for Bar type: ``` struct Foo { a Bar b *Bar } ``` Type implementation (`func (t Bar) GetBSON()` ) would crash on `Foo.b = nil` value encoding. Reference implementation (`func (t *Bar) GetBSON()` ) would not call on `Foo.a` value encoding. After this change --- For type implementation `func (t Bar) GetBSON()` would not call on `Foo.b = nil` value encoding. In this case `nil` value would be seariazied as `nil` BSON value. For reference implementation `func (t *Bar) GetBSON()` would call even on `Foo.a` value encoding. * Minor refactoring * readme: credit @bozaro (#47)
bachue
pushed a commit
to bachue/mgo
that referenced
this pull request
Dec 20, 2017
* add DropAllIndexes() method (go-mgo#25) Create a new method to drop all the indexes of a collection in a single call * readme: credit @feliixx for go-mgo#25 (#26) * send metadata during handshake (#28) fix [#484](https://github.com/go-mgo/mgo/issues/484) Annotate connections with metadata provided by the connecting client. informations send: { "aplication": { // optional "name": "myAppName" } "driver": { "name": "mgo", "version": "v2" }, "os": { "type": runtime.GOOS, "architecture": runtime.GOARCH } } to set "application.name", add `appname` param in options of string connection URI, for example : "mongodb://localhost:27017?appname=myAppName" * Update README to add appName (go-mgo#32) * docs: elaborate on what appName does * readme: add appName to changes * add method CreateView() (go-mgo#33) Fix go-mgo#30. Thanks to @feliixx for the time and effort. * readme: credit @feliixx in the README (go-mgo#36) * Don't panic on indexed int64 fields (#23) * Stop all db instances after tests (go-mgo#462) If all tests pass, the builds for mongo earlier than 2.6 are still failing. Running a clean up fixes the issue. * fixing int64 type failing when getting indexes and trying to type them * requested changes relating to case statement and panic * Update README.md to credit @mapete94. * tests: ensure indexed int64 fields do not cause a panic in Indexes() See: * globalsign#23 * https://github.com/go-mgo/mgo/issues/475 * go-mgo#476 * Add collation option to collection.Create() (go-mgo#37) - Allow specifying the default collation for the collection when creating it. - Add some documentation to query.Collation() method. fix go-mgo#29 * Test against MongoDB 3.4.x (go-mgo#35) * test against MongoDB 3.4.x * tests: use listIndexes to assert index state for 3.4+ * make test pass against v3.4.x - skip `TestViewWithCollation` because of SERVER-31049, cf: https://jira.mongodb.org/browse/SERVER-31049 - add versionAtLeast() method in init.js script to better detect server version fixes go-mgo#31 * Introduce constants for BSON element types (go-mgo#41) * bson.Unmarshal returns time in UTC (go-mgo#42) * readme: add missing features / credit * Adds missing collation feature description (by @feliixx). * Adds missing 3.4 tests description (by @feliixx). * Adds BSON constants description (by @bozaro). * Adds UTC time.Time unmarshalling (by @gazoon). * fix golint, go vet and gofmt warnings (#44) Fixes #43 * readme: credit @feliixx (#46) * Fix GetBSON() method usage (go-mgo#40) * Fix GetBSON() method usage Original issue --- You can't use type with custom GetBSON() method mixed with structure field type and structure field reference type. For example, you can't create custom GetBSON() for Bar type: ``` struct Foo { a Bar b *Bar } ``` Type implementation (`func (t Bar) GetBSON()` ) would crash on `Foo.b = nil` value encoding. Reference implementation (`func (t *Bar) GetBSON()` ) would not call on `Foo.a` value encoding. After this change --- For type implementation `func (t Bar) GetBSON()` would not call on `Foo.b = nil` value encoding. In this case `nil` value would be seariazied as `nil` BSON value. For reference implementation `func (t *Bar) GetBSON()` would call even on `Foo.a` value encoding. * Minor refactoring * readme: credit @bozaro (#47) * Improve cursorData struct unmarshaling speed (#49) This change remove full BSON decoding on: - parsing to `bson.Raw` and `bson.DocElem` fields; - skipping unused BSON fields. * readme: credit @bozaro and @idy (go-mgo#53) * readme: credit @bozaro and @idy * readme: add @idy to contributor list * do not lock while writing to a socket (#52) (#54) fix go-mgo#51
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
A new command called "repairCursor" was added to the server in 2.7.8 (see https://jira.mongodb.org/browse/SERVER-15544 for details). We need support for this in the mgo driver so that tools which involve database repair operations (like mongodump) can access it.
It's implemented as a command which returns a cursor; the server performs a best-effort attempt to recover any data from damaged data files and return the documents it finds on the cursor - so, it could potentially return multiple copies of the same document. I've written the test to account for this behavior.