Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat[bmqstoragetool]: PrintManager #529

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

waldgange
Copy link
Collaborator

@waldgange waldgange commented Nov 24, 2024

In order to make the output of the storagetool more grep-friendly or/and to be able to parse it the following changes have been done:

  • a separate entity responsible for printing has been indroduced - the PrintManager
  • the existing printing logic has been moved to one of the implementations of the PrintManager
  • two new implementations of the PrintManager have been added: JsonLine (each Record is printed as a single-line json) and JsonPretty (each Record is printed as a multi-line json).
  • a new parameter to specify the print mode has been added:
--print-mode             <print mode> 
          can be one of the following {human|json-prety|json-line}
          (default: human)

@pniedzielski pniedzielski self-assigned this Dec 2, 2024
@waldgange waldgange force-pushed the grepfriend branch 5 times, most recently from de766a9 to 00c89fe Compare February 3, 2025 14:25
@waldgange waldgange marked this pull request as ready for review February 3, 2025 14:32
@waldgange waldgange requested a review from a team as a code owner February 3, 2025 14:32
@@ -44,8 +44,10 @@ Filters::Filters(const bsl::vector<bsl::string>& queueKeys,
mqbu::StorageKey key;
bsl::vector<bsl::string>::const_iterator uriIt = queueUris.cbegin();
for (; uriIt != queueUris.cend(); ++uriIt) {
if (queueMap.findKeyByUri(&key, *uriIt)) {
d_queueKeys.insert(key);
bsl::optional<mqbu::StorageKey> key = queueMap.findKeyByUri(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need optional here? Parameters::validateQueueNames guaranties that queue name is present in queueMap.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you suggesting to change the code to something like this:
d_queueKeys.insert(queueMap.findKeyByUri(&key, *uriIt).value()) ?
Without checking if the optional really has the value? I agree that all the URIs are meant to be valid at this point. But are we winning anything with this potentially dangerous code? A temporary optional inside the function will be constructed anyway. But RVO will prevent it from extra copying to outside the function. The second copying will happen when we insert the value. The only way to avoid it is to return a const reference (which is not convinient, as we can't check if it's valid or dangling without an extra bool variable) or a pointer (which is not safe to return) to the internal value of the queueMap. Also mqbu::StorageKey is a very lightweight object. sizeof(mqbu::StorageKey) is only 5 bytes, sizeof(bsl::optional<mqbu::StorageKey>) is 6 bytes which is less than even a pointer. So it's not worth optimizing anythng here.

@@ -76,6 +76,9 @@ int moveToLowerBound(mqbs::JournalFileIterator* jit,
}
left = jit->recordIndex();
}
static int i = 0;
bsl::cout << (++i) << " advance(" << bsl::max((right - left) / 2, 1ULL)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Debug?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. I removed this

@@ -2068,6 +2356,12 @@ static void test24_summaryWithQueueDetailsTest()
bmqtst::TestHelper::printTestName(
"OUTPUT SUMMARY WITH QUEUE DETAILS TEST");

bmqtst::TestHelperUtil::ignoreCheckDefAlloc() = true;
// Disable default allocator check for this test because
// EXPECT_CALL(PrinterMock::printRecordSummary(bsls::Types::Uint64,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Debug?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No. It's a useful comment. It describes what causes the default allocator use.


PRINTER_TYPE1 printer(ostream, &fields);
{
bsl::ostringstream s;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
bsl::ostringstream s;
bsl::ostringstream s(allocator);

@@ -17,7 +17,7 @@
#include <mqbs_filestoreprotocolprinter.h>

// MQB
#include <mqbs_filestoreprotocolutil.h>
// #include <mqbs_filestoreprotocolutil.h>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove if this include file not needed

Signed-off-by: Anton Pryakhin <[email protected]>
Signed-off-by: Anton Pryakhin <[email protected]>
Copy link

@bmq-oss-ci bmq-oss-ci bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Build 2454 of commit fdeaad4 has completed with FAILURE

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants