Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Documentation on settings and warning message when using a documents archive file #264

Closed
ppf2 opened this issue Apr 12, 2017 · 1 comment
Labels
:Docs Changes to the documentation enhancement Improves the status quo :Track Management New operations, changes in the track format, track download changes and the like
Milestone

Comments

@ppf2
Copy link
Member

ppf2 commented Apr 12, 2017

Per the documentation:

http://esrally.readthedocs.io/en/latest/track.html?highlight=compressed_size_in_bytes#indices

document-count (optional): Number of documents in the documents file. This number will be used to verify that all documents have been indexed successfully.
compressed-bytes (optional): The size in bytes of the compressed document file. This number is used to show users how much data will be downloaded by Rally and also to check whether the download is complete.
uncompressed-bytes (optional): The size in bytes of the documents file after decompression.

The above settings are optional. But based on the code comments, these are needed if a document archive file is provided for proper recording of progress and verification of download and uncompression:

        :param number_of_documents: The number of documents in the benchmark document. Needed for proper progress reporting. Only needed if
         a document_archive is given.
        :param compressed_size_in_bytes: The compressed size in bytes of the benchmark document. Needed for verification of the download and
         user reporting. Only needed if a document_archive is given.
        :param uncompressed_size_in_bytes: The size in bytes of the benchmark document after decompressing it. Only needed if a
        document_archive is given.

If these settings are not provided, then when using an archive file, it will throw this ambiguous warning message:

[WARNING] None of the types for index [index_name] defines documents. Please check that you either don't want to index data or parameter sources are defined for indexing.

Because these settings are all initialized to be 0, and we check to make sure that they are >0 when determining if there is any valid documents for the type.

It will be nice to sync up the documentation with the comments in the code, and perhaps a more intuitive warning message?

@ppf2 ppf2 added the :Docs Changes to the documentation label Apr 12, 2017
@danielmitterdorfer danielmitterdorfer added this to the 0.5.x milestone Apr 13, 2017
@danielmitterdorfer danielmitterdorfer added :Track Management New operations, changes in the track format, track download changes and the like enhancement Improves the status quo labels May 24, 2017
@danielmitterdorfer danielmitterdorfer modified the milestones: 0.5.4, 0.5.x May 24, 2017
@danielmitterdorfer
Copy link
Member

document-count is still needed but the other two settings are optional now. I've improved the validation code in the track loader and updated the documentation so it matches the implementation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Docs Changes to the documentation enhancement Improves the status quo :Track Management New operations, changes in the track format, track download changes and the like
Projects
None yet
Development

No branches or pull requests

2 participants