Add 'Tracker' track-generation tool #875
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
drawlerr
added
:Track Management
added 3 commits
January 17, 2020 11:09
Pylint warns against "self imports" unless the importing code is part of a module as defined by the presence of __init__.py.
Refactored Tracker to support multiple indexes on a single Track, as in http_logs. Also including fix for logging issue introduced in the move from private repo to Rally.
added 7 commits
January 21, 2020 09:43
- Change elasticsearch import to occur later instead of at init - Use proper logger instead of root logger - Use Rally's console progress reporter instead of naked print
drawlerr
commented
Jan 22, 2020
|
@elasticmachine test this |
|
@elasticmachine run tests |
danielmitterdorfer
requested changes
Feb 11, 2020
There was a problem hiding this comment.
Thanks for the tool; it will be really helpful to our users. I did a first pass. Some high-level comments, more comments inline:
- The Tracker unit tests are not run when invoking
make test. For consistency we should maybe move thetrackermodule intoesrallyand have the tests for Tracker in the top-leveltestsdirectory. Also Tracker tests are currently not excluded from the Rally binary. - IMHO there should be an integration test for it.
- None of the
__init__.pyfiles contains the standard license header and I think we should add them to all files, regardless whether they are empty now. It's too easy to forget later on. - I think we should revisit the concept of dumping all log output to the terminal and instead provide some high-level information (and use the
consoleabstraction from Rally for this). Having the log output written to the current directory makes sense to me though. - I see that you opted for the most flexible challenge format but we should keep the track as simple as possible and generate just one
track.jsonfile instead of having separate files for challenges. IMHO we should even avoid the challenge block and just specify thescheduleon top level. - It would be nice if we would also prepare the track for
--test-modealready. - When something goes wrong, the user is presented the complete trace (e.g. when an index is not found). We should provide more helpful error messages for cases that we can anticipate.
- We are extracting data from indices one by one so if anything goes wrong (e.g. a typo in one of the index names) we might have an inconsistent local state and would need to redo everything again (including having to potentially download and compress multiple GB of data). IMHO we should at least verify that all of the specified indices exist and abort early.
- The docs here are a start but I think we need to provide more examples and cross-reference it to other parts of the docs. There is also no example how you'd actually execute such a generated track.
- We should surround parameters with
[]in log messages (similar to Elasticsearch and Rally) to better separate them from the rest of the text and be able to identify whitespaces and empty values. estracker --versiononly dumps an error message mentioned required parameters but it should show a version output that is consistent withesrally --version(We will need this in error reports)- We should provide pointers for users where to look for help (compare what Rally does when it encounters an error).
- I am not so sure about the inconsistent usage of the classic path access APIs and the introduction of the Path object API introduced here. We should embrace either one or the other in the code base but not both.
tracker/tracker.py
Outdated
|
|
||
| parser.add_argument("--target-hosts", default="", required=True, help="Elasticsearch host(s) to connect to") | ||
| parser.add_argument("--client-options", default=opts.ClientOptions.DEFAULT_CLIENT_OPTIONS, help="Elasticsearch client options") | ||
| parser.add_argument("--indices", nargs='+', required=True, help="Indices to include in track") |
There was a problem hiding this comment.
Use double-quotes instead of single-quotes (I will only mention this once here but you use single-quotes in a lot of places).
There was a problem hiding this comment.
Hopefully I got most of them in 0848be0
tracker/tracker.py
Outdated
| corpus_vars = corpus.extract(client, outpath, index_name) | ||
| corpora.append(corpus_vars) | ||
|
|
||
| template_vars.update({ |
There was a problem hiding this comment.
Declare template_vars here so we can avoid the update call?
There was a problem hiding this comment.
Addressed in 4be9f72
tracker/corpus.py
Outdated
| return template_vars(index, outpath, comp_outpath, total_docs) | ||
|
|
||
|
|
||
| def render_progress(progress, cur, total, freq): |
There was a problem hiding this comment.
Great idea; I like the progress indicator a lot! :)
tracker/corpus.py
Outdated
| def render_progress(progress, cur, total, freq): | ||
| if cur % freq == 0 or total - cur < freq: | ||
| percent = (cur * 100) / total | ||
| progress.print("Extracting documents...", "{n}/{total_docs} ({percent:.1f}%)".format(n=cur, total_docs=total, percent=percent)) |
There was a problem hiding this comment.
How about we mention the index name in the output message?
danielmitterdorfer
added
enhancement
added 7 commits
February 19, 2020 09:22
- Use 7.6.0 instead of random_dist. Not planning on supporting pre-typeless-ES right now. - Use random trackname - cleanup IT track on completion
�[1m============================= test session starts ==============================�[0m platform darwin -- Python 3.5.8, pytest-5.2.0, py-1.8.1, pluggy-0.13.1 -- /Users/dlawler/Code/rally/.venv/bin/python3 cachedir: .pytest_cache benchmark: 3.2.2 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000) rootdir: /Users/dlawler/Code/rally, inifile: setup.cfg plugins: benchmark-3.2.2 �[1mcollecting ... �[0mcollected 696 items tests/client_test.py::EsClientFactoryTests::test_create_http_connection �[32mPASSED�[0m�[36m [ 0%]�[0m tests/client_test.py::EsClientFactoryTests::test_create_https_connection_only_verify_self_signed_server_certificate �[32mPASSED�[0m�[36m [ 0%]�[0m tests/client_test.py::EsClientFactoryTests::test_create_https_connection_unverified_certificate �[32mPASSED�[0m�[36m [ 0%]�[0m tests/client_test.py::EsClientFactoryTests::test_create_https_connection_unverified_certificate_present_client_certificates �[32mPASSED�[0m�[36m [ 0%]�[0m tests/client_test.py::EsClientFactoryTests::test_create_https_connection_verify_self_signed_server_and_client_certificate �[32mPASSED�[0m�[36m [ 0%]�[0m tests/client_test.py::EsClientFactoryTests::test_create_https_connection_verify_server �[32mPASSED�[0m�[36m [ 0%]�[0m tests/client_test.py::EsClientFactoryTests::test_raises_error_when_only_one_of_client_cert_and_client_key_defined �[32mPASSED�[0m�[36m [ 1%]�[0m tests/client_test.py::RestLayerTests::test_dont_retry_eternally_on_transport_errors �[32mPASSED�[0m�[36m [ 1%]�[0m tests/client_test.py::RestLayerTests::test_retries_on_transport_errors �[32mPASSED�[0m�[36m [ 1%]�[0m tests/client_test.py::RestLayerTests::test_ssl_error �[32mPASSED�[0m�[36m [ 1%]�[0m tests/client_test.py::RestLayerTests::test_successfully_waits_for_rest_layer �[32mPASSED�[0m�[36m [ 1%]�[0m tests/config_test.py::ConfigTests::test_add_all_in_section �[32mPASSED�[0m�[36m [ 1%]�[0m tests/config_test.py::ConfigTests::test_load_all_opts_in_section �[32mPASSED�[0m�[36m [ 1%]�[0m tests/config_test.py::ConfigTests::test_load_existing_config �[32mPASSED�[0m�[36m [ 2%]�[0m tests/config_test.py::ConfigTests::test_load_non_existing_config �[32mPASSED�[0m�[36m [ 2%]�[0m tests/config_test.py::AutoLoadConfigTests::test_can_create_non_existing_config �[32mPASSED�[0m�[36m [ 2%]�[0m tests/config_test.py::AutoLoadConfigTests::test_can_load_and_amend_existing_config �[32mPASSED�[0m�[36m [ 2%]�[0m tests/config_test.py::AutoLoadConfigTests::test_can_migrate_outdated_config �[32mPASSED�[0m�[36m [ 2%]�[0m tests/config_test.py::ConfigFactoryTests::test_create_advanced_config �[32mPASSED�[0m�[36m [ 2%]�[0m tests/config_test.py::ConfigFactoryTests::test_create_simple_config �[32mPASSED�[0m�[36m [ 2%]�[0m tests/config_test.py::ConfigMigrationTests::test_does_not_migrate_outdated_config �[32mPASSED�[0m�[36m [ 3%]�[0m tests/config_test.py::ConfigMigrationTests::test_migrate_from_12_to_13_with_gradle_and_jdk8_ask_user_and_skip �[32mPASSED�[0m�[36m [ 3%]�[0m tests/config_test.py::ConfigMigrationTests::test_migrate_from_12_to_13_with_gradle_and_jdk8_ask_user_enter_valid �[32mPASSED�[0m�[36m [ 3%]�[0m tests/config_test.py::ConfigMigrationTests::test_migrate_from_12_to_13_with_gradle_and_jdk8_autodetect_jdk9 �[32mPASSED�[0m�[36m [ 3%]�[0m tests/config_test.py::ConfigMigrationTests::test_migrate_from_12_to_13_with_gradle_and_jdk9 �[32mPASSED�[0m�[36m [ 3%]�[0m tests/config_test.py::ConfigMigrationTests::test_migrate_from_12_to_13_without_gradle �[32mPASSED�[0m�[36m [ 3%]�[0m tests/config_test.py::ConfigMigrationTests::test_migrate_from_13_to_14_with_gradle_and_jdk10 �[32mPASSED�[0m�[36m [ 3%]�[0m tests/config_test.py::ConfigMigrationTests::test_migrate_from_13_to_14_with_gradle_and_jdk8_ask_user_and_skip �[32mPASSED�[0m�[36m [ 4%]�[0m tests/config_test.py::ConfigMigrationTests::test_migrate_from_13_to_14_with_gradle_and_jdk8_ask_user_enter_valid �[32mPASSED�[0m�[36m [ 4%]�[0m tests/config_test.py::ConfigMigrationTests::test_migrate_from_13_to_14_with_gradle_and_jdk8_autodetect_jdk10 �[32mPASSED�[0m�[36m [ 4%]�[0m tests/config_test.py::ConfigMigrationTests::test_migrate_from_13_to_14_without_gradle �[32mPASSED�[0m�[36m [ 4%]�[0m tests/config_test.py::ConfigMigrationTests::test_migrate_from_14_to_15_with_gradle �[32mPASSED�[0m�[36m [ 4%]�[0m tests/config_test.py::ConfigMigrationTests::test_migrate_from_14_to_15_with_source_plugin_definition �[32mPASSED�[0m�[36m [ 4%]�[0m tests/config_test.py::ConfigMigrationTests::test_migrate_from_14_to_15_without_gradle �[32mPASSED�[0m�[36m [ 4%]�[0m tests/config_test.py::ConfigMigrationTests::test_migrate_from_15_to_16 �[32mPASSED�[0m�[36m [ 5%]�[0m tests/config_test.py::ConfigMigrationTests::test_migrate_from_16_to_17 �[32mPASSED�[0m�[36m [ 5%]�[0m tests/config_test.py::ConfigMigrationTests::test_migrate_from_earliest_supported_to_latest �[32mPASSED�[0m�[36m [ 5%]�[0m tests/metrics_test.py::ExtractUserTagsTests::test_extracts_proper_user_tags �[32mPASSED�[0m�[36m [ 5%]�[0m tests/metrics_test.py::ExtractUserTagsTests::test_missing_comma_raises_error �[32mPASSED�[0m�[36m [ 5%]�[0m tests/metrics_test.py::ExtractUserTagsTests::test_missing_value_raises_error �[32mPASSED�[0m�[36m [ 5%]�[0m tests/metrics_test.py::ExtractUserTagsTests::test_no_tags_returns_empty_dict �[32mPASSED�[0m�[36m [ 5%]�[0m tests/metrics_test.py::EsClientTests::test_config_opts_parsing �[32mPASSED�[0m�[36m [ 6%]�[0m tests/metrics_test.py::EsClientTests::test_fails_after_too_many_errors �[32mPASSED�[0m�[36m [ 6%]�[0m tests/metrics_test.py::EsClientTests::test_raises_rally_error_on_unknown_problems �[32mPASSED�[0m�[36m [ 6%]�[0m tests/metrics_test.py::EsClientTests::test_raises_sytem_setup_error_on_authentication_problems �[32mPASSED�[0m�[36m [ 6%]�[0m tests/metrics_test.py::EsClientTests::test_raises_sytem_setup_error_on_authorization_problems �[32mPASSED�[0m�[36m [ 6%]�[0m tests/metrics_test.py::EsClientTests::test_raises_sytem_setup_error_on_connection_problems �[32mPASSED�[0m�[36m [ 6%]�[0m tests/metrics_test.py::EsClientTests::test_retries_on_various_transport_errors �[32mPASSED�[0m�[36m [ 6%]�[0m tests/metrics_test.py::EsMetricsTests::test_get_error_rate_additional_unknown_key �[32mPASSED�[0m�[36m [ 7%]�[0m tests/metrics_test.py::EsMetricsTests::test_get_error_rate_explicit_one �[32mPASSED�[0m�[36m [ 7%]�[0m tests/metrics_test.py::EsMetricsTests::test_get_error_rate_explicit_zero �[32mPASSED�[0m�[36m [ 7%]�[0m tests/metrics_test.py::EsMetricsTests::test_get_error_rate_implicit_one �[32mPASSED�[0m�[36m [ 7%]�[0m tests/metrics_test.py::EsMetricsTests::test_get_error_rate_implicit_zero �[32mPASSED�[0m�[36m [ 7%]�[0m tests/metrics_test.py::EsMetricsTests::test_get_error_rate_mixed �[32mPASSED�[0m�[36m [ 7%]�[0m tests/metrics_test.py::EsMetricsTests::test_get_mean �[32mPASSED�[0m�[36m [ 7%]�[0m tests/metrics_test.py::EsMetricsTests::test_get_median �[32mPASSED�[0m�[36m [ 8%]�[0m tests/metrics_test.py::EsMetricsTests::test_get_per_node_value �[32mPASSED�[0m�[36m [ 8%]�[0m tests/metrics_test.py::EsMetricsTests::test_get_value �[32mPASSED�[0m�[36m [ 8%]�[0m tests/metrics_test.py::EsMetricsTests::test_put_doc_no_meta_data �[32mPASSED�[0m�[36m [ 8%]�[0m tests/metrics_test.py::EsMetricsTests::test_put_doc_with_metadata �[32mPASSED�[0m�[36m [ 8%]�[0m tests/metrics_test.py::EsMetricsTests::test_put_value_with_explicit_timestamps �[32mPASSED�[0m�[36m [ 8%]�[0m tests/metrics_test.py::EsMetricsTests::test_put_value_with_meta_info �[32mPASSED�[0m�[36m [ 8%]�[0m tests/metrics_test.py::EsMetricsTests::test_put_value_without_meta_info �[32mPASSED�[0m�[36m [ 9%]�[0m tests/metrics_test.py::EsRaceStoreTests::test_does_not_find_missing_race_by_race_id �[32mPASSED�[0m�[36m [ 9%]�[0m tests/metrics_test.py::EsRaceStoreTests::test_find_existing_race_by_race_id �[32mPASSED�[0m�[36m [ 9%]�[0m tests/metrics_test.py::EsRaceStoreTests::test_store_race �[32mPASSED�[0m�[36m [ 9%]�[0m tests/metrics_test.py::EsResultsStoreTests::test_store_results �[32mPASSED�[0m�[36m [ 9%]�[0m tests/metrics_test.py::InMemoryMetricsStoreTests::test_externalize_and_bulk_add �[32mPASSED�[0m�[36m [ 9%]�[0m tests/metrics_test.py::InMemoryMetricsStoreTests::test_get_error_rate_by_sample_type �[32mPASSED�[0m�[36m [ 9%]�[0m tests/metrics_test.py::InMemoryMetricsStoreTests::test_get_error_rate_mixed �[32mPASSED�[0m�[36m [ 10%]�[0m tests/metrics_test.py::InMemoryMetricsStoreTests::test_get_error_rate_zero_without_samples �[32mPASSED�[0m�[36m [ 10%]�[0m tests/metrics_test.py::InMemoryMetricsStoreTests::test_get_mean �[32mPASSED�[0m�[36m [ 10%]�[0m tests/metrics_test.py::InMemoryMetricsStoreTests::test_get_median �[32mPASSED�[0m�[36m [ 10%]�[0m tests/metrics_test.py::InMemoryMetricsStoreTests::test_get_percentile �[32mPASSED�[0m�[36m [ 10%]�[0m tests/metrics_test.py::InMemoryMetricsStoreTests::test_get_value �[32mPASSED�[0m�[36m [ 10%]�[0m tests/metrics_test.py::InMemoryMetricsStoreTests::test_meta_data_per_document �[32mPASSED�[0m�[36m [ 10%]�[0m tests/metrics_test.py::FileRaceStoreTests::test_race_not_found �[32mPASSED�[0m�[36m [ 11%]�[0m tests/metrics_test.py::FileRaceStoreTests::test_store_race �[32mPASSED�[0m�[36m [ 11%]�[0m tests/metrics_test.py::StatsCalculatorTests::test_calculate_global_stats �[32mPASSED�[0m�[36m [ 11%]�[0m tests/metrics_test.py::StatsCalculatorTests::test_calculate_system_stats �[32mPASSED�[0m�[36m [ 11%]�[0m tests/metrics_test.py::GlobalStatsTests::test_as_flat_list �[32mPASSED�[0m�[36m [ 11%]�[0m tests/metrics_test.py::SystemStatsTests::test_as_flat_list �[32mPASSED�[0m�[36m [ 11%]�[0m tests/racecontrol_test.py::RaceControlTests::test_fails_without_benchmark_only_pipeline_in_docker �[32mPASSED�[0m�[36m [ 11%]�[0m tests/racecontrol_test.py::RaceControlTests::test_finds_available_pipelines �[32mPASSED�[0m�[36m [ 12%]�[0m tests/racecontrol_test.py::RaceControlTests::test_passes_benchmark_only_pipeline_in_docker �[32mPASSED�[0m�[36m [ 12%]�[0m tests/racecontrol_test.py::RaceControlTests::test_prevents_running_an_unknown_pipeline �[32mPASSED�[0m�[36m [ 12%]�[0m tests/racecontrol_test.py::RaceControlTests::test_runs_a_known_pipeline �[32mPASSED�[0m�[36m [ 12%]�[0m tests/reporter_test.py::FormatterTests::test_formats_as_csv �[32mPASSED�[0m�[36m [ 12%]�[0m tests/reporter_test.py::FormatterTests::test_formats_as_markdown �[32mPASSED�[0m�[36m [ 12%]�[0m tests/telemetry_test.py::TelemetryTests::test_merges_options_set_by_different_devices �[32mPASSED�[0m�[36m [ 12%]�[0m tests/telemetry_test.py::StartupTimeTests::test_store_calculated_metrics �[32mPASSED�[0m�[36m [ 13%]�[0m tests/telemetry_test.py::JfrTests::test_sets_options_for_java_11_or_above_custom_recording_template �[32mPASSED�[0m�[36m [ 13%]�[0m tests/telemetry_test.py::JfrTests::test_sets_options_for_java_11_or_above_default_recording_template �[32mPASSED�[0m�[36m [ 13%]�[0m tests/telemetry_test.py::JfrTests::test_sets_options_for_java_9_or_10_custom_recording_template �[32mPASSED�[0m�[36m [ 13%]�[0m tests/telemetry_test.py::JfrTests::test_sets_options_for_java_9_or_10_default_recording_template �[32mPASSED�[0m�[36m [ 13%]�[0m tests/telemetry_test.py::JfrTests::test_sets_options_for_pre_java_9_custom_recording_template �[32mPASSED�[0m�[36m [ 13%]�[0m tests/telemetry_test.py::JfrTests::test_sets_options_for_pre_java_9_default_recording_template �[32mPASSED�[0m�[36m [ 13%]�[0m tests/telemetry_test.py::GcTests::test_sets_options_for_java_9_or_above �[32mPASSED�[0m�[36m [ 14%]�[0m tests/telemetry_test.py::GcTests::test_sets_options_for_pre_java_9 �[32mPASSED�[0m�[36m [ 14%]�[0m tests/telemetry_test.py::HeapdumpTests::test_generates_heap_dump �[32mPASSED�[0m�[36m [ 14%]�[0m tests/telemetry_test.py::CcrStatsTests::test_negative_sample_interval_forbidden �[32mPASSED�[0m�[36m [ 14%]�[0m tests/telemetry_test.py::CcrStatsTests::test_wrong_cluster_name_in_ccr_stats_indices_forbidden �[32mPASSED�[0m�[36m [ 14%]�[0m tests/telemetry_test.py::CcrStatsRecorderTests::test_raises_exception_on_transport_error �[32mPASSED�[0m�[36m [ 14%]�[0m tests/telemetry_test.py::CcrStatsRecorderTests::test_stores_default_ccr_stats �[32mPASSED�[0m�[36m [ 14%]�[0m tests/telemetry_test.py::CcrStatsRecorderTests::test_stores_default_ccr_stats_many_shards �[32mPASSED�[0m�[36m [ 15%]�[0m tests/telemetry_test.py::CcrStatsRecorderTests::test_stores_filtered_ccr_stats �[32mPASSED�[0m�[36m [ 15%]�[0m tests/telemetry_test.py::RecoveryStatsTests::test_no_metrics_if_no_pending_recoveries �[32mPASSED�[0m�[36m [ 15%]�[0m tests/telemetry_test.py::RecoveryStatsTests::test_stores_multi_index_multi_shard_stats �[32mPASSED�[0m�[36m [ 15%]�[0m tests/telemetry_test.py::RecoveryStatsTests::test_stores_single_shard_stats �[32mPASSED�[0m�[36m [ 15%]�[0m tests/telemetry_test.py::NodeStatsTests::test_prints_warning_using_node_stats �[32mPASSED�[0m�[36m [ 15%]�[0m tests/telemetry_test.py::NodeStatsRecorderTests::test_exception_when_include_indices_metrics_not_valid �[32mPASSED�[0m�[36m [ 15%]�[0m tests/telemetry_test.py::NodeStatsRecorderTests::test_flatten_indices_fields �[32mPASSED�[0m�[36m [ 16%]�[0m tests/telemetry_test.py::NodeStatsRecorderTests::test_negative_sample_interval_forbidden �[32mPASSED�[0m�[36m [ 16%]�[0m tests/telemetry_test.py::NodeStatsRecorderTests::test_stores_all_nodes_stats �[32mPASSED�[0m�[36m [ 16%]�[0m tests/telemetry_test.py::NodeStatsRecorderTests::test_stores_default_nodes_stats �[32mPASSED�[0m�[36m [ 16%]�[0m tests/telemetry_test.py::NodeStatsRecorderTests::test_stores_selected_indices_metrics_from_nodes_stats �[32mPASSED�[0m�[36m [ 16%]�[0m tests/telemetry_test.py::ClusterEnvironmentInfoTests::test_resilient_if_error_response �[32mPASSED�[0m�[36m [ 16%]�[0m tests/telemetry_test.py::ClusterEnvironmentInfoTests::test_stores_cluster_level_metrics_on_attach �[32mPASSED�[0m�[36m [ 16%]�[0m tests/telemetry_test.py::NodeEnvironmentInfoTests::test_stores_node_level_metrics �[32mPASSED�[0m�[36m [ 17%]�[0m tests/telemetry_test.py::ExternalEnvironmentInfoTests::test_fallback_when_host_not_available �[32mPASSED�[0m�[36m [ 17%]�[0m tests/telemetry_test.py::ExternalEnvironmentInfoTests::test_resilient_if_error_response �[32mPASSED�[0m�[36m [ 17%]�[0m tests/telemetry_test.py::ExternalEnvironmentInfoTests::test_stores_all_node_metrics_on_attach �[32mPASSED�[0m�[36m [ 17%]�[0m tests/telemetry_test.py::DiskIoTests::test_diskio_disk_io_counters �[32mPASSED�[0m�[36m [ 17%]�[0m tests/telemetry_test.py::DiskIoTests::test_diskio_process_io_counters �[32mPASSED�[0m�[36m [ 17%]�[0m tests/telemetry_test.py::DiskIoTests::test_diskio_writes_metrics_if_available �[32mPASSED�[0m�[36m [ 17%]�[0m tests/telemetry_test.py::JvmStatsSummaryTests::test_stores_only_diff_of_gc_times �[32mPASSED�[0m�[36m [ 18%]�[0m tests/telemetry_test.py::IndexStatsTests::test_stores_available_index_stats �[32mPASSED�[0m�[36m [ 18%]�[0m tests/telemetry_test.py::MlBucketProcessingTimeTests::test_empty_result_does_not_store_metrics �[32mPASSED�[0m�[36m [ 18%]�[0m tests/telemetry_test.py::MlBucketProcessingTimeTests::test_error_on_retrieval_does_not_store_metrics �[32mPASSED�[0m�[36m [ 18%]�[0m tests/telemetry_test.py::MlBucketProcessingTimeTests::test_result_is_stored �[32mPASSED�[0m�[36m [ 18%]�[0m tests/telemetry_test.py::IndexSizeTests::test_stores_index_size_for_data_paths �[32mPASSED�[0m�[36m [ 18%]�[0m tests/telemetry_test.py::IndexSizeTests::test_stores_nothing_if_no_data_path �[32mPASSED�[0m�[36m [ 18%]�[0m tests/time_test.py::TimeTests::test_millis_conversion_roughly_in_expected_range �[32mPASSED�[0m�[36m [ 19%]�[0m tests/time_test.py::TimeTests::test_split_time_increases �[32mPASSED�[0m�[36m [ 19%]�[0m tests/time_test.py::TimeTests::test_total_time_roughly_in_expected_range �[32mPASSED�[0m�[36m [ 19%]�[0m tests/driver/driver_test.py::DriverTests::test_assign_drivers_round_robin �[32mPASSED�[0m�[36m [ 19%]�[0m tests/driver/driver_test.py::DriverTests::test_client_reaches_join_point_others_still_executing �[32mPASSED�[0m�[36m [ 19%]�[0m tests/driver/driver_test.py::DriverTests::test_client_reaches_join_point_which_completes_parent �[32mPASSED�[0m�[36m [ 19%]�[0m tests/driver/driver_test.py::DriverTests::test_start_benchmark_and_prepare_track �[32mPASSED�[0m�[36m [ 19%]�[0m tests/driver/driver_test.py::AllocatorTests::test_a_task_completes_the_parallel_structure �[32mPASSED�[0m�[36m [ 20%]�[0m tests/driver/driver_test.py::AllocatorTests::test_allocates_mixed_tasks �[32mPASSED�[0m�[36m [ 20%]�[0m tests/driver/driver_test.py::AllocatorTests::test_allocates_more_tasks_than_clients �[32mPASSED�[0m�[36m [ 20%]�[0m tests/driver/driver_test.py::AllocatorTests::test_allocates_one_task �[32mPASSED�[0m�[36m [ 20%]�[0m tests/driver/driver_test.py::AllocatorTests::test_allocates_two_parallel_tasks �[32mPASSED�[0m�[36m [ 20%]�[0m tests/driver/driver_test.py::AllocatorTests::test_allocates_two_serial_tasks �[32mPASSED�[0m�[36m [ 20%]�[0m tests/driver/driver_test.py::AllocatorTests::test_considers_number_of_clients_per_subtask �[32mPASSED�[0m�[36m [ 20%]�[0m tests/driver/driver_test.py::MetricsAggregationTests::test_different_sample_types �[32mPASSED�[0m�[36m [ 21%]�[0m tests/driver/driver_test.py::MetricsAggregationTests::test_single_metrics_aggregation �[32mPASSED�[0m�[36m [ 21%]�[0m tests/driver/driver_test.py::SchedulerTests::test_finite_schedule_with_progress_indication �[32mPASSED�[0m�[36m [ 21%]�[0m tests/driver/driver_test.py::SchedulerTests::test_infinite_schedule_without_progress_indication �[32mPASSED�[0m�[36m [ 21%]�[0m tests/driver/driver_test.py::SchedulerTests::test_schedule_defaults_to_iteration_based �[32mPASSED�[0m�[36m [ 21%]�[0m tests/driver/driver_test.py::SchedulerTests::test_schedule_for_time_based �[32mPASSED�[0m�[36m [ 21%]�[0m tests/driver/driver_test.py::SchedulerTests::test_schedule_for_warmup_time_based �[32mPASSED�[0m�[36m [ 21%]�[0m tests/driver/driver_test.py::SchedulerTests::test_schedule_param_source_determines_iterations_including_warmup �[32mPASSED�[0m�[36m [ 22%]�[0m tests/driver/driver_test.py::SchedulerTests::test_schedule_param_source_determines_iterations_no_warmup �[32mPASSED�[0m�[36m [ 22%]�[0m tests/driver/driver_test.py::SchedulerTests::test_schedule_with_progress_determined_by_runner �[32mPASSED�[0m�[36m [ 22%]�[0m tests/driver/driver_test.py::SchedulerTests::test_search_task_one_client �[32mPASSED�[0m�[36m [ 22%]�[0m tests/driver/driver_test.py::SchedulerTests::test_search_task_two_clients �[32mPASSED�[0m�[36m [ 22%]�[0m tests/driver/driver_test.py::ExecutorTests::test_cancel_execute_schedule �[32mPASSED�[0m�[36m [ 22%]�[0m tests/driver/driver_test.py::ExecutorTests::test_execute_schedule_aborts_on_error �[32mPASSED�[0m�[36m [ 22%]�[0m tests/driver/driver_test.py::ExecutorTests::test_execute_schedule_in_throughput_mode �[32mPASSED�[0m�[36m [ 23%]�[0m tests/driver/driver_test.py::ExecutorTests::test_execute_schedule_throughput_throttled �[32mPASSED�[0m�[36m [ 23%]�[0m tests/driver/driver_test.py::ExecutorTests::test_execute_schedule_with_progress_determined_by_runner �[32mPASSED�[0m�[36m [ 23%]�[0m tests/driver/driver_test.py::ExecutorTests::test_execute_single_dict �[32mPASSED�[0m�[36m [ 23%]�[0m tests/driver/driver_test.py::ExecutorTests::test_execute_single_no_return_value �[32mPASSED�[0m�[36m [ 23%]�[0m tests/driver/driver_test.py::ExecutorTests::test_execute_single_tuple �[32mPASSED�[0m�[36m [ 23%]�[0m tests/driver/driver_test.py::ExecutorTests::test_execute_single_with_connection_error �[32mPASSED�[0m�[36m [ 23%]�[0m tests/driver/driver_test.py::ExecutorTests::test_execute_single_with_http_400 �[32mPASSED�[0m�[36m [ 24%]�[0m tests/driver/driver_test.py::ExecutorTests::test_execute_single_with_key_error �[32mPASSED�[0m�[36m [ 24%]�[0m tests/driver/driver_test.py::ProfilerTests::test_profiler_is_a_transparent_wrapper �[32mPASSED�[0m�[36m [ 24%]�[0m tests/driver/runner_test.py::RegisterRunnerTests::test_multi_cluster_runner_class_should_be_wrapped �[32mPASSED�[0m�[36m [ 24%]�[0m tests/driver/runner_test.py::RegisterRunnerTests::test_multi_cluster_runner_class_with_context_manager_should_be_wrapped_with_context_manager_enabled �[32mPASSED�[0m�[36m [ 24%]�[0m tests/driver/runner_test.py::RegisterRunnerTests::test_runner_function_should_be_wrapped �[32mPASSED�[0m�[36m [ 24%]�[0m tests/driver/runner_test.py::RegisterRunnerTests::test_single_cluster_runner_class_should_be_wrapped �[32mPASSED�[0m�[36m [ 25%]�[0m tests/driver/runner_test.py::RegisterRunnerTests::test_single_cluster_runner_class_with_context_manager_should_be_wrapped_with_context_manager_enabled �[32mPASSED�[0m�[36m [ 25%]�[0m tests/driver/runner_test.py::BulkIndexRunnerTests::test_bulk_index_error �[32mPASSED�[0m�[36m [ 25%]�[0m tests/driver/runner_test.py::BulkIndexRunnerTests::test_bulk_index_error_no_shards �[32mPASSED�[0m�[36m [ 25%]�[0m tests/driver/runner_test.py::BulkIndexRunnerTests::test_bulk_index_missing_params �[32mPASSED�[0m�[36m [ 25%]�[0m tests/driver/runner_test.py::BulkIndexRunnerTests::test_bulk_index_success_with_metadata �[32mPASSED�[0m�[36m [ 25%]�[0m tests/driver/runner_test.py::BulkIndexRunnerTests::test_bulk_index_success_without_metadata_and_without_doc_type �[32mPASSED�[0m�[36m [ 25%]�[0m tests/driver/runner_test.py::BulkIndexRunnerTests::test_bulk_index_success_without_metadata_with_doc_type �[32mPASSED�[0m�[36m [ 26%]�[0m tests/driver/runner_test.py::BulkIndexRunnerTests::test_mixed_bulk_with_detailed_stats �[32mPASSED�[0m�[36m [ 26%]�[0m tests/driver/runner_test.py::BulkIndexRunnerTests::test_mixed_bulk_with_simple_stats �[32mPASSED�[0m�[36m [ 26%]�[0m tests/driver/runner_test.py::ForceMergeRunnerTests::test_force_merge_override_request_timeout �[32mPASSED�[0m�[36m [ 26%]�[0m tests/driver/runner_test.py::ForceMergeRunnerTests::test_force_merge_with_defaults �[32mPASSED�[0m�[36m [ 26%]�[0m tests/driver/runner_test.py::ForceMergeRunnerTests::test_force_merge_with_params �[32mPASSED�[0m�[36m [ 26%]�[0m tests/driver/runner_test.py::ForceMergeRunnerTests::test_optimize_with_defaults �[32mPASSED�[0m�[36m [ 26%]�[0m tests/driver/runner_test.py::ForceMergeRunnerTests::test_optimize_with_params �[32mPASSED�[0m�[36m [ 27%]�[0m tests/driver/runner_test.py::QueryRunnerTests::test_query_hits_total_as_number �[32mPASSED�[0m�[36m [ 27%]�[0m tests/driver/runner_test.py::QueryRunnerTests::test_query_match_all �[32mPASSED�[0m�[36m [ 27%]�[0m tests/driver/runner_test.py::QueryRunnerTests::test_query_match_all_doc_type_fallback �[32mPASSED�[0m�[36m [ 27%]�[0m tests/driver/runner_test.py::QueryRunnerTests::test_query_match_only_request_body_defined �[32mPASSED�[0m�[36m [ 27%]�[0m tests/driver/runner_test.py::QueryRunnerTests::test_query_match_using_request_params �[32mPASSED�[0m�[36m [ 27%]�[0m tests/driver/runner_test.py::QueryRunnerTests::test_scroll_query_cannot_clear_scroll �[32mPASSED�[0m�[36m [ 27%]�[0m tests/driver/runner_test.py::QueryRunnerTests::test_scroll_query_early_termination �[32mPASSED�[0m�[36m [ 28%]�[0m tests/driver/runner_test.py::QueryRunnerTests::test_scroll_query_no_request_cache �[32mPASSED�[0m�[36m [ 28%]�[0m tests/driver/runner_test.py::QueryRunnerTests::test_scroll_query_only_one_page �[32mPASSED�[0m�[36m [ 28%]�[0m tests/driver/runner_test.py::QueryRunnerTests::test_scroll_query_only_one_page_only_request_body_defined �[32mPASSED�[0m�[36m [ 28%]�[0m tests/driver/runner_test.py::QueryRunnerTests::test_scroll_query_request_all_pages �[32mPASSED�[0m�[36m [ 28%]�[0m tests/driver/runner_test.py::QueryRunnerTests::test_scroll_query_with_explicit_number_of_pages �[32mPASSED�[0m�[36m [ 28%]�[0m tests/driver/runner_test.py::PutPipelineRunnerTests::test_create_pipeline �[32mPASSED�[0m�[36m [ 28%]�[0m tests/driver/runner_test.py::PutPipelineRunnerTests::test_param_body_mandatory �[32mPASSED�[0m�[36m [ 29%]�[0m tests/driver/runner_test.py::PutPipelineRunnerTests::test_param_id_mandatory �[32mPASSED�[0m�[36m [ 29%]�[0m tests/driver/runner_test.py::ClusterHealthRunnerTests::test_accepts_better_cluster_status �[32mPASSED�[0m�[36m [ 29%]�[0m tests/driver/runner_test.py::ClusterHealthRunnerTests::test_rejects_relocating_shards �[32mPASSED�[0m�[36m [ 29%]�[0m tests/driver/runner_test.py::ClusterHealthRunnerTests::test_rejects_unknown_cluster_status �[32mPASSED�[0m�[36m [ 29%]�[0m tests/driver/runner_test.py::ClusterHealthRunnerTests::test_waits_for_expected_cluster_status �[32mPASSED�[0m�[36m [ 29%]�[0m tests/driver/runner_test.py::CreateIndexRunnerTests::test_creates_multiple_indices �[32mPASSED�[0m�[36m [ 29%]�[0m tests/driver/runner_test.py::CreateIndexRunnerTests::test_param_indices_mandatory �[32mPASSED�[0m�[36m [ 30%]�[0m tests/driver/runner_test.py::DeleteIndexRunnerTests::test_deletes_all_indices �[32mPASSED�[0m�[36m [ 30%]�[0m tests/driver/runner_test.py::DeleteIndexRunnerTests::test_deletes_existing_indices �[32mPASSED�[0m�[36m [ 30%]�[0m tests/driver/runner_test.py::CreateIndexTemplateRunnerTests::test_create_index_templates �[32mPASSED�[0m�[36m [ 30%]�[0m tests/driver/runner_test.py::CreateIndexTemplateRunnerTests::test_param_templates_mandatory �[32mPASSED�[0m�[36m [ 30%]�[0m tests/driver/runner_test.py::DeleteIndexTemplateRunnerTests::test_deletes_all_index_templates �[32mPASSED�[0m�[36m [ 30%]�[0m tests/driver/runner_test.py::DeleteIndexTemplateRunnerTests::test_deletes_only_existing_index_templates �[32mPASSED�[0m�[36m [ 30%]�[0m tests/driver/runner_test.py::DeleteIndexTemplateRunnerTests::test_param_templates_mandatory �[32mPASSED�[0m�[36m [ 31%]�[0m tests/driver/runner_test.py::CreateMlDatafeedTests::test_create_ml_datafeed �[32mPASSED�[0m�[36m [ 31%]�[0m tests/driver/runner_test.py::CreateMlDatafeedTests::test_create_ml_datafeed_fallback �[32mPASSED�[0m�[36m [ 31%]�[0m tests/driver/runner_test.py::DeleteMlDatafeedTests::test_delete_ml_datafeed �[32mPASSED�[0m�[36m [ 31%]�[0m tests/driver/runner_test.py::DeleteMlDatafeedTests::test_delete_ml_datafeed_fallback �[32mPASSED�[0m�[36m [ 31%]�[0m tests/driver/runner_test.py::StartMlDatafeedTests::test_start_ml_datafeed_with_body �[32mPASSED�[0m�[36m [ 31%]�[0m tests/driver/runner_test.py::StartMlDatafeedTests::test_start_ml_datafeed_with_body_fallback �[32mPASSED�[0m�[36m [ 31%]�[0m tests/driver/runner_test.py::StartMlDatafeedTests::test_start_ml_datafeed_with_params �[32mPASSED�[0m�[36m [ 32%]�[0m tests/driver/runner_test.py::CreateMlJobTests::test_create_ml_job �[32mPASSED�[0m�[36m [ 32%]�[0m tests/driver/runner_test.py::CreateMlJobTests::test_create_ml_job_fallback �[32mPASSED�[0m�[36m [ 32%]�[0m tests/driver/runner_test.py::DeleteMlJobTests::test_delete_ml_job �[32mPASSED�[0m�[36m [ 32%]�[0m tests/driver/runner_test.py::DeleteMlJobTests::test_delete_ml_job_fallback �[32mPASSED�[0m�[36m [ 32%]�[0m tests/driver/runner_test.py::OpenMlJobTests::test_open_ml_job �[32mPASSED�[0m�[36m [ 32%]�[0m tests/driver/runner_test.py::OpenMlJobTests::test_open_ml_job_fallback �[32mPASSED�[0m�[36m [ 32%]�[0m tests/driver/runner_test.py::RawRequestRunnerTests::test_issue_create_index �[32mPASSED�[0m�[36m [ 33%]�[0m tests/driver/runner_test.py::RawRequestRunnerTests::test_issue_delete_index �[32mPASSED�[0m�[36m [ 33%]�[0m tests/driver/runner_test.py::RawRequestRunnerTests::test_issue_msearch �[32mPASSED�[0m�[36m [ 33%]�[0m tests/driver/runner_test.py::RawRequestRunnerTests::test_issue_request_with_defaults �[32mPASSED�[0m�[36m [ 33%]�[0m tests/driver/runner_test.py::SleepTests::test_missing_parameter �[32mPASSED�[0m�[36m [ 33%]�[0m tests/driver/runner_test.py::SleepTests::test_sleep �[32mPASSED�[0m�[36m [ 33%]�[0m tests/driver/runner_test.py::DeleteSnapshotRepositoryTests::test_delete_snapshot_repository �[32mPASSED�[0m�[36m [ 33%]�[0m tests/driver/runner_test.py::CreateSnapshotRepositoryTests::test_create_snapshot_repository �[32mPASSED�[0m�[36m [ 34%]�[0m tests/driver/runner_test.py::RestoreSnapshotTests::test_restore_snapshot �[32mPASSED�[0m�[36m [ 34%]�[0m tests/driver/runner_test.py::RestoreSnapshotTests::test_restore_snapshot_with_body �[32mPASSED�[0m�[36m [ 34%]�[0m tests/driver/runner_test.py::IndicesRecoveryTests::test_indices_recovery_already_finished �[32mPASSED�[0m�[36m [ 34%]�[0m tests/driver/runner_test.py::IndicesRecoveryTests::test_waits_for_ongoing_indices_recovery �[32mPASSED�[0m�[36m [ 34%]�[0m tests/driver/runner_test.py::ShrinkIndexTests::test_shrink_index_derives_shrink_node �[32mPASSED�[0m�[36m [ 34%]�[0m tests/driver/runner_test.py::ShrinkIndexTests::test_shrink_index_with_shrink_node �[32mPASSED�[0m�[36m [ 34%]�[0m tests/driver/runner_test.py::PutSettingsTests::test_put_settings �[32mPASSED�[0m�[36m [ 35%]�[0m tests/driver/runner_test.py::RetryTests::test_assumes_success_if_runner_returns_non_dict �[32mPASSED�[0m�[36m [ 35%]�[0m tests/driver/runner_test.py::RetryTests::test_does_not_retry_on_application_error_if_not_wanted �[32mPASSED�[0m�[36m [ 35%]�[0m tests/driver/runner_test.py::RetryTests::test_does_not_retry_on_timeout_if_not_wanted �[32mPASSED�[0m�[36m [ 35%]�[0m tests/driver/runner_test.py::RetryTests::test_is_does_not_retry_on_success �[32mPASSED�[0m�[36m [ 35%]�[0m tests/driver/runner_test.py::RetryTests::test_is_transparent_on_application_error_when_no_retries �[32mPASSED�[0m�[36m [ 35%]�[0m tests/driver/runner_test.py::RetryTests::test_is_transparent_on_exception_when_no_retries �[32mPASSED�[0m�[36m [ 35%]�[0m tests/driver/runner_test.py::RetryTests::test_is_transparent_on_success_when_no_retries �[32mPASSED�[0m�[36m [ 36%]�[0m tests/driver/runner_test.py::RetryTests::test_retries_mixed_timeout_and_application_errors �[32mPASSED�[0m�[36m [ 36%]�[0m tests/driver/runner_test.py::RetryTests::test_retries_on_application_error_if_wanted �[32mPASSED�[0m�[36m [ 36%]�[0m tests/driver/runner_test.py::RetryTests::test_retries_on_timeout_if_wanted_and_raises_if_no_recovery �[32mPASSED�[0m�[36m [ 36%]�[0m tests/driver/runner_test.py::RetryTests::test_retries_on_timeout_if_wanted_and_returns_first_call �[32mPASSED�[0m�[36m [ 36%]�[0m tests/driver/runner_test.py::RetryTests::test_retries_until_success �[32mPASSED�[0m�[36m [ 36%]�[0m tests/driver/scheduler_test.py::WaitTimeTests::test_calculate_wait_time_based_on_target_interval �[32mPASSED�[0m�[36m [ 36%]�[0m tests/driver/scheduler_test.py::WaitTimeTests::test_calculate_wait_time_based_on_throughput �[32mPASSED�[0m�[36m [ 37%]�[0m tests/driver/scheduler_test.py::WaitTimeTests::test_calculate_wait_time_zero �[32mPASSED�[0m�[36m [ 37%]�[0m tests/driver/scheduler_test.py::WaitTimeTests::test_either_target_throughput_or_target_interval �[32mPASSED�[0m�[36m [ 37%]�[0m tests/driver/scheduler_test.py::DeterministicSchedulerTests::test_schedule_matches_expected_target_throughput �[32mPASSED�[0m�[36m [ 37%]�[0m tests/driver/scheduler_test.py::DeterministicSchedulerTests::test_schedule_unlimited �[32mPASSED�[0m�[36m [ 37%]�[0m tests/driver/scheduler_test.py::PoissonSchedulerTests::test_schedule_matches_expected_target_throughput �[32mPASSED�[0m�[36m [ 37%]�[0m tests/driver/scheduler_test.py::PoissonSchedulerTests::test_schedule_unlimited �[32mPASSED�[0m�[36m [ 37%]�[0m tests/mechanic/java_resolver_test.py::JavaResolverTests::test_resolves_java_home_for_bundled_jdk �[32mPASSED�[0m�[36m [ 38%]�[0m tests/mechanic/java_resolver_test.py::JavaResolverTests::test_resolves_java_home_for_default_runtime_jdk �[32mPASSED�[0m�[36m [ 38%]�[0m tests/mechanic/java_resolver_test.py::JavaResolverTests::test_resolves_java_home_for_specific_runtime_jdk �[32mPASSED�[0m�[36m [ 38%]�[0m tests/mechanic/launcher_test.py::ProcessLauncherTests::test_bundled_jdk_not_in_path �[32mPASSED�[0m�[36m [ 38%]�[0m tests/mechanic/launcher_test.py::ProcessLauncherTests::test_daemon_start_stop �[32mPASSED�[0m�[36m [ 38%]�[0m tests/mechanic/launcher_test.py::ProcessLauncherTests::test_daemon_stop_with_already_terminated_process �[32mPASSED�[0m�[36m [ 38%]�[0m tests/mechanic/launcher_test.py::ProcessLauncherTests::test_env_options_order �[32mPASSED�[0m�[36m [ 38%]�[0m tests/mechanic/launcher_test.py::DockerLauncherTests::test_container_not_started �[32mPASSED�[0m�[36m [ 39%]�[0m tests/mechanic/launcher_test.py::DockerLauncherTests::test_starts_container_successfully �[32mPASSED�[0m�[36m [ 39%]�[0m tests/mechanic/launcher_test.py::DockerLauncherTests::test_stops_container_successfully �[32mPASSED�[0m�[36m [ 39%]�[0m tests/mechanic/mechanic_test.py::HostHandlingTests::test_converts_valid_hosts �[32mPASSED�[0m�[36m [ 39%]�[0m tests/mechanic/mechanic_test.py::HostHandlingTests::test_extract_all_node_ips �[32mPASSED�[0m�[36m [ 39%]�[0m tests/mechanic/mechanic_test.py::HostHandlingTests::test_groups_nodes_by_host �[32mPASSED�[0m�[36m [ 39%]�[0m tests/mechanic/mechanic_test.py::HostHandlingTests::test_rejects_hosts_with_unexpected_properties �[32mPASSED�[0m�[36m [ 39%]�[0m tests/mechanic/mechanic_test.py::MechanicTests::test_start_stop_nodes �[32mPASSED�[0m�[36m [ 40%]�[0m tests/mechanic/provisioner_test.py::BareProvisionerTests::test_prepare_distribution_ge_63_with_plugins �[32mPASSED�[0m�[36m [ 40%]�[0m tests/mechanic/provisioner_test.py::BareProvisionerTests::test_prepare_distribution_lt_63_with_plugins �[32mPASSED�[0m�[36m [ 40%]�[0m tests/mechanic/provisioner_test.py::BareProvisionerTests::test_prepare_without_plugins �[32mPASSED�[0m�[36m [ 40%]�[0m tests/mechanic/provisioner_test.py::ElasticsearchInstallerTests::test_invokes_hook_no_java_home �[32mPASSED�[0m�[36m [ 40%]�[0m tests/mechanic/provisioner_test.py::ElasticsearchInstallerTests::test_invokes_hook_with_java_home �[32mPASSED�[0m�[36m [ 40%]�[0m tests/mechanic/provisioner_test.py::ElasticsearchInstallerTests::test_prepare_default_data_paths �[32mPASSED�[0m�[36m [ 40%]�[0m tests/mechanic/provisioner_test.py::ElasticsearchInstallerTests::test_prepare_user_provided_data_path �[32mPASSED�[0m�[36m [ 41%]�[0m tests/mechanic/provisioner_test.py::PluginInstallerTests::test_install_plugin_successfully �[32mPASSED�[0m�[36m [ 41%]�[0m tests/mechanic/provisioner_test.py::PluginInstallerTests::test_install_plugin_with_bundled_jdk �[32mPASSED�[0m�[36m [ 41%]�[0m tests/mechanic/provisioner_test.py::PluginInstallerTests::test_install_plugin_with_io_error �[32mPASSED�[0m�[36m [ 41%]�[0m tests/mechanic/provisioner_test.py::PluginInstallerTests::test_install_plugin_with_unknown_error �[32mPASSED�[0m�[36m [ 41%]�[0m tests/mechanic/provisioner_test.py::PluginInstallerTests::test_install_unknown_plugin �[32mPASSED�[0m�[36m [ 41%]�[0m tests/mechanic/provisioner_test.py::PluginInstallerTests::test_invokes_hook_no_java_home �[32mPASSED�[0m�[36m [ 41%]�[0m tests/mechanic/provisioner_test.py::PluginInstallerTests::test_invokes_hook_with_java_home �[32mPASSED�[0m�[36m [ 42%]�[0m tests/mechanic/provisioner_test.py::PluginInstallerTests::test_pass_plugin_properties �[32mPASSED�[0m�[36m [ 42%]�[0m tests/mechanic/provisioner_test.py::DockerProvisionerTests::test_provisioning_with_defaults �[32mPASSED�[0m�[36m [ 42%]�[0m tests/mechanic/provisioner_test.py::DockerProvisionerTests::test_provisioning_with_variables �[32mPASSED�[0m�[36m [ 42%]�[0m tests/mechanic/provisioner_test.py::CleanupTests::test_cleanup �[32mPASSED�[0m�[36m [ 42%]�[0m tests/mechanic/provisioner_test.py::CleanupTests::test_preserves �[32mPASSED�[0m�[36m [ 42%]�[0m tests/mechanic/supplier_test.py::RevisionExtractorTests::test_invalid_revisions �[32mPASSED�[0m�[36m [ 42%]�[0m tests/mechanic/supplier_test.py::RevisionExtractorTests::test_multiple_revisions �[32mPASSED�[0m�[36m [ 43%]�[0m tests/mechanic/supplier_test.py::RevisionExtractorTests::test_single_revision �[32mPASSED�[0m�[36m [ 43%]�[0m tests/mechanic/supplier_test.py::SourceRepositoryTests::test_checkout_current �[32mPASSED�[0m�[36m [ 43%]�[0m tests/mechanic/supplier_test.py::SourceRepositoryTests::test_checkout_revision �[32mPASSED�[0m�[36m [ 43%]�[0m tests/mechanic/supplier_test.py::SourceRepositoryTests::test_checkout_revision_for_local_only_repo �[32mPASSED�[0m�[36m [ 43%]�[0m tests/mechanic/supplier_test.py::SourceRepositoryTests::test_checkout_ts �[32mPASSED�[0m�[36m [ 43%]�[0m tests/mechanic/supplier_test.py::SourceRepositoryTests::test_intial_checkout_latest �[32mPASSED�[0m�[36m [ 43%]�[0m tests/mechanic/supplier_test.py::SourceRepositoryTests::test_is_commit_hash �[32mPASSED�[0m�[36m [ 44%]�[0m tests/mechanic/supplier_test.py::SourceRepositoryTests::test_is_not_commit_hash �[32mPASSED�[0m�[36m [ 44%]�[0m tests/mechanic/supplier_test.py::BuilderTests::test_build_on_jdk_10 �[32mPASSED�[0m�[36m [ 44%]�[0m tests/mechanic/supplier_test.py::BuilderTests::test_build_on_jdk_8 �[32mPASSED�[0m�[36m [ 44%]�[0m tests/mechanic/supplier_test.py::TemplateRendererTests::test_uses_derived_values �[32mPASSED�[0m�[36m [ 44%]�[0m tests/mechanic/supplier_test.py::TemplateRendererTests::test_uses_provided_values �[32mPASSED�[0m�[36m [ 44%]�[0m tests/mechanic/supplier_test.py::ElasticsearchSourceSupplierTests::test_add_elasticsearch_binary �[32mPASSED�[0m�[36m [ 44%]�[0m tests/mechanic/supplier_test.py::ElasticsearchSourceSupplierTests::test_build �[32mPASSED�[0m�[36m [ 45%]�[0m tests/mechanic/supplier_test.py::ElasticsearchSourceSupplierTests::test_no_build �[32mPASSED�[0m�[36m [ 45%]�[0m tests/mechanic/supplier_test.py::ElasticsearchSourceSupplierTests::test_raises_error_on_missing_car_variable �[32mPASSED�[0m�[36m [ 45%]�[0m tests/mechanic/supplier_test.py::ExternalPluginSourceSupplierTests::test_add_binary_built_along_elasticsearch �[32mPASSED�[0m�[36m [ 45%]�[0m tests/mechanic/supplier_test.py::ExternalPluginSourceSupplierTests::test_along_es_plugin_keeps_build_dir �[32mPASSED�[0m�[36m [ 45%]�[0m tests/mechanic/supplier_test.py::ExternalPluginSourceSupplierTests::test_invalid_config_duplicate_source �[32mPASSED�[0m�[36m [ 45%]�[0m tests/mechanic/supplier_test.py::ExternalPluginSourceSupplierTests::test_invalid_config_no_source �[32mPASSED�[0m�[36m [ 45%]�[0m tests/mechanic/supplier_test.py::ExternalPluginSourceSupplierTests::test_resolve_plugin_binary_built_standalone �[32mPASSED�[0m�[36m [ 46%]�[0m tests/mechanic/supplier_test.py::ExternalPluginSourceSupplierTests::test_standalone_plugin_overrides_build_dir �[32mPASSED�[0m�[36m [ 46%]�[0m tests/mechanic/supplier_test.py::CorePluginSourceSupplierTests::test_resolve_plugin_binary �[32mPASSED�[0m�[36m [ 46%]�[0m tests/mechanic/supplier_test.py::PluginDistributionSupplierTests::test_resolve_plugin_url �[32mPASSED�[0m�[36m [ 46%]�[0m tests/mechanic/supplier_test.py::CreateSupplierTests::test_create_suppliers_for_es_and_plugin_source_build �[32mPASSED�[0m�[36m [ 46%]�[0m tests/mechanic/supplier_test.py::CreateSupplierTests::test_create_suppliers_for_es_distribution_plugin_source_build �[32mPASSED�[0m�[36m [ 46%]�[0m tests/mechanic/supplier_test.py::CreateSupplierTests::test_create_suppliers_for_es_distribution_plugin_source_skip �[32mPASSED�[0m�[36m [ 46%]�[0m tests/mechanic/supplier_test.py::CreateSupplierTests::test_create_suppliers_for_es_missing_distribution_plugin_source_skip �[32mPASSED�[0m�[36m [ 47%]�[0m tests/mechanic/supplier_test.py::CreateSupplierTests::test_create_suppliers_for_es_only_config �[32mPASSED�[0m�[36m [ 47%]�[0m tests/mechanic/supplier_test.py::CreateSupplierTests::test_derive_supply_requirements_es_and_plugin_source_build �[32mPASSED�[0m�[36m [ 47%]�[0m tests/mechanic/supplier_test.py::CreateSupplierTests::test_derive_supply_requirements_es_distribution �[32mPASSED�[0m�[36m [ 47%]�[0m tests/mechanic/supplier_test.py::CreateSupplierTests::test_derive_supply_requirements_es_distribution_and_plugin_source_build �[32mPASSED�[0m�[36m [ 47%]�[0m tests/mechanic/supplier_test.py::CreateSupplierTests::test_derive_supply_requirements_es_distribution_and_plugin_source_skip �[32mPASSED�[0m�[36m [ 47%]�[0m tests/mechanic/supplier_test.py::CreateSupplierTests::test_derive_supply_requirements_es_source_build �[32mPASSED�[0m�[36m [ 47%]�[0m tests/mechanic/supplier_test.py::CreateSupplierTests::test_derive_supply_requirements_es_source_skip �[32mPASSED�[0m�[36m [ 48%]�[0m tests/mechanic/supplier_test.py::DistributionRepositoryTests::test_invalid_cache_value �[32mPASSED�[0m�[36m [ 48%]�[0m tests/mechanic/supplier_test.py::DistributionRepositoryTests::test_missing_cache �[32mPASSED�[0m�[36m [ 48%]�[0m tests/mechanic/supplier_test.py::DistributionRepositoryTests::test_missing_plugin_config �[32mPASSED�[0m�[36m [ 48%]�[0m tests/mechanic/supplier_test.py::DistributionRepositoryTests::test_missing_url �[32mPASSED�[0m�[36m [ 48%]�[0m tests/mechanic/supplier_test.py::DistributionRepositoryTests::test_plugin_config_with_default_url �[32mPASSED�[0m�[36m [ 48%]�[0m tests/mechanic/supplier_test.py::DistributionRepositoryTests::test_plugin_config_with_user_url �[32mPASSED�[0m�[36m [ 48%]�[0m tests/mechanic/supplier_test.py::DistributionRepositoryTests::test_release_repo_config_with_default_url �[32mPASSED�[0m�[36m [ 49%]�[0m tests/mechanic/supplier_test.py::DistributionRepositoryTests::test_release_repo_config_with_user_url �[32mPASSED�[0m�[36m [ 49%]�[0m tests/mechanic/team_test.py::CarLoaderTests::test_lists_car_names �[32mPASSED�[0m�[36m [ 49%]�[0m tests/mechanic/team_test.py::CarLoaderTests::test_load_car_with_install_hook �[32mPASSED�[0m�[36m [ 49%]�[0m tests/mechanic/team_test.py::CarLoaderTests::test_load_car_with_mixin_multiple_config_bases �[32mPASSED�[0m�[36m [ 49%]�[0m tests/mechanic/team_test.py::CarLoaderTests::test_load_car_with_mixin_single_config_base �[32mPASSED�[0m�[36m [ 49%]�[0m tests/mechanic/team_test.py::CarLoaderTests::test_load_car_with_multiple_bases_referring_same_install_hook �[32mPASSED�[0m�[36m [ 50%]�[0m tests/mechanic/team_test.py::CarLoaderTests::test_load_known_car �[32mPASSED�[0m�[36m [ 50%]�[0m tests/mechanic/team_test.py::CarLoaderTests::test_raises_error_if_more_than_one_different_install_hook �[32mPASSED�[0m�[36m [ 50%]�[0m tests/mechanic/team_test.py::CarLoaderTests::test_raises_error_on_empty_config_base �[32mPASSED�[0m�[36m [ 50%]�[0m tests/mechanic/team_test.py::CarLoaderTests::test_raises_error_on_missing_config_base �[32mPASSED�[0m�[36m [ 50%]�[0m tests/mechanic/team_test.py::CarLoaderTests::test_raises_error_on_unknown_car �[32mPASSED�[0m�[36m [ 50%]�[0m tests/mechanic/team_test.py::PluginLoaderTests::test_cannot_load_community_plugin_with_missing_config �[32mPASSED�[0m�[36m [ 50%]�[0m tests/mechanic/team_test.py::PluginLoaderTests::test_cannot_load_plugin_with_missing_config �[32mPASSED�[0m�[36m [ 51%]�[0m tests/mechanic/team_test.py::PluginLoaderTests::test_lists_plugins �[32mPASSED�[0m�[36m [ 51%]�[0m tests/mechanic/team_test.py::PluginLoaderTests::test_loads_community_plugin_without_configuration �[32mPASSED�[0m�[36m [ 51%]�[0m tests/mechanic/team_test.py::PluginLoaderTests::test_loads_configured_plugin �[32mPASSED�[0m�[36m [ 51%]�[0m tests/mechanic/team_test.py::PluginLoaderTests::test_loads_core_plugin �[32mPASSED�[0m�[36m [ 51%]�[0m tests/mechanic/team_test.py::BootstrapHookHandlerTests::test_cannot_register_for_unknown_phase �[32mPASSED�[0m�[36m [ 51%]�[0m tests/mechanic/team_test.py::BootstrapHookHandlerTests::test_loads_module �[32mPASSED�[0m�[36m [ 51%]�[0m tests/track/loader_test.py::SimpleTrackRepositoryTests::test_track_from_directory �[32mPASSED�[0m�[36m [ 52%]�[0m tests/track/loader_test.py::SimpleTrackRepositoryTests::test_track_from_directory_without_track �[32mPASSED�[0m�[36m [ 52%]�[0m tests/track/loader_test.py::SimpleTrackRepositoryTests::test_track_from_file �[32mPASSED�[0m�[36m [ 52%]�[0m tests/track/loader_test.py::SimpleTrackRepositoryTests::test_track_from_file_but_not_json �[32mPASSED�[0m�[36m [ 52%]�[0m tests/track/loader_test.py::SimpleTrackRepositoryTests::test_track_from_named_pipe �[32mPASSED�[0m�[36m [ 52%]�[0m tests/track/loader_test.py::SimpleTrackRepositoryTests::test_track_from_non_existing_path �[32mPASSED�[0m�[36m [ 52%]�[0m tests/track/loader_test.py::GitRepositoryTests::test_track_from_existing_repo �[32mPASSED�[0m�[36m [ 52%]�[0m tests/track/loader_test.py::TrackPreparationTests::test_decompresses_if_archive_available �[32mPASSED�[0m�[36m [ 53%]�[0m tests/track/loader_test.py::TrackPreparationTests::test_does_nothing_if_document_file_available �[32mPASSED�[0m�[36m [ 53%]�[0m tests/track/loader_test.py::TrackPreparationTests::test_download_document_archive_if_no_file_available �[32mPASSED�[0m�[36m [ 53%]�[0m tests/track/loader_test.py::TrackPreparationTests::test_download_document_file_if_no_file_available �[32mPASSED�[0m�[36m [ 53%]�[0m tests/track/loader_test.py::TrackPreparationTests::test_prepare_bundled_document_set_decompresses_compressed_docs �[32mPASSED�[0m�[36m [ 53%]�[0m tests/track/loader_test.py::TrackPreparationTests::test_prepare_bundled_document_set_does_nothing_if_no_document_files �[32mPASSED�[0m�[36m [ 53%]�[0m tests/track/loader_test.py::TrackPreparationTests::test_prepare_bundled_document_set_error_compressed_docs_wrong_size �[32mPASSED�[0m�[36m [ 53%]�[0m tests/track/loader_test.py::TrackPreparationTests::test_prepare_bundled_document_set_if_document_file_available �[32mPASSED�[0m�[36m [ 54%]�[0m tests/track/loader_test.py::TrackPreparationTests::test_prepare_bundled_document_set_uncompressed_docs_wrong_size �[32mPASSED�[0m�[36m [ 54%]�[0m tests/track/loader_test.py::TrackPreparationTests::test_raise_download_error_if_no_url_provided_and_file_missing �[32mPASSED�[0m�[36m [ 54%]�[0m tests/track/loader_test.py::TrackPreparationTests::test_raise_download_error_if_no_url_provided_and_wrong_file_size �[32mPASSED�[0m�[36m [ 54%]�[0m tests/track/loader_test.py::TrackPreparationTests::test_raise_download_error_if_offline �[32mPASSED�[0m�[36m [ 54%]�[0m tests/track/loader_test.py::TrackPreparationTests::test_raise_download_error_no_test_mode_file �[32mPASSED�[0m�[36m [ 54%]�[0m tests/track/loader_test.py::TrackPreparationTests::test_raise_download_error_on_connection_problems �[32mPASSED�[0m�[36m [ 54%]�[0m tests/track/loader_test.py::TrackPreparationTests::test_raise_error_if_compressed_does_not_contain_expected_document_file �[32mPASSED�[0m�[36m [ 55%]�[0m tests/track/loader_test.py::TrackPreparationTests::test_raise_error_on_wrong_uncompressed_file_size �[32mPASSED�[0m�[36m [ 55%]�[0m tests/track/loader_test.py::TrackPreparationTests::test_used_corpora �[32mPASSED�[0m�[36m [ 55%]�[0m tests/track/loader_test.py::TemplateSource::test_entrypoint_of_replace_includes �[32mPASSED�[0m�[36m [ 55%]�[0m tests/track/loader_test.py::TemplateSource::test_read_glob_files �[32mPASSED�[0m�[36m [ 55%]�[0m tests/track/loader_test.py::TemplateRenderTests::test_render_simple_template �[32mPASSED�[0m�[36m [ 55%]�[0m tests/track/loader_test.py::TemplateRenderTests::test_render_template_with_external_variables �[32mPASSED�[0m�[36m [ 55%]�[0m tests/track/loader_test.py::TemplateRenderTests::test_render_template_with_globbing �[32mPASSED�[0m�[36m [ 56%]�[0m tests/track/loader_test.py::TemplateRenderTests::test_render_template_with_variables �[32mPASSED�[0m�[36m [ 56%]�[0m tests/track/loader_test.py::CompleteTrackParamsTests::test_check_complete_track_params_contains_all_track_params �[32mPASSED�[0m�[36m [ 56%]�[0m tests/track/loader_test.py::CompleteTrackParamsTests::test_check_complete_track_params_does_not_fail_with_no_track_params �[32mPASSED�[0m�[36m [ 56%]�[0m tests/track/loader_test.py::CompleteTrackParamsTests::test_unused_user_defined_track_params �[32mPASSED�[0m�[36m [ 56%]�[0m tests/track/loader_test.py::CompleteTrackParamsTests::test_unused_user_defined_track_params_doesnt_fail_with_detaults �[32mPASSED�[0m�[36m [ 56%]�[0m tests/track/loader_test.py::TrackPostProcessingTests::test_post_processes_track_spec �[32mPASSED�[0m�[36m [ 56%]�[0m tests/track/loader_test.py::TrackPathTests::test_sets_absolute_path �[32mPASSED�[0m�[36m [ 57%]�[0m tests/track/loader_test.py::TrackFilterTests::test_create_filters_from_empty_filtered_tasks �[32mPASSED�[0m�[36m [ 57%]�[0m tests/track/loader_test.py::TrackFilterTests::test_create_filters_from_mixed_filtered_tasks �[32mPASSED�[0m�[36m [ 57%]�[0m tests/track/loader_test.py::TrackFilterTests::test_filters_exclude_tasks �[32mPASSED�[0m�[36m [ 57%]�[0m tests/track/loader_test.py::TrackFilterTests::test_filters_tasks �[32mPASSED�[0m�[36m [ 57%]�[0m tests/track/loader_test.py::TrackFilterTests::test_rejects_invalid_syntax �[32mPASSED�[0m�[36m [ 57%]�[0m tests/track/loader_test.py::TrackFilterTests::test_rejects_unknown_filter_type �[32mPASSED�[0m�[36m [ 57%]�[0m tests/track/loader_test.py::TrackSpecificationReaderTests::test_at_least_one_default_challenge �[32mPASSED�[0m�[36m [ 58%]�[0m tests/track/loader_test.py::TrackSpecificationReaderTests::test_auto_generates_challenge_from_schedule �[32mPASSED�[0m�[36m [ 58%]�[0m tests/track/loader_test.py::TrackSpecificationReaderTests::test_can_read_track_info �[32mPASSED�[0m�[36m [ 58%]�[0m tests/track/loader_test.py::TrackSpecificationReaderTests::test_description_is_optional �[32mPASSED�[0m�[36m [ 58%]�[0m tests/track/loader_test.py::TrackSpecificationReaderTests::test_document_count_mandatory_if_file_present �[32mPASSED�[0m�[36m [ 58%]�[0m tests/track/loader_test.py::TrackSpecificationReaderTests::test_exactly_one_default_challenge �[32mPASSED�[0m�[36m [ 58%]�[0m tests/track/loader_test.py::TrackSpecificationReaderTests::test_inline_operations �[32mPASSED�[0m�[36m [ 58%]�[0m tests/track/loader_test.py::TrackSpecificationReaderTests::test_load_invalid_index_body �[32mPASSED�[0m�[36m [ 59%]�[0m tests/track/loader_test.py::TrackSpecificationReaderTests::test_not_more_than_one_default_challenge_possible �[32mPASSED�[0m�[36m [ 59%]�[0m tests/track/loader_test.py::TrackSpecificationReaderTests::test_parallel_tasks_with_completed_by_set �[32mPASSED�[0m�[36m [ 59%]�[0m tests/track/loader_test.py::TrackSpecificationReaderTests::test_parallel_tasks_with_completed_by_set_multiple_tasks_match �[32mPASSED�[0m�[36m [ 59%]�[0m tests/track/loader_test.py::TrackSpecificationReaderTests::test_parallel_tasks_with_completed_by_set_no_task_matches �[32mPASSED�[0m�[36m [ 59%]�[0m tests/track/loader_test.py::TrackSpecificationReaderTests::test_parallel_tasks_with_default_clients_does_not_propagate �[32mPASSED�[0m�[36m [ 59%]�[0m tests/track/loader_test.py::TrackSpecificationReaderTests::test_parallel_tasks_with_default_values �[32mPASSED�[0m�[36m [ 59%]�[0m tests/track/loader_test.py::TrackSpecificationReaderTests::test_parse_challenge_and_challenges_are_defined �[32mPASSED�[0m�[36m [ 60%]�[0m tests/track/loader_test.py::TrackSpecificationReaderTests::test_parse_duplicate_explicit_task_names �[32mPASSED�[0m�[36m [ 60%]�[0m tests/track/loader_test.py::TrackSpecificationReaderTests::test_parse_duplicate_implicit_task_names �[32mPASSED�[0m�[36m [ 60%]�[0m tests/track/loader_test.py::TrackSpecificationReaderTests::test_parse_missing_challenge_or_challenges �[32mPASSED�[0m�[36m [ 60%]�[0m tests/track/loader_test.py::TrackSpecificationReaderTests::test_parse_unique_task_names �[32mPASSED�[0m�[36m [ 60%]�[0m tests/track/loader_test.py::TrackSpecificationReaderTests::test_parse_valid_track_specification �[32mPASSED�[0m�[36m [ 60%]�[0m tests/track/loader_test.py::TrackSpecificationReaderTests::test_parse_valid_track_specification_with_index_template �[32mPASSED�[0m�[36m [ 60%]�[0m tests/track/loader_test.py::TrackSpecificationReaderTests::test_parse_valid_without_types �[32mPASSED�[0m�[36m [ 61%]�[0m tests/track/loader_test.py::TrackSpecificationReaderTests::test_parse_with_mixed_warmup_iterations_and_measurement �[32mPASSED�[0m�[36m [ 61%]�[0m tests/track/loader_test.py::TrackSpecificationReaderTests::test_parse_with_mixed_warmup_time_period_and_iterations �[32mPASSED�[0m�[36m [ 61%]�[0m tests/track/loader_test.py::TrackSpecificationReaderTests::test_selects_sole_challenge_implicitly_as_default �[32mPASSED�[0m�[36m [ 61%]�[0m tests/track/loader_test.py::TrackSpecificationReaderTests::test_supports_target_interval �[32mPASSED�[0m�[36m [ 61%]�[0m tests/track/loader_test.py::TrackSpecificationReaderTests::test_supports_target_throughput �[32mPASSED�[0m�[36m [ 61%]�[0m tests/track/loader_test.py::TrackSpecificationReaderTests::test_unique_challenge_names �[32mPASSED�[0m�[36m [ 61%]�[0m tests/track/params_test.py::SliceTests::test_slice_with_slice_larger_than_source �[32mPASSED�[0m�[36m [ 62%]�[0m tests/track/params_test.py::SliceTests::test_slice_with_source_larger_than_slice �[32mPASSED�[0m�[36m [ 62%]�[0m tests/track/params_test.py::ConflictingIdsBuilderTests::test_no_id_conflicts �[32mPASSED�[0m�[36m [ 62%]�[0m tests/track/params_test.py::ConflictingIdsBuilderTests::test_random_conflicts �[32mPASSED�[0m�[36m [ 62%]�[0m tests/track/params_test.py::ConflictingIdsBuilderTests::test_sequential_conflicts �[32mPASSED�[0m�[36m [ 62%]�[0m tests/track/params_test.py::ActionMetaDataTests::test_generate_action_meta_data_typeless �[32mPASSED�[0m�[36m [ 62%]�[0m tests/track/params_test.py::ActionMetaDataTests::test_generate_action_meta_data_with_id_and_zero_conflict_probability �[32mPASSED�[0m�[36m [ 62%]�[0m tests/track/params_test.py::ActionMetaDataTests::test_generate_action_meta_data_with_id_conflicts �[32mPASSED�[0m�[36m [ 63%]�[0m tests/track/params_test.py::ActionMetaDataTests::test_generate_action_meta_data_with_id_conflicts_and_recency_bias �[32mPASSED�[0m�[36m [ 63%]�[0m tests/track/params_test.py::ActionMetaDataTests::test_generate_action_meta_data_without_id_conflicts �[32mPASSED�[0m�[36m [ 63%]�[0m tests/track/params_test.py::ActionMetaDataTests::test_source_file_action_meta_data �[32mPASSED�[0m�[36m [ 63%]�[0m tests/track/params_test.py::IndexDataReaderTests::test_read_bulk_larger_than_number_of_docs �[32mPASSED�[0m�[36m [ 63%]�[0m tests/track/params_test.py::IndexDataReaderTests::test_read_bulk_smaller_than_number_of_docs �[32mPASSED�[0m�[36m [ 63%]�[0m tests/track/params_test.py::IndexDataReaderTests::test_read_bulk_smaller_than_number_of_docs_and_multiple_clients �[32mPASSED�[0m�[36m [ 63%]�[0m tests/track/params_test.py::IndexDataReaderTests::test_read_bulk_with_external_id_and_zero_conflict_probability �[32mPASSED�[0m�[36m [ 64%]�[0m tests/track/params_test.py::IndexDataReaderTests::test_read_bulk_with_id_conflicts �[32mPASSED�[0m�[36m [ 64%]�[0m tests/track/params_test.py::IndexDataReaderTests::test_read_bulk_with_offset �[32mPASSED�[0m�[36m [ 64%]�[0m tests/track/params_test.py::IndexDataReaderTests::test_read_bulks_and_assume_metadata_line_in_source_file �[32mPASSED�[0m�[36m [ 64%]�[0m tests/track/params_test.py::InvocationGeneratorTests::test_build_conflicting_ids �[32mPASSED�[0m�[36m [ 64%]�[0m tests/track/params_test.py::InvocationGeneratorTests::test_calculate_bounds �[32mPASSED�[0m�[36m [ 64%]�[0m tests/track/params_test.py::InvocationGeneratorTests::test_calculate_non_multiple_bounds �[32mPASSED�[0m�[36m [ 64%]�[0m tests/track/params_test.py::InvocationGeneratorTests::test_calculate_number_of_bulks �[32mPASSED�[0m�[36m [ 65%]�[0m tests/track/params_test.py::InvocationGeneratorTests::test_iterator_chaining_respects_context_manager �[32mPASSED�[0m�[36m [ 65%]�[0m tests/track/params_test.py::BulkIndexParamSourceTests::test_create_valid_param_source �[32mPASSED�[0m�[36m [ 65%]�[0m tests/track/params_test.py::BulkIndexParamSourceTests::test_create_with_conflict_probability_not_numeric �[32mPASSED�[0m�[36m [ 65%]�[0m tests/track/params_test.py::BulkIndexParamSourceTests::test_create_with_conflict_probability_too_high �[32mPASSED�[0m�[36m [ 65%]�[0m tests/track/params_test.py::BulkIndexParamSourceTests::test_create_with_conflict_probability_too_low �[32mPASSED�[0m�[36m [ 65%]�[0m tests/track/params_test.py::BulkIndexParamSourceTests::test_create_with_conflict_probability_zero �[32mPASSED�[0m�[36m [ 65%]�[0m tests/track/params_test.py::BulkIndexParamSourceTests::test_create_with_fraction_larger_batch_size �[32mPASSED�[0m�[36m [ 66%]�[0m tests/track/params_test.py::BulkIndexParamSourceTests::test_create_with_fraction_smaller_batch_size �[32mPASSED�[0m�[36m [ 66%]�[0m tests/track/params_test.py::BulkIndexParamSourceTests::test_create_with_ingest_percentage_not_numeric �[32mPASSED�[0m�[36m [ 66%]�[0m tests/track/params_test.py::BulkIndexParamSourceTests::test_create_with_ingest_percentage_too_high �[32mPASSED�[0m�[36m [ 66%]�[0m tests/track/params_test.py::BulkIndexParamSourceTests::test_create_with_ingest_percentage_too_low �[32mPASSED�[0m�[36m [ 66%]�[0m tests/track/params_test.py::BulkIndexParamSourceTests::test_create_with_metadata_in_source_file_but_conflicts �[32mPASSED�[0m�[36m [ 66%]�[0m tests/track/params_test.py::BulkIndexParamSourceTests::test_create_with_negative_bulk_size �[32mPASSED�[0m�[36m [ 66%]�[0m tests/track/params_test.py::BulkIndexParamSourceTests::test_create_with_non_numeric_bulk_size �[32mPASSED�[0m�[36m [ 67%]�[0m tests/track/params_test.py::BulkIndexParamSourceTests::test_create_with_unknown_id_conflicts �[32mPASSED�[0m�[36m [ 67%]�[0m tests/track/params_test.py::BulkIndexParamSourceTests::test_create_with_unknown_on_conflict_setting �[32mPASSED�[0m�[36m [ 67%]�[0m tests/track/params_test.py::BulkIndexParamSourceTests::test_create_without_params �[32mPASSED�[0m�[36m [ 67%]�[0m tests/track/params_test.py::BulkIndexParamSourceTests::test_filters_corpora �[32mPASSED�[0m�[36m [ 67%]�[0m tests/track/params_test.py::BulkIndexParamSourceTests::test_ingests_all_documents_by_default �[32mPASSED�[0m�[36m [ 67%]�[0m tests/track/params_test.py::BulkIndexParamSourceTests::test_passes_all_corpora_by_default �[32mPASSED�[0m�[36m [ 67%]�[0m tests/track/params_test.py::BulkIndexParamSourceTests::test_raises_exception_if_no_corpus_matches �[32mPASSED�[0m�[36m [ 68%]�[0m tests/track/params_test.py::BulkIndexParamSourceTests::test_restricts_number_of_bulks_if_required �[32mPASSED�[0m�[36m [ 68%]�[0m tests/track/params_test.py::BulkDataGeneratorTests::test_generate_bulks_from_multiple_corpora �[32mPASSED�[0m�[36m [ 68%]�[0m tests/track/params_test.py::BulkDataGeneratorTests::test_generate_two_bulks �[32mPASSED�[0m�[36m [ 68%]�[0m tests/track/params_test.py::BulkDataGeneratorTests::test_internal_params_take_precedence �[32mPASSED�[0m�[36m [ 68%]�[0m tests/track/params_test.py::ParamsRegistrationTests::test_can_register_class_as_param_source �[32mPASSED�[0m�[36m [ 68%]�[0m tests/track/params_test.py::ParamsRegistrationTests::test_can_register_function_as_param_source �[32mPASSED�[0m�[36m [ 68%]�[0m tests/track/params_test.py::ParamsRegistrationTests::test_can_register_legacy_class_as_param_source �[32mPASSED�[0m�[36m [ 69%]�[0m tests/track/params_test.py::ParamsRegistrationTests::test_can_register_legacy_function_as_param_source �[32mPASSED�[0m�[36m [ 69%]�[0m tests/track/params_test.py::SleepParamSourceTests::test_duration_parameter_negative_number �[32mPASSED�[0m�[36m [ 69%]�[0m tests/track/params_test.py::SleepParamSourceTests::test_duration_parameter_wrong_type �[32mPASSED�[0m�[36m [ 69%]�[0m tests/track/params_test.py::SleepParamSourceTests::test_missing_duration_parameter �[32mPASSED�[0m�[36m [ 69%]�[0m tests/track/params_test.py::SleepParamSourceTests::test_param_source_passes_all_parameters �[32mPASSED�[0m�[36m [ 69%]�[0m tests/track/params_test.py::CreateIndexParamSourceTests::test_create_index_from_track_with_settings �[32mPASSED�[0m�[36m [ 69%]�[0m tests/track/params_test.py::CreateIndexParamSourceTests::test_create_index_from_track_without_settings �[32mPASSED�[0m�[36m [ 70%]�[0m tests/track/params_test.py::CreateIndexParamSourceTests::test_create_index_inline_with_body �[32mPASSED�[0m�[36m [ 70%]�[0m tests/track/params_test.py::CreateIndexParamSourceTests::test_create_index_inline_without_body �[32mPASSED�[0m�[36m [ 70%]�[0m tests/track/params_test.py::CreateIndexParamSourceTests::test_filter_index �[32mPASSED�[0m�[36m [ 70%]�[0m tests/track/params_test.py::DeleteIndexParamSourceTests::test_delete_index_by_name �[32mPASSED�[0m�[36m [ 70%]�[0m tests/track/params_test.py::DeleteIndexParamSourceTests::test_delete_index_from_track �[32mPASSED�[0m�[36m [ 70%]�[0m tests/track/params_test.py::DeleteIndexParamSourceTests::test_delete_no_index �[32mPASSED�[0m�[36m [ 70%]�[0m tests/track/params_test.py::DeleteIndexParamSourceTests::test_filter_index_from_track �[32mPASSED�[0m�[36m [ 71%]�[0m tests/track/params_test.py::CreateIndexTemplateParamSourceTests::test_create_index_template_from_track �[32mPASSED�[0m�[36m [ 71%]�[0m tests/track/params_test.py::CreateIndexTemplateParamSourceTests::test_create_index_template_inline �[32mPASSED�[0m�[36m [ 71%]�[0m tests/track/params_test.py::DeleteIndexTemplateParamSourceTests::test_delete_index_template_by_name �[32mPASSED�[0m�[36m [ 71%]�[0m tests/track/params_test.py::DeleteIndexTemplateParamSourceTests::test_delete_index_template_by_name_and_matching_indices �[32mPASSED�[0m�[36m [ 71%]�[0m tests/track/params_test.py::DeleteIndexTemplateParamSourceTests::test_delete_index_template_by_name_and_matching_indices_missing_index_pattern �[32mPASSED�[0m�[36m [ 71%]�[0m tests/track/params_test.py::DeleteIndexTemplateParamSourceTests::test_delete_index_template_from_track �[32mPASSED�[0m�[36m [ 71%]�[0m tests/track/params_test.py::SearchParamSourceTests::test_create_without_index �[32mPASSED�[0m�[36m [ 72%]�[0m tests/track/params_test.py::SearchParamSourceTests::test_passes_cache �[32mPASSED�[0m�[36m [ 72%]�[0m tests/track/params_test.py::SearchParamSourceTests::test_passes_request_parameters �[32mPASSED�[0m�[36m [ 72%]�[0m tests/track/params_test.py::SearchParamSourceTests::test_replaces_body_params �[32mPASSED�[0m�[36m [ 72%]�[0m tests/track/params_test.py::SearchParamSourceTests::test_user_specified_overrides_defaults �[32mPASSED�[0m�[36m [ 72%]�[0m tests/track/params_test.py::ForceMergeParamSourceTests::test_default_force_merge_index �[32mPASSED�[0m�[36m [ 72%]�[0m tests/track/params_test.py::ForceMergeParamSourceTests::test_force_merge_all_params �[32mPASSED�[0m�[36m [ 72%]�[0m tests/track/params_test.py::ForceMergeParamSourceTests::test_force_merge_index_by_name �[32mPASSED�[0m�[36m [ 73%]�[0m tests/track/params_test.py::ForceMergeParamSourceTests::test_force_merge_index_from_track �[32mPASSED�[0m�[36m [ 73%]�[0m tests/track/track_test.py::TrackTests::test_default_challenge_none_if_no_challenges �[32mPASSED�[0m�[36m [ 73%]�[0m tests/track/track_test.py::TrackTests::test_does_not_find_unknown_challenge �[32mPASSED�[0m�[36m [ 73%]�[0m tests/track/track_test.py::TrackTests::test_finds_challenge_by_name �[32mPASSED�[0m�[36m [ 73%]�[0m tests/track/track_test.py::TrackTests::test_finds_default_challenge �[32mPASSED�[0m�[36m [ 73%]�[0m tests/track/track_test.py::TrackTests::test_uses_default_challenge_if_no_name_given �[32mPASSED�[0m�[36m [ 73%]�[0m tests/track/track_test.py::IndexTests::test_matches_exactly �[32mPASSED�[0m�[36m [ 74%]�[0m tests/track/track_test.py::IndexTests::test_matches_if_catch_all_pattern_is_defined �[32mPASSED�[0m�[36m [ 74%]�[0m tests/track/track_test.py::IndexTests::test_matches_if_no_pattern_is_defined �[32mPASSED�[0m�[36m [ 74%]�[0m tests/track/track_test.py::IndexTests::test_str �[32mPASSED�[0m�[36m [ 74%]�[0m tests/track/track_test.py::DocumentCorpusTests::test_cannot_union_mixed_document_corpora �[32mPASSED�[0m�[36m [ 74%]�[0m tests/track/track_test.py::DocumentCorpusTests::test_do_not_filter �[32mPASSED�[0m�[36m [ 74%]�[0m tests/track/track_test.py::DocumentCorpusTests::test_filter_documents_by_format �[32mPASSED�[0m�[36m [ 75%]�[0m tests/track/track_test.py::DocumentCorpusTests::test_filter_documents_by_format_and_indices �[32mPASSED�[0m�[36m [ 75%]�[0m tests/track/track_test.py::DocumentCorpusTests::test_filter_documents_by_indices �[32mPASSED�[0m�[36m [ 75%]�[0m tests/track/track_test.py::DocumentCorpusTests::test_union_document_corpora_is_symmetric �[32mPASSED�[0m�[36m [ 75%]�[0m tests/track/track_test.py::DocumentCorpusTests::test_union_document_corpus_is_reflexive �[32mPASSED�[0m�[36m [ 75%]�[0m tests/utils/console_test.py::ConsoleFunctionTests::test_global_rally_running_in_docker_is_false �[32mPASSED�[0m�[36m [ 75%]�[0m tests/utils/console_test.py::ConsoleFunctionTests::test_global_rally_running_in_docker_is_false_if_unset �[32mPASSED�[0m�[36m [ 75%]�[0m tests/utils/console_test.py::ConsoleFunctionTests::test_global_rally_running_in_docker_is_true �[32mPASSED�[0m�[36m [ 76%]�[0m tests/utils/console_test.py::ConsoleFunctionTests::test_println_force_prints_even_when_quiet �[32mPASSED�[0m�[36m [ 76%]�[0m tests/utils/console_test.py::ConsoleFunctionTests::test_println_isquiet_and_randomized_docker_or_istty �[32mPASSED�[0m�[36m [ 76%]�[0m tests/utils/console_test.py::ConsoleFunctionTests::test_println_randomized_dockertrue_or_istty_and_isnotquiet �[32mPASSED�[0m�[36m [ 76%]�[0m tests/utils/console_test.py::TestCmdLineProgressReporter::test_print_when_isquiet_and_any_docker_or_istty[0] �[32mPASSED�[0m�[36m [ 76%]�[0m tests/utils/console_test.py::TestCmdLineProgressReporter::test_print_when_isquiet_and_any_docker_or_istty[1] �[32mPASSED�[0m�[36m [ 76%]�[0m tests/utils/console_test.py::TestCmdLineProgressReporter::test_print_when_isquiet_and_any_docker_or_istty[2] �[32mPASSED�[0m�[36m [ 76%]�[0m tests/utils/console_test.py::TestCmdLineProgressReporter::test_print_when_isquiet_and_any_docker_or_istty[3] �[32mPASSED�[0m�[36m [ 77%]�[0m tests/utils/console_test.py::TestCmdLineProgressReporter::test_print_when_isquiet_and_any_docker_or_istty[4] �[32mPASSED�[0m�[36m [ 77%]�[0m tests/utils/console_test.py::TestCmdLineProgressReporter::test_print_when_isquiet_and_any_docker_or_istty[5] �[32mPASSED�[0m�[36m [ 77%]�[0m tests/utils/console_test.py::TestCmdLineProgressReporter::test_print_when_isquiet_and_any_docker_or_istty[6] �[32mPASSED�[0m�[36m [ 77%]�[0m tests/utils/console_test.py::TestCmdLineProgressReporter::test_print_when_isquiet_a…
dliappis
reviewed
Feb 20, 2020
docs/tracker.rst
Outdated
| ----- | ||
| To generate a new track, invoke Tracker as follows (replacing the example parameters below):: | ||
|
|
||
| estracker --target-hosts=host:9200,host2:9200 --indices=index_1 index_2 --trackname=xtrack --outdir=mytracks |
There was a problem hiding this comment.
We should also document the --client-options e.g. with an example; scrapping real Cloud clusters will be THE most common use case.
There was a problem hiding this comment.
Also there's a typo here it's --track-name not --trackname
There was a problem hiding this comment.
Additionally the values for --indices should be enclosed in double quotes when dealing with multiple indices.
|
Looks like CI was aborted for Jenkins upgrade. @elasticmachine run tests |
added 10 commits
February 20, 2020 17:42
- don't dump 0 doc count indices, as Rally won't accept such a track - handle extraction failures appropriately with rollback - fix a few more root-logger usages - move track logic out of main()
…an one index is specified.
|
Closing in favor of #970 (which is based on this PR) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Tracker is a small Python tool for automatically generating a new Rally track from an existing repository
Relates #444