Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dlt_unit_test: Fix and improve quality of unit tests #494

Merged
merged 1 commit into from
Jun 14, 2023
Merged

dlt_unit_test: Fix and improve quality of unit tests #494

merged 1 commit into from
Jun 14, 2023

Conversation

minminlittleshrimp
Copy link
Collaborator

@minminlittleshrimp minminlittleshrimp commented Jun 6, 2023

  • Fix issues offline log unit test fail when not trigger gzip feature
  • Add timeout to prevent timeout silently ignored
  • Limit dlt_user final test suite to 5 seconds
  • Fix wrongly triggered gtests with environment (gateway+offline)

@minminlittleshrimp
Copy link
Collaborator Author

Hello @alexmohr
For the point: "Limit dlt_user final test suite to 5 seconds",
I would like to have your comment to make sure it is ok to
minimize the max runtime of the test suite.
Link to PR: #362
Thank you

@minminlittleshrimp
Copy link
Collaborator Author

minminlittleshrimp commented Jun 6, 2023

More detail about this fix:

Docker image: ubuntu-bionic 18.04 from dockerhub
Branch: master

Running a fully ctest verbose and observed:

6: Test command: /bin/sh "gtest_dlt_daemon_offline_log.sh" "gtest_dlt_daemon_offline_log"
6: Test timeout computed to be: 9.99988e+06
6/8 Test #6: gtest_dlt_daemon_offline_log ..............   Passed    0.00 sec

**Running gtest_dlt_daemon_offline_log.sh and then ./gtest_dlt_daemon_offline_log
Result:**
 [----------] 3 tests from t_dlt_logstorage_write_on_msg
[ RUN      ] t_dlt_logstorage_write_on_msg.normal
[       OK ] t_dlt_logstorage_write_on_msg.normal (1 ms)
[ RUN      ] t_dlt_logstorage_write_on_msg.gzip
[87723.288506]~DLT~31308~ERROR    ~dlt_logstorage_open_log_file: Unable to open log file.
/home/lum3hc/work/github/minmindlt/backup/dlt-daemon/tests/gtest_dlt_daemon_offline_log.cpp:1212: Failure
Value of: dlt_logstorage_prepare_on_msg(&config, &file_config, path, 1, &newest_file_name)
  Actual: -1
Expected: DLT_RETURN_OK
Which is: 0
Segmentation fault (core dumped)

=> timeout silently ignored

After applying the fix: Branch fix_gtest_offline_log

[----------] 2 tests from t_dlt_logstorage_sync_msg_cache
[ RUN      ] t_dlt_logstorage_sync_msg_cache.normal
[91714.776902]~DLT~16908~ERROR    ~Unable to calculate index from log file name. Reset to 001.
[       OK ] t_dlt_logstorage_sync_msg_cache.normal (0 ms)
[ RUN      ] t_dlt_logstorage_sync_msg_cache.null
[       OK ] t_dlt_logstorage_sync_msg_cache.null (0 ms)
[----------] 2 tests from t_dlt_logstorage_sync_msg_cache (0 ms total)

[----------] Global test environment tear-down
[==========] 114 tests from 59 test cases ran. (3 ms total)
[  PASSED  ] 114 tests.

@alexmohr
Copy link
Contributor

alexmohr commented Jun 6, 2023

Hello @alexmohr For the point: "Limit dlt_user final test suite to 5 seconds", I would like to have your comment to make sure it is ok to minimize the max runtime of the test suite. Link to PR: #362 Thank you

15s where a bit overly cautious, if there is something broken the test should fail in the 5s you set as timeout, if it doesn't break anything i'd increase to 7s or so just to have a bit more safety margin.

@minminlittleshrimp
Copy link
Collaborator Author

minminlittleshrimp commented Jun 9, 2023

The closed PR: #496 is now fixed with new commit version of this PR. To check the change, please kindly try to run $ctest -V and observe gtests running.

test 5
    Start 5: gtest_dlt_daemon_gateway

5: Test command: /bin/sh "-e" "-c" "                         sh /home/runner/work/dlt-daemon/dlt-daemon/build/tests/gtest_dlt_daemon_gateway.sh ;                          /home/runner/work/dlt-daemon/dlt-daemon/build/tests/gtest_dlt_daemon_gateway || TESTS_FAILED=1 && TESTS_FAILED=0                          sleep 0.5 ;                          killall -v dlt-daemon || echo 'Could not find any daemon to kill' ;
                         exit $TESTS_FAILED"
5: Working Directory: /home/runner/work/dlt-daemon/dlt-daemon/build/tests
5: Environment variables: 
5:  PATH=/home/runner/work/dlt-daemon/dlt-daemon/build/src/daemon:/home/runner/.local/bin:/opt/pipx_bin:/home/runner/.cargo/bin:/home/runner/.config/composer/vendor/bin:/usr/local/.ghcup/bin:/home/runner/.dotnet/tools:/snap/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
5:  DLT_UT_DAEMON_PATH=/home/runner/work/dlt-daemon/dlt-daemon/build/src/daemon/dlt-daemon
5: Test timeout computed to be: 0
5: Cleaning up dlt-daemon instances
5: Initializing test
5: Restarting dlt-daemons
5: Cannot open configuration file: /usr/local/etc/dlt.conf
5: [  385.163634]~DLT~ 3613~NOTICE   ~Starting DLT Daemon; DLT Package Version: 2.18.9 STABLE, Package Revision: , build on Jun  9 2023 04:46:49
5: -SYSTEMD -SYSTEMD_WATCHDOG -TEST -SHM
5: 
5: [  385.163661]~DLT~ 3613~NOTICE   ~Daemon mode
5: [  386.167746]~DLT~ 3616~NOTICE   ~Starting DLT Daemon; DLT Package Version: 2.18.9 STABLE, Package Revision: , build on Jun  9 2023 04:46:49
5: -SYSTEMD -SYSTEMD_WATCHDOG -TEST -SHM
5: 
5: [  386.167779]~DLT~ 3616~NOTICE   ~Daemon mode
5: Execute: gtest_dlt_daemon_gateway unit test
5: [==========] Running 46 tests from 20 test cases.
5: [----------] Global test environment set-up.
5: [----------] 2 tests from t_dlt_gateway_init
5: [ RUN      ] t_dlt_gateway_init.normal
5: [  386.175521]~DLT~ 3619~INFO     ~dlt_gateway_init()
5: [  386.175536]~DLT~ 3619~INFO     ~dlt_gateway_configure()
5: [  386.175574]~DLT~ 3619~WARNING  ~Missing General section in gateway. Using default interval 1 (secs)
5: [  386.175586]~DLT~ 3619~WARNING  ~Entry does not exist in section: SendPeriodicControl
5: [  386.175588]~DLT~ 3619~WARNING  ~Using default for SendPeriodicControl.
5: [  386.175590]~DLT~ 3619~INFO     ~dlt_gateway_store_connection()
5: [  386.175595]~DLT~ 3619~INFO     ~dlt_gateway_establish_connections()
5: [  386.175780]~DLT~ 3619~INFO     ~dlt_client_connect: Connected to DLT daemon (127.0.0.1)
5: [  386.175844]~DLT~ 3619~INFO     ~Activate connection type: 11
5: [  386.175851]~DLT~ 3619~INFO     ~dlt_gateway_send_control_message()
5: [  386.175915]~DLT~ 3619~INFO     ~dlt_client_send_ctrl_msg: Control message forwarded : DLT_SERVICE_ID_GET_LOG_INFO
5: [  386.175920]~DLT~ 3619~INFO     ~dlt_gateway_send_control_message()
5: [  386.175958]~DLT~ 3619~INFO     ~dlt_client_send_ctrl_msg: Control message forwarded : DLT_SERVICE_ID_GET_SOFTWARE_VERSION
5: [       OK ] t_dlt_gateway_init.normal (1 ms)

+ Fix issues offline log unit test fail when not trigger gzip feature
+ Add timeout to prevent timeout silently ignored
+ Limit dlt_user final test suite to 5 seconds
+ Fix wrongly triggered gtests with environment (gateway+offline)

Signed-off-by: Luu Quang Minh <[email protected]>
Copy link
Collaborator

@michael-methner michael-methner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. Thank you very much.

@michael-methner michael-methner merged commit 81ac18f into COVESA:master Jun 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants