Build farmer handoff 2019-03-01

[sloretz]

Outgoing @sloretz from #160
Incoming @dirk-thomas

Known flaky tests and recurring failures

• test_params_yaml fails on all platforms test_cli.test_params_yaml flaky #166, test_params_yaml flaky #155

• linux_nightly_coverage qt_gui_cpp test fails linux_nightly_coverage qt_gui_cpp test fails  #158

• rviz tests are failing on aarch64 linux repeated (original contributors aware) rviz tests failing on aarch64 linux repeated #144

• pendulum tests fail on aarch64 opensplice Linux Opensplice pendulum tests fail on aarch64 #133

• launch testing failures on linux repeated Launch tests on Linux fail on repeated #126

• Services/parameter tests with Fast-RTPS are flaky on aarch64 Linux aarch64 test_services with Fast-RTPS sometimes times out #136

• Lots of test failures on windows debug

  • Recent increase documented in Increase in test failures on windows debug with actions tests #167
  Failures in https://ci.ros2.org/job/nightly_win_deb/1175

  message_filters.test.directed
  message_filters.test.test_approxsync
  message_filters.test.test_message_filters_cache
  projectroot.directed.py
  projectroot.gtest_avoid_ros_namespace_conventions_qos__rmw_connext_cpp
  projectroot.gtest_avoid_ros_namespace_conventions_qos__rmw_fastrtps_cpp
  projectroot.gtest_executor__rmw_connext_cpp
  projectroot.gtest_executor__rmw_fastrtps_cpp
  projectroot.gtest_local_parameters__rmw_connext_cpp
  projectroot.gtest_local_parameters__rmw_fastrtps_cpp
  projectroot.gtest_multiple_service_calls__rmw_connext_cpp
  projectroot.gtest_multiple_service_calls__rmw_fastrtps_cpp
  projectroot.gtest_multithreaded__rmw_connext_cpp
  projectroot.gtest_multithreaded__rmw_fastrtps_cpp
  projectroot.gtest_publisher__rmw_connext_cpp
  projectroot.gtest_publisher__rmw_fastrtps_cpp
  projectroot.gtest_services_in_constructor__rmw_connext_cpp
  projectroot.gtest_services_in_constructor__rmw_fastrtps_cpp
  projectroot.gtest_spin__rmw_connext_cpp
  projectroot.gtest_spin__rmw_fastrtps_cpp
  projectroot.gtest_subscription__rmw_connext_cpp
  projectroot.gtest_subscription__rmw_fastrtps_cpp
  projectroot.gtest_timer__rmw_connext_cpp
  projectroot.gtest_timer__rmw_fastrtps_cpp
  projectroot.gtest_waitable__rmw_connext_cpp
  projectroot.gtest_waitable__rmw_fastrtps_cpp
  projectroot.message_filters_msg_cache_unittest
  projectroot.message_filters_test_approximate_time_policy
  projectroot.message_filters_test_chain
  projectroot.message_filters_test_exact_time_policy
  projectroot.message_filters_test_simple
  projectroot.message_filters_test_subscriber
  projectroot.message_filters_test_time_sequencer
  projectroot.message_filters_time_synchronizer_unittest
  projectroot.rqt_py_common
  projectroot.test_action_client_server__Fibonacci__rclcpp__rmw_connext_cpp
  projectroot.test_action_client_server__Fibonacci__rclcpp__rmw_fastrtps_cpp
  projectroot.test_action_client_server__NestedMessage__rclcpp__rmw_connext_cpp
  projectroot.test_action_client_server__NestedMessage__rclcpp__rmw_fastrtps_cpp
  projectroot.test_approxsync.py
  projectroot.test_callback_exceptions
  projectroot.test_client
  projectroot.test_client_scope_consistency_cpp__rmw_connext_cpp
  projectroot.test_client_scope_consistency_cpp__rmw_fastrtps_cpp
  projectroot.test_client_scope_cpp__rmw_connext_cpp
  projectroot.test_client_scope_cpp__rmw_fastrtps_cpp
  projectroot.test_executor
  projectroot.test_externally_defined_services
  projectroot.test_find_weak_nodes
  projectroot.test_lifecycle_node
  projectroot.test_message_filters_cache.py
  projectroot.test_multi_threaded_executor
  projectroot.test_node
  projectroot.test_node_global_args
  projectroot.test_parameter_server_cpp__rmw_connext_cpp
  projectroot.test_parameter_server_cpp__rmw_fastrtps_cpp
  projectroot.test_publisher
  projectroot.test_publisher_subscription_count_api
  projectroot.test_register_custom_callbacks
  projectroot.test_server
  projectroot.test_service
  projectroot.test_services_cpp__rmw_connext_cpp
  projectroot.test_services_cpp__rmw_fastrtps_cpp
  projectroot.test_state_machine_info
  projectroot.test_subscription
  projectroot.test_subscription_publisher_count_api
  projectroot.tf2_ros_test_message_filter
  rclcpp.cppcheck.warning: nullPointer (src\rclcpp\clock.cpp:139)
  rmw_connext_shared_cpp.cppcheck.error: unknownMacro (C:\J\workspace\nightly_win_deb\ws\src\ros2\rmw_connext\rmw_connext_shared_cpp\include\rmw_connext_shared_cpp\wait.hpp:51)
  rmw_connext_shared_cpp.cppcheck.error: unknownMacro (include\rmw_connext_shared_cpp\wait.hpp:51)
  rqt_py_common.test.test_rqt_ros_graph
  sros2.test.test_policy_to_permissions
  test_communication.build.test_communication.test_action_client_server__Fibonacci__rclcpp__rmw_connext_cpp_Debug.test_action_client_server
  test_communication.build.test_communication.test_action_client_server__Fibonacci__rclcpp__rmw_fastrtps_cpp_Debug.test_action_client_server
  test_communication.build.test_communication.test_action_client_server__NestedMessage__rclcpp__rmw_connext_cpp_Debug.test_action_client_server
  test_communication.build.test_communication.test_action_client_server__NestedMessage__rclcpp__rmw_fastrtps_cpp_Debug.test_action_client_server
  test_rclcpp.build.test_rclcpp.test_client_scope_consistency_cpp__rmw_connext_cpp_Debug.test_client_scope_consistency_cpp
  test_rclcpp.build.test_rclcpp.test_client_scope_consistency_cpp__rmw_fastrtps_cpp_Debug.test_client_scope_consistency_cpp
  test_rclcpp.build.test_rclcpp.test_client_scope_cpp__rmw_connext_cpp_Debug.test_client_scope_cpp
  test_rclcpp.build.test_rclcpp.test_client_scope_cpp__rmw_fastrtps_cpp_Debug.test_client_scope_cpp
  test_rclcpp.build.test_rclcpp.test_parameter_server_cpp__rmw_connext_cpp_Debug.test_parameter_server_cpp
  test_rclcpp.build.test_rclcpp.test_parameter_server_cpp__rmw_fastrtps_cpp_Debug.test_parameter_server_cpp
  test_rclcpp.build.test_rclcpp.test_services_cpp__rmw_connext_cpp_Debug.test_services_cpp
  test_rclcpp.build.test_rclcpp.test_services_cpp__rmw_fastrtps_cpp_Debug.test_services_cpp
  tf2_ros.tf2_ros_test_message_filter.gtest.missing_result
  

• Many instances of stuck colcon/build processes after a previous job fails on windows Job failed to delete workspace #161

  • killing the process tree fixes the issue

Outstanding buildfarmer tasks

• Periodic upgrades on Mac and Windows
  • mini1/mini2/mini3/icecube/portable upgraded Feb 28, 2019
  • windshield upgraded Mar 01, 2019
• Profile memory usage of bridge compilation job building the bridge requires a lot of memory #106 (failures currently resolved by increasing instance size)

[dirk-thomas]

Installed OpenSplice 6.9.190227OSS (r2, updated tag/release from today) on all macOS and Windows nodes.

[dirk-thomas]

Installed OpenSplice 6.9.190403OSS on all macOS and Windows nodes.

[dirk-thomas]

ros2/rclcpp#682 fixes 12 test failures on Windows debug.

[dirk-thomas]

Installed Visual Studio 2019 on all Windows nodes.

[dirk-thomas]

Upgraded CMake to version 3.14.1 on all Windows nodes.

[dirk-thomas]

Updated TightVNC server to 2.8.11 on icecube 🤞

[dirk-thomas]

Upgraded OpenCV to 3.4.6 (for VS 2017 and VS 2019) on all Windows nodes.

[dirk-thomas]

Closing in favor of #178.