[MacOS] Job hanging in the middle of the compilation

[mikaelarguedas]

This doesnt seem related to the other hangs we've seen (where parallel build hang on missing packages, or qemu hangs on CMake) so I'm opening a separate issue

http://ci.ros2.org/job/ci_osx/3151

[nuclearsandwich]

https://ci.ros2.org/job/nightly_osx_release/741 is another hang on the same host though in another part of the build (and a different configuration). Both of these occurred on dosa and both during a build phase though not during CMake generation which is where we usually see hangs in qemu.

[sloretz]

#88 is another case of a hang on dosa nightly_osx_repeated #1015

[nuclearsandwich]

In the archive linked is a copy of the workspace log directory and the open files and ports of the running cmake processes from build https://ci.ros2.org/job/ci_osx/4007/ rviz_ogre_vendor_fail.tar.gz

[nuclearsandwich]

from the above log, the last lines in rviz_ogre_vendor/stderr_stdout.log are

-- Downloading...
   dst='/Users/osrf/jenkins-agent/workspace/ci_osx/ws/build/rviz_ogre_vendor/ogre-master-ca665a6-prefix/src/v1.10.11.zip'
   timeout='none'
-- Using src='https://github.com/OGRECave/ogre/archive/v1.10.11.zip'

When inspecting the build space du returns a static filesize of 19M (du without the -h was used to determine that the file size was not slowly increasing over a period of several minutes.

du -h build/rviz_ogre_vendor/ogre-master-ca665a6-prefix/src/v1.10.11.zip                                                                                                                                                                                         
 19M    build/rviz_ogre_vendor/ogre-master-ca665a6-prefix/src/v1.10.11.zip

On a local download if the archive via the same URL the archive size upon completion was 127M.

I notice in the source that the ExternalProject_Add lacks a download timeout. Perhaps we should add one, even if it's looong?

https://github.com/ros2/rviz/blob/6d2af5e82018359ef7e86210336d86391b2c3d5f/rviz_ogre_vendor/CMakeLists.txt#L124-L146

[mikaelarguedas]

Interesting, there was a timeout before but it was apparently too short for some users: ros2/rviz#231.

Maybe we should go back and add a long timeout instead of just removing it completely

[dirk-thomas]

Assuming this issue was resolved by reintroducing the timeout (since there were no further comments on this issue since then) I will go ahead and close this ticket. Please feel free to comment on the closed ticket in case this happens again with more information.

---

Closed during #168.