Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Segmentation Fault when saving ICP map #479

Closed
alexge233 opened this issue Mar 15, 2017 · 30 comments
Closed

Segmentation Fault when saving ICP map #479

alexge233 opened this issue Mar 15, 2017 · 30 comments

Comments

@alexge233
Copy link

I'm using mrpt 1.5 from an Ubuntu PPA.

The ICP builder exposes some methods which AFAIK allow the CSimpleMap to be saved, however they all SEGFAULT when I try:

  • saveCurrentMapToFile segfaults right away
  • setCurrentMapFile when the builder is destroyed, it tries to save the map file
  • getCurrentlyBuiltMap does not fill the map, which when I try to save segfaults also

Am I using this wrongly, or is this a bug?
Running under gdb and backtracing pinpoints to mrpt::obs::CSensoryFrame::writeToStream which in turn hints towards a dynamic_cast.

@jolting
Copy link
Member

jolting commented Mar 15, 2017

Can you post a code snippet of what you are actually doing? Also if you are feeling particularly adventurous try it against mrpt-2.0. The pointer and serialization code was overhauled and I want to see what kind of exception you get.

@jlblancoc
Copy link
Member

Hi,
I just tested the app icp-slam and it correctly generates all map files, CSimpleMap, etc.

I agree with @jolting : you would need to share more details on your application to help finding out what's wrong...

@alexge233
Copy link
Author

Hi @jolting it is not an exception but a segmentation fault, I've tried 2 variations of the same thing:

mrpt::maps::CSimpleMap out_map;
__builder__.getCurrentlyBuiltMap(out_map);
out_map.saveToFile("simplemap.gz");

I've also tried setting the current map file setCurrentMapFile when starting ICP map builder,
and then saving it:

__builder__.initialize(__map__);

where __map__ and __builder__ are private members, and then later saving it as before (this actually works).

@jlblancoc the maps are generated, especially the multimetric maps, its the simple map that fails to save to disk, when using getCurrentlyBuiltMap.

Please bear in mind I've put the code in a ROS node, but the gdb output is consistent:

Thread 12 "mario_node" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffbaffd700 (LWP 9266)]
0x00007ffff4363f93 in __dynamic_cast () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
(gdb) bt
#0  0x00007ffff4363f93 in __dynamic_cast () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#1  0x00007ffff5ea06de in mrpt::obs::CSensoryFrame::writeToStream(mrpt::utils::CStream&, int*) const () from /usr/lib/x86_64-linux-gnu/libmrpt-obs.so.1.5
#2  0x00007ffff6ca513e in mrpt::utils::CStream::WriteObject(mrpt::utils::CSerializable const*) () from /usr/lib/x86_64-linux-gnu/libmrpt-base.so.1.5
#3  0x00007ffff6ca5629 in mrpt::utils::CStream::operator<<(mrpt::utils::CSerializable const&) () from /usr/lib/x86_64-linux-gnu/libmrpt-base.so.1.5
#4  0x00007ffff5e2837a in mrpt::maps::CSimpleMap::writeToStream(mrpt::utils::CStream&, int*) const () from /usr/lib/x86_64-linux-gnu/libmrpt-obs.so.1.5
#5  0x00007ffff6ca513e in mrpt::utils::CStream::WriteObject(mrpt::utils::CSerializable const*) () from /usr/lib/x86_64-linux-gnu/libmrpt-base.so.1.5
#6  0x00007ffff6ca5629 in mrpt::utils::CStream::operator<<(mrpt::utils::CSerializable const&) () from /usr/lib/x86_64-linux-gnu/libmrpt-base.so.1.5
#7  0x00007ffff5e2c9ff in mrpt::maps::CSimpleMap::saveToFile(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const () from /usr/lib/x86_64-linux-gnu/libmrpt-obs.so.1.5

@jolting
Copy link
Member

jolting commented Mar 16, 2017 via email

@alexge233
Copy link
Author

alexge233 commented Mar 16, 2017

@jolting Hi, yes I'm using PPA

http://ppa.launchpad.net/joseluisblancoc/mrpt/ubuntu xenial/main amd64 Packages:

libmrpt-base1.5 - Mobile Robot Programming Toolkit - base library
libmrpt-detectors1.5 - Mobile Robot Programming Toolkit - detectors library
libmrpt-gui1.5 - Mobile Robot Programming Toolkit - gui library
libmrpt-hmtslam1.5 - Mobile Robot Programming Toolkit - hmtslam library
libmrpt-hwdrivers1.5 - Mobile Robot Programming Toolkit - hwdrivers library
libmrpt-kinematics1.5 - Mobile Robot Programming Toolkit - kinematics library
libmrpt-maps1.5 - Mobile Robot Programming Toolkit - maps library
libmrpt-nav1.5 - Mobile Robot Programming Toolkit - nav library
libmrpt-obs1.5 - Mobile Robot Programming Toolkit - obs library
libmrpt-opengl1.5 - Mobile Robot Programming Toolkit - opengl library
libmrpt-slam1.5 - Mobile Robot Programming Toolkit - slam library
libmrpt-tfest1.5 - Mobile Robot Programming Toolkit - tfest library
libmrpt-topography1.5 - Mobile Robot Programming Toolkit - topography library
libmrpt-vision1.5 - Mobile Robot Programming Toolkit - vision library
libmrpt-graphslam1.5 - Mobile Robot Programming Toolkit - graphslam library
libmrpt-graphs1.5 - Mobile Robot Programming Toolkit - graphs library

However there is no dbg package for 1.5 as far as I can see, only the older version (1.3?) which brakes the current package.

@jolting
Copy link
Member

jolting commented Mar 16, 2017

Try my ppa if you are using 16.04.

https://launchpad.net/~jolting/+archive/ubuntu/mrpt-daily

Note: I can't guarantee stability of my PPA. Only use it for debugging.

@alexge233
Copy link
Author

alexge233 commented Mar 17, 2017

HI @jolting I've tried your ppa, but it still doesn't install dbg packages:

/usr/bin/ld: cannot find -lmrpt-base-dbg
/usr/bin/ld: cannot find -lmrpt-maps-dbg
/usr/bin/ld: cannot find -lmrpt-obs-dbg
/usr/bin/ld: cannot find -lmrpt-gui-dbg
/usr/bin/ld: cannot find -lmrpt-opengl-dbg
/usr/bin/ld: cannot find -lmrpt-vision-dbg
/usr/bin/ld: cannot find -lmrpt-graphs-dbg
/usr/bin/ld: cannot find -lmrpt-hwdrivers-dbg
/usr/bin/ld: cannot find -lmrpt-nav-dbg
/usr/bin/ld: cannot find -lmrpt-slam-dbg
/usr/bin/ld: cannot find -lmrpt-graphslam-dbg
/usr/bin/ld: cannot find -lmrpt-hmtslam-dbg
collect2: error: ld returned 1 exit status

When I try with ppa:joseluisblancoc/mrpt I keep getting the following error:

The following packages have unmet dependencies.
 libmrpt-dbg : Depends: libmrpt-dev (= 1:1.5.0~snapshot20160512-1218-git-ebcee4ef-xenial-1~ppa1~xenial) but 1:1.5.0~snapshot20170307-0309-git-61268233-xenial-1~ppa1~xenial is to be installed
E: Unable to correct problems, you have held broken packages.

@jlblancoc
Copy link
Member

Did you installed this one?

sudo apt install libmrpt-dbg

@alexge233
Copy link
Author

yes I did, I even removed the cached cmake build stuff just to be sure...
Should I use both the ppa's (yours and Jose's?)

@jlblancoc
Copy link
Member

jlblancoc commented Mar 17, 2017

Ah! Let's @jolting give his opinion here, but I think we had a misunderstood, sorry:
Debian -dbg packages only include symbol information, but not the MRPT -dbg libraries.
Please, build in normal / "Release" CMake configuration, but with the Debian -dbg package installed.

Then, gdb should be able, so goes on the theory, to find the symbols and print a more informative stack trace.

@alexge233
Copy link
Author

I'm a bit confused, should I build it from source with debug symbols? AFAIK the ppa packages do not include debug symbols.

@jolting
Copy link
Member

jolting commented Mar 17, 2017

#211

Add don't use debug libs macro to your cmake file.

@alexge233
Copy link
Author

Hi, the only thing I'm adding is the cmake flag -DCMAKE_BUILD_TYPE=Debug. I don't know where the ld errors come from, I'm going to guess from your cmake module?

@jolting
Copy link
Member

jolting commented Mar 20, 2017

Did you download a newer build?

The latest one was built 7 hours ago.
1:1.5.0~snapshot201703200708-git-4f1f736-201703201102~ubuntu16.04.1

The -dbg issue should have been fixed 3 days ago with this commit.
ea55898

@alexge233
Copy link
Author

@jolting I'm away this week so can't access the robot; I'll try rebuilding from github master branch next week. Many thanks!

@alexge233
Copy link
Author

@jolting Hi I'm back this week. I'm trying to build from github (master branch) but it keeps failing:

/usr/bin/ld: cannot find -lassimp-mrpt

Should I try the one from your repository instead?

@jolting
Copy link
Member

jolting commented Mar 27, 2017

Yes, you can try the repo on 16.04.

To address your issue try installing libassimp-dev. I'm not sure what the problem is with the non-system version of assimp. I'll have to look at this.

@alexge233
Copy link
Author

Sorry for taking so long to reply, here is what happens:

[CMetricMapBuilderICP|INFO |13:10:21.3311] [CMetricMapBuilderICP]   Fit:97.3% Itr:8 In 0.00ms 
[CMetricMapBuilderICP|INFO |13:10:21.3751] [CMetricMapBuilder::saveCurrentMapToFile] Saving current map to 'icpmap' ...
pure virtual method called
terminate called without an active exception

Thread 13 "mario_node" received signal SIGABRT, Aborted.
[Switching to Thread 0x7fffaf7fe700 (LWP 6165)]
0x00007ffff22f9428 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54
54      ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0  0x00007ffff22f9428 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54
#1  0x00007ffff22fb02a in __GI_abort () at abort.c:89
#2  0x00007ffff293284d in __gnu_cxx::__verbose_terminate_handler() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#3  0x00007ffff29306b6 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#4  0x00007ffff2930701 in std::terminate() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#5  0x00007ffff293123f in __cxa_pure_virtual () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#6  0x00007ffff6c667f2 in mrpt::utils::CStream::WriteObject (this=this@entry=0x7fffaf7fd180, o=0x7fffaeffc2e0) at /build/mrpt-gP7Bdz/mrpt-1.5.0~snapshot201704030626-git-bff6d01/libs/base/src/utils/CStream.cpp:193
#7  0x00007ffff6c66d09 in mrpt::utils::CStream::operator<< (this=this@entry=0x7fffaf7fd180, obj=...) at /build/mrpt-gP7Bdz/mrpt-1.5.0~snapshot201704030626-git-bff6d01/libs/base/src/utils/CStream.cpp:220
#8  0x00007ffff5dc5469 in mrpt::obs::CSensoryFrame::writeToStream (this=0x7fff9c001220, out=..., version=<optimised out>) at /build/mrpt-gP7Bdz/mrpt-1.5.0~snapshot201704030626-git-bff6d01/libs/obs/src/CSensoryFrame.cpp:100
#9  0x00007ffff6c6681e in mrpt::utils::CStream::WriteObject (this=this@entry=0x7fffaf7fd180, o=o@entry=0x7fff9c001220) at /build/mrpt-gP7Bdz/mrpt-1.5.0~snapshot201704030626-git-bff6d01/libs/base/src/utils/CStream.cpp:200
#10 0x00007ffff6c66d09 in mrpt::utils::CStream::operator<< (this=0x7fffaf7fd180, obj=...) at /build/mrpt-gP7Bdz/mrpt-1.5.0~snapshot201704030626-git-bff6d01/libs/base/src/utils/CStream.cpp:220
#11 0x00007ffff5e5e38a in mrpt::maps::CSimpleMap::writeToStream (this=0x7fffaf7fd190, out=..., version=<optimised out>) at /build/mrpt-gP7Bdz/mrpt-1.5.0~snapshot201704030626-git-bff6d01/libs/obs/src/CSimpleMap.cpp:274
#12 0x00007ffff6c6681e in mrpt::utils::CStream::WriteObject (this=this@entry=0x7fffaf7fd180, o=o@entry=0x7fffaf7fd190) at /build/mrpt-gP7Bdz/mrpt-1.5.0~snapshot201704030626-git-bff6d01/libs/base/src/utils/CStream.cpp:200
#13 0x00007ffff6c66d09 in mrpt::utils::CStream::operator<< (this=this@entry=0x7fffaf7fd180, obj=...) at /build/mrpt-gP7Bdz/mrpt-1.5.0~snapshot201704030626-git-bff6d01/libs/base/src/utils/CStream.cpp:220
#14 0x00007ffff5ae6bff in mrpt::slam::CMetricMapBuilder::saveCurrentMapToFile (this=<optimised out>, fileName="icpmap", compressGZ=<optimised out>) at /build/mrpt-gP7Bdz/mrpt-1.5.0~snapshot201704030626-git-bff6d01/libs/slam/src/slam/CMetricMapBuilder.cpp:92
#15 0x000000000041c65f in icp_builder::save() const ()
#16 0x000000000041de84 in icp_builder::publish_pose(ros::TimerEvent const&) ()
#17 0x00007ffff7b094eb in ros::TimerManager<ros::Time, ros::Duration, ros::TimerEvent>::TimerQueueCallback::call() () from /opt/ros/kinetic/lib/libroscpp.so
#18 0x00007ffff7b2ecf0 in ros::CallbackQueue::callOneCB(ros::CallbackQueue::TLS*) () from /opt/ros/kinetic/lib/libroscpp.so
#19 0x00007ffff7b2f8e4 in ros::CallbackQueue::callOne(ros::WallDuration) () from /opt/ros/kinetic/lib/libroscpp.so
#20 0x00007ffff7b87ad5 in ros::AsyncSpinnerImpl::threadFunc() () from /opt/ros/kinetic/lib/libroscpp.so
#21 0x00007ffff1c885d5 in ?? () from /usr/lib/x86_64-linux-gnu/libboost_thread.so.1.58.0
#22 0x00007ffff1a616ba in start_thread (arg=0x7fffaf7fe700) at pthread_create.c:333
#23 0x00007ffff23ca82d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

@jolting
Copy link
Member

jolting commented Apr 21, 2017

CSensoryFrame Can't be serialized, which probably means that some observation you inserted into the sensory frame is invalid.
https://github.com/MRPT/mrpt/blob/master/libs/obs/src/CSensoryFrame.cpp#L100

https://github.com/MRPT/mrpt/blob/master/libs/base/src/utils/CStream.cpp#L193
I think __cxa_pure_virtual means it's trying to invoke writeToStream on an invalid object.

Go up to function 6 and print o.

#6  0x00007ffff6c667f2 in mrpt::utils::CStream::WriteObject (this=this@entry=0x7fffaf7fd180, o=0x7fffaeffc2e0) at /build/mrpt-gP7Bdz/mrpt-1.5.0~snapshot201704030626-git-bff6d01/libs/base/src/utils/CStream.cpp:193

@alexge233
Copy link
Author

The observation was a CObservation2DRangeScanPtr and it should be valid because the ICP builder works properly, unless I've malformed its data. I'm afraid I haven't got access to the robot until Monday, but I'll get back to you on this, including the way the observation is populated. Thanks for the help!

@jolting
Copy link
Member

jolting commented Apr 21, 2017

In that context the type of o is CSerializable *, which should point to CObservation2DRangeScan. If o is really pointing to a valid CObservation2DRangeScan, I can't see how __cxa_pure_virtual is getting called.

Is it possible that you are assigning a raw pointer to a smart pointer twice?

@alexge233
Copy link
Author

Hi,

I am populating observation from a ros laserscan message. I allocate an object, and then assign a pointer using that object, and finally insert the observation. ICP works fine, I get no segfaults, crashes or leaks, the only exception is when I use that particular method.

The (overall and simplified) version of this is:

void sick_561::populate_lazer(const sensor_msgs::LaserScan::ConstPtr & data) 
{
    auto obs = mrpt::obs::CObservation2DRangeScan();
    obs.loadFromVectors(data->ranges.size(), 
                                       data->ranges.data(), 
                                       (char*)data->intensities.data());
    auto ptr = mrpt::obs::CObservation2DRangeScanPtr();
    ptr.setFromPointerDoNotFreeAtDtor(&obs);
    builder__.processObservation(ptr);
}

The ICP class wrapper then goes on to write the map, at which point I get the segmentation fault:

builder__.saveCurrentMapToFile("icpmap", true);

The other method saveCurrentEstimationToImage("icp_map", true); works fine,
so does me accessing the underlying maps:

builder__->getcurrentlyBuiltMetricMap()->m_gridMaps[0]->saveAsBitmapFile("gridmap.png");
builder__->getcurrentlyBuiltMetricMap()->m_pointsMaps[0]->save3D_to_text_file("pointsmap.txt");

I went to mrpt::utils::CStream::WriteObject and printing o shows:

$1 = (const mrpt::utils::CSerializable *) 0x7fffb4f1afb0

I did: print *(const mrpt::utils::CSerializable *) 0x7fffb4f1afb0 and the result was:

$2 = {<mrpt::utils::CObject> = {_vptr.CObject = 0x7ffff60f4840 <vtable for mrpt::maps::CSimpleMap+16>,
 static classCObject = {
      className = 0x7ffff6e76e53 "CObject", ptrCreateObject = 0x0, getBaseClass = 0x0}}, static class
CSerializable = {
    className = 0x7ffff6d66278 "CSerializable", ptrCreateObject = 0x0,
    getBaseClass = 0x7ffff6c5f3e0 <mrpt::utils::CSerializable::_GetBaseClass()>}}
(gdb)

Does that help?

@jolting
Copy link
Member

jolting commented Apr 25, 2017

You're not at the right stack level.

Assuming that old stack trace is somewhat relevant...
You're printing it here:

#12 0x00007ffff6c6681e in mrpt::utils::CStream::WriteObject (this=this@entry=0x7fffaf7fd180, o=o@entry=0x7fffaf7fd190) at /build/mrpt-gP7Bdz/mrpt-1.5.0~snapshot201704030626-git-bff6d01/libs/base/src/utils/CStream.cpp:200

The useful data is here:

#6  0x00007ffff6c667f2 in mrpt::utils::CStream::WriteObject (this=this@entry=0x7fffaf7fd180, o=0x7fffaeffc2e0) at /build/mrpt-gP7Bdz/mrpt-1.5.0~snapshot201704030626-git-bff6d01/libs/base/src/utils/CStream.cpp:193

@jlblancoc
Copy link
Member

Hi Alex

Your problem is that you're populating the smart pointer with a pointer to a local variable, which is allocated the stack! This is one of the biggest DONT with smart pointers ;-)

Just create the smart pointer FIRST, with something like:

auto obs = CObservation2DRangeScan::Create();

Cheers

@alexge233
Copy link
Author

@jlblancoc this should not be a problem, unless the builder requires a copy, in which case it should make a deep copy. From what I understand the method expects a heap-allocated object which the builder will deallocate?

@jolting
Copy link
Member

jolting commented Apr 25, 2017

@alexge233 The purpose of the shared pointer is to prevent the overhead of the deep copy. It just stores a copy of the the smart pointer, so it's best just to let the smart pointer manage the lifetime of the object. Basically the object must exist after processObservation until the builder decides it's done with it. In general the shared pointer should not use setFromPointerDoNotFreeAtDtor. If I remember correctly we removed it from mrpt-2.0.

This stack allocates the object:

auto obs = mrpt::obs::CObservation2DRangeScan();

This makes the shared pointer from a stack allocated object. This is bad

    ptr.setFromPointerDoNotFreeAtDtor(&obs);

Try this:

    auto ptr =  CObservation2DRangeScan::Create();
    ptr->loadFromVectors(data->ranges.size(), 
                                       data->ranges.data(), 
                                       (char*)data->intensities.data());
    builder__.processObservation(ptr);

I find that smart pointers are a bit burdensome to learn. I've attempted to teach them a several times to classmates with varying levels of success. I had an idea for mrpt-2.0 about adding a version that does not depend on pointers. Instead it would depend on c++11 move semantics/copy semantics. A move would be zero copy and a copy would then have an overhead of a deep copy. Personally, I find these concepts simpler and easier to express. Unfortunately, we aren't there yet in the 2.0 branch. Also, this idea might be trading one problem for another because many developers haven't quite grasped move semantics yet.

@alexge233
Copy link
Author

@jolting thanks for the help!

I'm not criticising guys, we appreciate a lot the effort you've put into this library. If however you intend to use smart pointers, may I suggest you go down the STL route instead? For example, if you use an std::unique_ptr it is quite clear what is expected.

Following the documentation from doxygen for 1.5.0 its wasn't clear if those methods are expecting an object (copyable), a heap-allocated object or whatnot; the fact you that you have setFromPointerDoNotFreeAtDtor implies you're expecting me to manage the lifetime of the object memory.

Even worse, I was using a scoped object on purpose, assuming you made a copy of it and ICP was working fine, it wasn't until I called the method saveCurrentMapToFile that I got a segfault, thereby adding to the confusion. I am guessing my observations were being used properly for building the map, but not indexed and when I called that method it eventually led to the segfault because they had been invalidated.

Again, many thanks for the help!
Regards,
Alex

@jlblancoc
Copy link
Member

jlblancoc commented Apr 25, 2017 via email

@jolting
Copy link
Member

jolting commented Apr 26, 2017

@alexge233 Feel free to help test mrpt-2.0.

@alexge233
Copy link
Author

@jolting is it in your ppa or should I build it from source? We're mostly interested in ICP and visual slam. Do you have a gitter room for mrpt?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants