getting malloc error while dealing with dates.

[yohaimagen]

Description of the problem

I am getting the following error while dealing with plotting time-series(dates as x-axis):
python(34471,0x1169cd5c0) malloc: *** error for object 0x7fcd964cf330: pointer being freed was not allocated
python(34471,0x1169cd5c0) malloc: *** set a breakpoint in malloc_error_break to debug
I am not sure it may be an upstream GMT issue

Full code that generated the error

import numpy as np
import pandas as pd
import pygmt as gmt

t = pd.date_range(pd.to_datetime('03-13-2021'), pd.to_datetime('04-10-2021'))

fig = gmt.Figure()
gmt.config(FORMAT_DATE_MAP='dd-mm')
fig.basemap(
    projection='X10/5',
    region=[t[0], t[-1], -0.1, 30],
    frame=True
)
fig.plot(
    x=t,
    y=np.ones(t.shape[0])
)
fig.show()

the script above works when changing the fig.plot to be:

fig.plot(
    x=t,
    y=np.arange(t.shape[0])
)

but fails with np.zeros, np.random.rand, and my real-world data.

Full error message

python(35111,0x10ef505c0) malloc: *** error for object 0x7fb538cc7ec0: pointer being freed was not allocated
python(35111,0x10ef505c0) malloc: *** set a breakpoint in malloc_error_break to debug
Abort trap: 6

System information

PyGMT information:
  version: v0.5.0
System information:
  python: 3.10.1 | packaged by conda-forge | (main, Dec 16 2021, 18:00:56) [Clang 11.1.0 ]
  executable: /Users/yohai/anaconda3/envs/ibaraki_proj/bin/python
  machine: macOS-10.14.6-x86_64-i386-64bit
Dependency information:
  numpy: 1.21.4
  pandas: 1.3.5
  xarray: 0.20.2
  netCDF4: 1.5.8
  packaging: 21.3
  ghostscript: 9.50
  gmt: 6.1.0
GMT library information:
  binary dir: /Users/yohai/anaconda3/envs/ibaraki_proj/bin
  cores: 12
  grid layout: rows
  library path: /Users/yohai/anaconda3/envs/ibaraki_proj/lib/libgmt.dylib
  padding: 2
  plugin dir: /Users/yohai/anaconda3/envs/ibaraki_proj/lib/gmt/plugins
  share dir: /Users/yohai/anaconda3/envs/ibaraki_proj/share/gmt
  version: 6.3.0

[maxrjones]

Thanks for the issue report - we should raise an error for this case. You need to specify the symbol parameters when using plot if they are not provided as part of your data:

import numpy as np
import pandas as pd
import pygmt as gmt

t = pd.date_range(pd.to_datetime('03-13-2021'), pd.to_datetime('04-10-2021'))

fig = gmt.Figure()
gmt.config(FORMAT_DATE_MAP='dd-mm')
fig.basemap(
    projection='X10/5',
    region=[t[0], t[-1], -0.1, 30],
    frame=True
)
fig.plot(
    x=t,
    y=np.ones(t.shape[0]),
    style="c0.4c",
    pen="1p",
    color="red3"
)
fig.show()

[seisman]

You need to specify the symbol parameters when using plot if they are not provided as part of your data:

If the symbol parameter style is not given, it means users want to plot a line. So the script from @yohaimagen should work. Am I misunderstand anything?

[yohaimagen]

That is what I thought, I attended to plot a line.

[maxrjones]

Thanks for the clarification, I'll work on debugging the issue.

[seisman]

This looks like an upstream bug. Here is a smaller script to reproduce the issue:

import pygmt

fig = pygmt.Figure()
fig.plot(
    projection='X10/5',
    region="2021-03-01/2021-03-04/-0.1/30",
    frame=True,
    x=["2021-03-01", "2021-03-02", "2021-03-03", "2021-03-04"],
    y=[0., 1., 2., 3.],
    #y=[0, 1, 2, 3],
)
fig.savefig("map.png", show=True)

y=[0., 1., 2., 3.] crashes but y=[0, 1, 2, 3] works.

Ping @meghanrjones or @PaulWessel to see if you have time debugging it.

[maxrjones]

I'll try again to debug tomorrow. I ran into a wall last time, if that happens again I will at least post what I find.

[maxrjones]

GMT changes the method from GMT_IN|GMT_IS_REFERENCE to GMT_IN|GMT_IS_DUPLICATE for any cases with non-double data (see https://github.com/GenericMappingTools/gmt/blob/a8bbbcf59ef6d4d20a456278f52605328df046a6/src/gmt_api.c#L9020-L9022). The data are duplicated for y=[0, 1, 2, 3], but not for y=[0., 1., 2., 3.], which is why one crashes but not the other.

So, there is a problem with passing GMT_DATASET structures via GMT_IS_REFERENCE. I found that this crash started after GenericMappingTools/gmt#5953 was merged. @PaulWessel, would you be able to take another look at the memory management for this case?

Here is the simple example that fails for this case:

import pygmt

fig = pygmt.Figure()
fig.plot(
    projection='X10/5',
    region="2021-03-01/2021-03-04/-0.1/30",
    frame=True,
    x=["2021-03-01", "2021-03-02", "2021-03-03", "2021-03-04"],
    y=[0., 1., 2., 3.],
    #y=[0, 1, 2, 3],
)

[PaulWessel]

I would love to debug this one but apparently the conda stuff is still Intel only, which makes it pretty hard to debug on M1. But if you can do this you should be able to determine what is the difference between the two cases. I dont understand my a double array gets duplicated (double is the natural type of datasets). A re you sure it is not the integer array that requires duplication - surely it does?

[maxrjones]

I would love to debug this one but apparently the conda stuff is still Intel only, which makes it pretty hard to debug on M1. But if you can do this you should be able to determine what is the difference between the two cases. I dont understand my a double array gets duplicated (double is the natural type of datasets). A re you sure it is not the integer array that requires duplication - surely it does?

sorry, copy-paste error when making the comment. Yes, double arrays are referenced (and crash). int arrays are duplicated (without issue).

[seisman]

I would love to debug this one but apparently the conda stuff is still Intel only, which makes it pretty hard to debug on M1.

What exactly did you mean? Installing GMT from conda works well on my M1.

[PaulWessel]

My experience has been that my conda installation (I have used miniconda but same with conda) the binaries are all Intel and things run under Rosetta2. Thus, when I try to debug I get failures due to mixed architectures....
Are you able to actually follow the PyGMT debug guidelines on our site and set stop points in Xcode and it works on M1?

[seisman]

I haven't tried to debug PyGMT in Xcode on M1.

[maxrjones]

This was fixed by GenericMappingTools/gmt#6521. I added a C API test so I don't think we need another regression test in the PyGMT repository.

[weiji14]

I've triggered a GMT dev version bump at conda-forge/gmt-feedstock#203, so once that's merged in a few hours (and assuming you don't mind using an unstable GMT version), you can either:

1. Run conda install -c conda-forge/label/dev gmt=6.4.0
2. Build GMT from source (see https://github.com/GenericMappingTools/gmt/blob/master/BUILDING.md) and set GMT_LIBRARY_PATH to the correct folder following https://www.pygmt.org/dev/install.html#finding-the-gmt-shared-library.

Either method should give you the latest patched version of GMT. Let us know if it works or if you need any additional clarification.