Parallelization: switch from multiprocessing to joblib

[Ziqi-Li]

This PR replaces the repvious multiprocessing.pool based parallelization to joblib based. Users only need to specify the n_jobs parameter instead of passing a pool.

Example use:
GWR

n_jobs = 8
bw = Sel_BW(coords, y, X, n_jobs= n_jobs).search()
GWR(coords, y, X, bw, n_jobs=n_jobs)

MGWR

n_jobs = -1 #use all
mgwr_selector = Sel_BW(coords, y, X, multi=True, n_jobs=n_jobs)
mgwr_bw = mgwr_selector.search()
mgwr_results = MGWR(coords, y, X, selector=mgwr_selector, n_jobs=n_jobs).fit()

The test sets are updated to reflect the new interface. The notebook example is also updated and the effectiveness can be compared here: joblib-based (new) vs. mp-based (old)

[martinfleis]

Sorry for being late but I made two comments in the code.

At least the one on hard-breaking change would be good to resolve imho.

[martinfleis]

I know I am coming late to the party but would you consider using -1 as a default? That is quite common across ML world and it is what users generally expect.

[knaaptime]

i was curious what others' opinion was on this... i tend to default to -1 personally, but joblib itself [indirectly] defaults to 1, and in esda we do both (join counts are conservative, G defaults to -1, etc)

[knaaptime]

i think scikit usually defaults to 1 so i dont think there's a standard expectation in ML world

[martinfleis]

I don't mind either as long as it is documented (which it is). But for heavily parallelisable code like this one, I tend to prefer parallel execution by default.

[Ziqi-Li]

Yeah, I was only looking at scikit-learn and adapts to what they have. I actually personally prefer -1 as the default.

[martinfleis]

This is a hard-breaking change we should avoid. I suggest keeping the keyword and warning when it is not None.

[martinfleis]

Another note, looking at the notebook. The section Effectivenss of n_jobs has a typo in the title but more importantly, can be misleading. The curve applies to a machine that likely has 8 effective cores and would look very differently on one with more or less. I think it deserves at least a line of a comment pointing this out.

[Ziqi-Li]

Another note, looking at the notebook. The section Effectivenss of n_jobs has a typo in the title but more importantly, can be misleading. The curve applies to a machine that likely has 8 effective cores and would look very differently on one with more or less. I think it deserves at least a line of a comment pointing this out.

Nice catch on the typo. Will fix it.

I think the curve looks like this because 1) the data is still small 2) the linear part of the computation limits it from further scaling. Here the curves are just showing the parallelization works, the specific scalability depends on so many factors.

[martinfleis]

The linear part of the computation limits it from further scaling

Only because of the number of available cores, no? If you ran the same code on a 32 core CPU, the minimum would be at 31-32, not 8 like here.

[Ziqi-Li]

The linear part of the computation limits it from further scaling

Only because of the number of available cores, no? If you ran the same code on a 32 core CPU, the minimum would be at 31-32, not 8 like here.

I forgot to mention I have 12 physical cores on the machine. Will you suggest to add a note in notebook saying this is based on a 12-core machine.

[martinfleis]

Will you suggest to add a note in notebook saying this is based on a 12-core machine.

Yes, that is what I meant.

It is interesting that it does not scale linearly to 12 cores...