BUG: pandas 1.2.0 and Pyarrow [0.16.0, 1.0.0) are incompatible for some column types

[ADraginda]

#35259 added optional importing from pyarrow. The currently listed minimum version of pyarrow is 0.15.1, and the logic of said PR guards against importing attributes from pyarrow.compute because it is not available in 0.15.1

Problem: pyarrow added the compute module in 0.16.0 but attributes imported by pandas are not available in that module until 1.0.0

Therefore: with pandas 1.2.0 and merging a DataFrame (with an Array String column?) :

Pyarrow Version	Behavior
not installed	works
< 0.15.1	not supported
[0.15.1, 0.16.0).	works (pyarrow.compute.equal not used?)
[0.16.0, 1.0.0)	ArgumentError
[1.0.0, 2.0.0 (latest)]	works (using pyarrow.compute.equal)

Adding another try/except around the comparison functions in pandas string_arrow.py will change the table of functionality to:

Pyarrow Version	Behavior
not installed	works
< 0.15.1	not supported
[0.15.1, 1.0.0).	works (pyarrow.compute.equal not used?)
[1.0.0, 2.0.0 (latest)]	works (using pyarrow.compute.equal)

[jreback]

see my comments on the PR. what causes this error exactly?

e.g. what is

Therefore: with pandas 1.2.0 and merging a DataFrame (with an Array String column?) :

[ADraginda]

@jreback I replied in the PR as well but here is a reproduction:

You can reproduce with pyarrow 0.17.1 (that's what I'm on, but breaks for other versions too) with the following example. Seems to be related to merging tow tables, one on a column of type string and the other of type object.

import pandas as pd

example_dict = {'i': {0: 'foo'}, 'j': {0: 1}}

foo = pd.DataFrame(example_dict)
foo['i'] = foo['i'].astype(pd.StringDtype.name)
foo['j'] = foo['j'].astype(pd.Int64Dtype.name)

bar = pd.DataFrame(example_dict)
bar['j'] = bar['j'].astype(pd.Int64Dtype.name)

foo.merge(bar)