[Python] Could not read encrypted metadata via pq.read_table

[heyuqi1970]

Describe the bug, including details regarding any error messages, version, and platform.

os: macos 11.7.10 (20G1427)
python: 3.9.7
pyarrow: 16.0.0

when I use pq.read_table with decryption_properties parameter, I get the following error。
And I can use pq.ParquetFile with decryption_properties to read same encrypted file.

Traceback (most recent call last):
  File "tt.py", line 98, in <module>
    table = pq.read_table("yellow_cryp.parquet", memory_map=True, decryption_properties=decryption_properties)
  File "/venv/lib/python3.9/site-packages/pyarrow/parquet/core.py", line 1762, in read_table
    dataset = ParquetDataset(
  File "/venv/lib/python3.9/site-packages/pyarrow/parquet/core.py", line 1329, in __init__
    [fragment], schema=schema or fragment.physical_schema,
  File "pyarrow/_dataset.pyx", line 1431, in pyarrow._dataset.Fragment.physical_schema.__get__
  File "pyarrow/error.pxi", line 154, in pyarrow.lib.pyarrow_internal_check_status
  File "pyarrow/error.pxi", line 91, in pyarrow.lib.check_status
OSError: Could not open Parquet input source 'yellow_cryp.parquet': Could not read encrypted metadata, no decryption found in reader's properties

Component(s)

Parquet, Python

[mapleFU]

@adamreeve @wgtmac do you have some infor for this issue?

[wgtmac]

This error happens when reading a parquet file with encrypted footer without providing decryption properties. From the issue description, it looks like the properties are not properly passed by the python dataset api.

@heyuqi1970 Could you please try the latest 17.0.0 release? If it still happens, please share the code to reproduce it if possible.

cc @tolleybot I'm not sure if you have fixed a similar issue.

[heyuqi1970]

@wgtmac
THX， 17.0.0 addresses this issue.

[wgtmac]

Thanks for your confirmation! @heyuqi1970