Update gaia astroquery to handle new output formats due to ESAC TAP 9.6.0 library (gaiapcr-1299)

[cosmoJFH]

Recently ESAC has release the new 9.6.0 TAP library, that has been tested in the gaia DEVELOPMENT environment (https://geadev.esac.esa.int/). The changes in the TAP library imply that:

1. now for the output format votable, an uncompressed votable file is returned. With Content-Type: application/x-votable+xml
2. the new output format votable_gzip was introduced, to return compressed votable files. With Content-Encoding: gzip and Content-Type: application/x-votable+gzip
3. for the output format ecsv, a compressed file is returned. With Content-Encoding: gzip and Content-Type: text/ecsv+gzip.

This implies the following issues in astroquery:

from astroquery.gaia import GaiaClass

gaia = GaiaClass(gaia_tap_server='https://geadev.esac.esa.int/')

job = gaia.launch_job("select top 100 solution_id,ref_epoch,ra_dec_corr,astrometric_n_obs_al, matched_transits,duplicated_source,phot_variable_flag from gaiadr3.gaia_source order by source_id", dump_to_file=True, output_format='votable_gzip')

job = gaia.launch_job("select top 100 solution_id,ref_epoch,ra_dec_corr,astrometric_n_obs_al, matched_transits,duplicated_source,phot_variable_flag from gaiadr3.gaia_source order by source_id", dump_to_file=True, output_format='ecsv')

The name of the output files are 1698154295940D-result.vot.gz.gz and 698154295940D-result.ecsv.gz.gz, respectively.

The issue is located in:

1. the method get_file_from_header in utils/tap/conn/tapconn.py. The function astroquery/utils/tap/conn/tapconn.py/get_file_from_header is called from astroquery/utils/tap/conn/tapconn.py/get_suitable_output_file
2. get_suitable_output_file_name_for_current_output_format in utils/tap/taputils.py. This is only used in utils/tap/core.py:
   1. launch_job
   2. launch_job_async

Note the the proposed solution is backward compatible for those ESAC archives that do not use the new 9.6.0 TAP library.

Note that the new 9.6.0 TAP library will use in the gaia PRODUCTION environment by February 2024.

cc @esdc-esac-esa-int

[pep8speaks]

Hello @cosmoJFH! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2024-01-04 21:48:47 UTC

[codecov]

Codecov Report

Attention: 6 lines in your changes are missing coverage. Please review.

Comparison is base (5600762) 67.23% compared to head (0d97538) 67.30%.
Report is 4 commits behind head on main.

Files	Patch %	Lines
astroquery/utils/tap/conn/tapconn.py	0.00%	5 Missing ⚠️
astroquery/utils/tap/taputils.py	50.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2907      +/-   ##
==========================================
+ Coverage   67.23%   67.30%   +0.06%     
==========================================
  Files         235      235              
  Lines       18155    18135      -20     
==========================================
- Hits        12207    12206       -1     
+ Misses       5948     5929      -19     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[bsipocz]

Thank you so much of the improvement and the nice cleanup, too!

[bsipocz]

My review comments were in fact so minor that I went ahead and addressed them, no need to have another round on this PR.

Thanks again @cosmoJFH!