makedist: update cvmfs package repo urls with fallback #105
Conversation
|
@DrDaveD I don't think I touched the ppc64 downloads from copr, but those are failing - could you take a look? |
|
I think you did break it, because it works for me on an el8 machine using the master branch makedist but I get the same failure as the CI check when I use your PR's makedist. |
|
Right, thanks - found it, just stumbled over the |
|
Fixed another typo, this should be ready now. However I did find that our s3 package server supports listings under the alternative url https://cvmrepo.s3-website.cern.ch/cvmrepo - if you prefer I can check if I can make that listing behave the same as the apache IndexDocument , that would let us avoid the .latest_cvmfs_version_prod file |
I don't feel strongly about it, because it is more efficient to just read a single file but it is more standard to do a listing. Is the creation of the version file automated, so it won't be forgotten and won't be any extra work for the CernVM team to maintain? If so then maybe we might as well stick with it. |
|
@vvolkl Do you consider this to be complete now? |
vvolkl
changed the title
|
Hi Dave, no, giving this some more thought I think that getting the version from the directory listing instead of a special file is better, as it's the more generic solution. I've already set up http://cvmrepo.s3-website.cern.ch/cvmrepo/yum/cvmfs/EL/9/x86_64/ in a way that serves listings, and will update this PR this week. |
vvolkl
force-pushed
the
ha-cvmrepo
branch
2 times, most recently
from
1932ed6 to
096cad9
Compare
vvolkl
changed the title
|
@DrDaveD ready for review again |
vvolkl
force-pushed
the
ha-cvmrepo
branch
2 times, most recently
from
e943c9e to
44d5fbe
Compare
Adds a fallback URL for the cvmfs package download. makedist currently looks for the latest package in the listing returned by package servers; since the s3 bucket we are mainly using (and which is the most reliable) does not support listings, this mechanism now relies on reading a small file .latest_cvmfs_version_prod (https://cvmrepo.web.cern.ch/cvmrepo/.latest_cvmfs_version_prod) containing a string with the latest version. I'll add this to our testing to ensure it is kept up to date. Obviously this won't work if you wanted to add mirrors maintained by a third party - one possibility would be to hardcode the latest version in the script. The downloads for the non-default config packages is kept the same, except for a small fix for the egi download. The ecsft.cern.ch url is removed alltogether, as this is the most likely one to cause issues.
Adds a fallback URL for the cvmfs package download. makedist currently looks for the latest package in the listing returned by package servers; since the s3 bucket we are mainly using (and which is the most reliable) does not support listings, this mechanism now relies on reading a small file .latest_cvmfs_version_prod (https://cvmrepo.web.cern.ch/cvmrepo/.latest_cvmfs_version_prod) containing a string with the latest version. I'll add this to our testing to ensure it is kept up to date.
Obviously this won't work if you wanted to add mirrors maintained by a third party - one possibility would be to hardcode the latest version in the script.
The downloads for the non-default config packages is kept the same, except for a small fix for the egi download: Fixes #104
The ecsft.cern.ch url is removed alltogether, as this is the most likely one to cause issues.
Tested only on rhel9-x86_64
I'll add a yum repo and support for SUSE (and fedora) packages shortly - it seems like suse anyway needs to have proper testing (#87), so it is currently disabled.