control-service: add amazon rds ca certificates to data job image

[mrMoZ1]

what:
Adding amazon rds ca certificates to secure job image

why:
Data job users require ca certificates to connect to amazon databases

testing:
building images locally and inspecting certificate output:
List truncated because of 100+ certificates
`38.45 subject=C = US, O = "Amazon Web Services, Inc.", OU = Amazon RDS, ST = WA, CN = Amazon RDS eu-west-1 Root CA RSA4096 G1, L = Seattle

38.46 subject=C = US, O = "Amazon Web Services, Inc.", OU = Amazon RDS, ST = WA, CN = Amazon RDS eu-central-1 Root CA RSA4096 G1, L = Seattle

38.46 subject=C = US, O = "Amazon Web Services, Inc.", OU = Amazon RDS, ST = WA, CN = Amazon RDS eu-north-1 Root CA RSA2048 G1, L = Seattle
`

[gabrielgeorgiev1]

not sure I understand this change, if users need a particular certificate, is it not their responsibility to include it as part of their job or download it?

[mrMoZ1]

not sure I understand this change, if users need a particular certificate, is it not their responsibility to include it as part of their job or download it?

Job users can't download or add anything in the secure images except for the tmp folders.

[antoniivanov]

not sure I understand this change, if users need a particular certificate, is it not their responsibility to include it as part of their job or download it?

Job users can't download or add anything in the secure images except for the tmp folders.

That's not 100% true.

A) They can download it to temp directory and then point REQUESTS_CA_BUNDLE (if they use requests library) or SSL_CERT_DIR (for openssl) to that location. Now they need to do that every execution and it's not very nice and good user experience.

B) user should install certifi in requrements.txt (https://pypi.org/project/certifi/) first. It provides Mozilla’s carefully curated collection of Root Certificates that is more extensive that the one that comes with default python version. It should be tried before resorting to adding more custom certificates.

C) Similarly to certifi if user can install aws-certifi (there is not such but could be created).

I would definitely want to know if certifi has been tried and works before proceeding with this change.

[mivanov1988]

not sure I understand this change, if users need a particular certificate, is it not their responsibility to include it as part of their job or download it?

Job users can't download or add anything in the secure images except for the tmp folders.

That's not 100% true.

A) They can download it to temp directory and then point REQUESTS_CA_BUNDLE (if they use requests library) or SSL_CERT_DIR (for openssl) to that location. Now they need to do that every execution and it's not very nice and good user experience.

B) user should install certifi in requrements.txt (https://pypi.org/project/certifi/) first. It provides Mozilla’s carefully curated collection of Root Certificates that is more extensive that the one that comes with default python version. It should be tried before resorting to adding more custom certificates.

C) Similarly to certifi if user can install aws-certifi (there is not such but could be created).

I would definitely want to know if certifi has been tried and works before proceeding with this change.

In order to speed up the process can you assist us with this matter? A customer is currently facing a blockage related to it.

[antoniivanov]

In order to speed up the process can you assist us with this matter? A customer is currently facing a blockage related to it.

I've approved it. But advice the user the try adding certifi to see if it would change anything.

[mrMoZ1]

In order to speed up the process can you assist us with this matter? A customer is currently facing a blockage related to it.

I've approved it. But advice the user the try adding certifi to see if it would change anything.

The customer tried a few things to get this to work - one of them was installing the certifi library.

Here is what the customer tried on their own and it didn't work:

I can’t seem to get this working. The file system is read only and certificates are not an allowed file type that I can add to the data job. Even saving the certificate as a .txt the file gets recognized as a cert.

I even tried adding the certificate as a variable in python and creating an in memory file and it wouldn’t work.

Either we need to allow certificate files or I need these in the container already.