Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

control-service: refactor service user doc #2436

Merged
merged 32 commits into from
Nov 20, 2023
Merged
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
35b0117
control-service: refactor service user doc
Jul 20, 2023
5eaaf1c
control-service: comments
Jul 25, 2023
239ad92
Merge main into person/mzhivkov/update-ecr-doc
github-actions[bot] Nov 13, 2023
78ddb1b
Merge main into person/mzhivkov/update-ecr-doc
github-actions[bot] Nov 13, 2023
7909beb
Merge main into person/mzhivkov/update-ecr-doc
github-actions[bot] Nov 13, 2023
a48ba0e
Merge main into person/mzhivkov/update-ecr-doc
github-actions[bot] Nov 13, 2023
2d345ba
Merge main into person/mzhivkov/update-ecr-doc
github-actions[bot] Nov 13, 2023
1776d38
Merge main into person/mzhivkov/update-ecr-doc
github-actions[bot] Nov 13, 2023
6f97fb9
Merge main into person/mzhivkov/update-ecr-doc
github-actions[bot] Nov 13, 2023
c742713
Merge main into person/mzhivkov/update-ecr-doc
github-actions[bot] Nov 13, 2023
5120e4e
Merge main into person/mzhivkov/update-ecr-doc
github-actions[bot] Nov 14, 2023
631499c
Merge main into person/mzhivkov/update-ecr-doc
github-actions[bot] Nov 14, 2023
9575301
Merge main into person/mzhivkov/update-ecr-doc
github-actions[bot] Nov 14, 2023
5d8e3d8
Merge main into person/mzhivkov/update-ecr-doc
github-actions[bot] Nov 14, 2023
20341bd
Merge main into person/mzhivkov/update-ecr-doc
github-actions[bot] Nov 15, 2023
abf0299
Merge main into person/mzhivkov/update-ecr-doc
github-actions[bot] Nov 15, 2023
dd7f3ae
Merge main into person/mzhivkov/update-ecr-doc
github-actions[bot] Nov 15, 2023
9ab3f2d
Merge main into person/mzhivkov/update-ecr-doc
github-actions[bot] Nov 15, 2023
571cefc
Merge main into person/mzhivkov/update-ecr-doc
github-actions[bot] Nov 16, 2023
f51b6cd
Merge main into person/mzhivkov/update-ecr-doc
github-actions[bot] Nov 16, 2023
ce96f78
Merge main into person/mzhivkov/update-ecr-doc
github-actions[bot] Nov 16, 2023
7c9ac9e
Merge main into person/mzhivkov/update-ecr-doc
github-actions[bot] Nov 16, 2023
fd7840c
Merge main into person/mzhivkov/update-ecr-doc
github-actions[bot] Nov 17, 2023
cd4ac82
Merge main into person/mzhivkov/update-ecr-doc
github-actions[bot] Nov 17, 2023
26c513f
Merge main into person/mzhivkov/update-ecr-doc
github-actions[bot] Nov 17, 2023
0cb7ad9
Merge main into person/mzhivkov/update-ecr-doc
github-actions[bot] Nov 17, 2023
eeaf535
Merge main into person/mzhivkov/update-ecr-doc
github-actions[bot] Nov 18, 2023
fabbb5d
Merge main into person/mzhivkov/update-ecr-doc
github-actions[bot] Nov 19, 2023
2d8182b
Merge main into person/mzhivkov/update-ecr-doc
github-actions[bot] Nov 20, 2023
b59c162
Merge main into person/mzhivkov/update-ecr-doc
github-actions[bot] Nov 20, 2023
50fd9b4
Merge main into person/mzhivkov/update-ecr-doc
github-actions[bot] Nov 20, 2023
6fabc92
comments
Nov 20, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -286,18 +286,35 @@ datajobs.aws.secretAccessKey=${AWS_ACCESS_KEY_SECRET}

# Variables for using the Service Account pattern. Must be set if the data job builder is to use
# an AWS ECR repository accessed by an IAM user instead of long term credentials.
# The AWS Service Account pattern is a design pattern used to manage access to AWS resources.
# The pattern involves creating an AWS Identity and Access Management (IAM)
# user account with a unique access key and secret key. The access key and secret key are then
# stored in a secure location, such as an encrypted file, on the EC2 instance.
# user account with a unique access key and secret key.
# The application running on the EC2 instance can then use the access key and secret key
# to authenticate with AWS services and make API calls on behalf of the IAM user account.
# By using a separate IAM user account for each application, the principle of least privilege is
# enforced and the attack surface is reduced.
# By using a separate IAM user, the principle of least privilege is enforced.
# https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html
# These need to be stored in different variables than the AWS credentials - datajobs.aws.accessKeyId
# and secretAccessKey used by the control service, because if the K8S cluster is hosted on a EKS
# instance, some API calls to the instance made by the control-service could fail.
# The service user variables are different than the AWS credentials - datajobs.aws.accessKeyId
# and secretAccessKey used by the control service.
#
# Necessary steps to setup Service Account:
# - Create a Service Account in Amazon (regular account that will be granted permissions to ECR repo) e.g:
# https://docs.aws.amazon.com/IAM/latest/UserGuide/id_users_create.html#id_users_create_console
# e.g - IAM user: svc.example-service-user
# - Create a role in your AWS account (account where your ECR repository lives) - https://docs.aws.amazon.com/cli/latest/reference/iam/create-role.html
# e.g - IAMRole: svc.example-service-user, the role must have the same name as your service user.
# More information on what the example-service-user document should include can be found in the above link
# which describes the process of creating a role.
# - Create a trust relationship on the role to the control account - edit your newly created role's Trust Relationship:
# This is the trust relationship in your main account which is associated with the service user. Or
# the role ARN - which is used to delegate access to resources securely.
# More information on how to create and use role ARN can be found in the below mentioned documents.
# https://aws.amazon.com/blogs/security/how-to-use-external-id-when-granting-access-to-your-aws-resources/
# https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_elements_condition.html.
# - Verify access to the linked account:
# aws sts get-caller-identity with the Service Account credentials - this is the account for which
# a role was created in the main account (where the Elastic Container Repository is stored)
# Any AWS account can be created for the purpose of being a service account.
# If everything is configured correctly you need to fill in the below values and ECR repository
# access through the control-service (when deploying data job images) will go through the service account.
# datajobs.aws.assumeIAMRole tells the control-service if the Service Account pattern should be used.
datajobs.aws.assumeIAMRole=${DATAJOBS_AWS_ASSUME_IAM_ROLE:false}
# All flags must be set if the datajobs.aws.assumeIAMRole flag is true
Expand Down