Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Migrate Farallon hub to this repo #368

Closed
7 tasks done
yuvipanda opened this issue Apr 28, 2021 · 15 comments
Closed
7 tasks done

Migrate Farallon hub to this repo #368

yuvipanda opened this issue Apr 28, 2021 · 15 comments
Assignees
@damianavila
Labels
Task Actions that don't involve changing our code or docs.

Comments

@yuvipanda
Copy link
Member

yuvipanda commented Apr 28, 2021 •
edited
Loading

Background

The Farallon institute JupyterHub is run from github.com/2i2c-org/pangeo-hubs/, and is not well automated. We should move that to this repository!

#366 tackles fully setting up on AWS, so this goal will focus only on migrating the helm config + deployment to this repository.

Expected timeline

???

Steps to complete this goal
@yuvipanda yuvipanda mentioned this issue Apr 28, 2021
Closed
6 tasks
@damianavila damianavila self-assigned this Apr 30, 2021
@choldgraf
Copy link
Member

Just a quick note on this - I think we should prioritize this below #363 - so I'm attaching a label to mark this as such :-)

@yuvipanda yuvipanda mentioned this issue May 3, 2021
Merged
4 tasks
@choldgraf choldgraf added Task Actions that don't involve changing our code or docs. and removed type: goal labels May 4, 2021
@yuvipanda yuvipanda mentioned this issue May 6, 2021
Merged
@yuvipanda yuvipanda mentioned this issue May 11, 2021
Closed
@yuvipanda
Copy link
Member Author

Once #391 is merged, @damianavila is going to go through moving the farallon hub over to the new kops based setup. That will help complete this.

@yuvipanda yuvipanda mentioned this issue May 14, 2021
Merged
@damianavila
Copy link
Contributor

A little reminder to re-revert #400 once the above is complete.

@yuvipanda
Copy link
Member Author

We should archive https://github.com/2i2c-org/pangeo-hubs once this is complete

This was referenced May 15, 2021
Merged
Closed
This was referenced Jun 22, 2021
Closed
Merged
Closed
yuvipanda added a commit to yuvipanda/pilot-hubs that referenced this issue Jun 28, 2021
@yuvipanda
farallon: List authenticated users
82ef571 
In the older setup
(https://github.com/2i2c-org/pangeo-hubs/blob/5115e56a60da49513891d362daf52933c7186698/deployments/farallon/config/common.yaml#L13),
we used a GitHub org to control who can login. Our auth0 support
does not support GitHub org based access control yet - so
they are listed here.

For now, new users will need to be added via the JupyterHub UI:
https://pilot.2i2c.org/en/latest/admin/howto/manage-users.html

Ref 2i2c-org#368
@yuvipanda yuvipanda mentioned this issue Jun 28, 2021
Merged
@yuvipanda
Copy link
Member Author

I've carefully deleted the old EFS instance. I'll now carefully delete the old cluster

@yuvipanda
Copy link
Member Author

I've deleted the staging and prod hubs by hand:

helm -n farallon-staging delete farallon-staging
helm -n farallon-prod delete farallon-prod

This ensures that any external cloud resources created by kubernetes (disks for the hub db dir, load balancer for the proxy-public service) are deleted

@yuvipanda
Copy link
Member Author

I've enabled EFS Lifecycles on the new EFS setup too (https://docs.aws.amazon.com/efs/latest/ug/lifecycle-management-efs.html) - this moves infrequently accessed files to slightly slower storage for significant cost benefits. The previous EFS had this enabled

@yuvipanda
Copy link
Member Author

I deleted the old cluster with:

export KOPS_STATE_STORE=s3://2i2c-farallon-pangeo-kops
kops delete cluster farallon-2i2c.k8s.local --yes

@damianavila
Copy link
Contributor

This ensures that any external cloud resources created by kubernetes (disks for the hub db dir, load balancer for the proxy-public service) are deleted

I have seen this issue as well with CloudFormation stuff, particularly with the load balancer.
We should document this practice, @yuvipanda, WDYT?

@yuvipanda
Copy link
Member Author

@damianavila yeah! Perhaps documented as part of decomissioning a hub / cluster?

yuvipanda added a commit to 2i2c-org/farallon-image that referenced this issue Jul 5, 2021
@yuvipanda
Remove everything except the image
0ec098a 
farallon hub is now run out of https://github.com/2i2c-org/pilot-hubs/,
and we can just set up the image via this.

Ref 2i2c-org/infrastructure#368
@yuvipanda
Copy link
Member Author

https://github.com/2i2c-org/pangeo-hubs is now just the user image

@damianavila
Copy link
Contributor

For the record, the repo was renamed as well as farallon-image

@GeorgianaElena
Copy link
Member

Once #391 is merged, @damianavila is going to go through moving the farallon hub over to the new kops based setup. That will help complete this.

A little reminder to re-revert #400 once the above is complete.

I was wondering where are we with this and if it's ok to re-revevert #400?
Ref: #507

@damianavila
Copy link
Contributor

@GeorgianaElena, I think we need to solve #381 to close this one and before revert #400.

@yuvipanda
Copy link
Member Author

I'm going to close this, as the hub itself is migrated. #381 tracks the general problem that is preventing CD from happening longer term

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@damianavila damianavila

Labels
Task Actions that don't involve changing our code or docs.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@yuvipanda @damianavila @choldgraf @GeorgianaElena