Go, CRUD, AWS

Go, CRUD, AWS is a project I am working on to get a better understanding of Go(Golang), CRUD functionality, and HTTP while hosting it all on AWS. The system works as a storage system for a user's game results in Apex Legends, Fortnite, and Heroes of the Storm. The user can retrieve their uploaded results and produce a table for each specific game.

Design

Design 1

Design 2

Design 3

Current Design:

System is hosted on AWS with route 53, 3 ALBs, 3 ECS clusters, 2 API gateways, various Lambda functions, RDS, & DynamoDB tables.

Web Gateway

• AWS API Gateway
• Proxies all requests to web specific Lambda functions

Web Fargate/Functions

• Handles all User/Client interaction
• Interacts with App gateway
• Runs on Go
• Stores all HTML files (CSS & images served from an S3 bucket)
• Manages cookie
• Web fargate can failover to App gateway if App or CRUD fargate is not healthy

App Gateway

• AWS API Gateway
• Proxies all requests to appropriate App/CRUD Lambda function

App Fargate/Functions

• Runs on Go
• Interacts with Credentials DB
• Only handles account specific requests

CRUD Fargate/Functions

• Runs on Go
• Interacts with Results DB
• Only handles results specific requests
• Independent function for each part of C.R.U.D.

Credentials DB

• Database for account credentials
• Runs on PostgreSQL

Results DB

• Database for game results
• Individual table for each game

Key Packages Used

• AWS - Session, DynamoDB, DynamoDB Attribute: Used for all AWS interactions/API calls
• Creds: Package created to store DB credentials and generate them when needed
• Game: Package created to contain structs for different games and a generator for each
• JSON & Bytes: For passing data between servers
• Mux Router & HTTP: Powered all HTTP interactions & routing on servers
• Template: Used to serve all Web server content

TODO

• Encrypt Passwords for data store

• Used bcrypt to salt and hash passwords for storage in DB
• Is not encrypting the password, but solves the issue of storing passwords in plain text

• Implement a cookie for smoother results viewing

• Added a cookie that is created when user signs in that expires after 24 hours
• Cookie required to access pages past sign-in/creation
• Streamlined process for viewing game results by leveraging a cookie
• Now skips a webpage that requests user's username to retrieve results
• Ensures user can only see their own results

• Setup fail-over *Used with Current and Design 2

• To be done at the server level by the go applications
• Server will check if primary path server is up; if it is not, it will send request to the fail-over API Gateway backed by Lambda
• If CRUD Server is down, requests will still go to App Server prior to fail-over gateway
• If App Server is down, all requests will go to fail-over gateway and will not reach CRUD server even if it is up

• Setup RR scheme & auto scaling policy for Web server/gateway

• Used a weighted routing policy to deliver content between an API gateway and an ALB infront of the web ECS cluster
• Health check is setup to ensure web server is healthy before requests are sent there
• Weight for web server is 4 and API is 1
• Have autoscaling disabled for servers behind ALBs, but have it configured so it can be used if necessary

• Error handling & edge cases
• Incorporate more games

Acknowledgements

Some resources that I found very helpful:

• Requests/JSON/Forms
• Go on AWS
• HTML Parsing
• Generate HTML content
• AWS Lambda Go Api Proxy
• go.rice - embedding HTML files
• golang - Docker Hub - to containerize server contents