Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

initial pass at adding tls support to mysql #82

Merged
merged 8 commits into from
May 19, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 21 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,20 +23,29 @@ It currently supports **MySQL**, **PostgreSQL** and **MSSQL**.
with a Crossplane provider (e.g. a [CloudSQLInstance] with provider-gcp) or you can
create for an existing server as follows:

```
kubectl create secret generic db-conn \
--from-literal=username=admin \
--from-literal=password='t0ps3cr3t' \
--from-literal=endpoint=my.sql-server.com \
--from-literal=port=3306
```
```
kubectl create secret generic db-conn \
--from-literal=username=admin \
--from-literal=password='t0ps3cr3t' \
--from-literal=endpoint=my.sql-server.com \
--from-literal=port=3306
```

2. Create managed resource for your SQL server flavor:

- **MySQL**: `Database`, `Grant`, `User` (See [the examples](examples/mysql))
- **PostgreSQL**: `Database`, `Grant`, `Extension`, `Role` (See [the examples](examples/postgresql))
- **MSSQL**: `Database`, `Grant`, `User` (See [the examples](examples/mssql))
- **MySQL**: `Database`, `Grant`, `User` (See [the examples](examples/mysql))
- **PostgreSQL**: `Database`, `Grant`, `Extension`, `Role` (See [the examples](examples/postgresql))
- **MSSQL**: `Database`, `Grant`, `User` (See [the examples](examples/mssql))

[Crossplane]: https://crossplane.io
[CloudSQLInstance]: https://doc.crds.dev/github.com/crossplane/provider-gcp/database.gcp.crossplane.io/CloudSQLInstance/[email protected]
[crossplane]: https://crossplane.io
[cloudsqlinstance]: https://doc.crds.dev/github.com/crossplane/provider-gcp/database.gcp.crossplane.io/CloudSQLInstance/[email protected]
[created automatically]: https://crossplane.io/docs/v1.5/concepts/managed-resources.html#connection-details

## Contributing

1. Fork the project and clone locally.
2. Create a branch with the changes.
3. Install go version 1.17.
4. Run `make` to initialize the "build". Make submodules used for CI/CD.
5. Run `make reviewable` to run code generation, linters, and tests.
6. Commit, push, and PR.
8 changes: 8 additions & 0 deletions apis/mysql/v1alpha1/provider_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,14 @@ import (
type ProviderConfigSpec struct {
// Credentials required to authenticate to this provider.
Credentials ProviderCredentials `json:"credentials"`
// tls=true enables TLS / SSL encrypted connection to the server.
// Use skip-verify if you want to use a self-signed or invalid certificate (server side)
// or use preferred to use TLS only when advertised by the server. This is similar
// to skip-verify, but additionally allows a fallback to a connection which is
// not encrypted. Neither skip-verify nor preferred add any reliable security.
// +kubebuilder:validation:Enum=true;skip-verify;preferred
// +optional
TLS *string `json:"tls"`
}

const (
Expand Down
5 changes: 5 additions & 0 deletions apis/mysql/v1alpha1/zz_generated.deepcopy.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion build
Submodule build updated 1 files
+2 −2 makelib/k8s_tools.mk
4 changes: 3 additions & 1 deletion cluster/local/integration_tests.sh
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@ echo "created cache dir at ${CACHE_PATH}"
docker tag "${BUILD_IMAGE}" "${PACKAGE_IMAGE}"
"${UP}" xpkg xp-extract --from-daemon "${PACKAGE_IMAGE}" -o "${CACHE_PATH}/${PACKAGE_NAME}.gz" && chmod 644 "${CACHE_PATH}/${PACKAGE_NAME}.gz"


# create kind cluster with extra mounts
echo_step "creating k8s cluster using kind"
KIND_CONFIG="$( cat <<EOF
Expand Down Expand Up @@ -171,7 +172,8 @@ EOF
echo "${INSTALL_YAML}" | "${KUBECTL}" apply -f -

echo_step "waiting for provider to be installed"
kubectl wait "provider.pkg.crossplane.io/${PACKAGE_NAME}" --for=condition=healthy --timeout=60s
"${KUBECTL}" wait "provider.pkg.crossplane.io/${PACKAGE_NAME}" --for=condition=healthy --timeout=60s


# printing the cache dir contents can be useful for troubleshooting failures
echo_step "check kind node cache dir contents"
Expand Down
2 changes: 2 additions & 0 deletions examples/mysql/config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,5 @@ spec:
connectionSecretRef:
namespace: default
name: db-conn
# tls one of preferred(default), skip-verify, or true
tls: preferred
7 changes: 7 additions & 0 deletions package/crds/mysql.sql.crossplane.io_providerconfigs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,13 @@ spec:
required:
- source
type: object
tls:
description: tls=true enables TLS / SSL encrypted connection to the server. Use skip-verify if you want to use a self-signed or invalid certificate (server side) or use preferred to use TLS only when advertised by the server. This is similar to skip-verify, but additionally allows a fallback to a connection which is not encrypted. Neither skip-verify nor preferred add any reliable security.
enum:
- true
- skip-verify
- preferred
type: string
required:
- credentials
type: object
Expand Down
32 changes: 26 additions & 6 deletions pkg/clients/mysql/mysql.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import (
"context"
"database/sql"
"fmt"
"net/url"
"strings"

"github.com/crossplane-contrib/provider-sql/pkg/clients/xsql"
Expand All @@ -21,24 +22,43 @@ type mySQLDB struct {
dsn string
endpoint string
port string
tls string
}

// New returns a new MySQL database client.
func New(creds map[string][]byte) xsql.DB {
func New(creds map[string][]byte, tls *string) xsql.DB {
// TODO(negz): Support alternative connection secret formats?
endpoint := string(creds[xpv1.ResourceCredentialsSecretEndpointKey])
port := string(creds[xpv1.ResourceCredentialsSecretPortKey])
username := string(creds[xpv1.ResourceCredentialsSecretUserKey])
password := string(creds[xpv1.ResourceCredentialsSecretPasswordKey])
if tls == nil {
defaultTLS := "preferred"
tls = &defaultTLS
}
dsn := DSN(username, password, endpoint, port, *tls)

return mySQLDB{
dsn: fmt.Sprintf("%s:%s@tcp(%s:%s)/",
creds[xpv1.ResourceCredentialsSecretUserKey],
creds[xpv1.ResourceCredentialsSecretPasswordKey],
endpoint,
port),
dsn: dsn,
endpoint: endpoint,
port: port,
tls: *tls,
}
}

// DSN returns the DSN URL
func DSN(username, password, endpoint, port, tls string) string {
MattMencel marked this conversation as resolved.
Show resolved Hide resolved
// Use net/url UserPassword to encode the username and password
// This will ensure that any special characters in the username or password
// are percent-encoded for use in the user info portion of the DSN URL
userInfo := url.UserPassword(username, password)
return fmt.Sprintf("%s@tcp(%s:%s)/?tls=%s",
userInfo,
endpoint,
port,
tls)
}

// ExecTx is unsupported in MySQL.
func (c mySQLDB) ExecTx(ctx context.Context, ql []xsql.Query) error {
return errors.Errorf(errNotSupported, "transactions")
Expand Down
24 changes: 24 additions & 0 deletions pkg/clients/mysql/mysql_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
package mysql

import (
"fmt"
"testing"
)

func TestDSNURLEscaping(t *testing.T) {
endpoint := "endpoint"
port := "3306"
user := "username"
rawPass := "password^"
encPass := "password%5E"
tls := "true"
dsn := DSN(user, rawPass, endpoint, port, tls)
if dsn != fmt.Sprintf("%s:%s@tcp(%s:%s)/?tls=%s",
user,
encPass,
endpoint,
port,
tls) {
t.Errorf("DSN string did not match expected output with URL encoded")
}
}
4 changes: 2 additions & 2 deletions pkg/controller/mysql/database/reconciler.go
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ func Setup(mgr ctrl.Manager, l logging.Logger) error {
type connector struct {
kube client.Client
usage resource.Tracker
newDB func(creds map[string][]byte) xsql.DB
newDB func(creds map[string][]byte, tls *string) xsql.DB
}

func (c *connector) Connect(ctx context.Context, mg resource.Managed) (managed.ExternalClient, error) {
Expand Down Expand Up @@ -110,7 +110,7 @@ func (c *connector) Connect(ctx context.Context, mg resource.Managed) (managed.E
return nil, errors.Wrap(err, errGetSecret)
}

return &external{db: c.newDB(s.Data)}, nil
return &external{db: c.newDB(s.Data, pc.Spec.TLS)}, nil
}

type external struct{ db xsql.DB }
Expand Down
2 changes: 1 addition & 1 deletion pkg/controller/mysql/database/reconciler_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ func TestConnect(t *testing.T) {
type fields struct {
kube client.Client
usage resource.Tracker
newDB func(creds map[string][]byte) xsql.DB
newDB func(creds map[string][]byte, tls *string) xsql.DB
}

type args struct {
Expand Down
4 changes: 2 additions & 2 deletions pkg/controller/mysql/grant/reconciler.go
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ func Setup(mgr ctrl.Manager, l logging.Logger) error {
type connector struct {
kube client.Client
usage resource.Tracker
newDB func(creds map[string][]byte) xsql.DB
newDB func(creds map[string][]byte, tls *string) xsql.DB
}

func (c *connector) Connect(ctx context.Context, mg resource.Managed) (managed.ExternalClient, error) {
Expand Down Expand Up @@ -123,7 +123,7 @@ func (c *connector) Connect(ctx context.Context, mg resource.Managed) (managed.E
}

return &external{
db: c.newDB(s.Data),
db: c.newDB(s.Data, pc.Spec.TLS),
kube: c.kube,
}, nil
}
Expand Down
2 changes: 1 addition & 1 deletion pkg/controller/mysql/grant/reconciler_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ func TestConnect(t *testing.T) {
type fields struct {
kube client.Client
usage resource.Tracker
newDB func(creds map[string][]byte) xsql.DB
newDB func(creds map[string][]byte, tls *string) xsql.DB
}

type args struct {
Expand Down
4 changes: 2 additions & 2 deletions pkg/controller/mysql/user/reconciler.go
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ func Setup(mgr ctrl.Manager, l logging.Logger) error {
type connector struct {
kube client.Client
usage resource.Tracker
newDB func(creds map[string][]byte) xsql.DB
newDB func(creds map[string][]byte, tls *string) xsql.DB
}

func (c *connector) Connect(ctx context.Context, mg resource.Managed) (managed.ExternalClient, error) {
Expand Down Expand Up @@ -118,7 +118,7 @@ func (c *connector) Connect(ctx context.Context, mg resource.Managed) (managed.E
}

return &external{
db: c.newDB(s.Data),
db: c.newDB(s.Data, pc.Spec.TLS),
kube: c.kube,
}, nil
}
Expand Down
2 changes: 1 addition & 1 deletion pkg/controller/mysql/user/reconciler_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ func TestConnect(t *testing.T) {
type fields struct {
kube client.Client
usage resource.Tracker
newDB func(creds map[string][]byte) xsql.DB
newDB func(creds map[string][]byte, tls *string) xsql.DB
}

type args struct {
Expand Down