Skip to content
This repository has been archived by the owner on Jul 6, 2022. It is now read-only.

Commit

Permalink
Parent child services (#243)
Browse files Browse the repository at this point in the history
* WIP Refactoring sqldb to POC (#142)

* Refactoring sqldb to POC #124

Removing the "existing server" logic from the current sqldb
module. This entails removing it from the provision/deprovision
file and removing it from the types. Also removed the config.go
file and updated the broker to not load that. We will need another
way to provide/register existing servers.

We will reimplement this behavior in a new service/service manager.

* fix minor lint failure

* Feature branch merge (#154)

* refactor event function signatures (#147)

* pass around instances and bindings instead of pointers

* Refactor instance encryption/decryption

Previously, encryption and decryption of certain instance
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to instance
objects, I wanted a way for modules to be able to extract
information from encrypted instance fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors instances and the storage layer so that
encryption occurs during instance marshaling and decryption occurs
during instance unmarshaling.

* Refactor binding encryption/decryption

Previously, encryption and decryption of certain binding
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to binding
objects, I wanted a way for modules to be able to extract
information from encrypted binding fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors bindings and the storage layer so that
encryption occurs during bindings marshaling and decryption occurs
during bindings unmarshaling.

* clean up a few errant lint errors

* pass instance to provisioning steps

* pass instance to deprovisioning steps

* make bind and unbind accept instances as arguments

* get defensive about possible unanticipated modifications to instances by module code

* pass instance to updating steps

* minor lint fixes

* refactor lifecycle tests

* Firewall rule parameterization (#152)

* Refactor to support allowing firewall rules.

Fixes: #146

Changes to suport optional parameters to allow specifying start
and end IP address for firewall rules on MySQL, PostgreSQL and
MSSQL. Previously, this defaulted to allow any connection. This
isn't ecure in practice, so allowing the customer to specify desired
rules. Also changed the default to 0.0.0.0 for both, which results in
only Azure internal addresses. Added some parameter validation
for the firewall values as well:

* Valid IPV4 addresses
* Start <= End

Extracted a method to build the arm template parameter map. Providing
empty strings is invalid for the ARM template, so only including
when non-empty.

Added parameters to the lifecycle tests to allow running them
from local machines (will need ao enhance the charts in helm-charts)

Added some tests.

* Fixing merge conflicts

* Test cleanup

* json field cleanup

* Resolving code review comments

* error message cleanup

* error message cleanup

* specify firewall rules for relational databased in example manifests

* Sqldb catalog poc merge master (#158)

* refactor event function signatures (#147)

* pass around instances and bindings instead of pointers

* Refactor instance encryption/decryption

Previously, encryption and decryption of certain instance
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to instance
objects, I wanted a way for modules to be able to extract
information from encrypted instance fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors instances and the storage layer so that
encryption occurs during instance marshaling and decryption occurs
during instance unmarshaling.

* Refactor binding encryption/decryption

Previously, encryption and decryption of certain binding
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to binding
objects, I wanted a way for modules to be able to extract
information from encrypted binding fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors bindings and the storage layer so that
encryption occurs during bindings marshaling and decryption occurs
during bindings unmarshaling.

* clean up a few errant lint errors

* pass instance to provisioning steps

* pass instance to deprovisioning steps

* make bind and unbind accept instances as arguments

* get defensive about possible unanticipated modifications to instances by module code

* pass instance to updating steps

* minor lint fixes

* refactor lifecycle tests

* Firewall rule parameterization (#152)

* Refactor to support allowing firewall rules.

Fixes: #146

Changes to suport optional parameters to allow specifying start
and end IP address for firewall rules on MySQL, PostgreSQL and
MSSQL. Previously, this defaulted to allow any connection. This
isn't ecure in practice, so allowing the customer to specify desired
rules. Also changed the default to 0.0.0.0 for both, which results in
only Azure internal addresses. Added some parameter validation
for the firewall values as well:

* Valid IPV4 addresses
* Start <= End

Extracted a method to build the arm template parameter map. Providing
empty strings is invalid for the ARM template, so only including
when non-empty.

Added parameters to the lifecycle tests to allow running them
from local machines (will need ao enhance the charts in helm-charts)

Added some tests.

* Fixing merge conflicts

* Test cleanup

* json field cleanup

* Resolving code review comments

* error message cleanup

* error message cleanup

* specify firewall rules for relational databased in example manifests

* Few minor corrections to the Quickstart (#136)

* Correcting the az ad sp delete command

* Update quickstart-minikube.md

Remove a leading $ from a few bash commands. copy paste on mac is messed up :-)

* Update quickstart-minikube.md

Clarification around logging in to WordPress

* restoring prompts in quickstart

* minor punctuation fix

* removing distracting echo command

* make prompts used consistently

* kick of redis tests sooner (#155)

* s/module lifecycle tests/service lifecycle tests/g (#156)

* make storage know how to deal with service-specific types (#153)

* make provision and deprovision steps receive a reference instance (#160)

* Merge master to feature (#175)

* refactor event function signatures (#147)

* pass around instances and bindings instead of pointers

* Refactor instance encryption/decryption

Previously, encryption and decryption of certain instance
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to instance
objects, I wanted a way for modules to be able to extract
information from encrypted instance fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors instances and the storage layer so that
encryption occurs during instance marshaling and decryption occurs
during instance unmarshaling.

* Refactor binding encryption/decryption

Previously, encryption and decryption of certain binding
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to binding
objects, I wanted a way for modules to be able to extract
information from encrypted binding fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors bindings and the storage layer so that
encryption occurs during bindings marshaling and decryption occurs
during bindings unmarshaling.

* clean up a few errant lint errors

* pass instance to provisioning steps

* pass instance to deprovisioning steps

* make bind and unbind accept instances as arguments

* get defensive about possible unanticipated modifications to instances by module code

* pass instance to updating steps

* minor lint fixes

* refactor lifecycle tests

* Firewall rule parameterization (#152)

* Refactor to support allowing firewall rules.

Fixes: #146

Changes to suport optional parameters to allow specifying start
and end IP address for firewall rules on MySQL, PostgreSQL and
MSSQL. Previously, this defaulted to allow any connection. This
isn't ecure in practice, so allowing the customer to specify desired
rules. Also changed the default to 0.0.0.0 for both, which results in
only Azure internal addresses. Added some parameter validation
for the firewall values as well:

* Valid IPV4 addresses
* Start <= End

Extracted a method to build the arm template parameter map. Providing
empty strings is invalid for the ARM template, so only including
when non-empty.

Added parameters to the lifecycle tests to allow running them
from local machines (will need ao enhance the charts in helm-charts)

Added some tests.

* Fixing merge conflicts

* Test cleanup

* json field cleanup

* Resolving code review comments

* error message cleanup

* error message cleanup

* specify firewall rules for relational databased in example manifests

* Few minor corrections to the Quickstart (#136)

* Correcting the az ad sp delete command

* Update quickstart-minikube.md

Remove a leading $ from a few bash commands. copy paste on mac is messed up :-)

* Update quickstart-minikube.md

Clarification around logging in to WordPress

* restoring prompts in quickstart

* minor punctuation fix

* removing distracting echo command

* make prompts used consistently

* kick of redis tests sooner (#155)

* s/module lifecycle tests/service lifecycle tests/g (#156)

* make storage know how to deal with service-specific types (#153)

* don't use codec where we don't need it anymore (#161)

* Updating CircleCI to define DOCKER_REPO env var (#167)

* Updating CircleCI to define REGISTRY env var

We previously set the REGISTRY environment variable in a deploy script.
When that was removed to streamline the release process, we need to
define it in Circle. Added two environment blocks to the publish-rc-images
and publish-release-images jobs.

Fixes: 145

* Qualified registry with docker.io

* Changed REGISTRY to DOCKER_REPO

* s/context/details/g (#163)

* make fake bind/unbind functionality not rely on standard context

* collapse standard provisioning context onto instance

* do away with (sort of) redundant standard provisioning parameters

* s/provisioning context/instance details/

* s/binding context/binding details/

* fix error handling when standard params aren't expected types

* add comment that adds some clarity to instance comparisons

* don't store credentials as their own field (#169)

* quickstart fixes (#174)

* adding missing docker pushes to release process

* minor merge nit fixes

* Decomposition of single service into multiple services (#176)

* Decomposition of single service into multiple services

1.) This renames the existing service manager into a service to
    implement the all in one scenario (existing)
2.) Next, two new services are added (vm only and db only)
3.) Each lifecycle operation go file (bind, deprovision, provision, unbind)
    now declares a method for each service manager struct
4.) Added new services to the catalog
5.) DB only is largely TBD implemented, took a crack at
    starting the provision.
6.) Refactored catalog/service interface to add GetBindable() method
7.) Modified tests to only execute binding related tests if plan
    is bindable

* Small spelling correction

* Implemented database only lifecycle methods and lint fix.

* DRY refactoring

* Review comments and lint fix

* add ref instance as instance attribute (#178)

* create a relationship between two services (#179)

* Refactor sqldb module to use parent instances in binding (#186)

* Refactor sqldb module to use parent instances in binding

Small refactor to decouple shared instance details and leverage
parent instance for binding/unbinding.

Further evolution for #124

* Updated error to be consistent.

* merge latest from master (#187)

* refactor event function signatures (#147)

* pass around instances and bindings instead of pointers

* Refactor instance encryption/decryption

Previously, encryption and decryption of certain instance
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to instance
objects, I wanted a way for modules to be able to extract
information from encrypted instance fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors instances and the storage layer so that
encryption occurs during instance marshaling and decryption occurs
during instance unmarshaling.

* Refactor binding encryption/decryption

Previously, encryption and decryption of certain binding
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to binding
objects, I wanted a way for modules to be able to extract
information from encrypted binding fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors bindings and the storage layer so that
encryption occurs during bindings marshaling and decryption occurs
during bindings unmarshaling.

* clean up a few errant lint errors

* pass instance to provisioning steps

* pass instance to deprovisioning steps

* make bind and unbind accept instances as arguments

* get defensive about possible unanticipated modifications to instances by module code

* pass instance to updating steps

* minor lint fixes

* refactor lifecycle tests

* Firewall rule parameterization (#152)

* Refactor to support allowing firewall rules.

Fixes: #146

Changes to suport optional parameters to allow specifying start
and end IP address for firewall rules on MySQL, PostgreSQL and
MSSQL. Previously, this defaulted to allow any connection. This
isn't ecure in practice, so allowing the customer to specify desired
rules. Also changed the default to 0.0.0.0 for both, which results in
only Azure internal addresses. Added some parameter validation
for the firewall values as well:

* Valid IPV4 addresses
* Start <= End

Extracted a method to build the arm template parameter map. Providing
empty strings is invalid for the ARM template, so only including
when non-empty.

Added parameters to the lifecycle tests to allow running them
from local machines (will need ao enhance the charts in helm-charts)

Added some tests.

* Fixing merge conflicts

* Test cleanup

* json field cleanup

* Resolving code review comments

* error message cleanup

* error message cleanup

* specify firewall rules for relational databased in example manifests

* Few minor corrections to the Quickstart (#136)

* Correcting the az ad sp delete command

* Update quickstart-minikube.md

Remove a leading $ from a few bash commands. copy paste on mac is messed up :-)

* Update quickstart-minikube.md

Clarification around logging in to WordPress

* restoring prompts in quickstart

* minor punctuation fix

* removing distracting echo command

* make prompts used consistently

* kick of redis tests sooner (#155)

* s/module lifecycle tests/service lifecycle tests/g (#156)

* make storage know how to deal with service-specific types (#153)

* don't use codec where we don't need it anymore (#161)

* Updating CircleCI to define DOCKER_REPO env var (#167)

* Updating CircleCI to define REGISTRY env var

We previously set the REGISTRY environment variable in a deploy script.
When that was removed to streamline the release process, we need to
define it in Circle. Added two environment blocks to the publish-rc-images
and publish-release-images jobs.

Fixes: 145

* Qualified registry with docker.io

* Changed REGISTRY to DOCKER_REPO

* s/context/details/g (#163)

* make fake bind/unbind functionality not rely on standard context

* collapse standard provisioning context onto instance

* do away with (sort of) redundant standard provisioning parameters

* s/provisioning context/instance details/

* s/binding context/binding details/

* fix error handling when standard params aren't expected types

* add comment that adds some clarity to instance comparisons

* don't store credentials as their own field (#169)

* quickstart fixes (#174)

* adding missing docker pushes to release process

* Adding build flags to inject the version and commit (#173)

* Adding build flags to inject the version and commit

This commit injects the version and commit to the OSBA binary. If there
is no version set, then sets the version to “devel”

* Using new variable for version

* Removing redundant setting for main.commit

* Printing version and commit on broker startup

* Adding fields to the log message on startup

* Removing build flags on the CLI builds

* broker doesn't need to hang on to codec (#180)

* remove quotes from devel pseudo-version string (#184)

Merging without CI.

* fix bug decoding tags (#183)

* move version info to avoid import cycle later (#185)

* Adding user-agent string to requests to ARM (#172)

* Adding user-agent string to requests to ARM

* get version for user agent string from version package

* use separate redis clients for storage and async engine (#182)

* Adding ability to filter lifecycle tests (#181)

* Adding ability to filter lifecycle tests

This adds the ability to filter lifecycle tests using an environment
variable (TEST_MODULES). Via this variable, the developer can provide
a comma delimited list of module names and only the test cases matching
that module will be enabled. If the variable is empty, all tests will run.
If an invalid module name is provided, no tests will run.

* fix typo and lint error

* remove unnecessary funtion

* Merge master (#189)

* refactor event function signatures (#147)

* pass around instances and bindings instead of pointers

* Refactor instance encryption/decryption

Previously, encryption and decryption of certain instance
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to instance
objects, I wanted a way for modules to be able to extract
information from encrypted instance fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors instances and the storage layer so that
encryption occurs during instance marshaling and decryption occurs
during instance unmarshaling.

* Refactor binding encryption/decryption

Previously, encryption and decryption of certain binding
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to binding
objects, I wanted a way for modules to be able to extract
information from encrypted binding fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors bindings and the storage layer so that
encryption occurs during bindings marshaling and decryption occurs
during bindings unmarshaling.

* clean up a few errant lint errors

* pass instance to provisioning steps

* pass instance to deprovisioning steps

* make bind and unbind accept instances as arguments

* get defensive about possible unanticipated modifications to instances by module code

* pass instance to updating steps

* minor lint fixes

* refactor lifecycle tests

* Firewall rule parameterization (#152)

* Refactor to support allowing firewall rules.

Fixes: #146

Changes to suport optional parameters to allow specifying start
and end IP address for firewall rules on MySQL, PostgreSQL and
MSSQL. Previously, this defaulted to allow any connection. This
isn't ecure in practice, so allowing the customer to specify desired
rules. Also changed the default to 0.0.0.0 for both, which results in
only Azure internal addresses. Added some parameter validation
for the firewall values as well:

* Valid IPV4 addresses
* Start <= End

Extracted a method to build the arm template parameter map. Providing
empty strings is invalid for the ARM template, so only including
when non-empty.

Added parameters to the lifecycle tests to allow running them
from local machines (will need ao enhance the charts in helm-charts)

Added some tests.

* Fixing merge conflicts

* Test cleanup

* json field cleanup

* Resolving code review comments

* error message cleanup

* error message cleanup

* specify firewall rules for relational databased in example manifests

* Few minor corrections to the Quickstart (#136)

* Correcting the az ad sp delete command

* Update quickstart-minikube.md

Remove a leading $ from a few bash commands. copy paste on mac is messed up :-)

* Update quickstart-minikube.md

Clarification around logging in to WordPress

* restoring prompts in quickstart

* minor punctuation fix

* removing distracting echo command

* make prompts used consistently

* kick of redis tests sooner (#155)

* s/module lifecycle tests/service lifecycle tests/g (#156)

* make storage know how to deal with service-specific types (#153)

* don't use codec where we don't need it anymore (#161)

* Updating CircleCI to define DOCKER_REPO env var (#167)

* Updating CircleCI to define REGISTRY env var

We previously set the REGISTRY environment variable in a deploy script.
When that was removed to streamline the release process, we need to
define it in Circle. Added two environment blocks to the publish-rc-images
and publish-release-images jobs.

Fixes: 145

* Qualified registry with docker.io

* Changed REGISTRY to DOCKER_REPO

* s/context/details/g (#163)

* make fake bind/unbind functionality not rely on standard context

* collapse standard provisioning context onto instance

* do away with (sort of) redundant standard provisioning parameters

* s/provisioning context/instance details/

* s/binding context/binding details/

* fix error handling when standard params aren't expected types

* add comment that adds some clarity to instance comparisons

* don't store credentials as their own field (#169)

* quickstart fixes (#174)

* adding missing docker pushes to release process

* Adding build flags to inject the version and commit (#173)

* Adding build flags to inject the version and commit

This commit injects the version and commit to the OSBA binary. If there
is no version set, then sets the version to “devel”

* Using new variable for version

* Removing redundant setting for main.commit

* Printing version and commit on broker startup

* Adding fields to the log message on startup

* Removing build flags on the CLI builds

* broker doesn't need to hang on to codec (#180)

* remove quotes from devel pseudo-version string (#184)

Merging without CI.

* fix bug decoding tags (#183)

* move version info to avoid import cycle later (#185)

* Adding user-agent string to requests to ARM (#172)

* Adding user-agent string to requests to ARM

* get version for user agent string from version package

* use separate redis clients for storage and async engine (#182)

* Adding ability to filter lifecycle tests (#181)

* Adding ability to filter lifecycle tests

This adds the ability to filter lifecycle tests using an environment
variable (TEST_MODULES). Via this variable, the developer can provide
a comma delimited list of module names and only the test cases matching
that module will be enabled. If the variable is empty, all tests will run.
If an invalid module name is provided, no tests will run.

* fix typo and lint error

* remove unnecessary funtion

* use redis namespaces (#188)

* use redis namespaces for instances and bindings

* use redis namespaces in async engine

* validate alias and parentAlias (#191)

* Update lifecycle tests to run mssql db only (#192)

* Update lifecycle tests to run mssql db only

1.) Small refactor to test_case_test.go to break provision into a
reusable function
2.) Modified setup function signature to include a *service.Instance
    to use as a parent instance
3.) Added setup function in mssql tests
4.) Added lifecycle test for DB only
5.) Fixed db only privision bug

Implemnts #124

* Lint fixes

* test refactor to support "child tests"

* Small rename

* Test description update

* handle storage aspects of instance aliases (#195)

* merge latest from master (#199)

* refactor event function signatures (#147)

* pass around instances and bindings instead of pointers

* Refactor instance encryption/decryption

Previously, encryption and decryption of certain instance
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to instance
objects, I wanted a way for modules to be able to extract
information from encrypted instance fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors instances and the storage layer so that
encryption occurs during instance marshaling and decryption occurs
during instance unmarshaling.

* Refactor binding encryption/decryption

Previously, encryption and decryption of certain binding
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to binding
objects, I wanted a way for modules to be able to extract
information from encrypted binding fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors bindings and the storage layer so that
encryption occurs during bindings marshaling and decryption occurs
during bindings unmarshaling.

* clean up a few errant lint errors

* pass instance to provisioning steps

* pass instance to deprovisioning steps

* make bind and unbind accept instances as arguments

* get defensive about possible unanticipated modifications to instances by module code

* pass instance to updating steps

* minor lint fixes

* refactor lifecycle tests

* Firewall rule parameterization (#152)

* Refactor to support allowing firewall rules.

Fixes: #146

Changes to suport optional parameters to allow specifying start
and end IP address for firewall rules on MySQL, PostgreSQL and
MSSQL. Previously, this defaulted to allow any connection. This
isn't ecure in practice, so allowing the customer to specify desired
rules. Also changed the default to 0.0.0.0 for both, which results in
only Azure internal addresses. Added some parameter validation
for the firewall values as well:

* Valid IPV4 addresses
* Start <= End

Extracted a method to build the arm template parameter map. Providing
empty strings is invalid for the ARM template, so only including
when non-empty.

Added parameters to the lifecycle tests to allow running them
from local machines (will need ao enhance the charts in helm-charts)

Added some tests.

* Fixing merge conflicts

* Test cleanup

* json field cleanup

* Resolving code review comments

* error message cleanup

* error message cleanup

* specify firewall rules for relational databased in example manifests

* Few minor corrections to the Quickstart (#136)

* Correcting the az ad sp delete command

* Update quickstart-minikube.md

Remove a leading $ from a few bash commands. copy paste on mac is messed up :-)

* Update quickstart-minikube.md

Clarification around logging in to WordPress

* restoring prompts in quickstart

* minor punctuation fix

* removing distracting echo command

* make prompts used consistently

* kick of redis tests sooner (#155)

* s/module lifecycle tests/service lifecycle tests/g (#156)

* make storage know how to deal with service-specific types (#153)

* don't use codec where we don't need it anymore (#161)

* Updating CircleCI to define DOCKER_REPO env var (#167)

* Updating CircleCI to define REGISTRY env var

We previously set the REGISTRY environment variable in a deploy script.
When that was removed to streamline the release process, we need to
define it in Circle. Added two environment blocks to the publish-rc-images
and publish-release-images jobs.

Fixes: 145

* Qualified registry with docker.io

* Changed REGISTRY to DOCKER_REPO

* s/context/details/g (#163)

* make fake bind/unbind functionality not rely on standard context

* collapse standard provisioning context onto instance

* do away with (sort of) redundant standard provisioning parameters

* s/provisioning context/instance details/

* s/binding context/binding details/

* fix error handling when standard params aren't expected types

* add comment that adds some clarity to instance comparisons

* don't store credentials as their own field (#169)

* quickstart fixes (#174)

* adding missing docker pushes to release process

* Adding build flags to inject the version and commit (#173)

* Adding build flags to inject the version and commit

This commit injects the version and commit to the OSBA binary. If there
is no version set, then sets the version to “devel”

* Using new variable for version

* Removing redundant setting for main.commit

* Printing version and commit on broker startup

* Adding fields to the log message on startup

* Removing build flags on the CLI builds

* broker doesn't need to hang on to codec (#180)

* remove quotes from devel pseudo-version string (#184)

Merging without CI.

* fix bug decoding tags (#183)

* move version info to avoid import cycle later (#185)

* Adding user-agent string to requests to ARM (#172)

* Adding user-agent string to requests to ARM

* get version for user agent string from version package

* use separate redis clients for storage and async engine (#182)

* Adding ability to filter lifecycle tests (#181)

* Adding ability to filter lifecycle tests

This adds the ability to filter lifecycle tests using an environment
variable (TEST_MODULES). Via this variable, the developer can provide
a comma delimited list of module names and only the test cases matching
that module will be enabled. If the variable is empty, all tests will run.
If an invalid module name is provided, no tests will run.

* fix typo and lint error

* remove unnecessary funtion

* use redis namespaces (#188)

* use redis namespaces for instances and bindings

* use redis namespaces in async engine

* start using lightweight-docker-go image for development tasks (#190)

* simplify ci config (#194)

* populate parent when retrieving instance (#200)

* parent child count (#201)

* fix arg name

* break up storage test cases better

* track number of children belonging to an instance

* Merge master into feature branch (#209)

* refactor event function signatures (#147)

* pass around instances and bindings instead of pointers

* Refactor instance encryption/decryption

Previously, encryption and decryption of certain instance
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to instance
objects, I wanted a way for modules to be able to extract
information from encrypted instance fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors instances and the storage layer so that
encryption occurs during instance marshaling and decryption occurs
during instance unmarshaling.

* Refactor binding encryption/decryption

Previously, encryption and decryption of certain binding
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to binding
objects, I wanted a way for modules to be able to extract
information from encrypted binding fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors bindings and the storage layer so that
encryption occurs during bindings marshaling and decryption occurs
during bindings unmarshaling.

* clean up a few errant lint errors

* pass instance to provisioning steps

* pass instance to deprovisioning steps

* make bind and unbind accept instances as arguments

* get defensive about possible unanticipated modifications to instances by module code

* pass instance to updating steps

* minor lint fixes

* refactor lifecycle tests

* Firewall rule parameterization (#152)

* Refactor to support allowing firewall rules.

Fixes: #146

Changes to suport optional parameters to allow specifying start
and end IP address for firewall rules on MySQL, PostgreSQL and
MSSQL. Previously, this defaulted to allow any connection. This
isn't ecure in practice, so allowing the customer to specify desired
rules. Also changed the default to 0.0.0.0 for both, which results in
only Azure internal addresses. Added some parameter validation
for the firewall values as well:

* Valid IPV4 addresses
* Start <= End

Extracted a method to build the arm template parameter map. Providing
empty strings is invalid for the ARM template, so only including
when non-empty.

Added parameters to the lifecycle tests to allow running them
from local machines (will need ao enhance the charts in helm-charts)

Added some tests.

* Fixing merge conflicts

* Test cleanup

* json field cleanup

* Resolving code review comments

* error message cleanup

* error message cleanup

* specify firewall rules for relational databased in example manifests

* Few minor corrections to the Quickstart (#136)

* Correcting the az ad sp delete command

* Update quickstart-minikube.md

Remove a leading $ from a few bash commands. copy paste on mac is messed up :-)

* Update quickstart-minikube.md

Clarification around logging in to WordPress

* restoring prompts in quickstart

* minor punctuation fix

* removing distracting echo command

* make prompts used consistently

* kick of redis tests sooner (#155)

* s/module lifecycle tests/service lifecycle tests/g (#156)

* make storage know how to deal with service-specific types (#153)

* don't use codec where we don't need it anymore (#161)

* Updating CircleCI to define DOCKER_REPO env var (#167)

* Updating CircleCI to define REGISTRY env var

We previously set the REGISTRY environment variable in a deploy script.
When that was removed to streamline the release process, we need to
define it in Circle. Added two environment blocks to the publish-rc-images
and publish-release-images jobs.

Fixes: 145

* Qualified registry with docker.io

* Changed REGISTRY to DOCKER_REPO

* s/context/details/g (#163)

* make fake bind/unbind functionality not rely on standard context

* collapse standard provisioning context onto instance

* do away with (sort of) redundant standard provisioning parameters

* s/provisioning context/instance details/

* s/binding context/binding details/

* fix error handling when standard params aren't expected types

* add comment that adds some clarity to instance comparisons

* don't store credentials as their own field (#169)

* quickstart fixes (#174)

* adding missing docker pushes to release process

* Adding build flags to inject the version and commit (#173)

* Adding build flags to inject the version and commit

This commit injects the version and commit to the OSBA binary. If there
is no version set, then sets the version to “devel”

* Using new variable for version

* Removing redundant setting for main.commit

* Printing version and commit on broker startup

* Adding fields to the log message on startup

* Removing build flags on the CLI builds

* broker doesn't need to hang on to codec (#180)

* remove quotes from devel pseudo-version string (#184)

Merging without CI.

* fix bug decoding tags (#183)

* move version info to avoid import cycle later (#185)

* Adding user-agent string to requests to ARM (#172)

* Adding user-agent string to requests to ARM

* get version for user agent string from version package

* use separate redis clients for storage and async engine (#182)

* Adding ability to filter lifecycle tests (#181)

* Adding ability to filter lifecycle tests

This adds the ability to filter lifecycle tests using an environment
variable (TEST_MODULES). Via this variable, the developer can provide
a comma delimited list of module names and only the test cases matching
that module will be enabled. If the variable is empty, all tests will run.
If an invalid module name is provided, no tests will run.

* fix typo and lint error

* remove unnecessary funtion

* use redis namespaces (#188)

* use redis namespaces for instances and bindings

* use redis namespaces in async engine

* start using lightweight-docker-go image for development tasks (#190)

* simplify ci config (#194)

* README work (#196)

* README formatting + one cmdline fix

* Link fixes

* Revert heading to title case

* append instead of replacing user-agent (#198)

* append instead of replacing user-agent

* fix lint errors

* add deferred task execution to async package (#208)

achieving this involved a significant refactor
of the entire package. the good news is that the
tests are improved beyond what we had before,
so i think we can be more confident in the
refactored package than the original.

* Provision and Deprovision enhancements to use new async features (#204)

* Enhancement of asynch engine

This PR enhances the asynch engine to enable the
concenpt of delayed or paused tasks. Tasks can be
added to the engine in a queue separate from the main
work queue. As delayed tasks are added, a  resumer is
notified and will periodically start the tasks up. Delayed
tasks can also be started by engine clients.

Using this, pkg/api/provision and deprovision were enhanced to
determine if a provision or deprovision action needs to wait for
another operation to finish. If so, they submit delayed tasks, either
wait_for_parent or wait_for_children, whichever is appropriate. Those
new jobs will make the same determination, and either readd themselves
as delayed tasks, or initiate the task that would otherwise originally
have been created.

* Added bad parent response message

* Merge master into feature branch (#209)

* refactor event function signatures (#147)

* pass around instances and bindings instead of pointers

* Refactor instance encryption/decryption

Previously, encryption and decryption of certain instance
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to instance
objects, I wanted a way for modules to be able to extract
information from encrypted instance fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors instances and the storage layer so that
encryption occurs during instance marshaling and decryption occurs
during instance unmarshaling.

* Refactor binding encryption/decryption

Previously, encryption and decryption of certain binding
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to binding
objects, I wanted a way for modules to be able to extract
information from encrypted binding fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors bindings and the storage layer so that
encryption occurs during bindings marshaling and decryption occurs
during bindings unmarshaling.

* clean up a few errant lint errors

* pass instance to provisioning steps

* pass instance to deprovisioning steps

* make bind and unbind accept instances as arguments

* get defensive about possible unanticipated modifications to instances by module code

* pass instance to updating steps

* minor lint fixes

* refactor lifecycle tests

* Firewall rule parameterization (#152)

* Refactor to support allowing firewall rules.

Fixes: #146

Changes to suport optional parameters to allow specifying start
and end IP address for firewall rules on MySQL, PostgreSQL and
MSSQL. Previously, this defaulted to allow any connection. This
isn't ecure in practice, so allowing the customer to specify desired
rules. Also changed the default to 0.0.0.0 for both, which results in
only Azure internal addresses. Added some parameter validation
for the firewall values as well:

* Valid IPV4 addresses
* Start <= End

Extracted a method to build the arm template parameter map. Providing
empty strings is invalid for the ARM template, so only including
when non-empty.

Added parameters to the lifecycle tests to allow running them
from local machines (will need ao enhance the charts in helm-charts)

Added some tests.

* Fixing merge conflicts

* Test cleanup

* json field cleanup

* Resolving code review comments

* error message cleanup

* error message cleanup

* specify firewall rules for relational databased in example manifests

* Few minor corrections to the Quickstart (#136)

* Correcting the az ad sp delete command

* Update quickstart-minikube.md

Remove a leading $ from a few bash commands. copy paste on mac is messed up :-)

* Update quickstart-minikube.md

Clarification around logging in to WordPress

* restoring prompts in quickstart

* minor punctuation fix

* removing distracting echo command

* make prompts used consistently

* kick of redis tests sooner (#155)

* s/module lifecycle tests/service lifecycle tests/g (#156)

* make storage know how to deal with service-specific types (#153)

* don't use codec where we don't need it anymore (#161)

* Updating CircleCI to define DOCKER_REPO env var (#167)

* Updating CircleCI to define REGISTRY env var

We previously set the REGISTRY environment variable in a deploy script.
When that was removed to streamline the release process, we need to
define it in Circle. Added two environment blocks to the publish-rc-images
and publish-release-images jobs.

Fixes: 145

* Qualified registry with docker.io

* Changed REGISTRY to DOCKER_REPO

* s/context/details/g (#163)

* make fake bind/unbind functionality not rely on standard context

* collapse standard provisioning context onto instance

* do away with (sort of) redundant standard provisioning parameters

* s/provisioning context/instance details/

* s/binding context/binding details/

* fix error handling when standard params aren't expected types

* add comment that adds some clarity to instance comparisons

* don't store credentials as their own field (#169)

* quickstart fixes (#174)

* adding missing docker pushes to release process

* Adding build flags to inject the version and commit (#173)

* Adding build flags to inject the version and commit

This commit injects the version and commit to the OSBA binary. If there
is no version set, then sets the version to “devel”

* Using new variable for version

* Removing redundant setting for main.commit

* Printing version and commit on broker startup

* Adding fields to the log message on startup

* Removing build flags on the CLI builds

* broker doesn't need to hang on to codec (#180)

* remove quotes from devel pseudo-version string (#184)

Merging without CI.

* fix bug decoding tags (#183)

* move version info to avoid import cycle later (#185)

* Adding user-agent string to requests to ARM (#172)

* Adding user-agent string to requests to ARM

* get version for user agent string from version package

* use separate redis clients for storage and async engine (#182)

* Adding ability to filter lifecycle tests (#181)

* Adding ability to filter lifecycle tests

This adds the ability to filter lifecycle tests using an environment
variable (TEST_MODULES). Via this variable, the developer can provide
a comma delimited list of module names and only the test cases matching
that module will be enabled. If the variable is empty, all tests will run.
If an invalid module name is provided, no tests will run.

* fix typo and lint error

* remove unnecessary funtion

* use redis namespaces (#188)

* use redis namespaces for instances and bindings

* use redis namespaces in async engine

* start using lightweight-docker-go image for development tasks (#190)

* simplify ci config (#194)

* README work (#196)

* README formatting + one cmdline fix

* Link fixes

* Revert heading to title case

* append instead of replacing user-agent (#198)

* append instead of replacing user-agent

* fix lint errors

* add deferred task execution to async package (#208)

achieving this involved a significant refactor
of the entire package. the good news is that the
tests are improved beyond what we had before,
so i think we can be more confident in the
refactored package than the original.

* Enhancement of asynch engine

This PR enhances the asynch engine to enable the
concenpt of delayed or paused tasks. Tasks can be
added to the engine in a queue separate from the main
work queue. As delayed tasks are added, a  resumer is
notified and will periodically start the tasks up. Delayed
tasks can also be started by engine clients.

Using this, pkg/api/provision and deprovision were enhanced to
determine if a provision or deprovision action needs to wait for
another operation to finish. If so, they submit delayed tasks, either
wait_for_parent or wait_for_children, whichever is appropriate. Those
new jobs will make the same determination, and either readd themselves
as delayed tasks, or initiate the task that would otherwise originally
have been created.

* Remove resumer changes

* removed references to submitdelayed task

* Fixed debug statements that got clobbered in merge

* Debug message

* Fixed response code

* Whitespace

* Code review comments

* Code review comments

* Refactor based on code-review comments.

* Code review comments

* Lint fixes

* Enhanced contrib/k8s/examples (#213)

* Enhanced contrib/k8s/examples to illustrate the use of the new
SQLDB catalog

* Spelling errors

* merge latest from master to poc branch (#232)

* refactor event function signatures (#147)

* pass around instances and bindings instead of pointers

* Refactor instance encryption/decryption

Previously, encryption and decryption of certain instance
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to instance
objects, I wanted a way for modules to be able to extract
information from encrypted instance fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors instances and the storage layer so that
encryption occurs during instance marshaling and decryption occurs
during instance unmarshaling.

* Refactor binding encryption/decryption

Previously, encryption and decryption of certain binding
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to binding
objects, I wanted a way for modules to be able to extract
information from encrypted binding fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors bindings and the storage layer so that
encryption occurs during bindings marshaling and decryption occurs
during bindings unmarshaling.

* clean up a few errant lint errors

* pass instance to provisioning steps

* pass instance to deprovisioning steps

* make bind and unbind accept instances as arguments

* get defensive about possible unanticipated modifications to instances by module code

* pass instance to updating steps

* minor lint fixes

* refactor lifecycle tests

* Firewall rule parameterization (#152)

* Refactor to support allowing firewall rules.

Fixes: #146

Changes to suport optional parameters to allow specifying start
and end IP address for firewall rules on MySQL, PostgreSQL and
MSSQL. Previously, this defaulted to allow any connection. This
isn't ecure in practice, so allowing the customer to specify desired
rules. Also changed the default to 0.0.0.0 for both, which results in
only Azure internal addresses. Added some parameter validation
for the firewall values as well:

* Valid IPV4 addresses
* Start <= End

Extracted a method to build the arm template parameter map. Providing
empty strings is invalid for the ARM template, so only including
when non-empty.

Added parameters to the lifecycle tests to allow running them
from local machines (will need ao enhance the charts in helm-charts)

Added some tests.

* Fixing merge conflicts

* Test cleanup

* json field cleanup

* Resolving code review comments

* error message cleanup

* error message cleanup

* specify firewall rules for relational databased in example manifests

* Few minor corrections to the Quickstart (#136)

* Correcting the az ad sp delete command

* Update quickstart-minikube.md

Remove a leading $ from a few bash commands. copy paste on mac is messed up :-)

* Update quickstart-minikube.md

Clarification around logging in to WordPress

* restoring prompts in quickstart

* minor punctuation fix

* removing distracting echo command

* make prompts used consistently

* kick of redis tests sooner (#155)

* s/module lifecycle tests/service lifecycle tests/g (#156)

* make storage know how to deal with service-specific types (#153)

* don't use codec where we don't need it anymore (#161)

* Updating CircleCI to define DOCKER_REPO env var (#167)

* Updating CircleCI to define REGISTRY env var

We previously set the REGISTRY environment variable in a deploy script.
When that was removed to streamline the release process, we need to
define it in Circle. Added two environment blocks to the publish-rc-images
and publish-release-images jobs.

Fixes: 145

* Qualified registry with docker.io

* Changed REGISTRY to DOCKER_REPO

* s/context/details/g (#163)

* make fake bind/unbind functionality not rely on standard context

* collapse standard provisioning context onto instance

* do away with (sort of) redundant standard provisioning parameters

* s/provisioning context/instance details/

* s/binding context/binding details/

* fix error handling when standard params aren't expected types

* add comment that adds some clarity to instance comparisons

* don't store credentials as their own field (#169)

* quickstart fixes (#174)

* adding missing docker pushes to release process

* Adding build flags to inject the version and commit (#173)

* Adding build flags to inject the version and commit

This commit injects the version and commit to the OSBA binary. If there
is no version set, then sets the version to “devel”

* Using new variable for version

* Removing redundant setting for main.commit

* Printing version and commit on broker startup

* Adding fields to the log message on startup

* Removing build flags on the CLI builds

* broker doesn't need to hang on to codec (#180)

* remove quotes from devel pseudo-version string (#184)

Merging without CI.

* fix bug decoding tags (#183)

* move version info to avoid import cycle later (#185)

* Adding user-agent string to requests to ARM (#172)

* Adding user-agent string to requests to ARM

* get version for user agent string from version package

* use separate redis clients for storage and async engine (#182)

* Adding ability to filter lifecycle tests (#181)

* Adding ability to filter lifecycle tests

This adds the ability to filter lifecycle tests using an environment
variable (TEST_MODULES). Via this variable, the developer can provide
a comma delimited list of module names and only the test cases matching
that module will be enabled. If the variable is empty, all tests will run.
If an invalid module name is provided, no tests will run.

* fix typo and lint error

* remove unnecessary funtion

* use redis namespaces (#188)

* use redis namespaces for instances and bindings

* use redis namespaces in async engine

* start using lightweight-docker-go image for development tasks (#190)

* simplify ci config (#194)

* README work (#196)

* README formatting + one cmdline fix

* Link fixes

* Revert heading to title case

* append instead of replacing user-agent (#198)

* append instead of replacing user-agent

* fix lint errors

* add deferred task execution to async package (#208)

achieving this involved a significant refactor
of the entire package. the good news is that the
tests are improved beyond what we had before,
so i think we can be more confident in the
refactored package than the original.

* reorganize async pakage/sub-packages more sanely (#210)

* change provision/deprovision task names (#211)

The are imperative and more accurate.

* Add async support for "follow-up" tasks (#212)

* make async job functions take task as argument

* make async job functions return tasks

* bonus: fix more task names

* make async task executor submit follow-up tasks

* make broker async processes leverage new async follow-up tasks feature

* ignore .vscode/ (#221)

* more async refactoring (#219)

* break up worker.go and worker_test.go into multiple files

* simplify heart

* simplify cleaner

* move heart from worker to engine

* remove receive/execute/watch from worker to engine

* let task receivers return right after fatal error

* let task executor return right after fatal error

* very minor comment cleanup

* fix lint errors by removing unused error types

* add svc and plan to instance during retrieval (#220)

* retrieve svc when instance is retrieved

* retrieve plan when instance is retrieved

* improve provision step function signature

* improve deprovision step function signature

* improve update step function signature

* lifecycles tests: set service and plan on instance

* Changes to mysql bind. (#222)

Change grant string in mysql bind to be bound to database
Removed global privs
Added Execute

Fixes #197

* Initial roadmap submission (#226)

* Initial Roadmap

* Publish initial roadmap for OSBA
Closes #109

* Spelling

* Review comments and wordsmithing.

* Refactor for generic filter capabilities (#218)

* WIP: Proposal to refactor for generic filter capabilities

1.) Create new api/filter structure
2.) Refactored authenticator stuff there
3.) Created new interfaces: filters.Filter and filters.Chain
4.) Made basic/always authenticator implement filters.Filter interface

* Fixed filename type-o.

* Fixed filename type-o

* Renamed Filter method to Execute.

* Added test for filter chain

* Refactor and style fixes

* ignore .vscode/ (#221)

* more async refactoring (#219)

* break up worker.go and worker_test.go into multiple files

* simplify heart

* simplify cleaner

* move heart from worker to engine

* remove receive/execute/watch from worker to engine

* let task receivers return right after fatal error

* let task executor return right after fatal error

* very minor comment cleanup

* fix lint errors by removing unused error types

* add svc and plan to instance during retrieval (#220)

* retrieve svc when instance is retrieved

* retrieve plan when instance is retrieved

* improve provision step function signature

* improve deprovision step function signature

* improve update step function signature

* lifecycles tests: set service and plan on instance

* changed filters package to filter

* Lint failure

* Refactor filter package a little and re-add Chain interface

* a few filter tweaks

* Change how the database name for cosmos db is generated (#228)

* Change the generation of the database name. A limitation in the current
ARM based CosmosDB provisoning limits the length of the database name to
50 characters, including the name provided, the location name and a dash.
In some situations, this means a UUID is invalid. If the length will be
too much, this patch will randomly generate a string of the available length
instead of using a UUID.

* Refactored to use generate package

* fix async cleaner (#233)

* bug fix: fix wait again on children not deprovisioned

* bug fix: it's ok for an instance's parent to not be found

* Code comments
  • Loading branch information
jeremyrickard authored Feb 6, 2018
1 parent eaa297e commit 612055e
Show file tree
Hide file tree
Showing 35 changed files with 2,081 additions and 485 deletions.
6 changes: 1 addition & 5 deletions cmd/broker/modules.go
Original file line number Diff line number Diff line change
Expand Up @@ -65,10 +65,6 @@ func initModules() error {
if err != nil {
return fmt.Errorf("error initializing mssql manager: %s", err)
}
msSQLConfig, err := sqldb.GetConfig()
if err != nil {
return fmt.Errorf("error parsing mssql configuration: %s", err)
}
cosmosDBManager, err := cd.NewManager()
if err != nil {
return fmt.Errorf("error initializing cosmosdb manager: %s", err)
Expand All @@ -93,7 +89,7 @@ func initModules() error {
servicebus.New(armDeployer, serviceBusManager),
eventhubs.New(armDeployer, eventHubManager),
keyvault.New(armDeployer, keyvaultManager),
sqldb.New(armDeployer, msSQLManager, msSQLConfig),
sqldb.New(armDeployer, msSQLManager),
cosmosdb.New(armDeployer, cosmosDBManager),
storage.New(armDeployer, storageManager),
search.New(armDeployer, searchManager),
Expand Down
25 changes: 25 additions & 0 deletions contrib/k8s/examples/sqldb/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# Azure SQL Database Examples

The examples in this directory show different ways to use the expanded Azure SQL Database catalog. The catalog now provides three services:
* azure-sqldb (All-in-One SQL Database Server VM and Database)
* azure-sqldb-vm-only (SQL Database Server VM Only)
* azure-sqldb-db-only (SQL Database only)

The use of the azure-sqldb-vm-only service and the azure-sqldb-db-only service allow you to independently provision the SQL Database VM and the Database itself. This also enables the creation of multiple databases on a single server VM.

This directory contains example Kubernetes manifests to exercise these services.

## sqldb-all-in-one

The `sqldb-all-in-one-instance.yaml` manifest will create an all-in-one Azure SQL Database that includes both the server VM and the database. The `sqldb-all-in-one-binding.yaml` will create a binding for the all in one service instance.

## sqldb-vm-only

The `sqldb-vm-only-instance.yaml` manifest will provision an instance of the `azure-sqldb-vm-only` service. This service is not bindable, so there is no corresponding binding manifest. An important element of this manifest is the `alias` parameter. This is used when provisioning an instance of the `azure-sqldb-db-only` service.

## sqldb-db-only

The `sqldb-db-only-instance.yaml` manifest will provision an instance of the `azure-sqldb-db-only` service. This service *requires* a parameter called `parentAlias`. The value of this parameter matches the `alias` parameter, which is defined in the `sqldb-vm-only-instance.yaml` manifest.

The `sqldb-db-only-instance-binding.yaml` manifest can then be used to create a service binding to the database only service instance created above.

9 changes: 9 additions & 0 deletions contrib/k8s/examples/sqldb/sqldb-all-in-one-binding.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
apiVersion: servicecatalog.k8s.io/v1beta1
kind: ServiceBinding
metadata:
name: all-in-one-sqldb-binding
namespace: default
spec:
instanceRef:
name: all-in-one-sqldb-instance
secretName: all-in-one-sqldb-secret
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
apiVersion: servicecatalog.k8s.io/v1beta1
kind: ServiceInstance
metadata:
name: my-sqldb-instance
name: all-in-one-sqldb-instance
namespace: default
spec:
clusterServiceClassExternalName: azure-sqldb
Expand Down
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
apiVersion: servicecatalog.k8s.io/v1beta1
kind: ServiceBinding
metadata:
name: my-sqldb-binding
name: db-only-sqldb-secret
namespace: default
spec:
instanceRef:
name: my-sqldb-instance
secretName: my-sqldb-secret
name: db-only-sqldb-instance
secretName: db-only-sqldb-secret
10 changes: 10 additions & 0 deletions contrib/k8s/examples/sqldb/sqldb-db-only-instance.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
apiVersion: servicecatalog.k8s.io/v1beta1
kind: ServiceInstance
metadata:
name: db-only-sqldb-instance
namespace: default
spec:
clusterServiceClassExternalName: azure-sqldb-db-only
clusterServicePlanExternalName: basic
parameters:
parentAlias: vm-only-sqldb
14 changes: 14 additions & 0 deletions contrib/k8s/examples/sqldb/sqldb-vm-only-instance.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
apiVersion: servicecatalog.k8s.io/v1beta1
kind: ServiceInstance
metadata:
name: vm-only-sqldb-instance
namespace: default
spec:
clusterServiceClassExternalName: azure-sqldb-vm-only
clusterServicePlanExternalName: sqldb-vm-only
parameters:
location: eastus
resourceGroup: demo
alias: vm-only-sqldb
firewallStartIPAddress: "0.0.0.0"
firewallEndIPAddress: "255.255.255.255"
41 changes: 34 additions & 7 deletions pkg/api/deprovision.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ package api
import (
"net/http"
"strconv"
"time"

"github.com/Azure/open-service-broker-azure/pkg/async"
"github.com/Azure/open-service-broker-azure/pkg/service"
Expand Down Expand Up @@ -119,13 +120,39 @@ func (s *server) deprovision(w http.ResponseWriter, r *http.Request) {
return
}

task := async.NewTask(
"executeDeprovisioningStep",
map[string]string{
"stepName": firstStepName,
"instanceID": instanceID,
},
)
childCount, err := s.store.GetInstanceChildCountByAlias(instance.Alias)
if err != nil {
logFields["step"] = firstStepName
logFields["error"] = err
log.WithFields(logFields).Error(
"deprovisioning error: error determining child count",
)
s.writeResponse(w, http.StatusInternalServerError, responseEmptyJSON)
}

var task async.Task
if childCount > 0 {
logFields["provisionedChildren"] = childCount
task = async.NewDelayedTask(
"checkChildrenStatuses",
map[string]string{
"instanceID": instanceID,
},
time.Minute*1,
)
log.WithFields(logFields).Debug("children not deprovisioned, waiting")
} else {
task = async.NewTask(
"executeDeprovisioningStep",
map[string]string{
"stepName": firstStepName,
"instanceID": instanceID,
},
)
log.WithFields(logFields).Debug(
"no provisioned children, starting deprovision",
)
}
if err = s.asyncEngine.SubmitTask(task); err != nil {
logFields["step"] = firstStepName
logFields["error"] = err
Expand Down
189 changes: 175 additions & 14 deletions pkg/api/provision.go
Original file line number Diff line number Diff line change
Expand Up @@ -196,6 +196,42 @@ func (s *server) provision(w http.ResponseWriter, r *http.Request) {
}
}

// Alias
alias := ""
aliasIface, ok := provisioningRequest.Parameters["alias"]
if ok {
alias, ok = aliasIface.(string)
if !ok {
s.handlePossibleValidationError(
service.NewValidationError(
"alias",
fmt.Sprintf(`"%v" is not a string`, locIface),
),
w,
logFields,
)
return
}
}

// Parent alias
parentAlias := ""
parentAliasIface, ok := provisioningRequest.Parameters["parentAlias"]
if ok {
parentAlias, ok = parentAliasIface.(string)
if !ok {
s.handlePossibleValidationError(
service.NewValidationError(
"parentAlias",
fmt.Sprintf(`"%v" is not a string`, parentAlias),
),
w,
logFields,
)
return
}
}

// Now service-specific parameters...
provisioningParameters := serviceManager.GetEmptyProvisioningParameters()
decoderConfig := &mapstructure.DecoderConfig{
Expand Down Expand Up @@ -284,7 +320,21 @@ func (s *server) provision(w http.ResponseWriter, r *http.Request) {
// If we get to here, we need to provision a new instance.

// Start by validating the location
err = s.validateLocation(location)
err = s.validateLocation(svc, location)
if err != nil {
s.handlePossibleValidationError(err, w, logFields)
return
}

// Validate alias (only applies if this service type has children)
err = s.validateAlias(svc, alias)
if err != nil {
s.handlePossibleValidationError(err, w, logFields)
return
}

// Validate parent alias (only applies if this service type has a parent)
err = s.validateParentAlias(svc, parentAlias)
if err != nil {
s.handlePossibleValidationError(err, w, logFields)
return
Expand Down Expand Up @@ -324,16 +374,29 @@ func (s *server) provision(w http.ResponseWriter, r *http.Request) {

instance = service.Instance{
InstanceID: instanceID,
Alias: alias,
ServiceID: provisioningRequest.ServiceID,
PlanID: provisioningRequest.PlanID,
ProvisioningParameters: provisioningParameters,
Status: service.InstanceStateProvisioning,
Location: location,
ResourceGroup: resourceGroup,
ParentAlias: parentAlias,
Tags: tags,
Details: serviceManager.GetEmptyInstanceDetails(),
Created: time.Now(),
}

waitForParent, err := s.isParentProvisioning(instance)
if err != nil {
logFields["error"] = err
log.WithFields(logFields).Error(
"provisioning error: error related to parent instance",
)
s.writeResponse(w, http.StatusBadRequest, responseParentInvalid)
return
}

if err = s.store.WriteInstance(instance); err != nil {
logFields["error"] = err
log.WithFields(logFields).Error(
Expand All @@ -343,13 +406,29 @@ func (s *server) provision(w http.ResponseWriter, r *http.Request) {
return
}

task := async.NewTask(
"executeProvisioningStep",
map[string]string{
"stepName": firstStepName,
"instanceID": instanceID,
},
)
var task async.Task
if waitForParent {
task = async.NewDelayedTask(
"checkParentStatus",
map[string]string{
"instanceID": instanceID,
},
time.Minute*1,
)
log.WithFields(logFields).Debug("parent not provisioned, waiting")
} else {
task = async.NewTask(
"executeProvisioningStep",
map[string]string{
"stepName": firstStepName,
"instanceID": instanceID,
},
)
log.WithFields(logFields).Debug(
"no need to wait for parent, starting provision",
)
}

if err = s.asyncEngine.SubmitTask(task); err != nil {
logFields["step"] = firstStepName
logFields["error"] = err
Expand All @@ -359,19 +438,101 @@ func (s *server) provision(w http.ResponseWriter, r *http.Request) {
s.writeResponse(w, http.StatusInternalServerError, responseEmptyJSON)
return
}

// If we get all the way to here, we've been successful!
s.writeResponse(w, http.StatusAccepted, responseProvisioningAccepted)

log.WithFields(logFields).Debug("asynchronous provisioning initiated")
}

func (s *server) validateLocation(location string) error {
if (location == "" && s.defaultAzureLocation == "") ||
(location != "" && !azure.IsValidLocation(location)) {
func (s *server) isParentProvisioning(instance service.Instance) (bool, error) {
//No parent, so no need to wait
if instance.ParentAlias == "" {
return false, nil
}

parent, parentFound, err := s.store.GetInstanceByAlias(instance.ParentAlias)

if err != nil {
log.WithFields(log.Fields{
"error": "waitforParent",
"instanceID": instance.InstanceID,
"parentAlias": instance.ParentAlias,
}).Error(
"bad provision request: unable to retrieve parent",
)
return false, err
}

//Parent has was not found, so wait for that that to occur
if !parentFound {
return true, nil
}

//If parent failed, we should not even attempt to provision this
if parent.Status == service.InstanceStateProvisioningFailed {
log.WithFields(log.Fields{
"error": "waitforParent",
"instanceID": instance.InstanceID,
"parentID": instance.Parent.InstanceID,
}).Info(
"bad provision request: parent failed provisioning",
)
return false, fmt.Errorf("error provisioning: parent provision failed")
}

//If parent is deprovisioning, we should not even attempt to provision this
if parent.Status == service.InstanceStateDeprovisioning {
log.WithFields(log.Fields{
"error": "waitforParent",
"instanceID": instance.InstanceID,
"parentID": instance.Parent.InstanceID,
}).Info(
"bad provision request: parent is deprovisioning",
)
return false, fmt.Errorf("error provisioning: parent is deprovisioning")
}

//If parent is provisioned, then no need to wait.
if parent.Status == service.InstanceStateProvisioned {
return false, nil
}

return true, nil
}

func (s *server) validateLocation(svc service.Service, location string) error {
// Validate location only if this is a "root" service type (i.e. has no
// parent)
if svc.GetParentServiceID() == "" {
if (location == "" && s.defaultAzureLocation == "") ||
(location != "" && !azure.IsValidLocation(location)) {
return service.NewValidationError(
"location",
fmt.Sprintf(`invalid location: "%s"`, location),
)
}
}
return nil
}

func (s *server) validateAlias(svc service.Service, alias string) error {
if svc.GetChildServiceID() != "" && alias == "" {
return service.NewValidationError(
"alias",
fmt.Sprintf(`invalid alias: "%s"`, alias),
)
}
return nil
}

func (s *server) validateParentAlias(
svc service.Service,
parentAlias string,
) error {
if svc.GetParentServiceID() != "" && parentAlias == "" {
return service.NewValidationError(
"location",
fmt.Sprintf(`invalid location: "%s"`, location),
"parentAlias",
fmt.Sprintf(`invalid parentAlias: "%s"`, parentAlias),
)
}
return nil
Expand Down
Loading

0 comments on commit 612055e

Please sign in to comment.