This repository has been archived by the owner on Jul 6, 2022. It is now read-only.
Provision and Deprovision enhancements to use new async features #204
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This PR enhances the asynch engine to enable the concenpt of delayed or paused tasks. Tasks can be added to the engine in a queue separate from the main work queue. As delayed tasks are added, a resumer is notified and will periodically start the tasks up. Delayed tasks can also be started by engine clients. Using this, pkg/api/provision and deprovision were enhanced to determine if a provision or deprovision action needs to wait for another operation to finish. If so, they submit delayed tasks, either wait_for_parent or wait_for_children, whichever is appropriate. Those new jobs will make the same determination, and either readd themselves as delayed tasks, or initiate the task that would otherwise originally have been created.
jeremyrickard
changed the title
jeremyrickard
changed the title
* refactor event function signatures (Azure#147) * pass around instances and bindings instead of pointers * Refactor instance encryption/decryption Previously, encryption and decryption of certain instance fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to instance objects, I wanted a way for modules to be able to extract information from encrypted instance fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors instances and the storage layer so that encryption occurs during instance marshaling and decryption occurs during instance unmarshaling. * Refactor binding encryption/decryption Previously, encryption and decryption of certain binding fields was accomplished through special getters and setters. To use these, you had to pass in a codec. This meant that any bit of code needing to access these fields needs access to the codec. Since I'm entertaining changes to the service.Module interface that will result in modules having access to binding objects, I wanted a way for modules to be able to extract information from encrypted binding fields _without_ needing them to be burdened with additional moving parts-- like codecs. So this commit refactors bindings and the storage layer so that encryption occurs during bindings marshaling and decryption occurs during bindings unmarshaling. * clean up a few errant lint errors * pass instance to provisioning steps * pass instance to deprovisioning steps * make bind and unbind accept instances as arguments * get defensive about possible unanticipated modifications to instances by module code * pass instance to updating steps * minor lint fixes * refactor lifecycle tests * Firewall rule parameterization (Azure#152) * Refactor to support allowing firewall rules. Fixes: Azure#146 Changes to suport optional parameters to allow specifying start and end IP address for firewall rules on MySQL, PostgreSQL and MSSQL. Previously, this defaulted to allow any connection. This isn't ecure in practice, so allowing the customer to specify desired rules. Also changed the default to 0.0.0.0 for both, which results in only Azure internal addresses. Added some parameter validation for the firewall values as well: * Valid IPV4 addresses * Start <= End Extracted a method to build the arm template parameter map. Providing empty strings is invalid for the ARM template, so only including when non-empty. Added parameters to the lifecycle tests to allow running them from local machines (will need ao enhance the charts in helm-charts) Added some tests. * Fixing merge conflicts * Test cleanup * json field cleanup * Resolving code review comments * error message cleanup * error message cleanup * specify firewall rules for relational databased in example manifests * Few minor corrections to the Quickstart (Azure#136) * Correcting the az ad sp delete command * Update quickstart-minikube.md Remove a leading $ from a few bash commands. copy paste on mac is messed up :-) * Update quickstart-minikube.md Clarification around logging in to WordPress * restoring prompts in quickstart * minor punctuation fix * removing distracting echo command * make prompts used consistently * kick of redis tests sooner (Azure#155) * s/module lifecycle tests/service lifecycle tests/g (Azure#156) * make storage know how to deal with service-specific types (Azure#153) * don't use codec where we don't need it anymore (Azure#161) * Updating CircleCI to define DOCKER_REPO env var (Azure#167) * Updating CircleCI to define REGISTRY env var We previously set the REGISTRY environment variable in a deploy script. When that was removed to streamline the release process, we need to define it in Circle. Added two environment blocks to the publish-rc-images and publish-release-images jobs. Fixes: 145 * Qualified registry with docker.io * Changed REGISTRY to DOCKER_REPO * s/context/details/g (Azure#163) * make fake bind/unbind functionality not rely on standard context * collapse standard provisioning context onto instance * do away with (sort of) redundant standard provisioning parameters * s/provisioning context/instance details/ * s/binding context/binding details/ * fix error handling when standard params aren't expected types * add comment that adds some clarity to instance comparisons * don't store credentials as their own field (Azure#169) * quickstart fixes (Azure#174) * adding missing docker pushes to release process * Adding build flags to inject the version and commit (Azure#173) * Adding build flags to inject the version and commit This commit injects the version and commit to the OSBA binary. If there is no version set, then sets the version to “devel” * Using new variable for version * Removing redundant setting for main.commit * Printing version and commit on broker startup * Adding fields to the log message on startup * Removing build flags on the CLI builds * broker doesn't need to hang on to codec (Azure#180) * remove quotes from devel pseudo-version string (Azure#184) Merging without CI. * fix bug decoding tags (Azure#183) * move version info to avoid import cycle later (Azure#185) * Adding user-agent string to requests to ARM (Azure#172) * Adding user-agent string to requests to ARM * get version for user agent string from version package * use separate redis clients for storage and async engine (Azure#182) * Adding ability to filter lifecycle tests (Azure#181) * Adding ability to filter lifecycle tests This adds the ability to filter lifecycle tests using an environment variable (TEST_MODULES). Via this variable, the developer can provide a comma delimited list of module names and only the test cases matching that module will be enabled. If the variable is empty, all tests will run. If an invalid module name is provided, no tests will run. * fix typo and lint error * remove unnecessary funtion * use redis namespaces (Azure#188) * use redis namespaces for instances and bindings * use redis namespaces in async engine * start using lightweight-docker-go image for development tasks (Azure#190) * simplify ci config (Azure#194) * README work (Azure#196) * README formatting + one cmdline fix * Link fixes * Revert heading to title case * append instead of replacing user-agent (Azure#198) * append instead of replacing user-agent * fix lint errors * add deferred task execution to async package (Azure#208) achieving this involved a significant refactor of the entire package. the good news is that the tests are improved beyond what we had before, so i think we can be more confident in the refactored package than the original.
This PR enhances the asynch engine to enable the concenpt of delayed or paused tasks. Tasks can be added to the engine in a queue separate from the main work queue. As delayed tasks are added, a resumer is notified and will periodically start the tasks up. Delayed tasks can also be started by engine clients. Using this, pkg/api/provision and deprovision were enhanced to determine if a provision or deprovision action needs to wait for another operation to finish. If so, they submit delayed tasks, either wait_for_parent or wait_for_children, whichever is appropriate. Those new jobs will make the same determination, and either readd themselves as delayed tasks, or initiate the task that would otherwise originally have been created.
…ker-azure into wait-for
krancour
reviewed
Jan 16, 2018
| return false, nil | ||
| } | ||
|
|
||
| parent, parentFound, err := s.store.GetInstanceByAlias(instance.ParentAlias) |
There was a problem hiding this comment.
Is this because instance.Parent will always be nil at this point in time?
There was a problem hiding this comment.
Yeah, at this point in time the instance.Parent is nil (we just created the instance) as we have not retrieved it from storage.
There was a problem hiding this comment.
I removed this from the task (but have not pushed).
krancour
reviewed
Jan 16, 2018
pkg/api/provision.go
Outdated
| return true, nil | ||
| } | ||
|
|
||
| if err != nil { |
There was a problem hiding this comment.
Knowing how GetInstanceByAlias is implemented, this is probably ok, but it might be better to code more defensively here by checking for an error before deciding to trust the value of parentFound.
krancour
reviewed
Jan 16, 2018
| if parent.Status == service.InstanceStateProvisioningFailed { | ||
| log.WithFields(log.Fields{ | ||
| "error": "waitforParent", | ||
| "instanceID": instance.InstanceID, |
There was a problem hiding this comment.
Let's include the parent ID in these log fields also.
There was a problem hiding this comment.
👍 where available, I'll add ID. IF ID is not known, I'll use Alias.
There was a problem hiding this comment.
IF ID is not known, I'll use Alias. 👍
krancour
reviewed
Jan 16, 2018
pkg/api/provision.go
Outdated
| }).Error( | ||
| "bad provision request: parent failed provisioning", | ||
| ) | ||
| return false, fmt.Errorf("error provisioning: parent provision failed") |
There was a problem hiding this comment.
imo, this should be severity INFO; not ERROR. My reasoning (which I have applied as consistently as I've been able throughout the rest of the broker) is that we use ERROR when the broker itself has experienced a problem. In this case, it has not. The broker refusing to provision a child for a parent that has already failed provisioning is actually the broker working as designed. INFO because is is worth knowing about, but not ERROR because this isn't evidence that anything is wrong with the broker.
krancour
reviewed
Jan 16, 2018
pkg/api/provision.go
Outdated
| log.WithFields(log.Fields{ | ||
| "error": "waitforParent", | ||
| "instanceID": instance.InstanceID, | ||
| }).Error( |
krancour
reviewed
Jan 16, 2018
pkg/broker/deprovision.go
Outdated
| @@ -89,7 +89,7 @@ func (b *broker) doDeprovisionStep( | |||
| "error loading persisted instance", | |||
| ) | |||
| } | |||
|
|
|||
There was a problem hiding this comment.
Nit: Not sure why space got added here.
krancour
reviewed
Jan 16, 2018
pkg/broker/provision.go
Outdated
| @@ -89,7 +89,7 @@ func (b *broker) doProvisionStep( | |||
| "error loading persisted instance", | |||
| ) | |||
| } | |||
|
|
|||
There was a problem hiding this comment.
Nit: Not sure why space got added here.
krancour
reviewed
Jan 16, 2018
pkg/broker/wait_for_children.go
Outdated
| ) error { | ||
|
|
||
| _, cancel := context.WithCancel(ctx) | ||
| defer cancel() |
There was a problem hiding this comment.
This looks like a mistake. This will cancel the context when this function returns, but this context isn't being passed to any functions. (It can't be, because we've used _ to refuse assigning it to any variable. If ctx, is passed to any functions below [I haven't looked yet], that is the ctx we were passed and that ctx won't be canceled when this function returns.
krancour
reviewed
Jan 16, 2018
pkg/broker/wait_for_parent.go
Outdated
| args map[string]string, | ||
| ) error { | ||
| _, cancel := context.WithCancel(ctx) | ||
| defer cancel() |
krancour
reviewed
Jan 16, 2018
pkg/api/deprovision.go
Outdated
| task = model.NewDelayedTask( | ||
| "waitForChildrenStep", | ||
| map[string]string{ | ||
| "deprovisionFirstStep": firstStepName, |
There was a problem hiding this comment.
For consistency with the more immediate provision/deprovision tasks, can this just be "stepName"?
There was a problem hiding this comment.
Actually-- strike that. We don't need to pass this information through the queue.
When the step that checks children's statuses wakes up, it retrieves the instance by ID. From that instance, we know the service and we can get the service's provisioner, which can tell us what its first step is.
There was a problem hiding this comment.
Yeah, this is totally doable. I passed this through just to avoid duplicating the lookup logic from the things in pkg/api (get service, get plan, get service manger, get deprovisioner/provisioner), but this is easily changed.
There was a problem hiding this comment.
I passed this through just to avoid duplicating the lookup logic from the things in pkg/api (get service, get plan, get service manger, get deprovisioner/provisioner), but this is easily changed.
I have a todo to look up service and plan when instances are retrieved from the database-- then we can stop doing that in a bunch of other places.
krancour
reviewed
Jan 16, 2018
pkg/api/provision.go
Outdated
| task = model.NewDelayedTask( | ||
| "waitForParentStep", | ||
| map[string]string{ | ||
| "provisionFirstStep": firstStepName, |
krancour
reviewed
Jan 16, 2018
pkg/api/deprovision.go
Outdated
| var task model.Task | ||
| if childCount > 0 { | ||
| task = model.NewDelayedTask( | ||
| "waitForChildrenStep", |
There was a problem hiding this comment.
Semantics: existing task names are things like deprovisionStep (emphasis on step) because that's what those tasks do-- they execute one step of the deprovisioning process. (Step is a "technical" term here, in a manner of speaking, because the provisioners and deprovisioners are literally composed of "steps.") I'm not sure how applicable "step" is to what we're doing here. waitForChildren (sans "step"), might be a more appropriate task name.
Part 2-- more semantics: waitForChildren still might not adequately describe the task. The task doesn't do much waiting. It's awake for a moment before either scheduling another task like itself or scheduling the first deprovisioning step. Maybe checkChildrensStatuses works?
Sorry to split hairs so much, but the naming of tasks (for instance) does fall into the category of things that we eventually can not change anymore, so I want to get it right.
krancour
reviewed
Jan 16, 2018
pkg/api/provision.go
Outdated
| var task model.Task | ||
| if waitForParent { | ||
| task = model.NewDelayedTask( | ||
| "waitForParentStep", |
krancour
reviewed
Jan 16, 2018
pkg/broker/check_children_status.go
Outdated
| ) | ||
| log.WithFields(log.Fields{ | ||
| "instanceID": instanceID, | ||
| }).Debug("children not deprovisioned, will wait again") |
There was a problem hiding this comment.
Nice message. Great for debugging.
krancour
reviewed
Jan 16, 2018
| plan, ok := svc.GetPlan(instance.PlanID) | ||
| if !ok { | ||
| // If we don't find the Service in the catalog, something is really wrong. | ||
| // (It should exist, because an instance with this serviceID exists.) |
There was a problem hiding this comment.
This comment is wrong-- and I realize it was probably wrong where it was copied from too. Can we fix it in just this spot for now?
krancour
reviewed
Jan 16, 2018
pkg/broker/check_children_status.go
Outdated
| ) | ||
| } | ||
| serviceManager := svc.GetServiceManager() | ||
| deprovisioner, dErr := serviceManager.GetDeprovisioner(plan) |
There was a problem hiding this comment.
Why dErr instead of the conventional err?
There was a problem hiding this comment.
Lint complained that it was shadowing the err declared outside of the conditional block (same in the other file).
krancour
reviewed
Jan 16, 2018
pkg/broker/check_parent_status.go
Outdated
| map[string]string{ | ||
| "instanceID": instanceID, | ||
| }, | ||
| time.Minute*5, |
There was a problem hiding this comment.
Applicable in a few other spots as well... can we bring this down to one minute (for now)? Speaking from a k8s perspective, there are already so many progressive back-offs in play. I worry how far k8s (which is waiting on us) backs off while we are potentially prolonging a process like this longer than we need to. In the future, we can either make this tunable or implement a progressive backoff of our own (that will require us to track how many times we've looped through this check task and we've already discussed doing that in a follow up).
|
CI passed, merging per Slack conversation. |
jeremyrickard
added a commit
that referenced
this pull request
Feb 6, 2018
* WIP Refactoring sqldb to POC (#142)
* Refactoring sqldb to POC #124
Removing the "existing server" logic from the current sqldb
module. This entails removing it from the provision/deprovision
file and removing it from the types. Also removed the config.go
file and updated the broker to not load that. We will need another
way to provide/register existing servers.
We will reimplement this behavior in a new service/service manager.
* fix minor lint failure
* Feature branch merge (#154)
* refactor event function signatures (#147)
* pass around instances and bindings instead of pointers
* Refactor instance encryption/decryption
Previously, encryption and decryption of certain instance
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to instance
objects, I wanted a way for modules to be able to extract
information from encrypted instance fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors instances and the storage layer so that
encryption occurs during instance marshaling and decryption occurs
during instance unmarshaling.
* Refactor binding encryption/decryption
Previously, encryption and decryption of certain binding
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to binding
objects, I wanted a way for modules to be able to extract
information from encrypted binding fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors bindings and the storage layer so that
encryption occurs during bindings marshaling and decryption occurs
during bindings unmarshaling.
* clean up a few errant lint errors
* pass instance to provisioning steps
* pass instance to deprovisioning steps
* make bind and unbind accept instances as arguments
* get defensive about possible unanticipated modifications to instances by module code
* pass instance to updating steps
* minor lint fixes
* refactor lifecycle tests
* Firewall rule parameterization (#152)
* Refactor to support allowing firewall rules.
Fixes: #146
Changes to suport optional parameters to allow specifying start
and end IP address for firewall rules on MySQL, PostgreSQL and
MSSQL. Previously, this defaulted to allow any connection. This
isn't ecure in practice, so allowing the customer to specify desired
rules. Also changed the default to 0.0.0.0 for both, which results in
only Azure internal addresses. Added some parameter validation
for the firewall values as well:
* Valid IPV4 addresses
* Start <= End
Extracted a method to build the arm template parameter map. Providing
empty strings is invalid for the ARM template, so only including
when non-empty.
Added parameters to the lifecycle tests to allow running them
from local machines (will need ao enhance the charts in helm-charts)
Added some tests.
* Fixing merge conflicts
* Test cleanup
* json field cleanup
* Resolving code review comments
* error message cleanup
* error message cleanup
* specify firewall rules for relational databased in example manifests
* Sqldb catalog poc merge master (#158)
* refactor event function signatures (#147)
* pass around instances and bindings instead of pointers
* Refactor instance encryption/decryption
Previously, encryption and decryption of certain instance
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to instance
objects, I wanted a way for modules to be able to extract
information from encrypted instance fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors instances and the storage layer so that
encryption occurs during instance marshaling and decryption occurs
during instance unmarshaling.
* Refactor binding encryption/decryption
Previously, encryption and decryption of certain binding
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to binding
objects, I wanted a way for modules to be able to extract
information from encrypted binding fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors bindings and the storage layer so that
encryption occurs during bindings marshaling and decryption occurs
during bindings unmarshaling.
* clean up a few errant lint errors
* pass instance to provisioning steps
* pass instance to deprovisioning steps
* make bind and unbind accept instances as arguments
* get defensive about possible unanticipated modifications to instances by module code
* pass instance to updating steps
* minor lint fixes
* refactor lifecycle tests
* Firewall rule parameterization (#152)
* Refactor to support allowing firewall rules.
Fixes: #146
Changes to suport optional parameters to allow specifying start
and end IP address for firewall rules on MySQL, PostgreSQL and
MSSQL. Previously, this defaulted to allow any connection. This
isn't ecure in practice, so allowing the customer to specify desired
rules. Also changed the default to 0.0.0.0 for both, which results in
only Azure internal addresses. Added some parameter validation
for the firewall values as well:
* Valid IPV4 addresses
* Start <= End
Extracted a method to build the arm template parameter map. Providing
empty strings is invalid for the ARM template, so only including
when non-empty.
Added parameters to the lifecycle tests to allow running them
from local machines (will need ao enhance the charts in helm-charts)
Added some tests.
* Fixing merge conflicts
* Test cleanup
* json field cleanup
* Resolving code review comments
* error message cleanup
* error message cleanup
* specify firewall rules for relational databased in example manifests
* Few minor corrections to the Quickstart (#136)
* Correcting the az ad sp delete command
* Update quickstart-minikube.md
Remove a leading $ from a few bash commands. copy paste on mac is messed up :-)
* Update quickstart-minikube.md
Clarification around logging in to WordPress
* restoring prompts in quickstart
* minor punctuation fix
* removing distracting echo command
* make prompts used consistently
* kick of redis tests sooner (#155)
* s/module lifecycle tests/service lifecycle tests/g (#156)
* make storage know how to deal with service-specific types (#153)
* make provision and deprovision steps receive a reference instance (#160)
* Merge master to feature (#175)
* refactor event function signatures (#147)
* pass around instances and bindings instead of pointers
* Refactor instance encryption/decryption
Previously, encryption and decryption of certain instance
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to instance
objects, I wanted a way for modules to be able to extract
information from encrypted instance fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors instances and the storage layer so that
encryption occurs during instance marshaling and decryption occurs
during instance unmarshaling.
* Refactor binding encryption/decryption
Previously, encryption and decryption of certain binding
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to binding
objects, I wanted a way for modules to be able to extract
information from encrypted binding fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors bindings and the storage layer so that
encryption occurs during bindings marshaling and decryption occurs
during bindings unmarshaling.
* clean up a few errant lint errors
* pass instance to provisioning steps
* pass instance to deprovisioning steps
* make bind and unbind accept instances as arguments
* get defensive about possible unanticipated modifications to instances by module code
* pass instance to updating steps
* minor lint fixes
* refactor lifecycle tests
* Firewall rule parameterization (#152)
* Refactor to support allowing firewall rules.
Fixes: #146
Changes to suport optional parameters to allow specifying start
and end IP address for firewall rules on MySQL, PostgreSQL and
MSSQL. Previously, this defaulted to allow any connection. This
isn't ecure in practice, so allowing the customer to specify desired
rules. Also changed the default to 0.0.0.0 for both, which results in
only Azure internal addresses. Added some parameter validation
for the firewall values as well:
* Valid IPV4 addresses
* Start <= End
Extracted a method to build the arm template parameter map. Providing
empty strings is invalid for the ARM template, so only including
when non-empty.
Added parameters to the lifecycle tests to allow running them
from local machines (will need ao enhance the charts in helm-charts)
Added some tests.
* Fixing merge conflicts
* Test cleanup
* json field cleanup
* Resolving code review comments
* error message cleanup
* error message cleanup
* specify firewall rules for relational databased in example manifests
* Few minor corrections to the Quickstart (#136)
* Correcting the az ad sp delete command
* Update quickstart-minikube.md
Remove a leading $ from a few bash commands. copy paste on mac is messed up :-)
* Update quickstart-minikube.md
Clarification around logging in to WordPress
* restoring prompts in quickstart
* minor punctuation fix
* removing distracting echo command
* make prompts used consistently
* kick of redis tests sooner (#155)
* s/module lifecycle tests/service lifecycle tests/g (#156)
* make storage know how to deal with service-specific types (#153)
* don't use codec where we don't need it anymore (#161)
* Updating CircleCI to define DOCKER_REPO env var (#167)
* Updating CircleCI to define REGISTRY env var
We previously set the REGISTRY environment variable in a deploy script.
When that was removed to streamline the release process, we need to
define it in Circle. Added two environment blocks to the publish-rc-images
and publish-release-images jobs.
Fixes: 145
* Qualified registry with docker.io
* Changed REGISTRY to DOCKER_REPO
* s/context/details/g (#163)
* make fake bind/unbind functionality not rely on standard context
* collapse standard provisioning context onto instance
* do away with (sort of) redundant standard provisioning parameters
* s/provisioning context/instance details/
* s/binding context/binding details/
* fix error handling when standard params aren't expected types
* add comment that adds some clarity to instance comparisons
* don't store credentials as their own field (#169)
* quickstart fixes (#174)
* adding missing docker pushes to release process
* minor merge nit fixes
* Decomposition of single service into multiple services (#176)
* Decomposition of single service into multiple services
1.) This renames the existing service manager into a service to
implement the all in one scenario (existing)
2.) Next, two new services are added (vm only and db only)
3.) Each lifecycle operation go file (bind, deprovision, provision, unbind)
now declares a method for each service manager struct
4.) Added new services to the catalog
5.) DB only is largely TBD implemented, took a crack at
starting the provision.
6.) Refactored catalog/service interface to add GetBindable() method
7.) Modified tests to only execute binding related tests if plan
is bindable
* Small spelling correction
* Implemented database only lifecycle methods and lint fix.
* DRY refactoring
* Review comments and lint fix
* add ref instance as instance attribute (#178)
* create a relationship between two services (#179)
* Refactor sqldb module to use parent instances in binding (#186)
* Refactor sqldb module to use parent instances in binding
Small refactor to decouple shared instance details and leverage
parent instance for binding/unbinding.
Further evolution for #124
* Updated error to be consistent.
* merge latest from master (#187)
* refactor event function signatures (#147)
* pass around instances and bindings instead of pointers
* Refactor instance encryption/decryption
Previously, encryption and decryption of certain instance
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to instance
objects, I wanted a way for modules to be able to extract
information from encrypted instance fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors instances and the storage layer so that
encryption occurs during instance marshaling and decryption occurs
during instance unmarshaling.
* Refactor binding encryption/decryption
Previously, encryption and decryption of certain binding
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to binding
objects, I wanted a way for modules to be able to extract
information from encrypted binding fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors bindings and the storage layer so that
encryption occurs during bindings marshaling and decryption occurs
during bindings unmarshaling.
* clean up a few errant lint errors
* pass instance to provisioning steps
* pass instance to deprovisioning steps
* make bind and unbind accept instances as arguments
* get defensive about possible unanticipated modifications to instances by module code
* pass instance to updating steps
* minor lint fixes
* refactor lifecycle tests
* Firewall rule parameterization (#152)
* Refactor to support allowing firewall rules.
Fixes: #146
Changes to suport optional parameters to allow specifying start
and end IP address for firewall rules on MySQL, PostgreSQL and
MSSQL. Previously, this defaulted to allow any connection. This
isn't ecure in practice, so allowing the customer to specify desired
rules. Also changed the default to 0.0.0.0 for both, which results in
only Azure internal addresses. Added some parameter validation
for the firewall values as well:
* Valid IPV4 addresses
* Start <= End
Extracted a method to build the arm template parameter map. Providing
empty strings is invalid for the ARM template, so only including
when non-empty.
Added parameters to the lifecycle tests to allow running them
from local machines (will need ao enhance the charts in helm-charts)
Added some tests.
* Fixing merge conflicts
* Test cleanup
* json field cleanup
* Resolving code review comments
* error message cleanup
* error message cleanup
* specify firewall rules for relational databased in example manifests
* Few minor corrections to the Quickstart (#136)
* Correcting the az ad sp delete command
* Update quickstart-minikube.md
Remove a leading $ from a few bash commands. copy paste on mac is messed up :-)
* Update quickstart-minikube.md
Clarification around logging in to WordPress
* restoring prompts in quickstart
* minor punctuation fix
* removing distracting echo command
* make prompts used consistently
* kick of redis tests sooner (#155)
* s/module lifecycle tests/service lifecycle tests/g (#156)
* make storage know how to deal with service-specific types (#153)
* don't use codec where we don't need it anymore (#161)
* Updating CircleCI to define DOCKER_REPO env var (#167)
* Updating CircleCI to define REGISTRY env var
We previously set the REGISTRY environment variable in a deploy script.
When that was removed to streamline the release process, we need to
define it in Circle. Added two environment blocks to the publish-rc-images
and publish-release-images jobs.
Fixes: 145
* Qualified registry with docker.io
* Changed REGISTRY to DOCKER_REPO
* s/context/details/g (#163)
* make fake bind/unbind functionality not rely on standard context
* collapse standard provisioning context onto instance
* do away with (sort of) redundant standard provisioning parameters
* s/provisioning context/instance details/
* s/binding context/binding details/
* fix error handling when standard params aren't expected types
* add comment that adds some clarity to instance comparisons
* don't store credentials as their own field (#169)
* quickstart fixes (#174)
* adding missing docker pushes to release process
* Adding build flags to inject the version and commit (#173)
* Adding build flags to inject the version and commit
This commit injects the version and commit to the OSBA binary. If there
is no version set, then sets the version to “devel”
* Using new variable for version
* Removing redundant setting for main.commit
* Printing version and commit on broker startup
* Adding fields to the log message on startup
* Removing build flags on the CLI builds
* broker doesn't need to hang on to codec (#180)
* remove quotes from devel pseudo-version string (#184)
Merging without CI.
* fix bug decoding tags (#183)
* move version info to avoid import cycle later (#185)
* Adding user-agent string to requests to ARM (#172)
* Adding user-agent string to requests to ARM
* get version for user agent string from version package
* use separate redis clients for storage and async engine (#182)
* Adding ability to filter lifecycle tests (#181)
* Adding ability to filter lifecycle tests
This adds the ability to filter lifecycle tests using an environment
variable (TEST_MODULES). Via this variable, the developer can provide
a comma delimited list of module names and only the test cases matching
that module will be enabled. If the variable is empty, all tests will run.
If an invalid module name is provided, no tests will run.
* fix typo and lint error
* remove unnecessary funtion
* Merge master (#189)
* refactor event function signatures (#147)
* pass around instances and bindings instead of pointers
* Refactor instance encryption/decryption
Previously, encryption and decryption of certain instance
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to instance
objects, I wanted a way for modules to be able to extract
information from encrypted instance fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors instances and the storage layer so that
encryption occurs during instance marshaling and decryption occurs
during instance unmarshaling.
* Refactor binding encryption/decryption
Previously, encryption and decryption of certain binding
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to binding
objects, I wanted a way for modules to be able to extract
information from encrypted binding fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors bindings and the storage layer so that
encryption occurs during bindings marshaling and decryption occurs
during bindings unmarshaling.
* clean up a few errant lint errors
* pass instance to provisioning steps
* pass instance to deprovisioning steps
* make bind and unbind accept instances as arguments
* get defensive about possible unanticipated modifications to instances by module code
* pass instance to updating steps
* minor lint fixes
* refactor lifecycle tests
* Firewall rule parameterization (#152)
* Refactor to support allowing firewall rules.
Fixes: #146
Changes to suport optional parameters to allow specifying start
and end IP address for firewall rules on MySQL, PostgreSQL and
MSSQL. Previously, this defaulted to allow any connection. This
isn't ecure in practice, so allowing the customer to specify desired
rules. Also changed the default to 0.0.0.0 for both, which results in
only Azure internal addresses. Added some parameter validation
for the firewall values as well:
* Valid IPV4 addresses
* Start <= End
Extracted a method to build the arm template parameter map. Providing
empty strings is invalid for the ARM template, so only including
when non-empty.
Added parameters to the lifecycle tests to allow running them
from local machines (will need ao enhance the charts in helm-charts)
Added some tests.
* Fixing merge conflicts
* Test cleanup
* json field cleanup
* Resolving code review comments
* error message cleanup
* error message cleanup
* specify firewall rules for relational databased in example manifests
* Few minor corrections to the Quickstart (#136)
* Correcting the az ad sp delete command
* Update quickstart-minikube.md
Remove a leading $ from a few bash commands. copy paste on mac is messed up :-)
* Update quickstart-minikube.md
Clarification around logging in to WordPress
* restoring prompts in quickstart
* minor punctuation fix
* removing distracting echo command
* make prompts used consistently
* kick of redis tests sooner (#155)
* s/module lifecycle tests/service lifecycle tests/g (#156)
* make storage know how to deal with service-specific types (#153)
* don't use codec where we don't need it anymore (#161)
* Updating CircleCI to define DOCKER_REPO env var (#167)
* Updating CircleCI to define REGISTRY env var
We previously set the REGISTRY environment variable in a deploy script.
When that was removed to streamline the release process, we need to
define it in Circle. Added two environment blocks to the publish-rc-images
and publish-release-images jobs.
Fixes: 145
* Qualified registry with docker.io
* Changed REGISTRY to DOCKER_REPO
* s/context/details/g (#163)
* make fake bind/unbind functionality not rely on standard context
* collapse standard provisioning context onto instance
* do away with (sort of) redundant standard provisioning parameters
* s/provisioning context/instance details/
* s/binding context/binding details/
* fix error handling when standard params aren't expected types
* add comment that adds some clarity to instance comparisons
* don't store credentials as their own field (#169)
* quickstart fixes (#174)
* adding missing docker pushes to release process
* Adding build flags to inject the version and commit (#173)
* Adding build flags to inject the version and commit
This commit injects the version and commit to the OSBA binary. If there
is no version set, then sets the version to “devel”
* Using new variable for version
* Removing redundant setting for main.commit
* Printing version and commit on broker startup
* Adding fields to the log message on startup
* Removing build flags on the CLI builds
* broker doesn't need to hang on to codec (#180)
* remove quotes from devel pseudo-version string (#184)
Merging without CI.
* fix bug decoding tags (#183)
* move version info to avoid import cycle later (#185)
* Adding user-agent string to requests to ARM (#172)
* Adding user-agent string to requests to ARM
* get version for user agent string from version package
* use separate redis clients for storage and async engine (#182)
* Adding ability to filter lifecycle tests (#181)
* Adding ability to filter lifecycle tests
This adds the ability to filter lifecycle tests using an environment
variable (TEST_MODULES). Via this variable, the developer can provide
a comma delimited list of module names and only the test cases matching
that module will be enabled. If the variable is empty, all tests will run.
If an invalid module name is provided, no tests will run.
* fix typo and lint error
* remove unnecessary funtion
* use redis namespaces (#188)
* use redis namespaces for instances and bindings
* use redis namespaces in async engine
* validate alias and parentAlias (#191)
* Update lifecycle tests to run mssql db only (#192)
* Update lifecycle tests to run mssql db only
1.) Small refactor to test_case_test.go to break provision into a
reusable function
2.) Modified setup function signature to include a *service.Instance
to use as a parent instance
3.) Added setup function in mssql tests
4.) Added lifecycle test for DB only
5.) Fixed db only privision bug
Implemnts #124
* Lint fixes
* test refactor to support "child tests"
* Small rename
* Test description update
* handle storage aspects of instance aliases (#195)
* merge latest from master (#199)
* refactor event function signatures (#147)
* pass around instances and bindings instead of pointers
* Refactor instance encryption/decryption
Previously, encryption and decryption of certain instance
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to instance
objects, I wanted a way for modules to be able to extract
information from encrypted instance fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors instances and the storage layer so that
encryption occurs during instance marshaling and decryption occurs
during instance unmarshaling.
* Refactor binding encryption/decryption
Previously, encryption and decryption of certain binding
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to binding
objects, I wanted a way for modules to be able to extract
information from encrypted binding fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors bindings and the storage layer so that
encryption occurs during bindings marshaling and decryption occurs
during bindings unmarshaling.
* clean up a few errant lint errors
* pass instance to provisioning steps
* pass instance to deprovisioning steps
* make bind and unbind accept instances as arguments
* get defensive about possible unanticipated modifications to instances by module code
* pass instance to updating steps
* minor lint fixes
* refactor lifecycle tests
* Firewall rule parameterization (#152)
* Refactor to support allowing firewall rules.
Fixes: #146
Changes to suport optional parameters to allow specifying start
and end IP address for firewall rules on MySQL, PostgreSQL and
MSSQL. Previously, this defaulted to allow any connection. This
isn't ecure in practice, so allowing the customer to specify desired
rules. Also changed the default to 0.0.0.0 for both, which results in
only Azure internal addresses. Added some parameter validation
for the firewall values as well:
* Valid IPV4 addresses
* Start <= End
Extracted a method to build the arm template parameter map. Providing
empty strings is invalid for the ARM template, so only including
when non-empty.
Added parameters to the lifecycle tests to allow running them
from local machines (will need ao enhance the charts in helm-charts)
Added some tests.
* Fixing merge conflicts
* Test cleanup
* json field cleanup
* Resolving code review comments
* error message cleanup
* error message cleanup
* specify firewall rules for relational databased in example manifests
* Few minor corrections to the Quickstart (#136)
* Correcting the az ad sp delete command
* Update quickstart-minikube.md
Remove a leading $ from a few bash commands. copy paste on mac is messed up :-)
* Update quickstart-minikube.md
Clarification around logging in to WordPress
* restoring prompts in quickstart
* minor punctuation fix
* removing distracting echo command
* make prompts used consistently
* kick of redis tests sooner (#155)
* s/module lifecycle tests/service lifecycle tests/g (#156)
* make storage know how to deal with service-specific types (#153)
* don't use codec where we don't need it anymore (#161)
* Updating CircleCI to define DOCKER_REPO env var (#167)
* Updating CircleCI to define REGISTRY env var
We previously set the REGISTRY environment variable in a deploy script.
When that was removed to streamline the release process, we need to
define it in Circle. Added two environment blocks to the publish-rc-images
and publish-release-images jobs.
Fixes: 145
* Qualified registry with docker.io
* Changed REGISTRY to DOCKER_REPO
* s/context/details/g (#163)
* make fake bind/unbind functionality not rely on standard context
* collapse standard provisioning context onto instance
* do away with (sort of) redundant standard provisioning parameters
* s/provisioning context/instance details/
* s/binding context/binding details/
* fix error handling when standard params aren't expected types
* add comment that adds some clarity to instance comparisons
* don't store credentials as their own field (#169)
* quickstart fixes (#174)
* adding missing docker pushes to release process
* Adding build flags to inject the version and commit (#173)
* Adding build flags to inject the version and commit
This commit injects the version and commit to the OSBA binary. If there
is no version set, then sets the version to “devel”
* Using new variable for version
* Removing redundant setting for main.commit
* Printing version and commit on broker startup
* Adding fields to the log message on startup
* Removing build flags on the CLI builds
* broker doesn't need to hang on to codec (#180)
* remove quotes from devel pseudo-version string (#184)
Merging without CI.
* fix bug decoding tags (#183)
* move version info to avoid import cycle later (#185)
* Adding user-agent string to requests to ARM (#172)
* Adding user-agent string to requests to ARM
* get version for user agent string from version package
* use separate redis clients for storage and async engine (#182)
* Adding ability to filter lifecycle tests (#181)
* Adding ability to filter lifecycle tests
This adds the ability to filter lifecycle tests using an environment
variable (TEST_MODULES). Via this variable, the developer can provide
a comma delimited list of module names and only the test cases matching
that module will be enabled. If the variable is empty, all tests will run.
If an invalid module name is provided, no tests will run.
* fix typo and lint error
* remove unnecessary funtion
* use redis namespaces (#188)
* use redis namespaces for instances and bindings
* use redis namespaces in async engine
* start using lightweight-docker-go image for development tasks (#190)
* simplify ci config (#194)
* populate parent when retrieving instance (#200)
* parent child count (#201)
* fix arg name
* break up storage test cases better
* track number of children belonging to an instance
* Merge master into feature branch (#209)
* refactor event function signatures (#147)
* pass around instances and bindings instead of pointers
* Refactor instance encryption/decryption
Previously, encryption and decryption of certain instance
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to instance
objects, I wanted a way for modules to be able to extract
information from encrypted instance fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors instances and the storage layer so that
encryption occurs during instance marshaling and decryption occurs
during instance unmarshaling.
* Refactor binding encryption/decryption
Previously, encryption and decryption of certain binding
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to binding
objects, I wanted a way for modules to be able to extract
information from encrypted binding fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors bindings and the storage layer so that
encryption occurs during bindings marshaling and decryption occurs
during bindings unmarshaling.
* clean up a few errant lint errors
* pass instance to provisioning steps
* pass instance to deprovisioning steps
* make bind and unbind accept instances as arguments
* get defensive about possible unanticipated modifications to instances by module code
* pass instance to updating steps
* minor lint fixes
* refactor lifecycle tests
* Firewall rule parameterization (#152)
* Refactor to support allowing firewall rules.
Fixes: #146
Changes to suport optional parameters to allow specifying start
and end IP address for firewall rules on MySQL, PostgreSQL and
MSSQL. Previously, this defaulted to allow any connection. This
isn't ecure in practice, so allowing the customer to specify desired
rules. Also changed the default to 0.0.0.0 for both, which results in
only Azure internal addresses. Added some parameter validation
for the firewall values as well:
* Valid IPV4 addresses
* Start <= End
Extracted a method to build the arm template parameter map. Providing
empty strings is invalid for the ARM template, so only including
when non-empty.
Added parameters to the lifecycle tests to allow running them
from local machines (will need ao enhance the charts in helm-charts)
Added some tests.
* Fixing merge conflicts
* Test cleanup
* json field cleanup
* Resolving code review comments
* error message cleanup
* error message cleanup
* specify firewall rules for relational databased in example manifests
* Few minor corrections to the Quickstart (#136)
* Correcting the az ad sp delete command
* Update quickstart-minikube.md
Remove a leading $ from a few bash commands. copy paste on mac is messed up :-)
* Update quickstart-minikube.md
Clarification around logging in to WordPress
* restoring prompts in quickstart
* minor punctuation fix
* removing distracting echo command
* make prompts used consistently
* kick of redis tests sooner (#155)
* s/module lifecycle tests/service lifecycle tests/g (#156)
* make storage know how to deal with service-specific types (#153)
* don't use codec where we don't need it anymore (#161)
* Updating CircleCI to define DOCKER_REPO env var (#167)
* Updating CircleCI to define REGISTRY env var
We previously set the REGISTRY environment variable in a deploy script.
When that was removed to streamline the release process, we need to
define it in Circle. Added two environment blocks to the publish-rc-images
and publish-release-images jobs.
Fixes: 145
* Qualified registry with docker.io
* Changed REGISTRY to DOCKER_REPO
* s/context/details/g (#163)
* make fake bind/unbind functionality not rely on standard context
* collapse standard provisioning context onto instance
* do away with (sort of) redundant standard provisioning parameters
* s/provisioning context/instance details/
* s/binding context/binding details/
* fix error handling when standard params aren't expected types
* add comment that adds some clarity to instance comparisons
* don't store credentials as their own field (#169)
* quickstart fixes (#174)
* adding missing docker pushes to release process
* Adding build flags to inject the version and commit (#173)
* Adding build flags to inject the version and commit
This commit injects the version and commit to the OSBA binary. If there
is no version set, then sets the version to “devel”
* Using new variable for version
* Removing redundant setting for main.commit
* Printing version and commit on broker startup
* Adding fields to the log message on startup
* Removing build flags on the CLI builds
* broker doesn't need to hang on to codec (#180)
* remove quotes from devel pseudo-version string (#184)
Merging without CI.
* fix bug decoding tags (#183)
* move version info to avoid import cycle later (#185)
* Adding user-agent string to requests to ARM (#172)
* Adding user-agent string to requests to ARM
* get version for user agent string from version package
* use separate redis clients for storage and async engine (#182)
* Adding ability to filter lifecycle tests (#181)
* Adding ability to filter lifecycle tests
This adds the ability to filter lifecycle tests using an environment
variable (TEST_MODULES). Via this variable, the developer can provide
a comma delimited list of module names and only the test cases matching
that module will be enabled. If the variable is empty, all tests will run.
If an invalid module name is provided, no tests will run.
* fix typo and lint error
* remove unnecessary funtion
* use redis namespaces (#188)
* use redis namespaces for instances and bindings
* use redis namespaces in async engine
* start using lightweight-docker-go image for development tasks (#190)
* simplify ci config (#194)
* README work (#196)
* README formatting + one cmdline fix
* Link fixes
* Revert heading to title case
* append instead of replacing user-agent (#198)
* append instead of replacing user-agent
* fix lint errors
* add deferred task execution to async package (#208)
achieving this involved a significant refactor
of the entire package. the good news is that the
tests are improved beyond what we had before,
so i think we can be more confident in the
refactored package than the original.
* Provision and Deprovision enhancements to use new async features (#204)
* Enhancement of asynch engine
This PR enhances the asynch engine to enable the
concenpt of delayed or paused tasks. Tasks can be
added to the engine in a queue separate from the main
work queue. As delayed tasks are added, a resumer is
notified and will periodically start the tasks up. Delayed
tasks can also be started by engine clients.
Using this, pkg/api/provision and deprovision were enhanced to
determine if a provision or deprovision action needs to wait for
another operation to finish. If so, they submit delayed tasks, either
wait_for_parent or wait_for_children, whichever is appropriate. Those
new jobs will make the same determination, and either readd themselves
as delayed tasks, or initiate the task that would otherwise originally
have been created.
* Added bad parent response message
* Merge master into feature branch (#209)
* refactor event function signatures (#147)
* pass around instances and bindings instead of pointers
* Refactor instance encryption/decryption
Previously, encryption and decryption of certain instance
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to instance
objects, I wanted a way for modules to be able to extract
information from encrypted instance fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors instances and the storage layer so that
encryption occurs during instance marshaling and decryption occurs
during instance unmarshaling.
* Refactor binding encryption/decryption
Previously, encryption and decryption of certain binding
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to binding
objects, I wanted a way for modules to be able to extract
information from encrypted binding fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors bindings and the storage layer so that
encryption occurs during bindings marshaling and decryption occurs
during bindings unmarshaling.
* clean up a few errant lint errors
* pass instance to provisioning steps
* pass instance to deprovisioning steps
* make bind and unbind accept instances as arguments
* get defensive about possible unanticipated modifications to instances by module code
* pass instance to updating steps
* minor lint fixes
* refactor lifecycle tests
* Firewall rule parameterization (#152)
* Refactor to support allowing firewall rules.
Fixes: #146
Changes to suport optional parameters to allow specifying start
and end IP address for firewall rules on MySQL, PostgreSQL and
MSSQL. Previously, this defaulted to allow any connection. This
isn't ecure in practice, so allowing the customer to specify desired
rules. Also changed the default to 0.0.0.0 for both, which results in
only Azure internal addresses. Added some parameter validation
for the firewall values as well:
* Valid IPV4 addresses
* Start <= End
Extracted a method to build the arm template parameter map. Providing
empty strings is invalid for the ARM template, so only including
when non-empty.
Added parameters to the lifecycle tests to allow running them
from local machines (will need ao enhance the charts in helm-charts)
Added some tests.
* Fixing merge conflicts
* Test cleanup
* json field cleanup
* Resolving code review comments
* error message cleanup
* error message cleanup
* specify firewall rules for relational databased in example manifests
* Few minor corrections to the Quickstart (#136)
* Correcting the az ad sp delete command
* Update quickstart-minikube.md
Remove a leading $ from a few bash commands. copy paste on mac is messed up :-)
* Update quickstart-minikube.md
Clarification around logging in to WordPress
* restoring prompts in quickstart
* minor punctuation fix
* removing distracting echo command
* make prompts used consistently
* kick of redis tests sooner (#155)
* s/module lifecycle tests/service lifecycle tests/g (#156)
* make storage know how to deal with service-specific types (#153)
* don't use codec where we don't need it anymore (#161)
* Updating CircleCI to define DOCKER_REPO env var (#167)
* Updating CircleCI to define REGISTRY env var
We previously set the REGISTRY environment variable in a deploy script.
When that was removed to streamline the release process, we need to
define it in Circle. Added two environment blocks to the publish-rc-images
and publish-release-images jobs.
Fixes: 145
* Qualified registry with docker.io
* Changed REGISTRY to DOCKER_REPO
* s/context/details/g (#163)
* make fake bind/unbind functionality not rely on standard context
* collapse standard provisioning context onto instance
* do away with (sort of) redundant standard provisioning parameters
* s/provisioning context/instance details/
* s/binding context/binding details/
* fix error handling when standard params aren't expected types
* add comment that adds some clarity to instance comparisons
* don't store credentials as their own field (#169)
* quickstart fixes (#174)
* adding missing docker pushes to release process
* Adding build flags to inject the version and commit (#173)
* Adding build flags to inject the version and commit
This commit injects the version and commit to the OSBA binary. If there
is no version set, then sets the version to “devel”
* Using new variable for version
* Removing redundant setting for main.commit
* Printing version and commit on broker startup
* Adding fields to the log message on startup
* Removing build flags on the CLI builds
* broker doesn't need to hang on to codec (#180)
* remove quotes from devel pseudo-version string (#184)
Merging without CI.
* fix bug decoding tags (#183)
* move version info to avoid import cycle later (#185)
* Adding user-agent string to requests to ARM (#172)
* Adding user-agent string to requests to ARM
* get version for user agent string from version package
* use separate redis clients for storage and async engine (#182)
* Adding ability to filter lifecycle tests (#181)
* Adding ability to filter lifecycle tests
This adds the ability to filter lifecycle tests using an environment
variable (TEST_MODULES). Via this variable, the developer can provide
a comma delimited list of module names and only the test cases matching
that module will be enabled. If the variable is empty, all tests will run.
If an invalid module name is provided, no tests will run.
* fix typo and lint error
* remove unnecessary funtion
* use redis namespaces (#188)
* use redis namespaces for instances and bindings
* use redis namespaces in async engine
* start using lightweight-docker-go image for development tasks (#190)
* simplify ci config (#194)
* README work (#196)
* README formatting + one cmdline fix
* Link fixes
* Revert heading to title case
* append instead of replacing user-agent (#198)
* append instead of replacing user-agent
* fix lint errors
* add deferred task execution to async package (#208)
achieving this involved a significant refactor
of the entire package. the good news is that the
tests are improved beyond what we had before,
so i think we can be more confident in the
refactored package than the original.
* Enhancement of asynch engine
This PR enhances the asynch engine to enable the
concenpt of delayed or paused tasks. Tasks can be
added to the engine in a queue separate from the main
work queue. As delayed tasks are added, a resumer is
notified and will periodically start the tasks up. Delayed
tasks can also be started by engine clients.
Using this, pkg/api/provision and deprovision were enhanced to
determine if a provision or deprovision action needs to wait for
another operation to finish. If so, they submit delayed tasks, either
wait_for_parent or wait_for_children, whichever is appropriate. Those
new jobs will make the same determination, and either readd themselves
as delayed tasks, or initiate the task that would otherwise originally
have been created.
* Remove resumer changes
* removed references to submitdelayed task
* Fixed debug statements that got clobbered in merge
* Debug message
* Fixed response code
* Whitespace
* Code review comments
* Code review comments
* Refactor based on code-review comments.
* Code review comments
* Lint fixes
* Enhanced contrib/k8s/examples (#213)
* Enhanced contrib/k8s/examples to illustrate the use of the new
SQLDB catalog
* Spelling errors
* merge latest from master to poc branch (#232)
* refactor event function signatures (#147)
* pass around instances and bindings instead of pointers
* Refactor instance encryption/decryption
Previously, encryption and decryption of certain instance
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to instance
objects, I wanted a way for modules to be able to extract
information from encrypted instance fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors instances and the storage layer so that
encryption occurs during instance marshaling and decryption occurs
during instance unmarshaling.
* Refactor binding encryption/decryption
Previously, encryption and decryption of certain binding
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to binding
objects, I wanted a way for modules to be able to extract
information from encrypted binding fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors bindings and the storage layer so that
encryption occurs during bindings marshaling and decryption occurs
during bindings unmarshaling.
* clean up a few errant lint errors
* pass instance to provisioning steps
* pass instance to deprovisioning steps
* make bind and unbind accept instances as arguments
* get defensive about possible unanticipated modifications to instances by module code
* pass instance to updating steps
* minor lint fixes
* refactor lifecycle tests
* Firewall rule parameterization (#152)
* Refactor to support allowing firewall rules.
Fixes: #146
Changes to suport optional parameters to allow specifying start
and end IP address for firewall rules on MySQL, PostgreSQL and
MSSQL. Previously, this defaulted to allow any connection. This
isn't ecure in practice, so allowing the customer to specify desired
rules. Also changed the default to 0.0.0.0 for both, which results in
only Azure internal addresses. Added some parameter validation
for the firewall values as well:
* Valid IPV4 addresses
* Start <= End
Extracted a method to build the arm template parameter map. Providing
empty strings is invalid for the ARM template, so only including
when non-empty.
Added parameters to the lifecycle tests to allow running them
from local machines (will need ao enhance the charts in helm-charts)
Added some tests.
* Fixing merge conflicts
* Test cleanup
* json field cleanup
* Resolving code review comments
* error message cleanup
* error message cleanup
* specify firewall rules for relational databased in example manifests
* Few minor corrections to the Quickstart (#136)
* Correcting the az ad sp delete command
* Update quickstart-minikube.md
Remove a leading $ from a few bash commands. copy paste on mac is messed up :-)
* Update quickstart-minikube.md
Clarification around logging in to WordPress
* restoring prompts in quickstart
* minor punctuation fix
* removing distracting echo command
* make prompts used consistently
* kick of redis tests sooner (#155)
* s/module lifecycle tests/service lifecycle tests/g (#156)
* make storage know how to deal with service-specific types (#153)
* don't use codec where we don't need it anymore (#161)
* Updating CircleCI to define DOCKER_REPO env var (#167)
* Updating CircleCI to define REGISTRY env var
We previously set the REGISTRY environment variable in a deploy script.
When that was removed to streamline the release process, we need to
define it in Circle. Added two environment blocks to the publish-rc-images
and publish-release-images jobs.
Fixes: 145
* Qualified registry with docker.io
* Changed REGISTRY to DOCKER_REPO
* s/context/details/g (#163)
* make fake bind/unbind functionality not rely on standard context
* collapse standard provisioning context onto instance
* do away with (sort of) redundant standard provisioning parameters
* s/provisioning context/instance details/
* s/binding context/binding details/
* fix error handling when standard params aren't expected types
* add comment that adds some clarity to instance comparisons
* don't store credentials as their own field (#169)
* quickstart fixes (#174)
* adding missing docker pushes to release process
* Adding build flags to inject the version and commit (#173)
* Adding build flags to inject the version and commit
This commit injects the version and commit to the OSBA binary. If there
is no version set, then sets the version to “devel”
* Using new variable for version
* Removing redundant setting for main.commit
* Printing version and commit on broker startup
* Adding fields to the log message on startup
* Removing build flags on the CLI builds
* broker doesn't need to hang on to codec (#180)
* remove quotes from devel pseudo-version string (#184)
Merging without CI.
* fix bug decoding tags (#183)
* move version info to avoid import cycle later (#185)
* Adding user-agent string to requests to ARM (#172)
* Adding user-agent string to requests to ARM
* get version for user agent string from version package
* use separate redis clients for storage and async engine (#182)
* Adding ability to filter lifecycle tests (#181)
* Adding ability to filter lifecycle tests
This adds the ability to filter lifecycle tests using an environment
variable (TEST_MODULES). Via this variable, the developer can provide
a comma delimited list of module names and only the test cases matching
that module will be enabled. If the variable is empty, all tests will run.
If an invalid module name is provided, no tests will run.
* fix typo and lint error
* remove unnecessary funtion
* use redis namespaces (#188)
* use redis namespaces for instances and bindings
* use redis namespaces in async engine
* start using lightweight-docker-go image for development tasks (#190)
* simplify ci config (#194)
* README work (#196)
* README formatting + one cmdline fix
* Link fixes
* Revert heading to title case
* append instead of replacing user-agent (#198)
* append instead of replacing user-agent
* fix lint errors
* add deferred task execution to async package (#208)
achieving this involved a significant refactor
of the entire package. the good news is that the
tests are improved beyond what we had before,
so i think we can be more confident in the
refactored package than the original.
* reorganize async pakage/sub-packages more sanely (#210)
* change provision/deprovision task names (#211)
The are imperative and more accurate.
* Add async support for "follow-up" tasks (#212)
* make async job functions take task as argument
* make async job functions return tasks
* bonus: fix more task names
* make async task executor submit follow-up tasks
* make broker async processes leverage new async follow-up tasks feature
* ignore .vscode/ (#221)
* more async refactoring (#219)
* break up worker.go and worker_test.go into multiple files
* simplify heart
* simplify cleaner
* move heart from worker to engine
* remove receive/execute/watch from worker to engine
* let task receivers return right after fatal error
* let task executor return right after fatal error
* very minor comment cleanup
* fix lint errors by removing unused error types
* add svc and plan to instance during retrieval (#220)
* retrieve svc when instance is retrieved
* retrieve plan when instance is retrieved
* improve provision step function signature
* improve deprovision step function signature
* improve update step function signature
* lifecycles tests: set service and plan on instance
* Changes to mysql bind. (#222)
Change grant string in mysql bind to be bound to database
Removed global privs
Added Execute
Fixes #197
* Initial roadmap submission (#226)
* Initial Roadmap
* Publish initial roadmap for OSBA
Closes #109
* Spelling
* Review comments and wordsmithing.
* Refactor for generic filter capabilities (#218)
* WIP: Proposal to refactor for generic filter capabilities
1.) Create new api/filter structure
2.) Refactored authenticator stuff there
3.) Created new interfaces: filters.Filter and filters.Chain
4.) Made basic/always authenticator implement filters.Filter interface
* Fixed filename type-o.
* Fixed filename type-o
* Renamed Filter method to Execute.
* Added test for filter chain
* Refactor and style fixes
* ignore .vscode/ (#221)
* more async refactoring (#219)
* break up worker.go and worker_test.go into multiple files
* simplify heart
* simplify cleaner
* move heart from worker to engine
* remove receive/execute/watch from worker to engine
* let task receivers return right after fatal error
* let task executor return right after fatal error
* very minor comment cleanup
* fix lint errors by removing unused error types
* add svc and plan to instance during retrieval (#220)
* retrieve svc when instance is retrieved
* retrieve plan when instance is retrieved
* improve provision step function signature
* improve deprovision step function signature
* improve update step function signature
* lifecycles tests: set service and plan on instance
* changed filters package to filter
* Lint failure
* Refactor filter package a little and re-add Chain interface
* a few filter tweaks
* Change how the database name for cosmos db is generated (#228)
* Change the generation of the database name. A limitation in the current
ARM based CosmosDB provisoning limits the length of the database name to
50 characters, including the name provided, the location name and a dash.
In some situations, this means a UUID is invalid. If the length will be
too much, this patch will randomly generate a string of the available length
instead of using a UUID.
* Refactored to use generate package
* fix async cleaner (#233)
* bug fix: fix wait again on children not deprovisioned
* bug fix: it's ok for an instance's parent to not be found
* Code comments
jeremyrickard
added a commit
to jeremyrickard/open-service-broker-azure
that referenced
this pull request
Feb 6, 2018
* WIP Refactoring sqldb to POC (#142)
* Refactoring sqldb to POC #124
Removing the "existing server" logic from the current sqldb
module. This entails removing it from the provision/deprovision
file and removing it from the types. Also removed the config.go
file and updated the broker to not load that. We will need another
way to provide/register existing servers.
We will reimplement this behavior in a new service/service manager.
* fix minor lint failure
* Feature branch merge (#154)
* refactor event function signatures (#147)
* pass around instances and bindings instead of pointers
* Refactor instance encryption/decryption
Previously, encryption and decryption of certain instance
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to instance
objects, I wanted a way for modules to be able to extract
information from encrypted instance fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors instances and the storage layer so that
encryption occurs during instance marshaling and decryption occurs
during instance unmarshaling.
* Refactor binding encryption/decryption
Previously, encryption and decryption of certain binding
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to binding
objects, I wanted a way for modules to be able to extract
information from encrypted binding fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors bindings and the storage layer so that
encryption occurs during bindings marshaling and decryption occurs
during bindings unmarshaling.
* clean up a few errant lint errors
* pass instance to provisioning steps
* pass instance to deprovisioning steps
* make bind and unbind accept instances as arguments
* get defensive about possible unanticipated modifications to instances by module code
* pass instance to updating steps
* minor lint fixes
* refactor lifecycle tests
* Firewall rule parameterization (#152)
* Refactor to support allowing firewall rules.
Fixes: #146
Changes to suport optional parameters to allow specifying start
and end IP address for firewall rules on MySQL, PostgreSQL and
MSSQL. Previously, this defaulted to allow any connection. This
isn't ecure in practice, so allowing the customer to specify desired
rules. Also changed the default to 0.0.0.0 for both, which results in
only Azure internal addresses. Added some parameter validation
for the firewall values as well:
* Valid IPV4 addresses
* Start <= End
Extracted a method to build the arm template parameter map. Providing
empty strings is invalid for the ARM template, so only including
when non-empty.
Added parameters to the lifecycle tests to allow running them
from local machines (will need ao enhance the charts in helm-charts)
Added some tests.
* Fixing merge conflicts
* Test cleanup
* json field cleanup
* Resolving code review comments
* error message cleanup
* error message cleanup
* specify firewall rules for relational databased in example manifests
* Sqldb catalog poc merge master (#158)
* refactor event function signatures (#147)
* pass around instances and bindings instead of pointers
* Refactor instance encryption/decryption
Previously, encryption and decryption of certain instance
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to instance
objects, I wanted a way for modules to be able to extract
information from encrypted instance fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors instances and the storage layer so that
encryption occurs during instance marshaling and decryption occurs
during instance unmarshaling.
* Refactor binding encryption/decryption
Previously, encryption and decryption of certain binding
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to binding
objects, I wanted a way for modules to be able to extract
information from encrypted binding fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors bindings and the storage layer so that
encryption occurs during bindings marshaling and decryption occurs
during bindings unmarshaling.
* clean up a few errant lint errors
* pass instance to provisioning steps
* pass instance to deprovisioning steps
* make bind and unbind accept instances as arguments
* get defensive about possible unanticipated modifications to instances by module code
* pass instance to updating steps
* minor lint fixes
* refactor lifecycle tests
* Firewall rule parameterization (#152)
* Refactor to support allowing firewall rules.
Fixes: #146
Changes to suport optional parameters to allow specifying start
and end IP address for firewall rules on MySQL, PostgreSQL and
MSSQL. Previously, this defaulted to allow any connection. This
isn't ecure in practice, so allowing the customer to specify desired
rules. Also changed the default to 0.0.0.0 for both, which results in
only Azure internal addresses. Added some parameter validation
for the firewall values as well:
* Valid IPV4 addresses
* Start <= End
Extracted a method to build the arm template parameter map. Providing
empty strings is invalid for the ARM template, so only including
when non-empty.
Added parameters to the lifecycle tests to allow running them
from local machines (will need ao enhance the charts in helm-charts)
Added some tests.
* Fixing merge conflicts
* Test cleanup
* json field cleanup
* Resolving code review comments
* error message cleanup
* error message cleanup
* specify firewall rules for relational databased in example manifests
* Few minor corrections to the Quickstart (#136)
* Correcting the az ad sp delete command
* Update quickstart-minikube.md
Remove a leading $ from a few bash commands. copy paste on mac is messed up :-)
* Update quickstart-minikube.md
Clarification around logging in to WordPress
* restoring prompts in quickstart
* minor punctuation fix
* removing distracting echo command
* make prompts used consistently
* kick of redis tests sooner (#155)
* s/module lifecycle tests/service lifecycle tests/g (#156)
* make storage know how to deal with service-specific types (#153)
* make provision and deprovision steps receive a reference instance (#160)
* Merge master to feature (#175)
* refactor event function signatures (#147)
* pass around instances and bindings instead of pointers
* Refactor instance encryption/decryption
Previously, encryption and decryption of certain instance
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to instance
objects, I wanted a way for modules to be able to extract
information from encrypted instance fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors instances and the storage layer so that
encryption occurs during instance marshaling and decryption occurs
during instance unmarshaling.
* Refactor binding encryption/decryption
Previously, encryption and decryption of certain binding
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to binding
objects, I wanted a way for modules to be able to extract
information from encrypted binding fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors bindings and the storage layer so that
encryption occurs during bindings marshaling and decryption occurs
during bindings unmarshaling.
* clean up a few errant lint errors
* pass instance to provisioning steps
* pass instance to deprovisioning steps
* make bind and unbind accept instances as arguments
* get defensive about possible unanticipated modifications to instances by module code
* pass instance to updating steps
* minor lint fixes
* refactor lifecycle tests
* Firewall rule parameterization (#152)
* Refactor to support allowing firewall rules.
Fixes: #146
Changes to suport optional parameters to allow specifying start
and end IP address for firewall rules on MySQL, PostgreSQL and
MSSQL. Previously, this defaulted to allow any connection. This
isn't ecure in practice, so allowing the customer to specify desired
rules. Also changed the default to 0.0.0.0 for both, which results in
only Azure internal addresses. Added some parameter validation
for the firewall values as well:
* Valid IPV4 addresses
* Start <= End
Extracted a method to build the arm template parameter map. Providing
empty strings is invalid for the ARM template, so only including
when non-empty.
Added parameters to the lifecycle tests to allow running them
from local machines (will need ao enhance the charts in helm-charts)
Added some tests.
* Fixing merge conflicts
* Test cleanup
* json field cleanup
* Resolving code review comments
* error message cleanup
* error message cleanup
* specify firewall rules for relational databased in example manifests
* Few minor corrections to the Quickstart (#136)
* Correcting the az ad sp delete command
* Update quickstart-minikube.md
Remove a leading $ from a few bash commands. copy paste on mac is messed up :-)
* Update quickstart-minikube.md
Clarification around logging in to WordPress
* restoring prompts in quickstart
* minor punctuation fix
* removing distracting echo command
* make prompts used consistently
* kick of redis tests sooner (#155)
* s/module lifecycle tests/service lifecycle tests/g (#156)
* make storage know how to deal with service-specific types (#153)
* don't use codec where we don't need it anymore (#161)
* Updating CircleCI to define DOCKER_REPO env var (#167)
* Updating CircleCI to define REGISTRY env var
We previously set the REGISTRY environment variable in a deploy script.
When that was removed to streamline the release process, we need to
define it in Circle. Added two environment blocks to the publish-rc-images
and publish-release-images jobs.
Fixes: 145
* Qualified registry with docker.io
* Changed REGISTRY to DOCKER_REPO
* s/context/details/g (#163)
* make fake bind/unbind functionality not rely on standard context
* collapse standard provisioning context onto instance
* do away with (sort of) redundant standard provisioning parameters
* s/provisioning context/instance details/
* s/binding context/binding details/
* fix error handling when standard params aren't expected types
* add comment that adds some clarity to instance comparisons
* don't store credentials as their own field (#169)
* quickstart fixes (#174)
* adding missing docker pushes to release process
* minor merge nit fixes
* Decomposition of single service into multiple services (#176)
* Decomposition of single service into multiple services
1.) This renames the existing service manager into a service to
implement the all in one scenario (existing)
2.) Next, two new services are added (vm only and db only)
3.) Each lifecycle operation go file (bind, deprovision, provision, unbind)
now declares a method for each service manager struct
4.) Added new services to the catalog
5.) DB only is largely TBD implemented, took a crack at
starting the provision.
6.) Refactored catalog/service interface to add GetBindable() method
7.) Modified tests to only execute binding related tests if plan
is bindable
* Small spelling correction
* Implemented database only lifecycle methods and lint fix.
* DRY refactoring
* Review comments and lint fix
* add ref instance as instance attribute (#178)
* create a relationship between two services (#179)
* Refactor sqldb module to use parent instances in binding (#186)
* Refactor sqldb module to use parent instances in binding
Small refactor to decouple shared instance details and leverage
parent instance for binding/unbinding.
Further evolution for #124
* Updated error to be consistent.
* merge latest from master (#187)
* refactor event function signatures (#147)
* pass around instances and bindings instead of pointers
* Refactor instance encryption/decryption
Previously, encryption and decryption of certain instance
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to instance
objects, I wanted a way for modules to be able to extract
information from encrypted instance fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors instances and the storage layer so that
encryption occurs during instance marshaling and decryption occurs
during instance unmarshaling.
* Refactor binding encryption/decryption
Previously, encryption and decryption of certain binding
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to binding
objects, I wanted a way for modules to be able to extract
information from encrypted binding fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors bindings and the storage layer so that
encryption occurs during bindings marshaling and decryption occurs
during bindings unmarshaling.
* clean up a few errant lint errors
* pass instance to provisioning steps
* pass instance to deprovisioning steps
* make bind and unbind accept instances as arguments
* get defensive about possible unanticipated modifications to instances by module code
* pass instance to updating steps
* minor lint fixes
* refactor lifecycle tests
* Firewall rule parameterization (#152)
* Refactor to support allowing firewall rules.
Fixes: #146
Changes to suport optional parameters to allow specifying start
and end IP address for firewall rules on MySQL, PostgreSQL and
MSSQL. Previously, this defaulted to allow any connection. This
isn't ecure in practice, so allowing the customer to specify desired
rules. Also changed the default to 0.0.0.0 for both, which results in
only Azure internal addresses. Added some parameter validation
for the firewall values as well:
* Valid IPV4 addresses
* Start <= End
Extracted a method to build the arm template parameter map. Providing
empty strings is invalid for the ARM template, so only including
when non-empty.
Added parameters to the lifecycle tests to allow running them
from local machines (will need ao enhance the charts in helm-charts)
Added some tests.
* Fixing merge conflicts
* Test cleanup
* json field cleanup
* Resolving code review comments
* error message cleanup
* error message cleanup
* specify firewall rules for relational databased in example manifests
* Few minor corrections to the Quickstart (#136)
* Correcting the az ad sp delete command
* Update quickstart-minikube.md
Remove a leading $ from a few bash commands. copy paste on mac is messed up :-)
* Update quickstart-minikube.md
Clarification around logging in to WordPress
* restoring prompts in quickstart
* minor punctuation fix
* removing distracting echo command
* make prompts used consistently
* kick of redis tests sooner (#155)
* s/module lifecycle tests/service lifecycle tests/g (#156)
* make storage know how to deal with service-specific types (#153)
* don't use codec where we don't need it anymore (#161)
* Updating CircleCI to define DOCKER_REPO env var (#167)
* Updating CircleCI to define REGISTRY env var
We previously set the REGISTRY environment variable in a deploy script.
When that was removed to streamline the release process, we need to
define it in Circle. Added two environment blocks to the publish-rc-images
and publish-release-images jobs.
Fixes: 145
* Qualified registry with docker.io
* Changed REGISTRY to DOCKER_REPO
* s/context/details/g (#163)
* make fake bind/unbind functionality not rely on standard context
* collapse standard provisioning context onto instance
* do away with (sort of) redundant standard provisioning parameters
* s/provisioning context/instance details/
* s/binding context/binding details/
* fix error handling when standard params aren't expected types
* add comment that adds some clarity to instance comparisons
* don't store credentials as their own field (#169)
* quickstart fixes (#174)
* adding missing docker pushes to release process
* Adding build flags to inject the version and commit (#173)
* Adding build flags to inject the version and commit
This commit injects the version and commit to the OSBA binary. If there
is no version set, then sets the version to “devel”
* Using new variable for version
* Removing redundant setting for main.commit
* Printing version and commit on broker startup
* Adding fields to the log message on startup
* Removing build flags on the CLI builds
* broker doesn't need to hang on to codec (#180)
* remove quotes from devel pseudo-version string (#184)
Merging without CI.
* fix bug decoding tags (#183)
* move version info to avoid import cycle later (#185)
* Adding user-agent string to requests to ARM (#172)
* Adding user-agent string to requests to ARM
* get version for user agent string from version package
* use separate redis clients for storage and async engine (#182)
* Adding ability to filter lifecycle tests (#181)
* Adding ability to filter lifecycle tests
This adds the ability to filter lifecycle tests using an environment
variable (TEST_MODULES). Via this variable, the developer can provide
a comma delimited list of module names and only the test cases matching
that module will be enabled. If the variable is empty, all tests will run.
If an invalid module name is provided, no tests will run.
* fix typo and lint error
* remove unnecessary funtion
* Merge master (#189)
* refactor event function signatures (#147)
* pass around instances and bindings instead of pointers
* Refactor instance encryption/decryption
Previously, encryption and decryption of certain instance
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to instance
objects, I wanted a way for modules to be able to extract
information from encrypted instance fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors instances and the storage layer so that
encryption occurs during instance marshaling and decryption occurs
during instance unmarshaling.
* Refactor binding encryption/decryption
Previously, encryption and decryption of certain binding
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to binding
objects, I wanted a way for modules to be able to extract
information from encrypted binding fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors bindings and the storage layer so that
encryption occurs during bindings marshaling and decryption occurs
during bindings unmarshaling.
* clean up a few errant lint errors
* pass instance to provisioning steps
* pass instance to deprovisioning steps
* make bind and unbind accept instances as arguments
* get defensive about possible unanticipated modifications to instances by module code
* pass instance to updating steps
* minor lint fixes
* refactor lifecycle tests
* Firewall rule parameterization (#152)
* Refactor to support allowing firewall rules.
Fixes: #146
Changes to suport optional parameters to allow specifying start
and end IP address for firewall rules on MySQL, PostgreSQL and
MSSQL. Previously, this defaulted to allow any connection. This
isn't ecure in practice, so allowing the customer to specify desired
rules. Also changed the default to 0.0.0.0 for both, which results in
only Azure internal addresses. Added some parameter validation
for the firewall values as well:
* Valid IPV4 addresses
* Start <= End
Extracted a method to build the arm template parameter map. Providing
empty strings is invalid for the ARM template, so only including
when non-empty.
Added parameters to the lifecycle tests to allow running them
from local machines (will need ao enhance the charts in helm-charts)
Added some tests.
* Fixing merge conflicts
* Test cleanup
* json field cleanup
* Resolving code review comments
* error message cleanup
* error message cleanup
* specify firewall rules for relational databased in example manifests
* Few minor corrections to the Quickstart (#136)
* Correcting the az ad sp delete command
* Update quickstart-minikube.md
Remove a leading $ from a few bash commands. copy paste on mac is messed up :-)
* Update quickstart-minikube.md
Clarification around logging in to WordPress
* restoring prompts in quickstart
* minor punctuation fix
* removing distracting echo command
* make prompts used consistently
* kick of redis tests sooner (#155)
* s/module lifecycle tests/service lifecycle tests/g (#156)
* make storage know how to deal with service-specific types (#153)
* don't use codec where we don't need it anymore (#161)
* Updating CircleCI to define DOCKER_REPO env var (#167)
* Updating CircleCI to define REGISTRY env var
We previously set the REGISTRY environment variable in a deploy script.
When that was removed to streamline the release process, we need to
define it in Circle. Added two environment blocks to the publish-rc-images
and publish-release-images jobs.
Fixes: 145
* Qualified registry with docker.io
* Changed REGISTRY to DOCKER_REPO
* s/context/details/g (#163)
* make fake bind/unbind functionality not rely on standard context
* collapse standard provisioning context onto instance
* do away with (sort of) redundant standard provisioning parameters
* s/provisioning context/instance details/
* s/binding context/binding details/
* fix error handling when standard params aren't expected types
* add comment that adds some clarity to instance comparisons
* don't store credentials as their own field (#169)
* quickstart fixes (#174)
* adding missing docker pushes to release process
* Adding build flags to inject the version and commit (#173)
* Adding build flags to inject the version and commit
This commit injects the version and commit to the OSBA binary. If there
is no version set, then sets the version to “devel”
* Using new variable for version
* Removing redundant setting for main.commit
* Printing version and commit on broker startup
* Adding fields to the log message on startup
* Removing build flags on the CLI builds
* broker doesn't need to hang on to codec (#180)
* remove quotes from devel pseudo-version string (#184)
Merging without CI.
* fix bug decoding tags (#183)
* move version info to avoid import cycle later (#185)
* Adding user-agent string to requests to ARM (#172)
* Adding user-agent string to requests to ARM
* get version for user agent string from version package
* use separate redis clients for storage and async engine (#182)
* Adding ability to filter lifecycle tests (#181)
* Adding ability to filter lifecycle tests
This adds the ability to filter lifecycle tests using an environment
variable (TEST_MODULES). Via this variable, the developer can provide
a comma delimited list of module names and only the test cases matching
that module will be enabled. If the variable is empty, all tests will run.
If an invalid module name is provided, no tests will run.
* fix typo and lint error
* remove unnecessary funtion
* use redis namespaces (#188)
* use redis namespaces for instances and bindings
* use redis namespaces in async engine
* validate alias and parentAlias (#191)
* Update lifecycle tests to run mssql db only (#192)
* Update lifecycle tests to run mssql db only
1.) Small refactor to test_case_test.go to break provision into a
reusable function
2.) Modified setup function signature to include a *service.Instance
to use as a parent instance
3.) Added setup function in mssql tests
4.) Added lifecycle test for DB only
5.) Fixed db only privision bug
Implemnts #124
* Lint fixes
* test refactor to support "child tests"
* Small rename
* Test description update
* handle storage aspects of instance aliases (#195)
* merge latest from master (#199)
* refactor event function signatures (#147)
* pass around instances and bindings instead of pointers
* Refactor instance encryption/decryption
Previously, encryption and decryption of certain instance
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to instance
objects, I wanted a way for modules to be able to extract
information from encrypted instance fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors instances and the storage layer so that
encryption occurs during instance marshaling and decryption occurs
during instance unmarshaling.
* Refactor binding encryption/decryption
Previously, encryption and decryption of certain binding
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to binding
objects, I wanted a way for modules to be able to extract
information from encrypted binding fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors bindings and the storage layer so that
encryption occurs during bindings marshaling and decryption occurs
during bindings unmarshaling.
* clean up a few errant lint errors
* pass instance to provisioning steps
* pass instance to deprovisioning steps
* make bind and unbind accept instances as arguments
* get defensive about possible unanticipated modifications to instances by module code
* pass instance to updating steps
* minor lint fixes
* refactor lifecycle tests
* Firewall rule parameterization (#152)
* Refactor to support allowing firewall rules.
Fixes: #146
Changes to suport optional parameters to allow specifying start
and end IP address for firewall rules on MySQL, PostgreSQL and
MSSQL. Previously, this defaulted to allow any connection. This
isn't ecure in practice, so allowing the customer to specify desired
rules. Also changed the default to 0.0.0.0 for both, which results in
only Azure internal addresses. Added some parameter validation
for the firewall values as well:
* Valid IPV4 addresses
* Start <= End
Extracted a method to build the arm template parameter map. Providing
empty strings is invalid for the ARM template, so only including
when non-empty.
Added parameters to the lifecycle tests to allow running them
from local machines (will need ao enhance the charts in helm-charts)
Added some tests.
* Fixing merge conflicts
* Test cleanup
* json field cleanup
* Resolving code review comments
* error message cleanup
* error message cleanup
* specify firewall rules for relational databased in example manifests
* Few minor corrections to the Quickstart (#136)
* Correcting the az ad sp delete command
* Update quickstart-minikube.md
Remove a leading $ from a few bash commands. copy paste on mac is messed up :-)
* Update quickstart-minikube.md
Clarification around logging in to WordPress
* restoring prompts in quickstart
* minor punctuation fix
* removing distracting echo command
* make prompts used consistently
* kick of redis tests sooner (#155)
* s/module lifecycle tests/service lifecycle tests/g (#156)
* make storage know how to deal with service-specific types (#153)
* don't use codec where we don't need it anymore (#161)
* Updating CircleCI to define DOCKER_REPO env var (#167)
* Updating CircleCI to define REGISTRY env var
We previously set the REGISTRY environment variable in a deploy script.
When that was removed to streamline the release process, we need to
define it in Circle. Added two environment blocks to the publish-rc-images
and publish-release-images jobs.
Fixes: 145
* Qualified registry with docker.io
* Changed REGISTRY to DOCKER_REPO
* s/context/details/g (#163)
* make fake bind/unbind functionality not rely on standard context
* collapse standard provisioning context onto instance
* do away with (sort of) redundant standard provisioning parameters
* s/provisioning context/instance details/
* s/binding context/binding details/
* fix error handling when standard params aren't expected types
* add comment that adds some clarity to instance comparisons
* don't store credentials as their own field (#169)
* quickstart fixes (#174)
* adding missing docker pushes to release process
* Adding build flags to inject the version and commit (#173)
* Adding build flags to inject the version and commit
This commit injects the version and commit to the OSBA binary. If there
is no version set, then sets the version to “devel”
* Using new variable for version
* Removing redundant setting for main.commit
* Printing version and commit on broker startup
* Adding fields to the log message on startup
* Removing build flags on the CLI builds
* broker doesn't need to hang on to codec (#180)
* remove quotes from devel pseudo-version string (#184)
Merging without CI.
* fix bug decoding tags (#183)
* move version info to avoid import cycle later (#185)
* Adding user-agent string to requests to ARM (#172)
* Adding user-agent string to requests to ARM
* get version for user agent string from version package
* use separate redis clients for storage and async engine (#182)
* Adding ability to filter lifecycle tests (#181)
* Adding ability to filter lifecycle tests
This adds the ability to filter lifecycle tests using an environment
variable (TEST_MODULES). Via this variable, the developer can provide
a comma delimited list of module names and only the test cases matching
that module will be enabled. If the variable is empty, all tests will run.
If an invalid module name is provided, no tests will run.
* fix typo and lint error
* remove unnecessary funtion
* use redis namespaces (#188)
* use redis namespaces for instances and bindings
* use redis namespaces in async engine
* start using lightweight-docker-go image for development tasks (#190)
* simplify ci config (#194)
* populate parent when retrieving instance (#200)
* parent child count (#201)
* fix arg name
* break up storage test cases better
* track number of children belonging to an instance
* Merge master into feature branch (#209)
* refactor event function signatures (#147)
* pass around instances and bindings instead of pointers
* Refactor instance encryption/decryption
Previously, encryption and decryption of certain instance
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to instance
objects, I wanted a way for modules to be able to extract
information from encrypted instance fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors instances and the storage layer so that
encryption occurs during instance marshaling and decryption occurs
during instance unmarshaling.
* Refactor binding encryption/decryption
Previously, encryption and decryption of certain binding
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to binding
objects, I wanted a way for modules to be able to extract
information from encrypted binding fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors bindings and the storage layer so that
encryption occurs during bindings marshaling and decryption occurs
during bindings unmarshaling.
* clean up a few errant lint errors
* pass instance to provisioning steps
* pass instance to deprovisioning steps
* make bind and unbind accept instances as arguments
* get defensive about possible unanticipated modifications to instances by module code
* pass instance to updating steps
* minor lint fixes
* refactor lifecycle tests
* Firewall rule parameterization (#152)
* Refactor to support allowing firewall rules.
Fixes: #146
Changes to suport optional parameters to allow specifying start
and end IP address for firewall rules on MySQL, PostgreSQL and
MSSQL. Previously, this defaulted to allow any connection. This
isn't ecure in practice, so allowing the customer to specify desired
rules. Also changed the default to 0.0.0.0 for both, which results in
only Azure internal addresses. Added some parameter validation
for the firewall values as well:
* Valid IPV4 addresses
* Start <= End
Extracted a method to build the arm template parameter map. Providing
empty strings is invalid for the ARM template, so only including
when non-empty.
Added parameters to the lifecycle tests to allow running them
from local machines (will need ao enhance the charts in helm-charts)
Added some tests.
* Fixing merge conflicts
* Test cleanup
* json field cleanup
* Resolving code review comments
* error message cleanup
* error message cleanup
* specify firewall rules for relational databased in example manifests
* Few minor corrections to the Quickstart (#136)
* Correcting the az ad sp delete command
* Update quickstart-minikube.md
Remove a leading $ from a few bash commands. copy paste on mac is messed up :-)
* Update quickstart-minikube.md
Clarification around logging in to WordPress
* restoring prompts in quickstart
* minor punctuation fix
* removing distracting echo command
* make prompts used consistently
* kick of redis tests sooner (#155)
* s/module lifecycle tests/service lifecycle tests/g (#156)
* make storage know how to deal with service-specific types (#153)
* don't use codec where we don't need it anymore (#161)
* Updating CircleCI to define DOCKER_REPO env var (#167)
* Updating CircleCI to define REGISTRY env var
We previously set the REGISTRY environment variable in a deploy script.
When that was removed to streamline the release process, we need to
define it in Circle. Added two environment blocks to the publish-rc-images
and publish-release-images jobs.
Fixes: 145
* Qualified registry with docker.io
* Changed REGISTRY to DOCKER_REPO
* s/context/details/g (#163)
* make fake bind/unbind functionality not rely on standard context
* collapse standard provisioning context onto instance
* do away with (sort of) redundant standard provisioning parameters
* s/provisioning context/instance details/
* s/binding context/binding details/
* fix error handling when standard params aren't expected types
* add comment that adds some clarity to instance comparisons
* don't store credentials as their own field (#169)
* quickstart fixes (#174)
* adding missing docker pushes to release process
* Adding build flags to inject the version and commit (#173)
* Adding build flags to inject the version and commit
This commit injects the version and commit to the OSBA binary. If there
is no version set, then sets the version to “devel”
* Using new variable for version
* Removing redundant setting for main.commit
* Printing version and commit on broker startup
* Adding fields to the log message on startup
* Removing build flags on the CLI builds
* broker doesn't need to hang on to codec (#180)
* remove quotes from devel pseudo-version string (#184)
Merging without CI.
* fix bug decoding tags (#183)
* move version info to avoid import cycle later (#185)
* Adding user-agent string to requests to ARM (#172)
* Adding user-agent string to requests to ARM
* get version for user agent string from version package
* use separate redis clients for storage and async engine (#182)
* Adding ability to filter lifecycle tests (#181)
* Adding ability to filter lifecycle tests
This adds the ability to filter lifecycle tests using an environment
variable (TEST_MODULES). Via this variable, the developer can provide
a comma delimited list of module names and only the test cases matching
that module will be enabled. If the variable is empty, all tests will run.
If an invalid module name is provided, no tests will run.
* fix typo and lint error
* remove unnecessary funtion
* use redis namespaces (#188)
* use redis namespaces for instances and bindings
* use redis namespaces in async engine
* start using lightweight-docker-go image for development tasks (#190)
* simplify ci config (#194)
* README work (#196)
* README formatting + one cmdline fix
* Link fixes
* Revert heading to title case
* append instead of replacing user-agent (#198)
* append instead of replacing user-agent
* fix lint errors
* add deferred task execution to async package (#208)
achieving this involved a significant refactor
of the entire package. the good news is that the
tests are improved beyond what we had before,
so i think we can be more confident in the
refactored package than the original.
* Provision and Deprovision enhancements to use new async features (#204)
* Enhancement of asynch engine
This PR enhances the asynch engine to enable the
concenpt of delayed or paused tasks. Tasks can be
added to the engine in a queue separate from the main
work queue. As delayed tasks are added, a resumer is
notified and will periodically start the tasks up. Delayed
tasks can also be started by engine clients.
Using this, pkg/api/provision and deprovision were enhanced to
determine if a provision or deprovision action needs to wait for
another operation to finish. If so, they submit delayed tasks, either
wait_for_parent or wait_for_children, whichever is appropriate. Those
new jobs will make the same determination, and either readd themselves
as delayed tasks, or initiate the task that would otherwise originally
have been created.
* Added bad parent response message
* Merge master into feature branch (#209)
* refactor event function signatures (#147)
* pass around instances and bindings instead of pointers
* Refactor instance encryption/decryption
Previously, encryption and decryption of certain instance
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to instance
objects, I wanted a way for modules to be able to extract
information from encrypted instance fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors instances and the storage layer so that
encryption occurs during instance marshaling and decryption occurs
during instance unmarshaling.
* Refactor binding encryption/decryption
Previously, encryption and decryption of certain binding
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to binding
objects, I wanted a way for modules to be able to extract
information from encrypted binding fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors bindings and the storage layer so that
encryption occurs during bindings marshaling and decryption occurs
during bindings unmarshaling.
* clean up a few errant lint errors
* pass instance to provisioning steps
* pass instance to deprovisioning steps
* make bind and unbind accept instances as arguments
* get defensive about possible unanticipated modifications to instances by module code
* pass instance to updating steps
* minor lint fixes
* refactor lifecycle tests
* Firewall rule parameterization (#152)
* Refactor to support allowing firewall rules.
Fixes: #146
Changes to suport optional parameters to allow specifying start
and end IP address for firewall rules on MySQL, PostgreSQL and
MSSQL. Previously, this defaulted to allow any connection. This
isn't ecure in practice, so allowing the customer to specify desired
rules. Also changed the default to 0.0.0.0 for both, which results in
only Azure internal addresses. Added some parameter validation
for the firewall values as well:
* Valid IPV4 addresses
* Start <= End
Extracted a method to build the arm template parameter map. Providing
empty strings is invalid for the ARM template, so only including
when non-empty.
Added parameters to the lifecycle tests to allow running them
from local machines (will need ao enhance the charts in helm-charts)
Added some tests.
* Fixing merge conflicts
* Test cleanup
* json field cleanup
* Resolving code review comments
* error message cleanup
* error message cleanup
* specify firewall rules for relational databased in example manifests
* Few minor corrections to the Quickstart (#136)
* Correcting the az ad sp delete command
* Update quickstart-minikube.md
Remove a leading $ from a few bash commands. copy paste on mac is messed up :-)
* Update quickstart-minikube.md
Clarification around logging in to WordPress
* restoring prompts in quickstart
* minor punctuation fix
* removing distracting echo command
* make prompts used consistently
* kick of redis tests sooner (#155)
* s/module lifecycle tests/service lifecycle tests/g (#156)
* make storage know how to deal with service-specific types (#153)
* don't use codec where we don't need it anymore (#161)
* Updating CircleCI to define DOCKER_REPO env var (#167)
* Updating CircleCI to define REGISTRY env var
We previously set the REGISTRY environment variable in a deploy script.
When that was removed to streamline the release process, we need to
define it in Circle. Added two environment blocks to the publish-rc-images
and publish-release-images jobs.
Fixes: 145
* Qualified registry with docker.io
* Changed REGISTRY to DOCKER_REPO
* s/context/details/g (#163)
* make fake bind/unbind functionality not rely on standard context
* collapse standard provisioning context onto instance
* do away with (sort of) redundant standard provisioning parameters
* s/provisioning context/instance details/
* s/binding context/binding details/
* fix error handling when standard params aren't expected types
* add comment that adds some clarity to instance comparisons
* don't store credentials as their own field (#169)
* quickstart fixes (#174)
* adding missing docker pushes to release process
* Adding build flags to inject the version and commit (#173)
* Adding build flags to inject the version and commit
This commit injects the version and commit to the OSBA binary. If there
is no version set, then sets the version to “devel”
* Using new variable for version
* Removing redundant setting for main.commit
* Printing version and commit on broker startup
* Adding fields to the log message on startup
* Removing build flags on the CLI builds
* broker doesn't need to hang on to codec (#180)
* remove quotes from devel pseudo-version string (#184)
Merging without CI.
* fix bug decoding tags (#183)
* move version info to avoid import cycle later (#185)
* Adding user-agent string to requests to ARM (#172)
* Adding user-agent string to requests to ARM
* get version for user agent string from version package
* use separate redis clients for storage and async engine (#182)
* Adding ability to filter lifecycle tests (#181)
* Adding ability to filter lifecycle tests
This adds the ability to filter lifecycle tests using an environment
variable (TEST_MODULES). Via this variable, the developer can provide
a comma delimited list of module names and only the test cases matching
that module will be enabled. If the variable is empty, all tests will run.
If an invalid module name is provided, no tests will run.
* fix typo and lint error
* remove unnecessary funtion
* use redis namespaces (#188)
* use redis namespaces for instances and bindings
* use redis namespaces in async engine
* start using lightweight-docker-go image for development tasks (#190)
* simplify ci config (#194)
* README work (#196)
* README formatting + one cmdline fix
* Link fixes
* Revert heading to title case
* append instead of replacing user-agent (#198)
* append instead of replacing user-agent
* fix lint errors
* add deferred task execution to async package (#208)
achieving this involved a significant refactor
of the entire package. the good news is that the
tests are improved beyond what we had before,
so i think we can be more confident in the
refactored package than the original.
* Enhancement of asynch engine
This PR enhances the asynch engine to enable the
concenpt of delayed or paused tasks. Tasks can be
added to the engine in a queue separate from the main
work queue. As delayed tasks are added, a resumer is
notified and will periodically start the tasks up. Delayed
tasks can also be started by engine clients.
Using this, pkg/api/provision and deprovision were enhanced to
determine if a provision or deprovision action needs to wait for
another operation to finish. If so, they submit delayed tasks, either
wait_for_parent or wait_for_children, whichever is appropriate. Those
new jobs will make the same determination, and either readd themselves
as delayed tasks, or initiate the task that would otherwise originally
have been created.
* Remove resumer changes
* removed references to submitdelayed task
* Fixed debug statements that got clobbered in merge
* Debug message
* Fixed response code
* Whitespace
* Code review comments
* Code review comments
* Refactor based on code-review comments.
* Code review comments
* Lint fixes
* Enhanced contrib/k8s/examples (#213)
* Enhanced contrib/k8s/examples to illustrate the use of the new
SQLDB catalog
* Spelling errors
* merge latest from master to poc branch (#232)
* refactor event function signatures (#147)
* pass around instances and bindings instead of pointers
* Refactor instance encryption/decryption
Previously, encryption and decryption of certain instance
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to instance
objects, I wanted a way for modules to be able to extract
information from encrypted instance fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors instances and the storage layer so that
encryption occurs during instance marshaling and decryption occurs
during instance unmarshaling.
* Refactor binding encryption/decryption
Previously, encryption and decryption of certain binding
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to binding
objects, I wanted a way for modules to be able to extract
information from encrypted binding fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors bindings and the storage layer so that
encryption occurs during bindings marshaling and decryption occurs
during bindings unmarshaling.
* clean up a few errant lint errors
* pass instance to provisioning steps
* pass instance to deprovisioning steps
* make bind and unbind accept instances as arguments
* get defensive about possible unanticipated modifications to instances by module code
* pass instance to updating steps
* minor lint fixes
* refactor lifecycle tests
* Firewall rule parameterization (#152)
* Refactor to support allowing firewall rules.
Fixes: #146
Changes to suport optional parameters to allow specifying start
and end IP address for firewall rules on MySQL, PostgreSQL and
MSSQL. Previously, this defaulted to allow any connection. This
isn't ecure in practice, so allowing the customer to specify desired
rules. Also changed the default to 0.0.0.0 for both, which results in
only Azure internal addresses. Added some parameter validation
for the firewall values as well:
* Valid IPV4 addresses
* Start <= End
Extracted a method to build the arm template parameter map. Providing
empty strings is invalid for the ARM template, so only including
when non-empty.
Added parameters to the lifecycle tests to allow running them
from local machines (will need ao enhance the charts in helm-charts)
Added some tests.
* Fixing merge conflicts
* Test cleanup
* json field cleanup
* Resolving code review comments
* error message cleanup
* error message cleanup
* specify firewall rules for relational databased in example manifests
* Few minor corrections to the Quickstart (#136)
* Correcting the az ad sp delete command
* Update quickstart-minikube.md
Remove a leading $ from a few bash commands. copy paste on mac is messed up :-)
* Update quickstart-minikube.md
Clarification around logging in to WordPress
* restoring prompts in quickstart
* minor punctuation fix
* removing distracting echo command
* make prompts used consistently
* kick of redis tests sooner (#155)
* s/module lifecycle tests/service lifecycle tests/g (#156)
* make storage know how to deal with service-specific types (#153)
* don't use codec where we don't need it anymore (#161)
* Updating CircleCI to define DOCKER_REPO env var (#167)
* Updating CircleCI to define REGISTRY env var
We previously set the REGISTRY environment variable in a deploy script.
When that was removed to streamline the release process, we need to
define it in Circle. Added two environment blocks to the publish-rc-images
and publish-release-images jobs.
Fixes: 145
* Qualified registry with docker.io
* Changed REGISTRY to DOCKER_REPO
* s/context/details/g (#163)
* make fake bind/unbind functionality not rely on standard context
* collapse standard provisioning context onto instance
* do away with (sort of) redundant standard provisioning parameters
* s/provisioning context/instance details/
* s/binding context/binding details/
* fix error handling when standard params aren't expected types
* add comment that adds some clarity to instance comparisons
* don't store credentials as their own field (#169)
* quickstart fixes (#174)
* adding missing docker pushes to release process
* Adding build flags to inject the version and commit (#173)
* Adding build flags to inject the version and commit
This commit injects the version and commit to the OSBA binary. If there
is no version set, then sets the version to “devel”
* Using new variable for version
* Removing redundant setting for main.commit
* Printing version and commit on broker startup
* Adding fields to the log message on startup
* Removing build flags on the CLI builds
* broker doesn't need to hang on to codec (#180)
* remove quotes from devel pseudo-version string (#184)
Merging without CI.
* fix bug decoding tags (#183)
* move version info to avoid import cycle later (#185)
* Adding user-agent string to requests to ARM (#172)
* Adding user-agent string to requests to ARM
* get version for user agent string from version package
* use separate redis clients for storage and async engine (#182)
* Adding ability to filter lifecycle tests (#181)
* Adding ability to filter lifecycle tests
This adds the ability to filter lifecycle tests using an environment
variable (TEST_MODULES). Via this variable, the developer can provide
a comma delimited list of module names and only the test cases matching
that module will be enabled. If the variable is empty, all tests will run.
If an invalid module name is provided, no tests will run.
* fix typo and lint error
* remove unnecessary funtion
* use redis namespaces (#188)
* use redis namespaces for instances and bindings
* use redis namespaces in async engine
* start using lightweight-docker-go image for development tasks (#190)
* simplify ci config (#194)
* README work (#196)
* README formatting + one cmdline fix
* Link fixes
* Revert heading to title case
* append instead of replacing user-agent (#198)
* append instead of replacing user-agent
* fix lint errors
* add deferred task execution to async package (#208)
achieving this involved a significant refactor
of the entire package. the good news is that the
tests are improved beyond what we had before,
so i think we can be more confident in the
refactored package than the original.
* reorganize async pakage/sub-packages more sanely (#210)
* change provision/deprovision task names (#211)
The are imperative and more accurate.
* Add async support for "follow-up" tasks (#212)
* make async job functions take task as argument
* make async job functions return tasks
* bonus: fix more task names
* make async task executor submit follow-up tasks
* make broker async processes leverage new async follow-up tasks feature
* ignore .vscode/ (#221)
* more async refactoring (#219)
* break up worker.go and worker_test.go into multiple files
* simplify heart
* simplify cleaner
* move heart from worker to engine
* remove receive/execute/watch from worker to engine
* let task receivers return right after fatal error
* let task executor return right after fatal error
* very minor comment cleanup
* fix lint errors by removing unused error types
* add svc and plan to instance during retrieval (#220)
* retrieve svc when instance is retrieved
* retrieve plan when instance is retrieved
* improve provision step function signature
* improve deprovision step function signature
* improve update step function signature
* lifecycles tests: set service and plan on instance
* Changes to mysql bind. (#222)
Change grant string in mysql bind to be bound to database
Removed global privs
Added Execute
Fixes #197
* Initial roadmap submission (#226)
* Initial Roadmap
* Publish initial roadmap for OSBA
Closes #109
* Spelling
* Review comments and wordsmithing.
* Refactor for generic filter capabilities (#218)
* WIP: Proposal to refactor for generic filter capabilities
1.) Create new api/filter structure
2.) Refactored authenticator stuff there
3.) Created new interfaces: filters.Filter and filters.Chain
4.) Made basic/always authenticator implement filters.Filter interface
* Fixed filename type-o.
* Fixed filename type-o
* Renamed Filter method to Execute.
* Added test for filter chain
* Refactor and style fixes
* ignore .vscode/ (#221)
* more async refactoring (#219)
* break up worker.go and worker_test.go into multiple files
* simplify heart
* simplify cleaner
* move heart from worker to engine
* remove receive/execute/watch from worker to engine
* let task receivers return right after fatal error
* let task executor return right after fatal error
* very minor comment cleanup
* fix lint errors by removing unused error types
* add svc and plan to instance during retrieval (#220)
* retrieve svc when instance is retrieved
* retrieve plan when instance is retrieved
* improve provision step function signature
* improve deprovision step function signature
* improve update step function signature
* lifecycles tests: set service and plan on instance
* changed filters package to filter
* Lint failure
* Refactor filter package a little and re-add Chain interface
* a few filter tweaks
* Change how the database name for cosmos db is generated (#228)
* Change the generation of the database name. A limitation in the current
ARM based CosmosDB provisoning limits the length of the database name to
50 characters, including the name provided, the location name and a dash.
In some situations, this means a UUID is invalid. If the length will be
too much, this patch will randomly generate a string of the available length
instead of using a UUID.
* Refactored to use generate package
* fix async cleaner (#233)
* bug fix: fix wait again on children not deprovisioned
* bug fix: it's ok for an instance's parent to not be found
* Code comments
krancour
pushed a commit
that referenced
this pull request
Feb 6, 2018
* Replacing empty json responses with error messages.
Issue #66 points out that our error messages don't help determine the
cause of operation failures. This PR adds an informative response to
locations where we are returning a 400 or a 409 so the user has some extra infromation
to debug the failure.
These should be inline with the Broker Errors section of the OSB spec.
* Adding error messages to operation failures
Fixes #66
* Parent child services (#243)
* WIP Refactoring sqldb to POC (#142)
* Refactoring sqldb to POC #124
Removing the "existing server" logic from the current sqldb
module. This entails removing it from the provision/deprovision
file and removing it from the types. Also removed the config.go
file and updated the broker to not load that. We will need another
way to provide/register existing servers.
We will reimplement this behavior in a new service/service manager.
* fix minor lint failure
* Feature branch merge (#154)
* refactor event function signatures (#147)
* pass around instances and bindings instead of pointers
* Refactor instance encryption/decryption
Previously, encryption and decryption of certain instance
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to instance
objects, I wanted a way for modules to be able to extract
information from encrypted instance fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors instances and the storage layer so that
encryption occurs during instance marshaling and decryption occurs
during instance unmarshaling.
* Refactor binding encryption/decryption
Previously, encryption and decryption of certain binding
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to binding
objects, I wanted a way for modules to be able to extract
information from encrypted binding fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors bindings and the storage layer so that
encryption occurs during bindings marshaling and decryption occurs
during bindings unmarshaling.
* clean up a few errant lint errors
* pass instance to provisioning steps
* pass instance to deprovisioning steps
* make bind and unbind accept instances as arguments
* get defensive about possible unanticipated modifications to instances by module code
* pass instance to updating steps
* minor lint fixes
* refactor lifecycle tests
* Firewall rule parameterization (#152)
* Refactor to support allowing firewall rules.
Fixes: #146
Changes to suport optional parameters to allow specifying start
and end IP address for firewall rules on MySQL, PostgreSQL and
MSSQL. Previously, this defaulted to allow any connection. This
isn't ecure in practice, so allowing the customer to specify desired
rules. Also changed the default to 0.0.0.0 for both, which results in
only Azure internal addresses. Added some parameter validation
for the firewall values as well:
* Valid IPV4 addresses
* Start <= End
Extracted a method to build the arm template parameter map. Providing
empty strings is invalid for the ARM template, so only including
when non-empty.
Added parameters to the lifecycle tests to allow running them
from local machines (will need ao enhance the charts in helm-charts)
Added some tests.
* Fixing merge conflicts
* Test cleanup
* json field cleanup
* Resolving code review comments
* error message cleanup
* error message cleanup
* specify firewall rules for relational databased in example manifests
* Sqldb catalog poc merge master (#158)
* refactor event function signatures (#147)
* pass around instances and bindings instead of pointers
* Refactor instance encryption/decryption
Previously, encryption and decryption of certain instance
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to instance
objects, I wanted a way for modules to be able to extract
information from encrypted instance fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors instances and the storage layer so that
encryption occurs during instance marshaling and decryption occurs
during instance unmarshaling.
* Refactor binding encryption/decryption
Previously, encryption and decryption of certain binding
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to binding
objects, I wanted a way for modules to be able to extract
information from encrypted binding fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors bindings and the storage layer so that
encryption occurs during bindings marshaling and decryption occurs
during bindings unmarshaling.
* clean up a few errant lint errors
* pass instance to provisioning steps
* pass instance to deprovisioning steps
* make bind and unbind accept instances as arguments
* get defensive about possible unanticipated modifications to instances by module code
* pass instance to updating steps
* minor lint fixes
* refactor lifecycle tests
* Firewall rule parameterization (#152)
* Refactor to support allowing firewall rules.
Fixes: #146
Changes to suport optional parameters to allow specifying start
and end IP address for firewall rules on MySQL, PostgreSQL and
MSSQL. Previously, this defaulted to allow any connection. This
isn't ecure in practice, so allowing the customer to specify desired
rules. Also changed the default to 0.0.0.0 for both, which results in
only Azure internal addresses. Added some parameter validation
for the firewall values as well:
* Valid IPV4 addresses
* Start <= End
Extracted a method to build the arm template parameter map. Providing
empty strings is invalid for the ARM template, so only including
when non-empty.
Added parameters to the lifecycle tests to allow running them
from local machines (will need ao enhance the charts in helm-charts)
Added some tests.
* Fixing merge conflicts
* Test cleanup
* json field cleanup
* Resolving code review comments
* error message cleanup
* error message cleanup
* specify firewall rules for relational databased in example manifests
* Few minor corrections to the Quickstart (#136)
* Correcting the az ad sp delete command
* Update quickstart-minikube.md
Remove a leading $ from a few bash commands. copy paste on mac is messed up :-)
* Update quickstart-minikube.md
Clarification around logging in to WordPress
* restoring prompts in quickstart
* minor punctuation fix
* removing distracting echo command
* make prompts used consistently
* kick of redis tests sooner (#155)
* s/module lifecycle tests/service lifecycle tests/g (#156)
* make storage know how to deal with service-specific types (#153)
* make provision and deprovision steps receive a reference instance (#160)
* Merge master to feature (#175)
* refactor event function signatures (#147)
* pass around instances and bindings instead of pointers
* Refactor instance encryption/decryption
Previously, encryption and decryption of certain instance
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to instance
objects, I wanted a way for modules to be able to extract
information from encrypted instance fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors instances and the storage layer so that
encryption occurs during instance marshaling and decryption occurs
during instance unmarshaling.
* Refactor binding encryption/decryption
Previously, encryption and decryption of certain binding
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to binding
objects, I wanted a way for modules to be able to extract
information from encrypted binding fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors bindings and the storage layer so that
encryption occurs during bindings marshaling and decryption occurs
during bindings unmarshaling.
* clean up a few errant lint errors
* pass instance to provisioning steps
* pass instance to deprovisioning steps
* make bind and unbind accept instances as arguments
* get defensive about possible unanticipated modifications to instances by module code
* pass instance to updating steps
* minor lint fixes
* refactor lifecycle tests
* Firewall rule parameterization (#152)
* Refactor to support allowing firewall rules.
Fixes: #146
Changes to suport optional parameters to allow specifying start
and end IP address for firewall rules on MySQL, PostgreSQL and
MSSQL. Previously, this defaulted to allow any connection. This
isn't ecure in practice, so allowing the customer to specify desired
rules. Also changed the default to 0.0.0.0 for both, which results in
only Azure internal addresses. Added some parameter validation
for the firewall values as well:
* Valid IPV4 addresses
* Start <= End
Extracted a method to build the arm template parameter map. Providing
empty strings is invalid for the ARM template, so only including
when non-empty.
Added parameters to the lifecycle tests to allow running them
from local machines (will need ao enhance the charts in helm-charts)
Added some tests.
* Fixing merge conflicts
* Test cleanup
* json field cleanup
* Resolving code review comments
* error message cleanup
* error message cleanup
* specify firewall rules for relational databased in example manifests
* Few minor corrections to the Quickstart (#136)
* Correcting the az ad sp delete command
* Update quickstart-minikube.md
Remove a leading $ from a few bash commands. copy paste on mac is messed up :-)
* Update quickstart-minikube.md
Clarification around logging in to WordPress
* restoring prompts in quickstart
* minor punctuation fix
* removing distracting echo command
* make prompts used consistently
* kick of redis tests sooner (#155)
* s/module lifecycle tests/service lifecycle tests/g (#156)
* make storage know how to deal with service-specific types (#153)
* don't use codec where we don't need it anymore (#161)
* Updating CircleCI to define DOCKER_REPO env var (#167)
* Updating CircleCI to define REGISTRY env var
We previously set the REGISTRY environment variable in a deploy script.
When that was removed to streamline the release process, we need to
define it in Circle. Added two environment blocks to the publish-rc-images
and publish-release-images jobs.
Fixes: 145
* Qualified registry with docker.io
* Changed REGISTRY to DOCKER_REPO
* s/context/details/g (#163)
* make fake bind/unbind functionality not rely on standard context
* collapse standard provisioning context onto instance
* do away with (sort of) redundant standard provisioning parameters
* s/provisioning context/instance details/
* s/binding context/binding details/
* fix error handling when standard params aren't expected types
* add comment that adds some clarity to instance comparisons
* don't store credentials as their own field (#169)
* quickstart fixes (#174)
* adding missing docker pushes to release process
* minor merge nit fixes
* Decomposition of single service into multiple services (#176)
* Decomposition of single service into multiple services
1.) This renames the existing service manager into a service to
implement the all in one scenario (existing)
2.) Next, two new services are added (vm only and db only)
3.) Each lifecycle operation go file (bind, deprovision, provision, unbind)
now declares a method for each service manager struct
4.) Added new services to the catalog
5.) DB only is largely TBD implemented, took a crack at
starting the provision.
6.) Refactored catalog/service interface to add GetBindable() method
7.) Modified tests to only execute binding related tests if plan
is bindable
* Small spelling correction
* Implemented database only lifecycle methods and lint fix.
* DRY refactoring
* Review comments and lint fix
* add ref instance as instance attribute (#178)
* create a relationship between two services (#179)
* Refactor sqldb module to use parent instances in binding (#186)
* Refactor sqldb module to use parent instances in binding
Small refactor to decouple shared instance details and leverage
parent instance for binding/unbinding.
Further evolution for #124
* Updated error to be consistent.
* merge latest from master (#187)
* refactor event function signatures (#147)
* pass around instances and bindings instead of pointers
* Refactor instance encryption/decryption
Previously, encryption and decryption of certain instance
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to instance
objects, I wanted a way for modules to be able to extract
information from encrypted instance fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors instances and the storage layer so that
encryption occurs during instance marshaling and decryption occurs
during instance unmarshaling.
* Refactor binding encryption/decryption
Previously, encryption and decryption of certain binding
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to binding
objects, I wanted a way for modules to be able to extract
information from encrypted binding fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors bindings and the storage layer so that
encryption occurs during bindings marshaling and decryption occurs
during bindings unmarshaling.
* clean up a few errant lint errors
* pass instance to provisioning steps
* pass instance to deprovisioning steps
* make bind and unbind accept instances as arguments
* get defensive about possible unanticipated modifications to instances by module code
* pass instance to updating steps
* minor lint fixes
* refactor lifecycle tests
* Firewall rule parameterization (#152)
* Refactor to support allowing firewall rules.
Fixes: #146
Changes to suport optional parameters to allow specifying start
and end IP address for firewall rules on MySQL, PostgreSQL and
MSSQL. Previously, this defaulted to allow any connection. This
isn't ecure in practice, so allowing the customer to specify desired
rules. Also changed the default to 0.0.0.0 for both, which results in
only Azure internal addresses. Added some parameter validation
for the firewall values as well:
* Valid IPV4 addresses
* Start <= End
Extracted a method to build the arm template parameter map. Providing
empty strings is invalid for the ARM template, so only including
when non-empty.
Added parameters to the lifecycle tests to allow running them
from local machines (will need ao enhance the charts in helm-charts)
Added some tests.
* Fixing merge conflicts
* Test cleanup
* json field cleanup
* Resolving code review comments
* error message cleanup
* error message cleanup
* specify firewall rules for relational databased in example manifests
* Few minor corrections to the Quickstart (#136)
* Correcting the az ad sp delete command
* Update quickstart-minikube.md
Remove a leading $ from a few bash commands. copy paste on mac is messed up :-)
* Update quickstart-minikube.md
Clarification around logging in to WordPress
* restoring prompts in quickstart
* minor punctuation fix
* removing distracting echo command
* make prompts used consistently
* kick of redis tests sooner (#155)
* s/module lifecycle tests/service lifecycle tests/g (#156)
* make storage know how to deal with service-specific types (#153)
* don't use codec where we don't need it anymore (#161)
* Updating CircleCI to define DOCKER_REPO env var (#167)
* Updating CircleCI to define REGISTRY env var
We previously set the REGISTRY environment variable in a deploy script.
When that was removed to streamline the release process, we need to
define it in Circle. Added two environment blocks to the publish-rc-images
and publish-release-images jobs.
Fixes: 145
* Qualified registry with docker.io
* Changed REGISTRY to DOCKER_REPO
* s/context/details/g (#163)
* make fake bind/unbind functionality not rely on standard context
* collapse standard provisioning context onto instance
* do away with (sort of) redundant standard provisioning parameters
* s/provisioning context/instance details/
* s/binding context/binding details/
* fix error handling when standard params aren't expected types
* add comment that adds some clarity to instance comparisons
* don't store credentials as their own field (#169)
* quickstart fixes (#174)
* adding missing docker pushes to release process
* Adding build flags to inject the version and commit (#173)
* Adding build flags to inject the version and commit
This commit injects the version and commit to the OSBA binary. If there
is no version set, then sets the version to “devel”
* Using new variable for version
* Removing redundant setting for main.commit
* Printing version and commit on broker startup
* Adding fields to the log message on startup
* Removing build flags on the CLI builds
* broker doesn't need to hang on to codec (#180)
* remove quotes from devel pseudo-version string (#184)
Merging without CI.
* fix bug decoding tags (#183)
* move version info to avoid import cycle later (#185)
* Adding user-agent string to requests to ARM (#172)
* Adding user-agent string to requests to ARM
* get version for user agent string from version package
* use separate redis clients for storage and async engine (#182)
* Adding ability to filter lifecycle tests (#181)
* Adding ability to filter lifecycle tests
This adds the ability to filter lifecycle tests using an environment
variable (TEST_MODULES). Via this variable, the developer can provide
a comma delimited list of module names and only the test cases matching
that module will be enabled. If the variable is empty, all tests will run.
If an invalid module name is provided, no tests will run.
* fix typo and lint error
* remove unnecessary funtion
* Merge master (#189)
* refactor event function signatures (#147)
* pass around instances and bindings instead of pointers
* Refactor instance encryption/decryption
Previously, encryption and decryption of certain instance
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to instance
objects, I wanted a way for modules to be able to extract
information from encrypted instance fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors instances and the storage layer so that
encryption occurs during instance marshaling and decryption occurs
during instance unmarshaling.
* Refactor binding encryption/decryption
Previously, encryption and decryption of certain binding
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to binding
objects, I wanted a way for modules to be able to extract
information from encrypted binding fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors bindings and the storage layer so that
encryption occurs during bindings marshaling and decryption occurs
during bindings unmarshaling.
* clean up a few errant lint errors
* pass instance to provisioning steps
* pass instance to deprovisioning steps
* make bind and unbind accept instances as arguments
* get defensive about possible unanticipated modifications to instances by module code
* pass instance to updating steps
* minor lint fixes
* refactor lifecycle tests
* Firewall rule parameterization (#152)
* Refactor to support allowing firewall rules.
Fixes: #146
Changes to suport optional parameters to allow specifying start
and end IP address for firewall rules on MySQL, PostgreSQL and
MSSQL. Previously, this defaulted to allow any connection. This
isn't ecure in practice, so allowing the customer to specify desired
rules. Also changed the default to 0.0.0.0 for both, which results in
only Azure internal addresses. Added some parameter validation
for the firewall values as well:
* Valid IPV4 addresses
* Start <= End
Extracted a method to build the arm template parameter map. Providing
empty strings is invalid for the ARM template, so only including
when non-empty.
Added parameters to the lifecycle tests to allow running them
from local machines (will need ao enhance the charts in helm-charts)
Added some tests.
* Fixing merge conflicts
* Test cleanup
* json field cleanup
* Resolving code review comments
* error message cleanup
* error message cleanup
* specify firewall rules for relational databased in example manifests
* Few minor corrections to the Quickstart (#136)
* Correcting the az ad sp delete command
* Update quickstart-minikube.md
Remove a leading $ from a few bash commands. copy paste on mac is messed up :-)
* Update quickstart-minikube.md
Clarification around logging in to WordPress
* restoring prompts in quickstart
* minor punctuation fix
* removing distracting echo command
* make prompts used consistently
* kick of redis tests sooner (#155)
* s/module lifecycle tests/service lifecycle tests/g (#156)
* make storage know how to deal with service-specific types (#153)
* don't use codec where we don't need it anymore (#161)
* Updating CircleCI to define DOCKER_REPO env var (#167)
* Updating CircleCI to define REGISTRY env var
We previously set the REGISTRY environment variable in a deploy script.
When that was removed to streamline the release process, we need to
define it in Circle. Added two environment blocks to the publish-rc-images
and publish-release-images jobs.
Fixes: 145
* Qualified registry with docker.io
* Changed REGISTRY to DOCKER_REPO
* s/context/details/g (#163)
* make fake bind/unbind functionality not rely on standard context
* collapse standard provisioning context onto instance
* do away with (sort of) redundant standard provisioning parameters
* s/provisioning context/instance details/
* s/binding context/binding details/
* fix error handling when standard params aren't expected types
* add comment that adds some clarity to instance comparisons
* don't store credentials as their own field (#169)
* quickstart fixes (#174)
* adding missing docker pushes to release process
* Adding build flags to inject the version and commit (#173)
* Adding build flags to inject the version and commit
This commit injects the version and commit to the OSBA binary. If there
is no version set, then sets the version to “devel”
* Using new variable for version
* Removing redundant setting for main.commit
* Printing version and commit on broker startup
* Adding fields to the log message on startup
* Removing build flags on the CLI builds
* broker doesn't need to hang on to codec (#180)
* remove quotes from devel pseudo-version string (#184)
Merging without CI.
* fix bug decoding tags (#183)
* move version info to avoid import cycle later (#185)
* Adding user-agent string to requests to ARM (#172)
* Adding user-agent string to requests to ARM
* get version for user agent string from version package
* use separate redis clients for storage and async engine (#182)
* Adding ability to filter lifecycle tests (#181)
* Adding ability to filter lifecycle tests
This adds the ability to filter lifecycle tests using an environment
variable (TEST_MODULES). Via this variable, the developer can provide
a comma delimited list of module names and only the test cases matching
that module will be enabled. If the variable is empty, all tests will run.
If an invalid module name is provided, no tests will run.
* fix typo and lint error
* remove unnecessary funtion
* use redis namespaces (#188)
* use redis namespaces for instances and bindings
* use redis namespaces in async engine
* validate alias and parentAlias (#191)
* Update lifecycle tests to run mssql db only (#192)
* Update lifecycle tests to run mssql db only
1.) Small refactor to test_case_test.go to break provision into a
reusable function
2.) Modified setup function signature to include a *service.Instance
to use as a parent instance
3.) Added setup function in mssql tests
4.) Added lifecycle test for DB only
5.) Fixed db only privision bug
Implemnts #124
* Lint fixes
* test refactor to support "child tests"
* Small rename
* Test description update
* handle storage aspects of instance aliases (#195)
* merge latest from master (#199)
* refactor event function signatures (#147)
* pass around instances and bindings instead of pointers
* Refactor instance encryption/decryption
Previously, encryption and decryption of certain instance
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to instance
objects, I wanted a way for modules to be able to extract
information from encrypted instance fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors instances and the storage layer so that
encryption occurs during instance marshaling and decryption occurs
during instance unmarshaling.
* Refactor binding encryption/decryption
Previously, encryption and decryption of certain binding
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to binding
objects, I wanted a way for modules to be able to extract
information from encrypted binding fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors bindings and the storage layer so that
encryption occurs during bindings marshaling and decryption occurs
during bindings unmarshaling.
* clean up a few errant lint errors
* pass instance to provisioning steps
* pass instance to deprovisioning steps
* make bind and unbind accept instances as arguments
* get defensive about possible unanticipated modifications to instances by module code
* pass instance to updating steps
* minor lint fixes
* refactor lifecycle tests
* Firewall rule parameterization (#152)
* Refactor to support allowing firewall rules.
Fixes: #146
Changes to suport optional parameters to allow specifying start
and end IP address for firewall rules on MySQL, PostgreSQL and
MSSQL. Previously, this defaulted to allow any connection. This
isn't ecure in practice, so allowing the customer to specify desired
rules. Also changed the default to 0.0.0.0 for both, which results in
only Azure internal addresses. Added some parameter validation
for the firewall values as well:
* Valid IPV4 addresses
* Start <= End
Extracted a method to build the arm template parameter map. Providing
empty strings is invalid for the ARM template, so only including
when non-empty.
Added parameters to the lifecycle tests to allow running them
from local machines (will need ao enhance the charts in helm-charts)
Added some tests.
* Fixing merge conflicts
* Test cleanup
* json field cleanup
* Resolving code review comments
* error message cleanup
* error message cleanup
* specify firewall rules for relational databased in example manifests
* Few minor corrections to the Quickstart (#136)
* Correcting the az ad sp delete command
* Update quickstart-minikube.md
Remove a leading $ from a few bash commands. copy paste on mac is messed up :-)
* Update quickstart-minikube.md
Clarification around logging in to WordPress
* restoring prompts in quickstart
* minor punctuation fix
* removing distracting echo command
* make prompts used consistently
* kick of redis tests sooner (#155)
* s/module lifecycle tests/service lifecycle tests/g (#156)
* make storage know how to deal with service-specific types (#153)
* don't use codec where we don't need it anymore (#161)
* Updating CircleCI to define DOCKER_REPO env var (#167)
* Updating CircleCI to define REGISTRY env var
We previously set the REGISTRY environment variable in a deploy script.
When that was removed to streamline the release process, we need to
define it in Circle. Added two environment blocks to the publish-rc-images
and publish-release-images jobs.
Fixes: 145
* Qualified registry with docker.io
* Changed REGISTRY to DOCKER_REPO
* s/context/details/g (#163)
* make fake bind/unbind functionality not rely on standard context
* collapse standard provisioning context onto instance
* do away with (sort of) redundant standard provisioning parameters
* s/provisioning context/instance details/
* s/binding context/binding details/
* fix error handling when standard params aren't expected types
* add comment that adds some clarity to instance comparisons
* don't store credentials as their own field (#169)
* quickstart fixes (#174)
* adding missing docker pushes to release process
* Adding build flags to inject the version and commit (#173)
* Adding build flags to inject the version and commit
This commit injects the version and commit to the OSBA binary. If there
is no version set, then sets the version to “devel”
* Using new variable for version
* Removing redundant setting for main.commit
* Printing version and commit on broker startup
* Adding fields to the log message on startup
* Removing build flags on the CLI builds
* broker doesn't need to hang on to codec (#180)
* remove quotes from devel pseudo-version string (#184)
Merging without CI.
* fix bug decoding tags (#183)
* move version info to avoid import cycle later (#185)
* Adding user-agent string to requests to ARM (#172)
* Adding user-agent string to requests to ARM
* get version for user agent string from version package
* use separate redis clients for storage and async engine (#182)
* Adding ability to filter lifecycle tests (#181)
* Adding ability to filter lifecycle tests
This adds the ability to filter lifecycle tests using an environment
variable (TEST_MODULES). Via this variable, the developer can provide
a comma delimited list of module names and only the test cases matching
that module will be enabled. If the variable is empty, all tests will run.
If an invalid module name is provided, no tests will run.
* fix typo and lint error
* remove unnecessary funtion
* use redis namespaces (#188)
* use redis namespaces for instances and bindings
* use redis namespaces in async engine
* start using lightweight-docker-go image for development tasks (#190)
* simplify ci config (#194)
* populate parent when retrieving instance (#200)
* parent child count (#201)
* fix arg name
* break up storage test cases better
* track number of children belonging to an instance
* Merge master into feature branch (#209)
* refactor event function signatures (#147)
* pass around instances and bindings instead of pointers
* Refactor instance encryption/decryption
Previously, encryption and decryption of certain instance
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to instance
objects, I wanted a way for modules to be able to extract
information from encrypted instance fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors instances and the storage layer so that
encryption occurs during instance marshaling and decryption occurs
during instance unmarshaling.
* Refactor binding encryption/decryption
Previously, encryption and decryption of certain binding
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to binding
objects, I wanted a way for modules to be able to extract
information from encrypted binding fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors bindings and the storage layer so that
encryption occurs during bindings marshaling and decryption occurs
during bindings unmarshaling.
* clean up a few errant lint errors
* pass instance to provisioning steps
* pass instance to deprovisioning steps
* make bind and unbind accept instances as arguments
* get defensive about possible unanticipated modifications to instances by module code
* pass instance to updating steps
* minor lint fixes
* refactor lifecycle tests
* Firewall rule parameterization (#152)
* Refactor to support allowing firewall rules.
Fixes: #146
Changes to suport optional parameters to allow specifying start
and end IP address for firewall rules on MySQL, PostgreSQL and
MSSQL. Previously, this defaulted to allow any connection. This
isn't ecure in practice, so allowing the customer to specify desired
rules. Also changed the default to 0.0.0.0 for both, which results in
only Azure internal addresses. Added some parameter validation
for the firewall values as well:
* Valid IPV4 addresses
* Start <= End
Extracted a method to build the arm template parameter map. Providing
empty strings is invalid for the ARM template, so only including
when non-empty.
Added parameters to the lifecycle tests to allow running them
from local machines (will need ao enhance the charts in helm-charts)
Added some tests.
* Fixing merge conflicts
* Test cleanup
* json field cleanup
* Resolving code review comments
* error message cleanup
* error message cleanup
* specify firewall rules for relational databased in example manifests
* Few minor corrections to the Quickstart (#136)
* Correcting the az ad sp delete command
* Update quickstart-minikube.md
Remove a leading $ from a few bash commands. copy paste on mac is messed up :-)
* Update quickstart-minikube.md
Clarification around logging in to WordPress
* restoring prompts in quickstart
* minor punctuation fix
* removing distracting echo command
* make prompts used consistently
* kick of redis tests sooner (#155)
* s/module lifecycle tests/service lifecycle tests/g (#156)
* make storage know how to deal with service-specific types (#153)
* don't use codec where we don't need it anymore (#161)
* Updating CircleCI to define DOCKER_REPO env var (#167)
* Updating CircleCI to define REGISTRY env var
We previously set the REGISTRY environment variable in a deploy script.
When that was removed to streamline the release process, we need to
define it in Circle. Added two environment blocks to the publish-rc-images
and publish-release-images jobs.
Fixes: 145
* Qualified registry with docker.io
* Changed REGISTRY to DOCKER_REPO
* s/context/details/g (#163)
* make fake bind/unbind functionality not rely on standard context
* collapse standard provisioning context onto instance
* do away with (sort of) redundant standard provisioning parameters
* s/provisioning context/instance details/
* s/binding context/binding details/
* fix error handling when standard params aren't expected types
* add comment that adds some clarity to instance comparisons
* don't store credentials as their own field (#169)
* quickstart fixes (#174)
* adding missing docker pushes to release process
* Adding build flags to inject the version and commit (#173)
* Adding build flags to inject the version and commit
This commit injects the version and commit to the OSBA binary. If there
is no version set, then sets the version to “devel”
* Using new variable for version
* Removing redundant setting for main.commit
* Printing version and commit on broker startup
* Adding fields to the log message on startup
* Removing build flags on the CLI builds
* broker doesn't need to hang on to codec (#180)
* remove quotes from devel pseudo-version string (#184)
Merging without CI.
* fix bug decoding tags (#183)
* move version info to avoid import cycle later (#185)
* Adding user-agent string to requests to ARM (#172)
* Adding user-agent string to requests to ARM
* get version for user agent string from version package
* use separate redis clients for storage and async engine (#182)
* Adding ability to filter lifecycle tests (#181)
* Adding ability to filter lifecycle tests
This adds the ability to filter lifecycle tests using an environment
variable (TEST_MODULES). Via this variable, the developer can provide
a comma delimited list of module names and only the test cases matching
that module will be enabled. If the variable is empty, all tests will run.
If an invalid module name is provided, no tests will run.
* fix typo and lint error
* remove unnecessary funtion
* use redis namespaces (#188)
* use redis namespaces for instances and bindings
* use redis namespaces in async engine
* start using lightweight-docker-go image for development tasks (#190)
* simplify ci config (#194)
* README work (#196)
* README formatting + one cmdline fix
* Link fixes
* Revert heading to title case
* append instead of replacing user-agent (#198)
* append instead of replacing user-agent
* fix lint errors
* add deferred task execution to async package (#208)
achieving this involved a significant refactor
of the entire package. the good news is that the
tests are improved beyond what we had before,
so i think we can be more confident in the
refactored package than the original.
* Provision and Deprovision enhancements to use new async features (#204)
* Enhancement of asynch engine
This PR enhances the asynch engine to enable the
concenpt of delayed or paused tasks. Tasks can be
added to the engine in a queue separate from the main
work queue. As delayed tasks are added, a resumer is
notified and will periodically start the tasks up. Delayed
tasks can also be started by engine clients.
Using this, pkg/api/provision and deprovision were enhanced to
determine if a provision or deprovision action needs to wait for
another operation to finish. If so, they submit delayed tasks, either
wait_for_parent or wait_for_children, whichever is appropriate. Those
new jobs will make the same determination, and either readd themselves
as delayed tasks, or initiate the task that would otherwise originally
have been created.
* Added bad parent response message
* Merge master into feature branch (#209)
* refactor event function signatures (#147)
* pass around instances and bindings instead of pointers
* Refactor instance encryption/decryption
Previously, encryption and decryption of certain instance
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to instance
objects, I wanted a way for modules to be able to extract
information from encrypted instance fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors instances and the storage layer so that
encryption occurs during instance marshaling and decryption occurs
during instance unmarshaling.
* Refactor binding encryption/decryption
Previously, encryption and decryption of certain binding
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to binding
objects, I wanted a way for modules to be able to extract
information from encrypted binding fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors bindings and the storage layer so that
encryption occurs during bindings marshaling and decryption occurs
during bindings unmarshaling.
* clean up a few errant lint errors
* pass instance to provisioning steps
* pass instance to deprovisioning steps
* make bind and unbind accept instances as arguments
* get defensive about possible unanticipated modifications to instances by module code
* pass instance to updating steps
* minor lint fixes
* refactor lifecycle tests
* Firewall rule parameterization (#152)
* Refactor to support allowing firewall rules.
Fixes: #146
Changes to suport optional parameters to allow specifying start
and end IP address for firewall rules on MySQL, PostgreSQL and
MSSQL. Previously, this defaulted to allow any connection. This
isn't ecure in practice, so allowing the customer to specify desired
rules. Also changed the default to 0.0.0.0 for both, which results in
only Azure internal addresses. Added some parameter validation
for the firewall values as well:
* Valid IPV4 addresses
* Start <= End
Extracted a method to build the arm template parameter map. Providing
empty strings is invalid for the ARM template, so only including
when non-empty.
Added parameters to the lifecycle tests to allow running them
from local machines (will need ao enhance the charts in helm-charts)
Added some tests.
* Fixing merge conflicts
* Test cleanup
* json field cleanup
* Resolving code review comments
* error message cleanup
* error message cleanup
* specify firewall rules for relational databased in example manifests
* Few minor corrections to the Quickstart (#136)
* Correcting the az ad sp delete command
* Update quickstart-minikube.md
Remove a leading $ from a few bash commands. copy paste on mac is messed up :-)
* Update quickstart-minikube.md
Clarification around logging in to WordPress
* restoring prompts in quickstart
* minor punctuation fix
* removing distracting echo command
* make prompts used consistently
* kick of redis tests sooner (#155)
* s/module lifecycle tests/service lifecycle tests/g (#156)
* make storage know how to deal with service-specific types (#153)
* don't use codec where we don't need it anymore (#161)
* Updating CircleCI to define DOCKER_REPO env var (#167)
* Updating CircleCI to define REGISTRY env var
We previously set the REGISTRY environment variable in a deploy script.
When that was removed to streamline the release process, we need to
define it in Circle. Added two environment blocks to the publish-rc-images
and publish-release-images jobs.
Fixes: 145
* Qualified registry with docker.io
* Changed REGISTRY to DOCKER_REPO
* s/context/details/g (#163)
* make fake bind/unbind functionality not rely on standard context
* collapse standard provisioning context onto instance
* do away with (sort of) redundant standard provisioning parameters
* s/provisioning context/instance details/
* s/binding context/binding details/
* fix error handling when standard params aren't expected types
* add comment that adds some clarity to instance comparisons
* don't store credentials as their own field (#169)
* quickstart fixes (#174)
* adding missing docker pushes to release process
* Adding build flags to inject the version and commit (#173)
* Adding build flags to inject the version and commit
This commit injects the version and commit to the OSBA binary. If there
is no version set, then sets the version to “devel”
* Using new variable for version
* Removing redundant setting for main.commit
* Printing version and commit on broker startup
* Adding fields to the log message on startup
* Removing build flags on the CLI builds
* broker doesn't need to hang on to codec (#180)
* remove quotes from devel pseudo-version string (#184)
Merging without CI.
* fix bug decoding tags (#183)
* move version info to avoid import cycle later (#185)
* Adding user-agent string to requests to ARM (#172)
* Adding user-agent string to requests to ARM
* get version for user agent string from version package
* use separate redis clients for storage and async engine (#182)
* Adding ability to filter lifecycle tests (#181)
* Adding ability to filter lifecycle tests
This adds the ability to filter lifecycle tests using an environment
variable (TEST_MODULES). Via this variable, the developer can provide
a comma delimited list of module names and only the test cases matching
that module will be enabled. If the variable is empty, all tests will run.
If an invalid module name is provided, no tests will run.
* fix typo and lint error
* remove unnecessary funtion
* use redis namespaces (#188)
* use redis namespaces for instances and bindings
* use redis namespaces in async engine
* start using lightweight-docker-go image for development tasks (#190)
* simplify ci config (#194)
* README work (#196)
* README formatting + one cmdline fix
* Link fixes
* Revert heading to title case
* append instead of replacing user-agent (#198)
* append instead of replacing user-agent
* fix lint errors
* add deferred task execution to async package (#208)
achieving this involved a significant refactor
of the entire package. the good news is that the
tests are improved beyond what we had before,
so i think we can be more confident in the
refactored package than the original.
* Enhancement of asynch engine
This PR enhances the asynch engine to enable the
concenpt of delayed or paused tasks. Tasks can be
added to the engine in a queue separate from the main
work queue. As delayed tasks are added, a resumer is
notified and will periodically start the tasks up. Delayed
tasks can also be started by engine clients.
Using this, pkg/api/provision and deprovision were enhanced to
determine if a provision or deprovision action needs to wait for
another operation to finish. If so, they submit delayed tasks, either
wait_for_parent or wait_for_children, whichever is appropriate. Those
new jobs will make the same determination, and either readd themselves
as delayed tasks, or initiate the task that would otherwise originally
have been created.
* Remove resumer changes
* removed references to submitdelayed task
* Fixed debug statements that got clobbered in merge
* Debug message
* Fixed response code
* Whitespace
* Code review comments
* Code review comments
* Refactor based on code-review comments.
* Code review comments
* Lint fixes
* Enhanced contrib/k8s/examples (#213)
* Enhanced contrib/k8s/examples to illustrate the use of the new
SQLDB catalog
* Spelling errors
* merge latest from master to poc branch (#232)
* refactor event function signatures (#147)
* pass around instances and bindings instead of pointers
* Refactor instance encryption/decryption
Previously, encryption and decryption of certain instance
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to instance
objects, I wanted a way for modules to be able to extract
information from encrypted instance fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors instances and the storage layer so that
encryption occurs during instance marshaling and decryption occurs
during instance unmarshaling.
* Refactor binding encryption/decryption
Previously, encryption and decryption of certain binding
fields was accomplished through special getters and setters.
To use these, you had to pass in a codec. This meant that any
bit of code needing to access these fields needs access to the
codec. Since I'm entertaining changes to the service.Module
interface that will result in modules having access to binding
objects, I wanted a way for modules to be able to extract
information from encrypted binding fields _without_ needing
them to be burdened with additional moving parts-- like codecs.
So this commit refactors bindings and the storage layer so that
encryption occurs during bindings marshaling and decryption occurs
during bindings unmarshaling.
* clean up a few errant lint errors
* pass instance to provisioning steps
* pass instance to deprovisioning steps
* make bind and unbind accept instances as arguments
* get defensive about possible unanticipated modifications to instances by module code
* pass instance to updating steps
* minor lint fixes
* refactor lifecycle tests
* Firewall rule parameterization (#152)
* Refactor to support allowing firewall rules.
Fixes: #146
Changes to suport optional parameters to allow specifying start
and end IP address for firewall rules on MySQL, PostgreSQL and
MSSQL. Previously, this defaulted to allow any connection. This
isn't ecure in practice, so allowing the customer to specify desired
rules. Also changed the default to 0.0.0.0 for both, which results in
only Azure internal addresses. Added some parameter validation
for the firewall values as well:
* Valid IPV4 addresses
* Start <= End
Extracted a method to build the arm template parameter map. Providing
empty strings is invalid for the ARM template, so only including
when non-empty.
Added parameters to the lifecycle tests to allow running them
from local machines (will need ao enhance the charts in helm-charts)
Added some tests.
* Fixing merge conflicts
* Test cleanup
* json field cleanup
* Resolving code review comments
* error message cleanup
* error message cleanup
* specify firewall rules for relational databased in example manifests
* Few minor corrections to the Quickstart (#136)
* Correcting the az ad sp delete command
* Update quickstart-minikube.md
Remove a leading $ from a few bash commands. copy paste on mac is messed up :-)
* Update quickstart-minikube.md
Clarification around logging in to WordPress
* restoring prompts in quickstart
* minor punctuation fix
* removing distracting echo command
* make prompts used consistently
* kick of redis tests sooner (#155)
* s/module lifecycle tests/service lifecycle tests/g (#156)
* make storage know how to deal with service-specific types (#153)
* don't use codec where we don't need it anymore (#161)
* Updating CircleCI to define DOCKER_REPO env var (#167)
* Updating CircleCI to define REGISTRY env var
We previously set the REGISTRY environment variable in a deploy script.
When that was removed to streamline the release process, we need to
define it in Circle. Added two environment blocks to the publish-rc-images
and publish-release-images jobs.
Fixes: 145
* Qualified registry with docker.io
* Changed REGISTRY to DOCKER_REPO
* s/context/details/g (#163)
* make fake bind/unbind functionality not rely on standard context
* collapse standard provisioning context onto instance
* do away with (sort of) redundant standard provisioning parameters
* s/provisioning context/instance details/
* s/binding context/binding details/
* fix error handling when standard params aren't expected types
* add comment that adds some clarity to instance comparisons
* don't store credentials as their own field (#169)
* quickstart fixes (#174)
* adding missing docker pushes to release process
* Adding build flags to inject the version and commit (#173)
* Adding build flags to inject the version and commit
This commit injects the version and commit to the OSBA binary. If there
is no version set, then sets the version to “devel”
* Using new variable for version
* Removing redundant setting for main.commit
* Printing version and commit on broker startup
* Adding fields to the log message on startup
* Removing build flags on the CLI builds
* broker doesn't need to hang on to codec (#180)
* remove quotes from devel pseudo-version string (#184)
Merging without CI.
* fix bug decoding tags (#183)
* move version info to avoid import cycle later (#185)
* Adding user-agent string to requests to ARM (#172)
* Adding user-agent string to requests to ARM
* get version for user agent string from version package
* use separate redis clients for storage and async engine (#182)
* Adding ability to filter lifecycle tests (#181)
* Adding ability to filter lifecycle tests
This adds the ability to filter lifecycle tests using an environment
variable (TEST_MODULES). Via this variable, the developer can provide
a comma delimited list of module names and only the test cases matching
that module will be enabled. If the variable is empty, all tests will run.
If an invalid module name is provided, no tests will run.
* fix typo and lint error
* remove unnecessary funtion
* use redis namespaces (#188)
* use redis namespaces for instances and bindings
* use redis namespaces in async engine
* start using lightweight-docker-go image for development tasks (#190)
* simplify ci config (#194)
* README work (#196)
* README formatting + one cmdline fix
* Link fixes
* Revert heading to title case
* append instead of replacing user-agent (#198)
* append instead of replacing user-agent
* fix lint errors
* add deferred task execution to async package (#208)
achieving this involved a significant refactor
of the entire package. the good news is that the
tests are improved beyond what we had before,
so i think we can be more confident in the
refactored package than the original.
* reorganize async pakage/sub-packages more sanely (#210)
* change provision/deprovision task names (#211)
The are imperative and more accurate.
* Add async support for "follow-up" tasks (#212)
* make async job functions take task as argument
* make async job functions return tasks
* bonus: fix more task names
* make async task executor submit follow-up tasks
* make broker async processes leverage new async follow-up tasks feature
* ignore .vscode/ (#221)
* more async refactoring (#219)
* break up worker.go and worker_test.go into multiple files
* simplify heart
* simplify cleaner
* move heart from worker to engine
* remove receive/execute/watch from worker to engine
* let task receivers return right after fatal error
* let task executor return right after fatal error
* very minor comment cleanup
* fix lint errors by removing unused error types
* add svc and plan to instance during retrieval (#220)
* retrieve svc when instance is retrieved
* retrieve plan when instance is retrieved
* improve provision step function signature
* improve deprovision step function signature
* improve update step function signature
* lifecycles tests: set service and plan on instance
* Changes to mysql bind. (#222)
Change grant string in mysql bind to be bound to database
Removed global privs
Added Execute
Fixes #197
* Initial roadmap submission (#226)
* Initial Roadmap
* Publish initial roadmap for OSBA
Closes #109
* Spelling
* Review comments and wordsmithing.
* Refactor for generic filter capabilities (#218)
* WIP: Proposal to refactor for generic filter capabilities
1.) Create new api/filter structure
2.) Refactored authenticator stuff there
3.) Created new interfaces: filters.Filter and filters.Chain
4.) Made basic/always authenticator implement filters.Filter interface
* Fixed filename type-o.
* Fixed filename type-o
* Renamed Filter method to Execute.
* Added test for filter chain
* Refactor and style fixes
* ignore .vscode/ (#221)
* more async refactoring (#219)
* break up worker.go and worker_test.go into multiple files
* simplify heart
* simplify cleaner
* move heart from worker to engine
* remove receive/execute/watch from worker to engine
* let task receivers return right after fatal error
* let task executor return right after fatal error
* very minor comment cleanup
* fix lint errors by removing unused error types
* add svc and plan to instance during retrieval (#220)
* retrieve svc when instance is retrieved
* retrieve plan when instance is retrieved
* improve provision step function signature
* improve deprovision step function signature
* improve update step function signature
* lifecycles tests: set service and plan on instance
* changed filters package to filter
* Lint failure
* Refactor filter package a little and re-add Chain interface
* a few filter tweaks
* Change how the database name for cosmos db is generated (#228)
* Change the generation of the database name. A limitation in the current
ARM based CosmosDB provisoning limits the length of the database name to
50 characters, including the name provided, the location name and a dash.
In some situations, this means a UUID is invalid. If the length will be
too much, this patch will randomly generate a string of the available length
instead of using a UUID.
* Refactored to use generate package
* fix async cleaner (#233)
* bug fix: fix wait again on children not deprovisioned
* bug fix: it's ok for an instance's parent to not be found
* Code comments
* Replacing empty json responses with error messages.
Issue #66 points out that our error messages don't help determine the
cause of operation failures. This PR adds an informative response to
locations where we are returning a 400 or a 409 so the user has some extra infromation
to debug the failure.
These should be inline with the Broker Errors section of the OSB spec.
* Adding error messages to operation failures
Fixes #66
* Fixed compile error after merge
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Using the updated features of the async engine, pkg/api/provision and deprovision were enhanced to determine if a provision or deprovision action needs to wait for
another operation to finish. If so, they submit delayed tasks, either
wait_for_parent or wait_for_children, whichever is appropriate. Those
new jobs will make the same determination, and either readd themselves
as delayed tasks, or initiate the task that would otherwise originally
have been created.
Provision was also enhanced to reject provision requests for things have parents that are currently deprovisioning or have failed.