Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

validate dependent jobs' parents - issue 148 #245

Merged
merged 7 commits into from
Feb 19, 2016

Conversation

giuliano108
Copy link
Contributor

This should fix #148 .

Catches two types of config errors:

  • Parents whose name is invalid (!= service_name.instance_name). Similar to what the JSON schema also enforces.
  • Parents that are referenced in a dependent job, but are not actually configured.

Note that the second validation won't let a "ServiceA child" depend on a "ServiceB parent". That's the best we can do without parsing the entire yelpsoa-configs tree.

In my opinion paasta_tools/cli/cmds/validate.py, in order to validate the Chronos config, needs to know too many details about ChronosJobConfig. If need be, I'm happy to move as much of that logic into paasta_tools/chronos_tools.py as possible, so that the latter is in charge of the validation and the former just prints out the outcome.

screen shot 2016-02-10 at 6 08 14 pm

@@ -205,16 +207,27 @@ def validate_chronos(service_path):
service=service, clusters=[cluster], instance_type=instance_type,
soa_dir=soa_dir):
cjc = load_chronos_job_config(service, instance, cluster, False, soa_dir)
checks_passed, check_msgs = cjc.validate()
cjcs[service + '.' + cjc.get_job_name()] = cjc
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks a little more complicated than I think it needs to be. While not super efficient, can it just be:

for parent in get_parents()
    if parent not in get_services_for_cluster(cluster=cluster, instance_type=chronos):
        # something

@giuliano108
Copy link
Contributor Author

This looks a little more complicated than I think it needs to be.

Yes. Using get_services_for_cluster also allows parent/dependent validation to work across services in the same cluster. In this new revision, I'm also making sure validation fails when a dependent job specifies itself as a parent.

screen shot 2016-02-15 at 6 42 53 pm

@@ -201,12 +203,25 @@ def validate_chronos(service_path):

returncode = True
for cluster in list_clusters(service, soa_dir, instance_type):
services_in_cluster = get_services_for_cluster(cluster=cluster, instance_type='chronos')
valid_services = set([name + '.' + instance for name, instance in services_in_cluster])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"%s%s%s" % (service, ".", instance)

@solarkennedy
Copy link
Contributor

lgtm

@Rob-Johnson
Copy link
Contributor

fix n ship

giuliano108 added a commit that referenced this pull request Feb 19, 2016
…sue-148

validate dependent jobs' parents - issue 148
@giuliano108 giuliano108 merged commit 6fc900c into master Feb 19, 2016
@asottile asottile deleted the validate_service_instance_parent-issue-148 branch May 31, 2016 21:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

we should validate that a given service.instance parent exists before trying to deploy
3 participants