Support for group tasks containing other group tasks

[leftink]

I am currently developing a build system where I could greatly benefit from multi-layer sub-tasking/group tasks.

The build system will have several top level "phases" that are mutually exclusive so I would like to keep the tasks grouped in that manner. One example of this would be something like the following:

sim
sim:compile
sim:compile:lib1
sim:compile:lib1:file1...sim:compile:lib1:fileN (contains action)
sim:compile:libN
sim:compile:libN:file1...sim:compile:libN:fileN (contains action)
sim:elaborate (contains action)
sim:run (contains action)

Sim is the top level group task, while sim:compile is a group task and sim:compile:lib(*) are embedded group tasks.

I could not figure out how to do this yielding dicts.

One reason I would like this grouping is so that "sim:elaborate" could use result_dep('sim:compile') that way if any library has to be recompiled, the elaborate step is re-executed.

result_dep() would also have to be updated to walk down through all groups to get a list of all tasks.

[schettino72]

A group task is actually implemented internally just using task_dep. You could do the same...
Regarding the name of the tasks. Sub-tasks can contain : in their name.

I would like to see some other ways to categorize tasks. maybe using tags...
But a hierarchy of group tasks is not how i would approach it.
I suggest you play a bit with the code and try to come up with something.

I am gonna close this but feel free to re-open if you have a more concrete idea and implementation.

thanks

[leftink]

@schettino72 I don't seem to be able to re-open, but there is problem with the approach you mentioned.

1. The "is_subtask" and "has_subtask" attributes don't get set appropriately which is a problem when want to use a function like result_dep() on a group task. I have even tried setting the basename and the name, but the code that processes this can't seem to handle it correctly.

2. The result_dep() needs to be fixed as well to walk through the complete hierarcy of group tasks to get the full lists of result dependencies.

I am not sure I agree with the notion that Hierarchy tasks don't make sense. We have several complex jobs that have complex sub-tasks, that make sense to make those sub-tasks.

Our build system, is actually doing multiple "top-level" tasks that are mutually exclusive and it is nice to have the hierarchy so a user can see what tasks go with each "command.

If you don't feel it is worth supporting then that is fine. I plan on fixing it for our company.

[schettino72]

ok. re-opening it... 😁

It would be great if you provide a full running example with dummy actions.
Are you working these changes in a public branch?

I would be definitely interested in integrating your changes if it works well.
And seeing your branch makes it easier for me to follow along and maybe make suggestions...

Feel free to ask me questions, although i will not have much time in near future.

[leftink]

Currently the changes will not be public, we are doing an Application on-top of Doit and so I was just going to handle this in my task loader, but we would prefer to make it part of the baseline code so I once I get things worked out, I will put something out on the public side for evaluation and updates.

[leftink]

I will close this. I think I understand now what you were referring to. I was able to write a generator that creates multiple groups that give me the control I desire.