Option to run batches unbatched

[mike-lawrence]

So I'm using stantargets to do SBC on two models with the same parameters just a different internal structure, and I'm using tar_map to iterate over a couple parameters that define the volume of data:

list(
	tar_map(
		values = as.list(expand_grid(
			n = c(1e1,1e2)
			, k = c(2,5,15)
		))
		, tar_stan_mcmc_rep_summary(
			name = out
			, stan_files = c(
				'./stan_code/mvn.stan'
				, './stan_code/pairwise.stan'
			)
			, data = generate_data(n,k)
			, variables = c('pop_means','pop_sds','cor_vec')
			, summaries = list(
				~posterior::quantile2(.x, probs = c(.1,.25,.5,.75,.9))
				, posterior::default_convergence_measures()
			)
			, batches = 1e3
			, reps = 1
			, chains = 4
			, parallel_chains = 4
			, init = .1
			, stdout = R.utils::nullfile()
			, stderr = R.utils::nullfile()
		)
	)
)

At present, this will run all 1e3 batches for a given combination of n, k, & model before moving on to the next combination. It would be useful to instead have things iterate over the combinations first, so processing all combinations for batch 1, then all combinations for batch 2, etc. That way the user can watch the results as they come in and catch any oddness earlier in the process.

[wlandau]

If you are talking about order of execution, you can set the priority argument of tar_stan_mcmc_rep_summary(). tar_map() might not let you automate this, so tar_eval() might be more convenient here: https://books.ropensci.org/targets/static.html#metaprogramming

[wlandau]

As a workaround, you could first run the pipeline with batches = 1 and let it completely finish. Then the next tar_make() will skip all those batches, so you will not have wasted much time.

Another workaround is just to assign random priorities to the target objects. Bit of a hack though. Sketch:

# _targets.R
# ...
targets <- tar_map(..., unlist = TRUE) %>%
  walk(~.x$settings$priority <- runif())

For most cases, I actually like the current behavior because homogeneity makes load balancing more efficient, which matters for persistent workers with tar_make_clustermq().

[mike-lawrence]

I ended up realizing that I could use tar_map to get the order behaviour I wanted (but I still had to set the priorties by hand):

(
	tar_map(
		values = as.list(
			expand_grid(
				iteration = 1:100 #sorts with iteration varying slowest
				, n = c(1e1,1e2)
				, k = c(2,5,15)
			)
			%>% select(n,k,iteration) #puts iteration at the end of target names
		)
		, tar_stan_mcmc_rep_summary(
			name = out
			, stan_files = c(
				'./stan_code/mvn.stan'
				, './stan_code/sufficient.stan'
			)
			, data = generate_data(n,k)
			, variables = c('pop_means','pop_sds','cor_vec')
			, summaries = list(
				~posterior::quantile2(.x, probs = c(.1,.25,.5,.75,.9))
				, posterior::default_convergence_measures()
			)
			, batches = 1
			, reps = 1
			, chains = 4
			, parallel_chains = 4
			, init = .1
			, stdout = R.utils::nullfile()
			, stderr = R.utils::nullfile()
		)
		, unlist = T
	)
) -> from_tar_map

num_targets = length(from_tar_map)
for(i in 1:num_targets){
	from_tar_map[[i]]$settings$priority = num_targets - i/num_targets
}

from_tar_map

[mike-lawrence]

Ah shoot, even that doesn't do what I was hoping; it still does all the iterations on one model before moving on to the next. hm...

[wlandau]

Any reason why batches = 1 followed by batches = 50 wouldn't work? Are you trying to break apart batches?

[wlandau]

A batch is a linear sequence of branches, so targets does not know how to run reps within batch in a different order or in parallel.