Proposal: Merge vars and env

[pd93]

One of the things I see a lot, is confusion about the differences between variables and environment variables. They essentially serve the same purpose (holding data), but they have a few differences:

• Both can be parsed into Task via the CLI as strings (though they have different syntaxes)
  • TASK_VAR=foo task ... (environment variable)
  • task ... VAR=foo (variable)
• Both can be defined in the Taskfile (using different keywords) at a global or task level.
  • vars can also be set when calling another task, but env cannot.
• vars can be of any type. env only supports strings (though sort of works with other scalar values).
• vars are output using templating syntax and an env is output using shell variable syntax.
  • $FOO (environment variable)
  • {{.FOO}} (variable)
• They have slightly different inheritance/scoping rules.
  • At this point, I'm not even sure what the differences are without investigating properly.
• Environment variables defined on the CLI/shell are available as variables, but ones defined using the env keyword are not.

I'm not convinced that any of these distinctions are necessary and I feel like they add an additional layer of confusion for both new users and people wanting to contribute to the project.

Therefore, I propose removing the env key entirely. vars would then inherit from the system environment and be overridable in Task. If a variable is not a string, it would be output using the stringer version of its value (%v). All of the features below would then apply to vars:

• Variables are only passed into Task via the shell environment (whether that be in an .rc file of a shell command).
  • No more task ... VAR=foo syntax.
• Variables can be defined at a global or task level and passed when calling dependant tasks.
• Variables can be of any type, though they are still limited to strings when passed via the environment variables.
• Variables are output using templating syntax OR shell variable syntax depending on the user's needs.
• Variables have a consistent inheritance/scoping model as defined in Proposal: Variable scope and inheritance #2035.

Before:

version: '3'

env:
  FOO: 'foo'
vars:
  BAR: 'bar'

tasks:
  default:
    cmds:
      - echo '$FOO'
      - echo '{{.FOO}}'
      - echo '$BAR'
      - echo '{{.BAR}}'

foo


bar

After:

version: '3'

vars:
  FOO: 'foo'
  BAR: 'bar'

tasks:
  default:
    cmds:
      - echo '$FOO'
      - echo '{{.FOO}}'
      - echo '$BAR'
      - echo '{{.BAR}}'

foo
foo
bar
bar

One caveat to this proposal is that sometimes temporary variables are needed to perform operations and we don't always want to pollute the environment with these variables. To solve this, we can allow variables to be marked as internal, just like we would with tasks. This makes variables consistent with tasks and fulfills the same purpose that variables previously did.

Since most Task commands execute in a fairly specific context, this shouldn't be needed the majority of the time.

version: '3'

vars:
  FOO: 'foo'
  BAR:
    value: 'bar'
    internal: true
  BAZ:
    sh: 'echo "baz"'
    internal: true
  QUX:
    ref: 'lower .BAZ'
    internal: true

tasks:
  default:
    cmds:
      - 'echo "foo: {{.FOO}}"'
      - 'echo "bar: {{.BAR}}"'
      - 'echo "baz: {{.BAZ}}"'
      - 'echo "qux: {{.QUX}}"'

foo: foo
bar:
baz:
qux:

[andreynering]

Hey @pd93,

Two years ago, I opened an issue proposing something different, but that has the same objetive of reducing the confusion between vars and envs:

• Idea: Consider having functions to access variables and envs #1065

In that proposal, instead of merging vars and envs, we'd be actually making them completely separate. The idea is that env would be exclusively used to make environment variables available to the actual commands. For interpolation and passing data between tasks, users should use vars.

Environment variables defined on the CLI/shell are available as variables, but ones defined using the env keyword are not.

With my proposal, no env would be automatically be recognized as a variable, in any circustance.

Both proposals are valid, and I'm open to discuss both. /cc @vmaerten Any thoughts?

[trulede]

One caveat to this proposal is that sometimes temporary variables are needed to perform operations and we don't always want to pollute the environment with these variables.

This is the crux of the problem. Are vars and envs really the same thing? Or are they actually different.

I've developed a "mental model" that vars are something internal, between tasks, and envs are something that comes from the calling process, and might be forwarded to a called process - typically credentials (or similar). Having envs distinguishable from vars ($foo vs {{.foo}}) would be something too - since differentiating between them is currently not possible.

Also, deciding what to "pass along" becomes important, since it may be necessary to restrict what a called task can discover from the callee task (credential leaks come to mind).

[vmaerten]

Hey @pd93 & @andreynering,

I'll start by quoting you:

Both proposals are valid, and I'm open to discussing both.

Merging envs and vars

I like this approach because it provides a single, unified concept, reducing confusion.
However, I have a question: if we completely remove env, how would we set an environment variable for a specific task? For example, something like this:

version: '3'

tasks:
  deploy:
    env:
      AWS_PROFILE: 'dev'
    cmds:
      - aws s3 ....
      - aws lambda ....
      - aws cloudfront invalid ...

Additionally, it would become harder to distinguish between what comes from vars and what comes from envs, especially since environment variables can also be inherited from .zshrc, .bashrc, etc.

Completely separate envs and vars

I really like this idea. The only "downside" is that we wouldn’t be able to write something like LOG_LEVEL=info task ..., but I think that's just a habit to unlearn.

As @andreynering said, both approaches are valid. My initial preference would be to separate them.

[pd93]

if we completely remove env, how would we set an environment variable for a specific task?

version: '3'

tasks:
  deploy:
    vars:
      AWS_PROFILE: 'dev'
    cmds:
      - aws s3 ....
      - aws lambda ....
      - aws cloudfront invalid ...

@vmaerten It would be exactly the same as now, just using vars instead. You would just need to think of all variables in Task as environment variables, with the benefit of having additional sugar in the templating system (like types).

I don't personally see the need for two separate concepts. They provide mostly the same functionality, but with minor differences. This is where the confusion comes from. I have had multiple people ask me what the differences actually are.

I'm not absolutely against separating them, but I'm not a fan of having to use functions to access variables and env vars as in #1065. It doesn't seem as intuitive as directly referencing one and goes against most templating systems I've used.

---

For reference, GitHub actions does have variables and environment variables, but variables are only inputs. You cannot set them in the action config files. In the configs, you are only able to set variables using the env keyword and you access them in templating by calling {{ env.FOO }}. I would actually be in favour of following this and updating this proposal to merge the two concepts into the env keyword instead of into vars.