[JENKINS-29817] Inject environment variables for a Job #56

recena · 2015-08-05T22:28:17Z

https://issues.jenkins-ci.org/browse/JENKINS-29817

Implement EnvInjectEnvVarsContributor
Add tests
Update the documentation (help information) for the Properties Content field

@reviewbybees

jglick · 2015-08-05T22:32:12Z

src/test/java/org/jenkinsci/plugins/envinject/EnvInjectEnvVarsJobTest.java

+        EnvVars environment = project.getEnvironment(jenkins.getInstance(), listener);
+
+        assertNotNull(environment.get("REPO"));
+        jenkins.assertBuildStatusSuccess(project.scheduleBuild2(0).get());


Running the build does not really accomplish anything. You could delete this line, or make the test demonstrate that the variable is also accessible to build steps. Which reminds me that it is likely that other code in this plugin is interpreting EnvInjectJobProperty on a Run (or AbstractBuild) that could probably now be deleted, since Run environments inherit from Job environments.

@jglick Indeed. I'm going to review the test.

@jglick Done

jenkinsadmin · 2015-08-06T00:01:47Z

Thank you for a pull request! Please check this document for how the Jenkins project handles pull requests

oleg-nenashev · 2015-08-06T05:31:22Z

src/main/java/org/jenkinsci/plugins/envinject/EnvInjectEnvVarsContributor.java

+
+    @Override
+    public void buildEnvironmentFor(Job job, EnvVars env, TaskListener listener) throws IOException, InterruptedException {
+        for (Object property : job.getProperties().values()) {


It's better to get properties by class without such cycles

@oleg-nenashev I'll take a look.

@oleg-nenashev I've done some changes related to your comment. Could you review it?

@oleg-nenashev I've reverted the changes because the tests were failing. I was trying this:

@Override public void buildEnvironmentFor(Job job, EnvVars env, TaskListener listener) throws IOException, InterruptedException { EnvInjectJobProperty jobProperty = (EnvInjectJobProperty) job.getProperty(EnvInjectJobProperty.class) EnvInjectJobPropertyInfo jobPropertyInfo = jobProperty.getInfo(); // Process "Properties Content" Map<String, String> result = jobPropertyInfo.getPropertiesContentMap(new HashMap<String, String>()); if (result != null) { env.putAll(result); } }

Null check is missing. If a job has no EnvInjectJobProperty, your test will fail with NPE

@oleg-nenashev AFAIK, if you are using the plugin (checkbox is clicked), there is always a EnvInjectJobProperty. And I've verified that job.getProperties() can never be null.

job.getProperty() may return null if you run the method agains the loaded job, which has been saved before the EnvInject installation.

job.getProperties() is never null, but IMO you're using a wrong pattern to lookup for properties. It may become performance-innefective in future Jenkins versions

@oleg-nenashev I don't know how job.getProperty() can be null if this method is invoked only if job.getProperties().values() returns values.

Do you prefer to use job.getProperty(EnvInjectJobProperty.class)?

Yes. And it can return the null value ;)

oleg-nenashev · 2015-08-06T05:40:13Z

I general I agree that such contributor mat be useful. On the other hand I cannot elaborate an impact of the change, because it may break the variable resolution order on existing installations. It may also cause a total breakage of existing jobs if they use Groovy scripts to inject variables and use build variables.

More tests and investigations are required before this PR becomes mergeable. I'm putting a 👎 on it now (as OSS contributor).

recena · 2015-08-06T06:32:52Z

@oleg-nenashev What tests do you propose to verify that improvement (very needed) does not breaks anything?

This PR has been tested with Subversion Plugin (master) in order to verify this kind of use cases JENKINS-29340 and works perfectly.

Basically, we need the environment variables defined by EnvInject availables in Job.getEnvironment().

oleg-nenashev · 2015-08-06T09:28:54Z

@recena
I'll try to respond on this evening.

oleg-nenashev · 2015-08-08T11:35:52Z

@recena
After digging into the code...
EnvInjectInfo: getPropertiesContentMap() works with property contents only (text area field). See https://github.com/jenkinsci/envinject-plugin/blob/master/src/main/java/org/jenkinsci/plugins/envinject/EnvInjectInfo.java#L42-L65. The current implementation ignores environment variables from property files, Groovy scripts and from Master node.

Such implementation may address a narrow case from JENKINS-29817, but it is incomplete. I would not vote for merging the fix in the current state, because it may confuse the plugin users => serious 🪲 . You should implement a new method, which would generate variables without a build context.

I'm also aware about the variables resolution order. buildEnvironmentFor for Jobs is being called before the resolution for runs. According to the brief code analysis, seems that the use-case with build parameter overrides will be broken by the change. The behavior is configurable via the ''Override build parameters" option. According to the manual testing, AbstractBuild:getEnvironment() returns wrong values after the change. The test has been added in #58

recena · 2015-08-10T22:45:15Z

@oleg-nenashev I'm going to implement the different way to provide variables (property files, groovy scripts, etc..). I don't understand why you say that this new extension breaks anything. The rest of tests are working fine. Maybe we should review these test, Isn't that so?

recena · 2015-08-10T22:54:19Z

@oleg-nenashev There are a lot of users using Subversion Plugin and EnvInject Plugin. We need the environment variables (injected by this plugin) availables, p.e, in polling process. What would it your proposal?

oleg-nenashev · 2015-08-11T06:23:58Z

I don't understand why you say that this new extension breaks anything

I just see a wrong output in the API method. This result needs more investigation. buildEnvironmentForJob() is being called before the Environment contributing actions, so probably the issue in overrides (or I'm doing something wrong in the debugger). I'll write a unit test and submit it.

I'm going to implement the different way to provide variables (property files, groovy scripts, etc..).

Please be aware about Groovy ones. I suspect that many Users rely on build runtime variables there. The plugin's code correctly handles null build references, but we should ensure that the newly introduced method is not failing horribly in the case of the misusage.

oleg-nenashev · 2015-08-11T07:51:38Z

The rest of tests are working fine. Maybe we should review these test, Isn't that so?

Unfortunately the plugin does not contain tests for EnvInjectJobProperty at all :(

recena · 2015-08-12T10:35:55Z

...n/resources/org/jenkinsci/plugins/envinject/EnvInjectJobProperty/help-propertiesContent.html

+    <p>All the properties name will be accessible as environment variables by their names. You can use or override the
+        properties specified in the above properties file.</p>
+
+    <p>These environment variables (<strong>only these</strong>) will be available in the context Job.</p>


@oleg-nenashev What do you think?

What is a context Job? I don't know such term. In any case it makes sense to add several sentences, which would allow users to understand what's going on here.

ghost · 2015-08-12T10:36:56Z

This pull request originates from a CloudBees employee. At CloudBees, we require that all pull requests be reviewed by other CloudBees employees before we seek to have the change accepted. If you want to learn more about our process please see this explanation.

recena · 2015-08-12T18:09:33Z

...n/resources/org/jenkinsci/plugins/envinject/EnvInjectJobProperty/help-propertiesContent.html

+    <ul>
+        <li>Validate Subversion Repository with a URL like this: <code>http://server/svn/$REPO</code></li>
+        <li>Subversion SCM Polling</li>
+    </ul>
 </div>


@oleg-nenashev Better?

My proposals:

... will be available in the Job environment independently from the build. There are following use-cases: SCM polling, validation of fields depending on environment variables (e.g. http://server/svn/$REPO), custom workspace management, etc. Limitations: The field may contain variables, which are not resolvable outside the Build context (e.g. BUILD_NUMBER, WORKSPACE, etc.), the plugin [injects empty value/ignores the field/or something else]). This behavior should be taken into account by job editors.

Please also add a proper test-case for the limitation

@oleg-nenashev Thanks so much for your help. I'm going to investigate the limitations.

@oleg-nenashev I don't understand what you mean with limitations. Could you show an example the variable defined in this field not resolvable? Do you mean something like this VAR1=$WORKSPACE?

@recena
Yes, WORKSPACE var is a valid example for this limitation

recena · 2015-08-12T22:41:34Z

@oleg-nenashev What do you thing about last commit?

oleg-nenashev · 2015-08-13T04:18:25Z

src/main/resources/org/jenkinsci/plugins/envinject/EnvInjectBuildWrapper/config.jelly

@@ -35,6 +21,20 @@
                value="${instance.info.groovyScriptContent}"/>
    </f:entry>

+    <f:entry field="propertiesFilePath"


Why did you move this section? How is it related to this PR?

@oleg-nenashev Indeed this change is not directly related to PR but I considered it would be helpful to show the form fields in the same order in both sections. Don't make sense to use a different order.

NIT: "Properties content" is the most widely used section, so I would use another direction. BTW, the interface should be reworked to show only the required section (extension point?). Such rework is not a part of the PR, so I'd prefer to submit it as another PR

@oleg-nenashev We can discuss about it out of the PR.

As a summary of the discussion, I still think it should be moved to another PR:

It is completely unrelated to the PR

The change impacts the usability and UX IMO, because it changes the original layout

oleg-nenashev · 2015-08-13T12:14:41Z

🐛 due to the unrelated change in src/main/resources/org/jenkinsci/plugins/envinject/EnvInjectBuildWrapper/config.jelly, which is not related to this PR. It should be decoupled to a separate pull-request

amuniz · 2015-08-13T12:28:17Z

@oleg-nenashev Is that really a bug? Is preventing to use the plugin somehow? Since it's not really a functional change and does not affect in any way to the behavior being modified in this PR, it's ok for me. Moreover it's a minor change that you can quick see that is only a code block moved some lines below.

oleg-nenashev · 2015-08-13T12:37:56Z

Please also address a comment regarding the Japanese help

oleg-nenashev · 2015-08-13T16:38:17Z

🐝
Could you please squash commits? After that we will be able to merge it according to the policy. I'm planning to release 0.92-beta1 on the weekend (or today if everything goes right)

recena · 2015-08-13T17:04:16Z

@oleg-nenashev done

recena · 2015-08-13T17:06:38Z

@oleg-nenashev By the way, Is there an official maintainer? /cc @gboissinot

[JENKINS-29817] Inject environment variables for a Job

oleg-nenashev · 2015-08-16T23:27:21Z

src/test/java/org/jenkinsci/plugins/envinject/EnvInjectEnvVarsContributorTest.java

+
+        TaskListener listener = jenkins.createTaskListener();
+        EnvVars environment = project.getEnvironment(jenkins.getInstance(), listener);
+        assertEquals("${WORKSPACE}", environment.get("VAR1"));


This check is failing on my local laptop. The result is null

fixed by clean build, please ignore

recena changed the title ~~[JENKINS-29817] Inject environment variables for a Job~~ [WiP] [JENKINS-29817] Inject environment variables for a Job Aug 5, 2015

jglick reviewed Aug 5, 2015
View reviewed changes

oleg-nenashev reviewed Aug 6, 2015
View reviewed changes

oleg-nenashev added the enhancement label Aug 9, 2015

oleg-nenashev mentioned this pull request Aug 11, 2015

Add new tests for EnvInjectJobProperty #58

Merged

recena changed the title ~~[WiP] [JENKINS-29817] Inject environment variables for a Job~~ [JENKINS-29817] Inject environment variables for a Job Aug 12, 2015

recena reviewed Aug 12, 2015
View reviewed changes

oleg-nenashev reviewed Aug 13, 2015
View reviewed changes

[JENKINS-29817] Inject environment variables for a Job

b34ae59

recena added a commit that referenced this pull request Aug 14, 2015

Merge pull request #56 from recena/JENKINS-29817

95c1f0e

[JENKINS-29817] Inject environment variables for a Job

recena merged commit 95c1f0e into jenkinsci:master Aug 14, 2015

recena mentioned this pull request Aug 15, 2015

[JENKINS-29340] ISVNAuthentication provider did not provide credentials (part. II) jenkinsci/subversion-plugin#131

Merged

oleg-nenashev reviewed Aug 16, 2015
View reviewed changes

oleg-nenashev mentioned this pull request Sep 28, 2017

[JENKINS-26583] - Attempt to tweak the injection priorities #124

Open

5 tasks

oleg-nenashev removed the enhancement label Jul 21, 2019

[JENKINS-29817] Inject environment variables for a Job #56

[JENKINS-29817] Inject environment variables for a Job #56

Conversation

recena commented Aug 5, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jenkinsadmin commented Aug 6, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

oleg-nenashev commented Aug 6, 2015

recena commented Aug 6, 2015

oleg-nenashev commented Aug 6, 2015

oleg-nenashev commented Aug 8, 2015

recena commented Aug 10, 2015

recena commented Aug 10, 2015

oleg-nenashev commented Aug 11, 2015

oleg-nenashev commented Aug 11, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ghost commented Aug 12, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

recena commented Aug 12, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

oleg-nenashev commented Aug 13, 2015

amuniz commented Aug 13, 2015

oleg-nenashev commented Aug 13, 2015

oleg-nenashev commented Aug 13, 2015

recena commented Aug 13, 2015

recena commented Aug 13, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment