Testing Shared Libraries that use Jenkins commands

[vle-tom]

I have been trying to test a shared library and went the route of wrapping it in a test pipeline and using this framework to exercise it. The library has a method that looks like

def getBuildProperties(){ return readFile('file.properties')} where readFile is a Jenkins command.

My test case is set up to register the method like so,

	helper.registerAllowedMethod(method("readFile", String.class), { file ->
	    return FileUtils.readFileToString(new File(file), Charset.forName("UTF-8"))
	})

The pipeline looks like

def execute() {
	def util = new Util()

	node() {
	
        stage('test') {
          readFile('file.properties') // works
          util.getBuildProperties() // does not work
        }
    }
}

The error I get is,

groovy.lang.MissingMethodException: No signature of method: com.company.Util.readFile() is applicable for argument types: (java.lang.String) values: [file.properties]
...

Is there something I'm missing here or is this not supported?
Is there a mechanism for testing a shared library in a standard unit test with the ability to have the Jenkins commands available, mocked or otherwise?

[stchar]

Hey, you forgot some lines in the pipeline

@Library('le_library')
import package.path.Util

Take a look on this one https://github.com/headcrabmeat/pipeline-sharedlib-testharness

[stchar]

Hi, I read it again =).
I think that you missed a reference to pipeline object in your getBuildProperties()
That it is why the mock works with the pipeline but not with the library
see https://jenkins.io/doc/book/pipeline/shared-libraries/#accessing-steps

so it should look like

// library class
class Util ... {
  Util(script) {
    this.script = script
  }

  def getBuildProperties(){ 
   return script.readFile('file.properties')
  }

// pipeline
def execute() {
	def util = new Util(this)

	node() {
	
        stage('test') {
          readFile('file.properties') // works
          util.getBuildProperties() // should work now )))
        }
    }
}

[attiand]

I think @vle-tom was trying to access steps outside of the scope of an enclosing class, i.e. the first example (src/org/foo/Zot.groovy) in https://jenkins.io/doc/book/pipeline/shared-libraries/#accessing-steps

i.e.

// src/com/company/Util.groovy
package com/company;

def getBuildProperties(){ 
   return script.readFile('file.properties')
}

this works for me in Jenkins but in the test environment I got the same error.

[Lance-Uppercut]

I'm having the exact same problem. Did you ever find a solution`?

[cbos]

We face the same issue. We have shared libaries were we combine a number of Jenkins default commands for example. We like to unit tests our own shared libraries.
That is not possible now, we are stuck with the error message as above.

[Lance-Uppercut]

I may have a work around. I havent tried it out with JenkinsPipelineUnit:
In the example with the Util class being loaded, inversion of control should be used.
Instead of

def execute() { 
 	def util = new Util() 
 	node() {....

Do

def execute() { 
 	def util = UtilFactory.getInstance()
 	node() {

then have a factory

class UtilFactory { 

     static Util instance
    static Util getInstance(){
        if(instance == null){
            instance = new Util()
        }else{
            return instance
        }
    }
}

which can override the Util class given:

class UtilForTesting {
boolean wasCalled= false
    String readFile(String fileName){
        //file reading code or mocked stuff for testing
        wasCalled=true
    }
}

Then when running a test, you can

@Test
public void whenCallingPipelineUtilClassIsUsed(){
    UtilForTesting utilForTesting = new UtilForTesting()
    UtilFactory .setInstance(utilForTesting)
    //run pipeline
    assertThat("Mocked Utils was not called", utilForTesting.getWasCalled(), is(true))
}

But that means that you have to override all pipeline methods the util class calls.

[giorgiosironi]

I think I am experiencing this too. I am able (sort-of) to load and run a vars/ step that uses echo or other Jenkins steps directly. I am not able to call other vars/ steps that rely on Jenkins steps from that one:

groovy.lang.MissingMethodException: No signature of method: sayHello.echo() is applicable for argument types: (java.lang.String) values: [Just hello]

It looks like a bug because registerAllowedMethod is built to allow this, but no matter what I set up with that I can't get past this exception.

[giorgiosironi]

But, I've seen this done in https://github.com/SAP/jenkins-library not sure what magic is necessary.

[luispollo]

If this helps anyone, I just spent an afternoon trying to figure this out, and what it boiled down for me was that including the vars directory from the library in your main source set is what broke it.

This works:

sourceSets {
    main {
        groovy {
            srcDirs = ['src']
        }
    }

    test {
        groovy {
            srcDirs = ['test']
        }
    }
}

This doesn't:

sourceSets {
    main {
        groovy {
            srcDirs = ['src', 'vars']
        }
    }

    test {
        groovy {
            srcDirs = ['test']
        }
    }
}

I guess it is because the library scripts in vars need to be compiled on-the-fly by the loader to inject those global symbols, and when you add them to the main source set they get pre-compiled (without any of those symbols). But I'm out of my depth here, just a guess... 😃

[luispollo]

P.S. And I just now found the same solution mentioned here: #25 (comment)

Oh well. Now we know. 😃

[jacob-keller]

I'm seeing what appears to be the same issue, but I actually don't have sourceSets defined in my build.gradle... I'm not sure what I'm doing wrong, but my shared library functions can't call steps like echo.

Is there some other way that the groovy library code could be pulled into the class path?

[jacob-keller]

I think my issue is because I'm trying to run steps inside of the src/* functions of my library. Is there a way to make that work?

[skatlapa]

@jacob-keller Hi, I am facing issues on the similar lines, I am using steps() and node() under src/* is that the reason? Official docs say this:

"Only entire pipelines can be defined in shared libraries as of this time. This can only be done in vars/*.groovy, and only in a call method. Only one Declarative Pipeline can be executed in a single build, and if you attempt to execute a second one, your build will fail as a result"

Should I move the steps, stages, etc. sections to vars/* ?

[skatlapa]

@giorgiosironi @luispollo @jacob-keller Hi! I am facing similar issues as folks in this thread. I am using declarative pipelines and debugged my issues to pipeline basic steps not being able to be recognized from src/ .
I am planning to restructure my libraries to move those parts into vars/ , but before that I just wanted to know if there's any way that I could get away with using pipeline steps in src/.
Any suggestions/tips are highly appreciated!

[jacob-keller]

@skatlapa Hmm. I don't remember if I got this resolved or simply worked around it.

I think the way I got something like this working, is that I created a pipeline script which pulls the steps and then I forward those into the src/* class via a class variable. This way, the pipeline in the test framework starts, gets the current steps, and assigns them to the class. Finally, all of my src/* code references full steps via "steps.", i.e. "steps.sh" or "steps.echo" or "steps.stage".

I believe that's how I got everything working, but I could be wrong. I'll try to double check if I can find any more details on this tomorrow.

[skatlapa]

@jacob-keller Thanks a lot for the response, I assume you did something like this: https://github.com/zowe/jenkins-library/blob/master/src/org/zowe/jenkins_shared_library/pipelines/base/Pipeline.groovy

I think that works too, but my current use case is pretty convoluted/complex. I could switch to using scripted pipelines to workaround but I am more inclined towards maintaining/get it working with declarative pipelines.

Also, could you let me know if you've used declarative pipelines as well? You can get back when you get a moment, thanks again!

[jacob-keller]

To clarify, you can do the "steps" setup from within a vars/* step itself, so then it could work within a declarative pipeline as well.

I haven't really used declarative pipelines myself, as most of the pipelines I wrote are from before the declarative style existed.

[skatlapa]

@jacob-keller Thanks for the clarification, I am aware of that. I have previously defined large(entire) parts of declarative pipelines under vars and called the scripts into the Jenkinsfiles and I had no problems at all.

I am trying to do something around the lines of this: https://gist.github.com/skatlapa/cd96b98bda9806cf134c03b6c30737d9
and I want to use the main src class method in most of my vars/scripts so I initially defined it under src/.

Notice the pipeline steps https://gist.github.com/skatlapa/cd96b98bda9806cf134c03b6c30737d9#file-srcmethod-groovy-L8 , node {..} here is not being recognized and get the error similar to the error(s) in the thread. I get the same error for steps {..}, retry(){}, etc.

I wanted to know a way to keep that method in src/ and make it work, also any other tips related or otherwise are appreciated. Please note I am using declarative style of Jenkinsfile.

[jacob-keller]

See https://jenkins.io/doc/book/pipeline/shared-libraries/#accessing-steps for some example of how to access steps in the src/* classes:

in the vars/* code:
@Library('utils') import org.foo.Utilities
def utils = new Utilities(this)
node {
utils.mvn 'clean package'
}

In the src/* classes:
package org.foo
class Utilities implements Serializable {
def steps
Utilities(steps) {this.steps = steps}
def mvn(args) {
steps.sh "${steps.tool 'Maven'}/bin/mvn -o ${args}"
}
}

Hope that helps. Basically you ultimately need to pass "this" or "this.steps" into the method somehow from the vars/* code, but the vast majority including actually calling steps can be done in src/* code.

[skatlapa]

@jacob-keller Thanks! That seems to make sense. I will give it a go and update here.

[skatlapa]

Update - That did not work for me, I may be calling/using scripts step incorrectly, I haven't gotten chance to dive deep. I'll get back to this the next week and keep you posted. Thanks!

[BernalCarlos]

@luispollo Removing the "vars" directory for the "srcDir", does work, but it breaks IntelliJ autocompletion for code inside the vars directory, do you know what else can be done?

[timbrown5]

I am trying to test the methods of a class without needing to create an entry in /var for each method.

I have a class:

package example

class ExampleClass {

  private final script

  ExampleClass(final script) {
    this.script = Objects.requireNonNull(script)
  }

  void sayHelloTo(String name) {
    script.echo("Hello there $name")
  }
}

Which I am trying to test like so:

package example

import com.lesfurets.jenkins.unit.BasePipelineTest
import org.junit.Before
import org.junit.Test

class TestExampleClass extends BasePipelineTest {

    @Override
    @Before
    void setUp() throws Exception {
        helper.scriptRoots += 'vars'
        super.setUp()
    }

    @Test
    void testExampleClassSayHelloTo() throws Exception {
        helper.registerAllowedMethod("echo", [Map.class], null)
        def exampleClass = new ExampleClass(helper)
        exampleClass.sayHelloTo("Fred")
    }
}

This seems to follow what @jacob-keller said above (passing the pipeline into the class and store it, then the pipeline methods through that).
I am still getting an error when running the test, even though I have registered a helper for the method I am calling:
Error:

testExampleClassSayHelloTo
groovy.lang.MissingMethodException: No signature of method: com.lesfurets.jenkins.unit.PipelineTestHelper.echo() is applicable for argument types: (org.codehaus.groovy.runtime.GStringImpl) values: [Hello there Fred]
Possible solutions: each(groovy.lang.Closure), macro(groovy.lang.Closure), wait(), init(), find(), every()
	at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.unwrap(ScriptBytecodeAdapter.java:72)
	at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:53)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:128)
	at example.ExampleClass.sayHelloTo(ExampleClass.groovy:12)
	at example.ExampleClass$sayHelloTo.call(Unknown Source)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:128)
	at example.TestExampleClass.testExampleClassSayHelloTo(TestExampleClass.groovy:21)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
...
...

Am I doing something wrong, or is this expected not to work?
I believe this is related to the OP, but if not I can raise a new issue.

[stchar]

@timbrown5,
You can try this instead of helper

def exampleClass = new ExampleClass(this)

But I'm not sure if it will work. If you're testing a shared library please see README for demo projects.

[timbrown5]

@stchar
I get a similar issue with ExampleClass(this).
I seem to have come full circle a little in this one. I worked around the above issue by testing ExampleClass by adding a script to vars which wraps it.

This allowed me to write some unit tests for a portion of my code. Then I started trying to test a groovy class in src which has closures inside it. Now I have hit this issue describe by @luispollo, above, which mainifests as:

groovy.lang.MissingMethodException: No signature of method: com.lesfurets.jenkins.unit.PipelineTestHelper.sshagent() is applicable for argument types: (ArrayList, xxxx.pipeline.os.UnixHelper$_create_conan_package_closure1) values: [[dummy_credential], xxxx.pipeline.os.UnixHelper$_create_conan_package_closure1@7ec12c86]
	at xxxx.pipeline.os.UnixHelper.create_conan_package(UnixHelper.groovy:24)
	at tests.xxxx.pipeline.os.UnixHelperTest.call create_conan_package(UnixHelperTest.groovy:36)

My plan was to use Groovy classes which contain common pipeline actions to help abstract away OS differences. Not sure this it's possible to write tests for this with the current implementation.

I will try writing wrappers scripts, in /vars, to wrap each method class and see if I can use those to write tests.

[philippart-s]

To test this kind of classes I do this way:

In my resources test I create a file foo.groovy

def version = '1.0'

return this

And my tests class:

class TestExampleClass extends BasePipelineTest  {

    Script steps = null

    @Override
    @Before
    void setUp() throws Exception {
        this.scriptRoots += "test/resources"
        this.scriptRoots += "src"
        super.setUp()

        LibraryConfiguration library = LibraryConfiguration.library().name("jenkins-ci")
                .retriever(new ProjectSourceRetreiver())
                .targetPath("N/A")
                .defaultVersion("master")
                .allowOverride(true)
                .implicit(true)
                .build();
        helper.registerSharedLibrary(library);
        helper.registerAllowedMethod("echo", [String.class]) { String message ->
            println(message)
        }
        steps = loadScript('foo.groovy')
    }

    @Test
    void testExampleClassSayHelloTo() {
        ExampleClass exampleClass  = new ExampleClass(steps)
        exampleClass.sayHelloTo('Stéphane')
    }
}

The output of my test:

Loading shared library jenkins-ci with version master
Hello there Stéphane

In my opinion the use of the foo script is a trick but it's the aonly way I found to test util method in a sharedlib that use jenkins methods (echo in the example).

If there is a best way to do that tell me !

[timbrown5]

@philippart-s Wow, that is a neat trick. I've need scratching my head for a while so thanks for the help 👍.

I had a little problem checking that it fixed my issue, so am adding my source below incase it helps anyone:

package example

import com.lesfurets.jenkins.unit.BasePipelineTest
import org.junit.Before
import org.junit.Test

import static org.assertj.core.api.Assertions.assertThat
import static com.lesfurets.jenkins.unit.global.lib.LibraryConfiguration.library
import static com.lesfurets.jenkins.unit.global.lib.ProjectSource.projectSource


class TestExampleClass extends BasePipelineTest  {

    Script steps = null

    @Override
    @Before
    void setUp() throws Exception {
        this.scriptRoots += "test/resources"
        this.scriptRoots += "src"
        super.setUp()

        def library = library().name("jenkins-ci")
                .defaultVersion('<notNeeded>')
                    .allowOverride(true)
                    .implicit(true)
                    .targetPath('<notNeeded>')
                    .retriever(projectSource())
                    .build()
        helper.registerSharedLibrary(library);
        helper.registerAllowedMethod("echo", [String.class]) { String message ->
            println(message)
        }
        // This looks in 'src' directory
        steps = loadScript('foo.groovy')
    }

    @Test
    void testExampleClassSayHelloTo() {
        ExampleClass exampleClass  = new ExampleClass(steps)
        assertThat(helper.callStack.findAll { call ->
            call.methodName == "echo"
        }.any { call ->
            callArgsToString(call).contains("Hello Stéphane")
        }).isFalse()
    }
}

Further to the other issue I found a way to get my use case above to work... use maven 🤢. I have ported my gradle project to maven and have the same test passing when using maven test and failing when using gradle test. I think this is why the guys at https://github.com/SAP/jenkins-library were write tests the same way I wanted to.
Not sure if this is a gradle bug, or incompatiblity, but it means the framework be used like this, so hopefully it can be made to work with gradle.gram

[stchar]

@timbrown5, could you please also share your build.gradle

[timbrown5]

@timbrown5, could you please also share your build.gradle
I will look at creating a sampel with both types of projects, although I have just realised Maven wasn't running tests with class name like xxxSpec, so maybe maven wasn't actually running the tests which were failing with gradle. I now have all all my tests working in both, so thinking my previous comment could be a false alarm.

[stchar]

@timbrown5, cool, this is a great starting point to a create wiki.

[philippart-s]

@timbrown5, cool, this is a great starting point to a create wiki.

I was planning to offer documentation if my approach was the right one, so if you want, I would like to participate in writing an article if that helps.

[timbrown5]

I was planning to offer documentation if my approach was the right one, so if you want, I would like to participate in writing an article if that helps.

That's fine by me 👍.
Will upload a Hello World style example (with gradle and maven) when I have one, which can hopefully help people. When I can to write tests I found a lot of the examples were too busy to get the core of what I wanted working.
I will also use this to see if I can reproduce the gradle issue I was havinng, or if it was a mistake on my part :).

[timbrown5]

example_jenkins_shared_library_tests.tar.gz
Here is the simple example from above. I think my 'gradle' problem was that I hadn't passed script into ExampleClass (I was still passing this). I think maven wasn;t running it because it was named ExampleClassSpec.groovy.

[mcascone]

What is this community's current best guideline as of today, May 2021, for testing Declarative pipelines that use a shared library?

I've built a shared library that is being adopted across my dev org. Each app's repo includes the library and passes in a few specific params, and the rest is done by the shared lib:

// github.mycompany.com/some-app-repo/Jenkinsfile
library 'shared-pipeline-library-repo'

servicePipeline {
	someParam = 'value'
}

// github.mycompany.com/shared-pipeline-library-repo
|-Jenkinsfile
├───jenkins-workflow
│       application.xml
│       unitTest.ps1
│
├───resources
│       pipelineHelpers.ps1
│       pipelineHelpers.Tests.ps1
│
└───vars
        addAllErrorsToInfoMessages.groovy
        addStageResultToInfoMessages.groovy
        addToInfoMessages.groovy
        deploy.groovy
        deployApproved.groovy
        doIntegrationTests.groovy
        doRestore.groovy
		servicePipeline.groovy

I've trimmed the list of files in vars/ and resources/ but you get the idea. The servicePipeline.groovy is the actual shared pipeline with all the stages and steps etc, written in Declarative Jenkins DSL. It follows the pattern detailed here:

def call(body) {
  // this is a jenkins-provided pattern to take in the pipeline params
  def pipelineParams = [:]
  body.resolveStrategy = Closure.DELEGATE_FIRST
  body.delegate = pipelineParams
  body()

  pipeline {
	// the pipeline
}

The jenkins-workflow folder is required to be in the application repos, where the shared pipeline looks for the unique build scripts and other support files. application.xml is one of those, and unitTest.ps1 is a quick call to invoke-pester to run the pester tests on the files in resources/
And, there's a Jenkinsfile in the shared library's repo itself, so it actually runs itself (this is when the unitTest.ps1 script comes in). So I'm using the library to test itself, which feels like a good idea.

My main question is, how can I test the groovy files and the pipeline itself? Is it possible in this current structure, or will I have to port everything into vars/ into proper groovy classes in src/? I have a background in object-oriented design from the C++ days, but I'm not a "java guy", although I've really come to love groovy. But I'd really like to avoid refactoring all of this if I can. I feel like I've read every article out there on JenkinsPipelineUnit, Spock, maven/gradle, etc, and I can't seem to find a way to do it without completely refactoring everything, and even then I don't really feel comfortable that I'd end up in a good place.

I've been able to set up the Pester tests for the powershell scripts fairly easily. In fact I've considered moving as much functionality out of the groovy files as I can, just so I can test it; but then the tradeoff is the loss of object permanence going in and out of powershell (since env vars can only be strings. Happy to explain this more if I'm not being clear). Is there a way to use inline assertions in the .groovy files in vars, or in similar companion test files similar to the pester style?

UPDATE May-20-2021

I've landed on a combination of methods to get to a functional state. I'm still having trouble with mocking pwsh, but I've actually got a working build and (very) basic tests running. As I work my way through this, I'm writing it down for future reference here: https://github.com/mcascone/test-jenk-unit. Please feel free to stop by and contribute comments, suggestions, better ways, and anything else!