Skip to content
This repository was archived by the owner on Nov 7, 2023. It is now read-only.

Commit

Permalink
Created new fork and commit mesasges to local branch to it:
Browse files Browse the repository at this point in the history 
commit dab42e57ce1a1b9ca1b2f8251f2d89127cd69d04
Author: tim.brown5 <[email protected]>
Date:   Mon Jan 27 17:40:43 2014 +0000

    Added the ability to specify a PollInterval value - to stop the plugin spamming the logs. Also Fixed some typos in the field titles in config.jelly.

commit 81558c521857bea4c0d0c6c951b759f8e28bdab4
Author: brownt <[email protected]>
Date:   Thu Jan 23 15:10:27 2014 +0000

    Changed the tile of check box from 'Block execution of remote builds while other builds are running.' to 'Block triggering of remote builds while other builds are running.' to better describe what it does.

commit b78f021cd0b7ddccd3e84fc2fc2758fd8d92a092
Author: brownt <[email protected]>
Date:   Thu Jan 23 15:08:25 2014 +0000

    Added to checkBoxes to allow toggling of the following fnctionality:
    - Block triggering of remote builds while other builds are running.
    - Block build step until remove build is complete.

    The first gets round an issue where multiple jobs are all trying to launch jobs at once and filling up the build queue on the remote server. This causes an issues because when jobs are in the queue you cannot get their status. Another way to solve this would be to assume that the job is in the build queue and 'qait' until you can access the job's build page.

    The second allows the user to make the local job block until the remote job has finished (and so set the status of the local job to that of the remote job). This is a fix for issue JENKINS-20828. If you are using this feature it's best to also use the feature above, as otherwise your builds will fail when remote builds you have triggered are waiting in the build queue.

    This works best if only one job is triggering builds on a remote job. This is because is two builds trigger a job at the same time it is possible for them both to think next next build is the one they triggered... A solution to this could be comparing the parameters??

commit 22983d13fc8a1b63b3686facfcd10d31d51f074a
Author: brownt <[email protected]>
Date:   Wed Jan 22 15:37:41 2014 +0000

    Added 'fix' as describe in: https://issues.jenkins-ci.org/browse/JENKINS-20828

    *Feel free to ignore this branch if you are working on this yourself. Also free free to use this code as you see fit.*

    "I have created a workaround/fix for this issue. My code does the following:

        get the nextBuildNumber from the job
        trigger the build
        check that next build number has increased by one - this means we know that the only job triggered is the one we triggers, otherwise we error
        then wait for the job with the build number above to start
        then wait for the job with the build number above to finish and return the result.

    I can share the code if you are interested
    *I also split out the HTTPRequest code into a method, so you can call different URLs with different methods (to simplify the communication with the remote server).

    I do agree however, it would be a lot easier if the trigger call returned the build number of the job it's just triggered. Could raise an issue to see if this can be done?
    "

    There are issues with this 'fix' when multiplem jobs are triggering the same remote job:
    1. It blocks all the time - would probably need to add a toggle button to turn the blocking on/off
    2. It returns failure if it tries to check on a remote job and the job has not started yet - as it goes to the job
    3. If two jobs are launched at the same time they can both pickup the same job ID - so they

    - These issues shouldn't affect quiet environments, but ones that kickoff a lot of the same jobs in a few seconds will see issues. There also might be issues if the remote job takes a long time. I cannot think of a way to get round these issues at the moment...

    To test this I setup multiple local jobs all triggering one remote job. When running them all one at a time there were no issues, but running them all at once there were issues.
  • Loading branch information
tim.brown5 committed Jan 28, 2014
1 parent 5aac90b commit d32c69d
Show file tree
Hide file tree
Showing 2 changed files with 252 additions and 35 deletions.
275 changes: 240 additions & 35 deletions src/main/java/org/jenkinsci/plugins/ParameterizedRemoteTrigger/RemoteBuildConfiguration.java
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,11 @@
import hudson.tasks.Builder;
import hudson.tasks.BuildStepDescriptor;
import net.sf.json.JSONObject;

import net.sf.json.JSONArray;
import net.sf.json.JSONSerializer;
//import net.sf.json.
//import net.sf.json.

import org.jenkinsci.plugins.tokenmacro.TokenMacro;
import org.apache.commons.lang.StringUtils;
import org.jenkinsci.plugins.tokenmacro.MacroEvaluationException;
Expand Down Expand Up @@ -44,7 +48,12 @@ public class RemoteBuildConfiguration extends Builder {
private final String token;
private final String remoteJenkinsName;
private final String job;
private final boolean shouldNotFailBuild;

private final boolean shouldNotFailBuild ;
private final int pollInterval;
private final boolean preventRemoteBuildQueue;
private final boolean blockBuildUntilComplete;

// "parameters" is the raw string entered by the user
private final String parameters;
// "parameterList" is the cleaned-up version of "parameters" (stripped out comments, character encoding, etc)
Expand All @@ -61,14 +70,17 @@ public class RemoteBuildConfiguration extends Builder {
private String queryString = "";

@DataBoundConstructor
public RemoteBuildConfiguration(String remoteJenkinsName, boolean shouldNotFailBuild, String job, String token,
public RemoteBuildConfiguration(String remoteJenkinsName, boolean shouldNotFailBuild, boolean preventRemoteBuildQueue, boolean blockBuildUntilComplete, int pollInterval, String job, String token,
String parameters, JSONObject overrideAuth) throws MalformedURLException {

this.token = token.trim();
this.remoteJenkinsName = remoteJenkinsName;
this.parameters = parameters;
this.job = job.trim();
this.shouldNotFailBuild = shouldNotFailBuild;
this.preventRemoteBuildQueue = preventRemoteBuildQueue;
this.blockBuildUntilComplete = blockBuildUntilComplete;
this.pollInterval = pollInterval;
if (overrideAuth != null && overrideAuth.has("auth")) {
this.overrideAuth = true;
this.auth.replaceBy(new Auth(overrideAuth.getJSONObject("auth")));
Expand All @@ -85,14 +97,17 @@ public RemoteBuildConfiguration(String remoteJenkinsName, boolean shouldNotFailB

}

public RemoteBuildConfiguration(String remoteJenkinsName, boolean shouldNotFailBuild, String job, String token,
public RemoteBuildConfiguration(String remoteJenkinsName, boolean shouldNotFailBuild, boolean preventRemoteBuildQueue, boolean blockBuildUntilComplete, int pollInterval, String job, String token,
String parameters) throws MalformedURLException {

this.token = token.trim();
this.remoteJenkinsName = remoteJenkinsName;
this.parameters = parameters;
this.job = job.trim();
this.shouldNotFailBuild = shouldNotFailBuild;
this.preventRemoteBuildQueue = preventRemoteBuildQueue;
this.blockBuildUntilComplete = blockBuildUntilComplete;
this.pollInterval = pollInterval;
this.overrideAuth = false;
this.auth.replaceBy(new Auth(null));

Expand Down Expand Up @@ -308,6 +323,31 @@ private String buildTriggerUrl(String job, String securityToken, Collection<Stri
return triggerUrlString;
}

/**
* Build the proper URL for GET calls
*
* All passed in string have already had their tokens replaced with real values.
*
* @param job
* Name of the remote job
* @param securityToken
* Security token used to trigger remote job
* @return fully formed, fully qualified remote trigger URL
*/
private String buildGetUrl (String job, String securityToken) {

RemoteJenkinsServer remoteServer = this.findRemoteHost(this.getRemoteJenkinsName());
String urlString = remoteServer.getAddress().toString();

urlString += "/job/";
urlString += this.encodeValue(job);

// don't try to include a security token in the URL if none is provided
if (!securityToken.equals("")) {
this.addToQueryString("token=" + encodeValue(securityToken));
}
return urlString;
}
/**
* Convenience function to mark the build as failed. It's intended to only be called from this.perform();
*
Expand All @@ -330,39 +370,191 @@ private void failBuild(Exception e, BuildListener listener) throws IOException {

@Override
public boolean perform(AbstractBuild build, Launcher launcher, BuildListener listener) throws IOException {

RemoteJenkinsServer remoteServer = this.findRemoteHost(this.getRemoteJenkinsName());

//Stores the status of the remote build
String buildStatusStr = "UNKNOWN";

if (remoteServer == null) {
this.failBuild(new Exception("No remote host is defined for this job."), listener);
return true;
}

//tokenize all variables and encode all variables, then build the fully-qualified trigger URL
List<String> parameters = getCleanedParameters();
parameters = replaceTokens(build, listener, parameters);
String jobName = replaceToken(build, listener, this.getJob());
String securityToken = replaceToken(build, listener, this.getToken());
String triggerUrlString = this.buildTriggerUrl(jobName, securityToken, parameters);


//Trigger remote job
//print out some debugging information to the console
listener.getLogger().println("URL: " + triggerUrlString);
listener.getLogger().println("Triggering this job: " + jobName);
listener.getLogger().println("Using this remote Jenkins config: " + this.getRemoteJenkinsName());

//get the ID of the Next Job to run.
if ( this.getPreventRemoteBuildQueue() ) {
listener.getLogger().println("Checking that the remote job " + jobName + " is not building.");
String preCheckUrlString = this.buildGetUrl( jobName , securityToken );
preCheckUrlString += "/lastBuild";
preCheckUrlString += "/api/json/";
JSONObject preCheckResponse = sendHTTPCall(preCheckUrlString, "POST", build, listener);

//check the latest build on the remote server to see if it's running - if so wait until it has stopped.
//if building is true then the build is running
//if result is null the build hasn't finished - but might not have started running.
while ( preCheckResponse.getBoolean("building") == true
|| preCheckResponse.getString("result") == null ) {
listener.getLogger().println("Build running - waiting for build to finish.");
preCheckResponse = sendHTTPCall(preCheckUrlString, "POST", build, listener);
listener.getLogger().println("Waiting for " + this.pollInterval + " seconds until next poll.");

//Sleep for 'pollInterval' seconds.
//Sleep takes miliseconds so need to convert this.pollInterval to milisecopnds (x 1000)
try {
Thread.sleep( this.pollInterval * 1000);
} catch (InterruptedException e) {
this.failBuild(e, listener);
}
}
listener.getLogger().println("Remote job remote job " + jobName + " is not building.");
} else {
listener.getLogger().println("Not checking if the remote job " + jobName + " is building.");
}

String queryUrlString = this.buildGetUrl( jobName, securityToken );
queryUrlString += "/api/json/";

listener.getLogger().println("Getting ID of next job to build. URL: " + queryUrlString);
JSONObject queryResponseObject = sendHTTPCall(queryUrlString, "POST", build, listener);
int nextBuildNumber = queryResponseObject.getInt( "nextBuildNumber" );
listener.getLogger().println("Getting ID of next job to build." );

if (this.getOverrideAuth()) {
listener.getLogger().println("Using job-level defined credentails in place of those from remote Jenkins config [" + this.getRemoteJenkinsName() + "]" );
}

listener.getLogger().println("Triggering remote job." );
JSONObject responseObject = sendHTTPCall(triggerUrlString, "POST", build, listener);

String jobURL = responseObject.getString( "url" );
int newNextBuildNumber = responseObject.getInt( "nextBuildNumber" ); // This should be nextBuildNumber + 1 OR there has been another job scheduled.

//This is only for Debug
if (newNextBuildNumber == (nextBuildNumber + 1)) {
listener.getLogger().println("DEBUG: No other jobs triggered" );
} else if( newNextBuildNumber > (nextBuildNumber + 1) ) {
listener.getLogger().println("DEBUG: WARNING Other jobs triggered," + newNextBuildNumber + ", " + nextBuildNumber );
} else {
listener.getLogger().println("DEBUG: WARNING Did not get the correct build number for the triggered job, previous nextBuildNumber:" + newNextBuildNumber + ", newNextBuildNumber" + nextBuildNumber );
}

//If we are told to block until remoteBuildComplete:
if ( this.getBlockBuildUntilComplete() ) {
listener.getLogger().println("Blocking local job until remote job completes" );
//Form the URL for the triggered job
String jobLocation = jobURL + nextBuildNumber + "/api/json";

buildStatusStr = getBuildStatus(jobLocation, build, listener);



while ( buildStatusStr.equals("not started") ) {
listener.getLogger().println("Waiting for remote build to start.");
listener.getLogger().println("Waiting for " + this.pollInterval + " seconds until next poll.");
buildStatusStr = getBuildStatus(jobLocation, build, listener);
//Sleep for 'pollInterval' seconds.
//Sleep takes miliseconds so need to convert this.pollInterval to milisecopnds (x 1000)
try {
Thread.sleep( this.pollInterval * 1000);
} catch (InterruptedException e) {
this.failBuild(e, listener);
}
}

listener.getLogger().println("Remote build started!");
while ( buildStatusStr.equals("running") ) {
listener.getLogger().println("Waiting for remote build to finish.");
listener.getLogger().println("Waiting for " + this.pollInterval + " seconds until next poll.");
buildStatusStr = getBuildStatus(jobLocation, build, listener);
//Sleep for 'pollInterval' seconds.
//Sleep takes miliseconds so need to convert this.pollInterval to milisecopnds (x 1000)
try {
Thread.sleep( this.pollInterval * 1000);
} catch (InterruptedException e) {
this.failBuild(e, listener);
}
}
listener.getLogger().println("Remote build finished with status " + buildStatusStr + ".");

// If build did not finish with 'success' then fail build step.
if ( !buildStatusStr.equals("SUCCESS") ) {
//failBuild will check if the 'shouldNotFailBuild' parameter is set or not, so will decide how to handle the failure.
this.failBuild(new Exception("The remote job did not succeed."), listener);
}
} else {
listener.getLogger().println("Not blocking local job until remote job completes - fire and forget." );
}

return true;
}

public String getBuildStatus ( String buildUrlString, AbstractBuild build, BuildListener listener ) throws IOException {
String buildStatus = "UNKNOWN";

RemoteJenkinsServer remoteServer = this.findRemoteHost(this.getRemoteJenkinsName());

if (remoteServer == null) {
this.failBuild(new Exception("No remote host is defined for this job."), listener);
return null;
}

//print out some debugging information to the console
listener.getLogger().println("Checking Status of this job: " + buildUrlString);
if (this.getOverrideAuth()) {
listener.getLogger().println("Using job-level defined credentails in place of those from remote Jenkins config [" + this.getRemoteJenkinsName() + "]" );
}
listener.getLogger().println("With these parameters: " + parameters.toString());

// uncomment the 2 lines below for debugging purposes only
// listener.getLogger().println("Fully Built URL: " + triggerUrlString);
// listener.getLogger().println("Token: " + securityToken);
JSONObject responseObject = sendHTTPCall( buildUrlString, "GET", build, listener);

HttpURLConnection connection = null;
//get the next build from the location

try {
URL triggerUrl = new URL(triggerUrlString);
connection = (HttpURLConnection) triggerUrl.openConnection();
//System.out.println( "\n\n##### JSON Object - Build!!! #####\n\n" );
//System.out.println( responseObject.toString() );

if ( responseObject.getString("result") == null && responseObject.getBoolean("building") == false ) {
//build not started
buildStatus = "not started";
} else if ( responseObject.getBoolean("building") ) {
//build running
buildStatus = "running";
} else if ( responseObject.getString("result") != null ) {
//build finished
buildStatus = responseObject.getString("result");
} else {
//Add additional else to check for unhandled conditions
listener.getLogger().println("WARNING: Unhandled condition!" );
}

return buildStatus;
}

public JSONObject sendHTTPCall ( String urlString, String requestType, AbstractBuild build, BuildListener listener ) throws IOException{
RemoteJenkinsServer remoteServer = this.findRemoteHost(this.getRemoteJenkinsName());

if (remoteServer == null) {
this.failBuild(new Exception("No remote host is defined for this job."), listener);
return null;
}

HttpURLConnection connection = null;

JSONObject responseObject = null;

try {
URL buildUrl = new URL( urlString );
connection = (HttpURLConnection) buildUrl.openConnection();

// if there is a username + apiToken defined for this remote host, then use it
String usernameTokenConcat = "";
Expand All @@ -381,29 +573,32 @@ public boolean perform(AbstractBuild build, Launcher launcher, BuildListener lis
byte[] encodedAuthKey = Base64.encodeBase64(usernameTokenConcat.getBytes());
connection.setRequestProperty("Authorization", "Basic " + new String(encodedAuthKey));
}

connection.setDoInput(true);
connection.setRequestProperty("Accept", "application/json");
connection.setRequestMethod("POST");
connection.setRequestMethod(requestType);
// wait up to 5 seconds for the connection to be open
connection.setConnectTimeout(5000);
connection.connect();

// TODO: right now this is just doing a "fire and forget", but would be nice to get some feedback from the
// remote server. To accomplish this we would need to poll some URL
// - http://jenkins.local/job/test/lastBuild/api/json


InputStream is = connection.getInputStream();

BufferedReader rd = new BufferedReader(new InputStreamReader(is));
// String line;
// StringBuffer response = new StringBuffer();

// while ((line = rd.readLine()) != null) {
// System.out.println(line);
// }
// rd.close();

String line;
//String response = "";
StringBuilder response = new StringBuilder();

while ((line = rd.readLine()) != null) {
//System.out.println(line);
response.append(line);

}
rd.close();

JSONSerializer serializer = new JSONSerializer();
//need to parse the data we get back into struct
responseObject = (JSONObject) serializer.toJSON( response.toString() );

} catch (IOException e) {
// something failed with the connection, so throw an exception to mark the build as failed.
this.failBuild(e, listener);
Expand All @@ -416,15 +611,13 @@ public boolean perform(AbstractBuild build, Launcher launcher, BuildListener lis
if (connection != null) {
connection.disconnect();
}

// and always clear the query string and remove some "global" values
this.clearQueryString();
// this.build = null;
// this.listener = null;

}

return true;
return responseObject;
}

/**
Expand Down Expand Up @@ -458,7 +651,19 @@ public String getJob() {
public boolean getShouldNotFailBuild() {
return this.shouldNotFailBuild;
}


public boolean getPreventRemoteBuildQueue() {
return this.preventRemoteBuildQueue;
}

public boolean getBlockBuildUntilComplete() {
return this.blockBuildUntilComplete;
}

public int getPollInterval() {
return this.pollInterval;
}

public String getToken() {
return this.token;
}
Expand Down
Loading

0 comments on commit d32c69d

Please sign in to comment.