should run a deployment to completion and then scale to zero [Conformance]

[0xmichalis]

Because of the latest changes in deployment logs

STEP: verifying the scale is updated on the deployment config
STEP: deploying a few more times
Jul 22 22:07:25.246: INFO: Running 'oc deploy --namespace=extended-test-cli-deployment-f3mzp-9tfjf --config=/tmp/openshift/extended-test-cli-deployment-f3mzp-9tfjf-user.kubeconfig --latest --follow deployment-test'
Jul 22 22:07:45.721: INFO: Error running &{/data/src/github.com/openshift/origin/_output/local/bin/linux/amd64/oc [oc deploy --namespace=extended-test-cli-deployment-f3mzp-9tfjf --config=/tmp/openshift/extended-test-cli-deployment-f3mzp-9tfjf-user.kubeconfig --latest --follow deployment-test] []   Started deployment #2
Error from server: The get operation against ReplicationController could not be completed at this time, please try again.
 Started deployment #2
Error from server: The get operation against ReplicationController could not be completed at this time, please try again.
 [] <nil> 0xc820d49f20 exit status 1 <nil> true [0xc82002f208 0xc82002f230 0xc82002f230] [0xc82002f208 0xc82002f230] [0xc82002f210 0xc82002f228] [0xa7ed60 0xa7eec0] 0xc8202c1080}:
Started deployment #2
Error from server: The get operation against ReplicationController could not be completed at this time, please try again.

• Failure [63.861 seconds]
deploymentconfigs
/data/src/github.com/openshift/origin/test/extended/deployments/deployments.go:598
  with test deployments
  /data/src/github.com/openshift/origin/test/extended/deployments/deployments.go:252
    should run a deployment to completion and then scale to zero [Conformance] [It]
    /data/src/github.com/openshift/origin/test/extended/deployments/deployments.go:251

    Expected error:
        <*exec.ExitError | 0xc820d49f80>: {
            ProcessState: {
                pid: 4354,
                status: 256,
                rusage: {
                    Utime: {Sec: 0, Usec: 120189},
                    Stime: {Sec: 0, Usec: 20722},
                    Maxrss: 28188,
                    Ixrss: 0,
                    Idrss: 0,
                    Isrss: 0,
                    Minflt: 6714,
                    Majflt: 0,
                    Nswap: 0,
                    Inblock: 0,
                    Oublock: 0,
                    Msgsnd: 0,
                    Msgrcv: 0,
                    Nsignals: 0,
                    Nvcsw: 2160,
                    Nivcsw: 15,
                },
            },
            Stderr: nil,
        }
        exit status 1
    not to have occurred

    /data/src/github.com/openshift/origin/test/extended/deployments/deployments.go:240
------------------------------

[smarterclayton]

What does "The Get operation cannot be completed at this time"? Is this the max rate limiter on the server? Do we have something hot looping?

[liggitt]

no, I think it's https://github.com/openshift/origin/blob/master/pkg/deploy/registry/deploylog/rest.go#L154-L159

latest, ok, err := registry.WaitForRunningDeployment(r.rn, target, r.timeout)
if err != nil {
    return nil, errors.NewBadRequest(fmt.Sprintf("unable to wait for deployment %s to run: %v", deployutil.LabelForDeployment(target), err))
}
if !ok {
    return nil, errors.NewServerTimeout(kapi.Resource("ReplicationController"), "get", 2)
}

[0xmichalis]

The client will retry on serverTimeout

[smarterclayton]

Well, the message shouldn't be written to output, so this looks like a minor bug there. If this is truly "timeout forever" then this flake is now "container did not start"

[smarterclayton]

Raising priority because this is now showing up fairly frequently in PRs

[smarterclayton]

#10176

[0xmichalis]

Well, the message shouldn't be written to output, so this looks like a minor bug there. If this is truly "timeout forever" then this flake is now "container did not start"

Is anybody familiar with the place in the client where server timeouts are retried? I couldn't find it. This is most probably a "deployer container did not start" error but I am not sure at which point should we intercept the message. All old test logs are dead so it would be really helpful next time this occurs to post the failure trap output.

[smarterclayton]

request.go

On Mon, Aug 15, 2016 at 11:23 AM, Michail Kargakis <[email protected]

wrote:

Well, the message shouldn't be written to output, so this looks like a
minor bug there. If this is truly "timeout forever" then this flake is now
"container did not start"

Is anybody familiar with the place in the client where server timeouts are
retried? I couldn't find it. This is most probably a "deployer container
did not start" error but I am not sure at which point should we intercept
the message. All old test logs are dead so it would be really helpful next
time this occurs to post the failure trap output.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#10002 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABG_p3Jn7JBDCZB2wGd865R6awbWoiRcks5qgISFgaJpZM4JTUtn
.