Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remove setting that's no longer needed #127

Closed
wants to merge 1 commit into from
Closed

Remove setting that's no longer needed #127

wants to merge 1 commit into from

Conversation

ycombinator
Copy link
Contributor

Now that elastic/kibana#77613 has been resolved, we can get rid of the workaround added in #111.

@ycombinator ycombinator requested a review from mtojek October 6, 2020 13:40
@ycombinator
Copy link
Contributor Author

ycombinator commented Oct 6, 2020 •
edited
Loading

I'm expecting CI to fail until the Kibana change is available in the Docker image. So let's keep this PR open and re-run CI until it passes consistently.

@elasticmachine
Copy link
Collaborator

elasticmachine commented Oct 6, 2020 •
edited
Loading

💔 Build Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: [Pull request #127 updated]

  • Start Time: 2020-10-12T18:57:32.452+0000

  • Duration: 12 min 31 sec

Steps errors 1

Expand to view the steps failures

  • Name: Check

    • Description: make check

    • Duration: 10 min 36 sec

    • Start Time: 2020-10-12T18:59:26.452+0000

    • log

Log output

Expand to view the last 100 lines of log output 

[2020-10-12T19:07:28.822Z] 2020/10/12 19:07:28 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T19:07:29.768Z] 2020/10/12 19:07:29 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T19:07:30.714Z] 2020/10/12 19:07:30 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T19:07:31.666Z] 2020/10/12 19:07:31 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T19:07:32.613Z] 2020/10/12 19:07:32 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T19:07:33.561Z] 2020/10/12 19:07:33 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T19:07:34.506Z] 2020/10/12 19:07:34 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T19:07:35.895Z] 2020/10/12 19:07:35 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T19:07:36.850Z] 2020/10/12 19:07:36 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T19:07:37.802Z] 2020/10/12 19:07:37 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T19:07:38.753Z] 2020/10/12 19:07:38 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T19:07:39.706Z] 2020/10/12 19:07:39 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T19:07:40.654Z] 2020/10/12 19:07:40 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T19:07:41.618Z] 2020/10/12 19:07:41 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T19:07:42.571Z] 2020/10/12 19:07:42 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T19:07:43.525Z] 2020/10/12 19:07:43 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T19:07:44.919Z] 2020/10/12 19:07:44 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T19:07:45.868Z] 2020/10/12 19:07:45 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T19:07:46.817Z] 2020/10/12 19:07:46 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T19:07:47.771Z] 2020/10/12 19:07:47 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T19:07:48.731Z] 2020/10/12 19:07:48 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T19:07:49.682Z] 2020/10/12 19:07:49 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T19:07:50.632Z] 2020/10/12 19:07:50 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T19:07:51.585Z] 2020/10/12 19:07:51 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T19:07:52.546Z] 2020/10/12 19:07:52 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T19:07:53.949Z] 2020/10/12 19:07:53 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T19:07:54.897Z] 2020/10/12 19:07:54 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T19:07:55.849Z] 2020/10/12 19:07:55 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T19:07:56.805Z] 2020/10/12 19:07:56 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T19:07:57.761Z] 2020/10/12 19:07:57 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T19:07:58.713Z] 2020/10/12 19:07:58 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T19:07:59.662Z] 2020/10/12 19:07:59 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T19:08:00.611Z] 2020/10/12 19:08:00 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T19:08:02.012Z] 2020/10/12 19:08:01 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T19:08:02.590Z] 2020/10/12 19:08:02 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T19:08:03.980Z] 2020/10/12 19:08:03 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T19:08:04.931Z] 2020/10/12 19:08:04 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T19:08:05.877Z] 2020/10/12 19:08:05 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T19:08:06.827Z] 2020/10/12 19:08:06 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T19:08:07.781Z] 2020/10/12 19:08:07 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T19:08:08.731Z] 2020/10/12 19:08:08 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T19:08:09.682Z] 2020/10/12 19:08:09 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T19:08:10.635Z] 2020/10/12 19:08:10 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T19:08:12.036Z] 2020/10/12 19:08:11 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T19:08:13.014Z] 2020/10/12 19:08:12 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T19:08:13.970Z] 2020/10/12 19:08:13 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T19:08:14.916Z] 2020/10/12 19:08:14 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T19:08:15.869Z] 2020/10/12 19:08:15 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T19:08:16.822Z] 2020/10/12 19:08:16 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T19:08:17.774Z] 2020/10/12 19:08:17 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T19:08:18.732Z] 2020/10/12 19:08:18 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T19:08:19.684Z] System test for apache/error data stream failed
[2020-10-12T19:08:19.684Z] 2020/10/12 19:08:19 DEBUG waiting for 30s before tearing down...
[2020-10-12T19:08:51.866Z] 2020/10/12 19:08:49 DEBUG reassigning original policy back to agent...
[2020-10-12T19:08:51.866Z] 2020/10/12 19:08:49 DEBUG PUT http://127.0.0.1:5601/api/fleet/agents/6846a485-aa8f-4af9-b8a9-0b2158a26c8d/reassign
[2020-10-12T19:08:51.866Z] 2020/10/12 19:08:49 DEBUG { "policy_id": "9c6180c0-0cbd-11eb-abea-1fae4748bbaa" }
[2020-10-12T19:08:52.440Z] 2020/10/12 19:08:52 DEBUG Wait until the policy (ID: 9c6180c0-0cbd-11eb-abea-1fae4748bbaa) is assigned to all agents...
[2020-10-12T19:08:54.361Z] 2020/10/12 19:08:54 DEBUG deleting test policy...
[2020-10-12T19:08:54.361Z] 2020/10/12 19:08:54 DEBUG POST http://127.0.0.1:5601/api/fleet/agent_policies/delete
[2020-10-12T19:08:54.361Z] 2020/10/12 19:08:54 DEBUG { "agentPolicyId": "fc31da90-0cbd-11eb-abea-1fae4748bbaa" }
[2020-10-12T19:08:57.688Z] 2020/10/12 19:08:57  INFO tearing down service...
[2020-10-12T19:08:57.688Z] 2020/10/12 19:08:57  INFO tearing down service using docker compose runner
[2020-10-12T19:08:57.688Z] 2020/10/12 19:08:57 DEBUG running command: /usr/local/bin/docker-compose -f /var/lib/jenkins/workspace/Beats_elastic-package_PR-127/src/github.com/elastic/elastic-package/test/packages/apache/_dev/deploy/docker/docker-compose.yml -p elastic-package-service down
[2020-10-12T19:08:58.266Z] Stopping elastic-package-service_apache_1 ... 
[2020-10-12T19:08:59.216Z] 
Stopping elastic-package-service_apache_1 ... done
Removing elastic-package-service_apache_1 ... 
[2020-10-12T19:08:59.216Z] 
Removing elastic-package-service_apache_1 ... done
Removing network elastic-package-service_default
[2020-10-12T19:08:59.216Z] 2020/10/12 19:08:59 DEBUG deleting data in data stream...
[2020-10-12T19:08:59.216Z] Error: error running package system tests: system test for apache/error data stream failed
[2020-10-12T19:08:59.216Z] + cleanup
[2020-10-12T19:08:59.216Z] + r=1
[2020-10-12T19:08:59.216Z] + elastic-package stack down -v
[2020-10-12T19:08:59.216Z] 2020/10/12 19:08:59 DEBUG Enable verbose logging
[2020-10-12T19:08:59.216Z] Take down the Elastic stack
[2020-10-12T19:08:59.216Z] 2020/10/12 19:08:59 DEBUG running command: /usr/local/bin/docker-compose -f /var/lib/jenkins/workspace/Beats_elastic-package_PR-127/.elastic-package/stack/snapshot.yml -p elastic-package-stack down
[2020-10-12T19:08:59.792Z] Stopping elastic-package-stack_elastic-agent_1    ... 
[2020-10-12T19:08:59.792Z] Stopping elastic-package-stack_kibana_1           ... 
[2020-10-12T19:08:59.792Z] Stopping elastic-package-stack_elasticsearch_1    ... 
[2020-10-12T19:08:59.792Z] Stopping elastic-package-stack_package-registry_1 ... 
[2020-10-12T19:09:02.529Z] 
Stopping elastic-package-stack_elastic-agent_1    ... done

Stopping elastic-package-stack_kibana_1           ... done

Stopping elastic-package-stack_package-registry_1 ... done

Stopping elastic-package-stack_elasticsearch_1    ... done
Removing elastic-package-stack_elastic-agent_is_ready_1    ... 
[2020-10-12T19:09:02.529Z] Removing elastic-package-stack_elastic-agent_1             ... 
[2020-10-12T19:09:02.529Z] Removing elastic-package-stack_kibana_is_ready_1           ... 
[2020-10-12T19:09:02.529Z] Removing elastic-package-stack_elasticsearch_is_ready_1    ... 
[2020-10-12T19:09:02.529Z] Removing elastic-package-stack_kibana_1                    ... 
[2020-10-12T19:09:02.529Z] Removing elastic-package-stack_package-registry_is_ready_1 ... 
[2020-10-12T19:09:02.529Z] Removing elastic-package-stack_elasticsearch_1             ... 
[2020-10-12T19:09:02.529Z] Removing elastic-package-stack_package-registry_1          ... 
[2020-10-12T19:09:02.529Z] 
Removing elastic-package-stack_package-registry_is_ready_1 ... done

Removing elastic-package-stack_kibana_1                    ... done

Removing elastic-package-stack_kibana_is_ready_1           ... done

Removing elastic-package-stack_elasticsearch_is_ready_1    ... done

Removing elastic-package-stack_package-registry_1          ... done

Removing elastic-package-stack_elastic-agent_is_ready_1    ... done

Removing elastic-package-stack_elasticsearch_1             ... done

Removing elastic-package-stack_elastic-agent_1             ... done
Removing network elastic-package-stack_default
[2020-10-12T19:09:02.792Z] Done
[2020-10-12T19:09:02.792Z] + exit 1
[2020-10-12T19:09:02.792Z] Makefile:29: recipe for target 'test-check-packages' failed
[2020-10-12T19:09:02.792Z] make: *** [test-check-packages] Error 1
[2020-10-12T19:09:03.140Z] Running on worker-395930 in /var/lib/jenkins/workspace/Beats_elastic-package_PR-127
[2020-10-12T19:09:03.334Z] [INFO] getVaultSecret: Getting secrets
[2020-10-12T19:09:03.376Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2020-10-12T19:09:05.676Z] + chmod 755 generate-build-data.sh
[2020-10-12T19:09:05.676Z] + ./generate-build-data.sh https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/elastic-package/PR-127/ https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/elastic-package/PR-127/runs/7 FAILURE 691363
[2020-10-12T19:09:05.677Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/elastic-package/PR-127/runs/7/steps/?limit=10000 -o steps-info.json
[2020-10-12T19:09:06.378Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/elastic-package/PR-127/runs/7/tests/?status=FAILED -o tests-errors.json
[2020-10-12T19:09:07.083Z] Retry 1/3 exited 22, retrying in 1 seconds...
[2020-10-12T19:09:08.534Z] Retry 2/3 exited 22, retrying in 2 seconds...

mtojek
mtojek approved these changes Oct 6, 2020
View reviewed changes
Copy link
Contributor

@mtojek mtojek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm happy to see if it fixes the issue!

@mtojek
Copy link
Contributor

mtojek commented Oct 7, 2020

/test

@mtojek
Copy link
Contributor

mtojek commented Oct 7, 2020

Unfortunately not yet :(

@mtojek
Copy link
Contributor

mtojek commented Oct 8, 2020

/test

@mtojek
Copy link
Contributor

mtojek commented Oct 8, 2020

@ycombinator Shouldn't this PR be passing? It's already a couple of days.

@ycombinator
Copy link
Contributor Author

Shouldn't this PR be passing? It's already a couple of days.

@mtojek It looks like the backport of the Kibana PR was merged 2 days ago: elastic/kibana#79657. I pulled down the latest Kibana 7.10.0-SNAPSHOT Docker image and it is 3 days old. So the change has not yet been released.

@ycombinator
Copy link
Contributor Author

/test

@ycombinator
Copy link
Contributor Author

CI is failing with this error:

[2020-10-11T16:13:11.371Z] Error: error running package system tests: could not create test policy: could not create policy; API status code = 404; response body = {"statusCode":404,"error":"Not Found","message":"Not Found"}

Will investigate tomorrow (Monday).

@mtojek
Copy link
Contributor

mtojek commented Oct 12, 2020

Looks like I fixed it in #133

/test

@mtojek
Copy link
Contributor

mtojek commented Oct 12, 2020

/test

@mtojek
Copy link
Contributor

mtojek commented Oct 12, 2020

It looks like it's still failing:

[2020-10-12T14:23:26.845Z] 2020/10/12 14:23:26 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T14:23:27.795Z] 2020/10/12 14:23:27 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T14:23:28.751Z] 2020/10/12 14:23:28 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T14:23:30.146Z] 2020/10/12 14:23:29 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T14:23:31.101Z] 2020/10/12 14:23:30 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T14:23:32.049Z] 2020/10/12 14:23:31 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T14:23:32.999Z] 2020/10/12 14:23:32 DEBUG found 0 hits in logs-apache.error-ep data stream
[2020-10-12T14:23:33.952Z] System test for apache/error data stream failed

I'm confused with the release cycle here...

@ycombinator
Copy link
Contributor Author

Yeah, I don't know why this is still failing. Looks like apache/access passed this time but apache/error failed. So this looks like the same flaky behavior we had before we introduced the setting being removed in this PR. Now I'm starting to wonder if we still need to keep that setting. Maybe elastic/kibana#77613 isn't fully resolved yet?

@mtojek
Copy link
Contributor

mtojek commented Oct 12, 2020

Yeah, I don't know why this is still failing. Looks like apache/access passed this time but apache/error failed. So this looks like the same flaky behavior we had before we introduced the setting being removed in this PR. Now I'm starting to wonder if we still need to keep that setting. Maybe elastic/kibana#77613 isn't fully resolved yet?

@nchaulet Would you mind taking a look again at this?

@mtojek
Copy link
Contributor

mtojek commented Oct 12, 2020

@ycombinator

Could you please rebase it against master? There were couple of breaking changes in the mean time.

@nchaulet nchaulet mentioned this pull request Oct 13, 2020
Merged
@ycombinator
Copy link
Contributor Author

Closing in favor of #139.

@ycombinator ycombinator deleted the cleanup-kibana-TODO branch October 13, 2020 18:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mtojek mtojek mtojek approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@ycombinator @elasticmachine @mtojek