Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kibana plugin install is pending on: Optimizing and caching browser bundles... #664

Closed
jasysadmin opened this issue Jul 4, 2018 · 34 comments
Assignees
Labels
type/bug Bug issue

Comments

@jasysadmin
Copy link

Kibana plugin install is pending on: Optimizing and caching browser bundles...
Elk 6.3.0
Wazuh App 3.3.1

If I cancel the plugin shows in Kibana, but after configuration it never really works.
image

If someone can give me a hand it would be nice :) Thank you!

@JuanjiJG JuanjiJG self-assigned this Jul 4, 2018
@JuanjiJG
Copy link
Contributor

JuanjiJG commented Jul 4, 2018

Hello @jasysadmin,

There are several factors to consider when installing a Kibana plugin such as our Wazuh app.

How many available RAM has the machine where Kibana is installed? We recommend a minimum of 4GB of RAM for a correct performance, but higher would be better (8GB for example).

Try again to reinstall the app, but this time let the optimization process finish properly. Follow these steps:

  1. Stop Kibana
systemctl stop kibana
  1. Uninstall the app
/usr/share/kibana/bin/kibana-plugin remove wazuh
  1. Remove the bundles
rm -rf /usr/share/kibana/optimize/bundles
  1. Increase the NodeJS memory limit
export NODE_OPTIONS="--max-old-space-size=3072"
  1. Install the app
/usr/share/kibana/bin/kibana-plugin install https://packages.wazuh.com/wazuhapp/wazuhapp-3.3.1_6.3.0.zip

Wait until the installation process finish. It might take several minutes, so please be patient.

  1. Restart Kibana
systemctl restart kibana

Let me know about the results.

Regards,
Juanjo

@jasysadmin
Copy link
Author

jasysadmin commented Jul 4, 2018

I have done that several times.

I have 8Gb of RAM, and Elk Stack installed, 4Gb to elasticsearch, 1Gb to logstash, the rest is used in kibana and linux os. Elk stack is a cluster and there is another machine (don't know if this has any impact).

Should I increase memory? Maybe increase NODE_OPTIONS to some higher value helps?

PS: Thank you for the very fast reply!

@JuanjiJG
Copy link
Contributor

JuanjiJG commented Jul 5, 2018

Hello again @jasysadmin,

I've been researching this problem, and it's quite hard to explain/reproduce, but sometimes Kibana can take even an hour to finish optimizing bundles.

Let's try something different. Just like you suggested, let's increase the export NODE_OPTIONS command up to 8000. Follo again the same steps from my previous message with this new memory limit.

Let me know about the results. I recommend you to be patient, let's see if the optimization process throws an error or something like that. Sorry for the inconveniences.

Regards,
Juanjo

@jasysadmin
Copy link
Author

No incoveniences, thanks for the community support.

I don't really work with nodejs but will try to report something that might help, even some other users. I waited 24hours for the plugin to install with no sucess. Retried the install with the following conclusions:

Removed wazuh plugin
Removed everything in /usr/share/kibana/optimize/bundles
Increased RAM to 12Gb (4Gb do ElasticSearch JVM; 1Gb logstash JVM)
Stopped Kibana

Plugin install with 8Gb, starts 2 processes:
kibana 3298 3133 81 09:59 pts/0 00:00:08 /usr/share/kibana/bin/../node/bin/node --max-old-space-size=8000 /usr/share/kibana/bin/../src/cli_plugin install file:/tmp/wazuhapp-3.3.1_6.3.0.zip
kibana 3311 3298 99 09:59 pts/0 00:00:04 /usr/share/kibana/node/bin/node /usr/share/kibana/src/cli --env.name=production --optimize.useBundleCache=false --server.autoListen=false --plugins.initialize=false --uiSettings.enabled=false

An strace on 3311 reveals:

stat("/usr/share/node_modules", 0x7fff8cb530e0) = -1 ENOENT (No such file or directory)
stat("/usr/node_modules", 0x7fff8cb530e0) = -1 ENOENT (No such file or directory)
stat("/node_modules", 0x7fff8cb530e0) = -1 ENOENT (No such file or directory)
stat("/home/kibana/.node_modules", 0x7fff8cb530e0) = -1 ENOENT (No such file or directory)
stat("/home/kibana/.node_libraries", 0x7fff8cb530e0) = -1 ENOENT (No such file or directory)
stat("/usr/share/kibana/node/lib/node", 0x7fff8cb530e0) = -1 ENOENT (No such file or directory)
stat("/usr/share/kibana/node_modules", {st_mode=S_IFDIR|0755, st_size=36864, ...}) = 0
--- Looks good, in the end it finds the node_modules directory

stat("/usr/share/kibana/node_modules/babel-preset-/usr/share/kibana/node_modules/babel-preset-react/lib/index.js", 0x7fff8cb530a0) = -1 ENOENT (No such file or directory)
stat("/usr/share/kibana/node_modules/babel-preset-/usr/share/kibana/node_modules/babel-preset-react/lib/index.js.js", 0x7fff8cb530a0) = -1 ENOENT (No such file or directory)
stat("/usr/share/kibana/node_modules/babel-preset-/usr/share/kibana/node_modules/babel-preset-react/lib/index.js.json", 0x7fff8cb530a0) = -1 ENOENT (No such file or directory)
stat("/usr/share/kibana/node_modules/babel-preset-/usr/share/kibana/node_modules/babel-preset-react/lib/index.js.node", 0x7fff8cb530a0) = -1 ENOENT (No such file or directory)
stat("/usr/share/kibana/node_modules/babel-preset-/usr/share/kibana/node_modules/babel-preset-react/lib/index.js.jsx", 0x7fff8cb530a0) = -1 ENOENT (No such file or directory)
stat("/usr/share/kibana/node_modules/babel-preset-/usr/share/kibana/node_modules/babel-preset-react/lib/index.js.es6", 0x7fff8cb530a0) = -1 ENOENT (No such file or directory)
stat("/usr/share/kibana/node_modules/babel-preset-/usr/share/kibana/node_modules/babel-preset-react/lib/index.js.es", 0x7fff8cb530a0) = -1 ENOENT (No such file or directory)
stat("/usr/share/kibana/node_modules/babel-preset-/usr/share/kibana/node_modules/babel-preset-react/lib/index.js.handlebars", 0x7fff8cb530a0) = -1 ENOENT (No such file or directory)
stat("/usr/share/kibana/node_modules/babel-preset-/usr/share/kibana/node_modules/babel-preset-react/lib/index.js.hbs", 0x7fff8cb530a0) = -1 ENOENT (No such file or directory)
--- Not good... it cannot find babel stuff (langiuage?) probably something missing after babel-preset-

stat("/usr/share/node_modules", 0x7fff8cb530a0) = -1 ENOENT (No such file or directory)
stat("/usr/node_modules", 0x7fff8cb530a0) = -1 ENOENT (No such file or directory)
stat("/node_modules", 0x7fff8cb530a0) = -1 ENOENT (No such file or directory)
stat("/home/kibana/.node_modules", 0x7fff8cb530a0) = -1 ENOENT (No such file or directory)
stat("/home/kibana/.node_libraries", 0x7fff8cb530a0) = -1 ENOENT (No such file or directory)
stat("/usr/share/kibana/node/lib/node", 0x7fff8cb530a0) = -1 ENOENT (No such file or directory)
--- Not good... does not find modules which are in /usr/share/kibana/node/lib/node_modules

Some time after, process appears to loop and wait a few seconds on:
epoll_wait(3, {{EPOLLIN, {u32=8, u64=8}}}, 1024, 4985) = 1
read(8, "\1\0\0\0\0\0\0\0", 1024) = 8
epoll_wait(3, {}, 1024, 0) = 0
epoll_wait(3,

Using lsof we can see that fd 3 is:
node 3311 kibana 3u a_inode 0,11 0 7369 [eventpoll]

Regards,
James

@jesusgn90
Copy link
Contributor

Hi @jasysadmin , from my view you have a corrupted installation. My suggestion at this point is to fully remove Kibana and fully remove Node.js (unless you are using Node.js for a different purpose) cause Kibana has its own Node.js interpreter , then reinstall it following our documentation, see https://documentation.wazuh.com/current/installation-guide/installing-elastic-stack/index.html, also you could find a more in deep information at the Elastic site https://www.elastic.co/guide/en/kibana/current/install.html.

Kibana will only listen on the loopback interface (localhost) by default. To set up Kibana to listen on all interfaces, edit the file /etc/kibana/kibana.yml uncommenting the setting server.host. Change the value to:
server.host: "0.0.0.0"

Restart Kibana:

systemctl restart kibana

Once done, don't install the Wazuh App yet, try to open Kibana in your browser to verify if it's working as expected. If your Kibana is now working then continue following our guide from https://documentation.wazuh.com/current/installation-guide/installing-elastic-stack/elastic_server_rpm.html#kibana and install our App.

If you can't run Kibana, please provide us a more in deep information about your environment pasting the output of the next commands:

# lscpu
# df -h
# node -v
# uname -a
# cat /etc/os-release

One or more of the above commands may not run in your machine, but paste the output anyway.

Regards,
Jesús

@jasysadmin
Copy link
Author

Found out that I was installing for Elk 6.3.0. Updated everything to 6.3.1 and used your wazuhapp for 6.3.1. Same situation, stuck in "Optimizing and caching browser bundles..."

Below is the info you asked...

lscpu

Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 4
On-line CPU(s) list: 0-3
Thread(s) per core: 1
Core(s) per socket: 4
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 63
Model name: Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz
Stepping: 2
CPU MHz: 3196.378
BogoMIPS: 6392.75
Hypervisor vendor: Xen
Virtualization type: full
L1d cache: 32K
L1i cache: 32K
L2 cache: 256K
L3 cache: 20480K
NUMA node0 CPU(s): 0-3

df -h
Filesystem Size Used Avail Use% Mounted on
devtmpfs 5.8G 0 5.8G 0% /dev
tmpfs 5.8G 0 5.8G 0% /dev/shm
tmpfs 5.8G 17M 5.8G 1% /run
tmpfs 5.8G 0 5.8G 0% /sys/fs/cgroup
/dev/mapper/ol-root00 3.0G 737M 2.3G 25% /
/dev/mapper/ol-usr 6.9G 5.2G 1.8G 75% /usr
/dev/xvda1 497M 355M 143M 72% /boot
/dev/mapper/data-data 130G 102G 29G 79% /data
/dev/mapper/ol-var 4.9G 3.3G 1.6G 68% /var
/dev/mapper/ol-home 997M 33M 965M 4% /home
/dev/mapper/ol-opt 997M 33M 965M 4% /opt
/dev/mapper/ol-tmp 997M 67M 931M 7% /tmp
tmpfs 1.2G 0 1.2G 0% /run/user/0

node -v

v6.14.2

Kibana installs its own node:

/usr/share/kibana/node/bin/node -v

v8.11.3

uname -a

Linux hostname 4.1.12-61.1.18.el7uek.x86_64 #2 SMP Fri Nov 4 15:48:30 PDT 2016 x86_64 x86_64 x86_64 GNU/Linux

cat /etc/os-release

NAME="Oracle Linux Server"
VERSION="7.3"
ID="ol"
VERSION_ID="7.3"
PRETTY_NAME="Oracle Linux Server 7.3"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:oracle:linux:7:3:server"
HOME_URL="https://linux.oracle.com/"
BUG_REPORT_URL="https://bugzilla.oracle.com/"

ORACLE_BUGZILLA_PRODUCT="Oracle Linux 7"
ORACLE_BUGZILLA_PRODUCT_VERSION=7.3
ORACLE_SUPPORT_PRODUCT="Oracle Linux"
ORACLE_SUPPORT_PRODUCT_VERSION=7.3

Thanks

@jesusgn90
Copy link
Contributor

Ok, what I've seen a bit weird is "Oracle Linux Server", I need to research a bit about a deployment using Oracle Linux Server because we usually test it under CentOS and Ubuntu. Once I test it, I'll let you know.

Regards,
Jesús

@jasysadmin
Copy link
Author

Thanks Jesus.

Oracle Linux is RHEL based with some updates on the kernel, shouldn't be a problem. Please note that I have the Elk 6.3.1 stack working for other purposes and it has been working just fine. Also, the wazuh index is getting filled properly with wazuh information from the agents. It is just the wazuh app for kibana that I cannot install.

@jesusgn90
Copy link
Contributor

Hello @jasysadmin , it's pretty weird cause we mainly test under CentOS 7. I've opened a discuss thread at https://discuss.elastic.co/t/oracle-linux/139489 to share your situation with the Elastic support team. Also it would be nice if you try to fully remove Kibana and reinstall it then try Kibana without install the Wazuh App to check if Kibana is working as expected.

Once I have news about your problem I'll let you know.

Regards,
Jesús

@jasysadmin
Copy link
Author

Kibana is fully functional, either before or after the wazuh plugin install. It is only the wazuh plugin that does not work.

Anything I can do to help please tell me.

Regards
James

@jesusgn90
Copy link
Contributor

I could not reproduce it. Also I'm tracking a thread at https://discuss.elastic.co/t/oracle-linux/139489/2 sharing your situation, maybe Kibana team know the trick. Any case I'll reply you here.

Regards,
Jesús

@jasysadmin
Copy link
Author

Thank you!

I tried to install another plugin and I have the same problem. It was the logtrail plugin and the same thing happened, it got stuck in Optimizing.... So this is really a kibana issue. I might try migrating from Oracle Linux to CentOS just to get it working, but this will be after my vacations :)

@jesusgn90
Copy link
Contributor

Ok @jasysadmin so we've concluded you have a Kibana issue with all kind of plugins regardless Wazuh... My bet is about Oracle Linux. If Kibana support team replies me, I'll copy here for your information. Have a nice holidays!

Regards,
Jesús

@athanatos64
Copy link

Hi All -

I have the same issue with RHEL 7.4 and 7.5 with 2 separate servers using Elasticsearch&Kibana 6.3.1 and 6.4.0. Everything is directly downloaded from elastic.co with verified hashes. X-pack is not enabled and running a single node cluster just for Wazuh logs. Basic RHEL installed with up-to-date patches.

CPU: 8 cores
RAM: 16 GB

Getting stuck on "Optimizing and caching browser bundles..."

I looked through kibana.log and got the following messages:
{"type":"log","@timestamp":"2018-09-05T20:34:05Z","tags":["warning","config","deprecation"],"pid":21552,"message":"uiSettings.enabled is deprecated and is no longer used"}
{"type":"log","@timestamp":"2018-09-05T20:34:14Z","tags":["info","optimize"],"pid":21552,"message":"Optimizing and caching bundles for graph, monitoring, login, logout, ml, dashboardViewer, apm, wazuh, kibana, stateSessionStorageRedirect, status_page and timelion. This may take a few minutes"}
{"type":"log","@timestamp":"2018-09-05T20:36:54Z","tags":["info","optimize"],"pid":21552,"message":"Optimization of bundles for graph, monitoring, login, logout, ml, dashboardViewer, apm, wazuh, kibana, stateSessionStorageRedirect, status_page and timelion complete in 160.16 seconds"}
{"type":"log","@timestamp":"2018-09-05T20:36:54Z","tags":["info"],"pid":21552,"message":"Plugin initialization disabled."}

I can only use 6.2.4 at the moment and cannot go to newer versions due to wazuh-kibana-app. I've waited for over 8 hours for the installation to finish before killing the process.

Is there a way to enable verbosity during the plugin install to figure out what's causing it/what it's stuck on?

6.2.4 version works just fine.

Thanks!

@JuanjiJG
Copy link
Contributor

JuanjiJG commented Sep 6, 2018

Hello @athanatos64,

You can start the Kibana binary in verbosity mode so you can check in real time what logs are being generated during plugin optimization process.

  1. Stop the Kibana service:
systemctl stop kibana
  1. Remove the optimization bundles:
rm -rf /usr/share/kibana/optimize/bundles
  1. Run the Kibana binary in foreground mode. The -c option is required to tell Kibana which configuration file to use:
/usr/share/kibana/bin/kibana --verbose -c /etc/kibana/kibana.yml
  1. At first, you'll se a message like this:
"message":"Optimizing and caching bundles for graph, monitoring, login, logout, ml, dashboardViewer, apm, wazuh, kibana, stateSessionStorageRedirect, status_page and timelion. This may take a few minutes"

You have to wait until you see this message:

"message":"Optimization of bundles for graph, monitoring, login, logout, ml, dashboardViewer, apm, wazuh, kibana, stateSessionStorageRedirect, status_page and timelion complete in 160.16 seconds"

This will tell us that the optimization process finished. Now you can close the Kibana binary execution using the Ctrl+C keyboard shortcut.

  1. Now you can restart the Kibana service:
systemctl restart kibana
  1. Wait some seconds and now try again to open the Kibana interface on an incognito window on your Internet browser.

Let us know if this helps.

Regards,
Juanjo

@wtibbitts
Copy link

I am experiencing the same issue on Ubuntu 16.04.

@jesusgn90
Copy link
Contributor

Hi @jwsomis and everybody here,

Since we are not able to reproduce it in an adequate way, we are in the process of gathering information on the environments in which this situation has occurred. If you would be kind enough to provide us with information about your environment, we would be very grateful.

Regards,
Jesús

@wtibbitts
Copy link

To add additional info to my comment, I tried this numerous times and found that it would generally eventually install sometimes taking only 20 minutes, but sometimes taking at least 5 hours. There was also one time where the process never exited, but after killing it, the plugin was installed.
I'd be happy to supply info about my environment. Just let me know what info you need.

@jesusgn90
Copy link
Contributor

Hi @jwsomis , what I meant with "info about your environment" is the OS, RAM, CPU, VM or real host, etc

All information is welcome for us.

Regards!

@athanatos64
Copy link

Here is the error that I got using verbosity mode in Kibana:

{"type":"error","@timestamp":"2018-09-13T14:20:52Z","tags":["fatal"],"pid":18950,"level":"fatal","error":{"message":"Optimizations failure.\n 6604 modules\n \n \u001b[1m\u001b[31mERROR in ../plugins/wazuh/public/directives/wz-menu/wz-menu.less\n Module build failed: EntryModuleNotFoundError: Entry module not found: Error: Can't resolve '/home/appl/kibana-6.4.0/plugins/wazuh/public/directives/wz-menu/wz-menu.less' in '/home/appl/kibana-6.4.0/plugins/wazuh/public/directives/wz-menu'\n at moduleFactory.create (/home/appl/kibana-6.4.0/node_modules/webpack/lib/Compilation.js:404:30)\n at factory (/home/appl/kibana-6.4.0/node_modules/webpack/lib/NormalModuleFactory.js:235:20)\n at resolver (/home/appl/kibana-6.4.0/node_modules/webpack/lib/NormalModuleFactory.js:60:20)\n at asyncLib.parallel (/home/appl/kibana-6.4.0/node_modules/webpack/lib/NormalModuleFactory.js:127:20)\n at /home/appl/kibana-6.4.0/node_modules/async/dist/async.js:3874:9\n at /home/appl/kibana-6.4.0/node_modules/async/dist/async.js:473:16\n at iteratorCallback (/home/appl/kibana-6.4.0/node_modules/async/dist/async.js:1048:13)\n at /home/appl/kibana-6.4.0/node_modules/async/dist/async.js:958:16\n at /home/appl/kibana-6.4.0/node_modules/async/dist/async.js:3871:13\n at resolvers.normal.resolve (/home/appl/kibana-6.4.0/node_modules/webpack/lib/NormalModuleFactory.js:119:22)\n at onError (/home/appl/kibana-6.4.0/node_modules/enhanced-resolve/lib/Resolver.js:65:10)\n at loggingCallbackWrapper (/home/appl/kibana-6.4.0/node_modules/enhanced-resolve/lib/createInnerCallback.js:31:19)\n at runAfter (/home/appl/kibana-6.4.0/node_modules/enhanced-resolve/lib/Resolver.js:158:4)\n at innerCallback (/home/appl/kibana-6.4.0/node_modules/enhanced-resolve/lib/Resolver.js:146:3)\n at loggingCallbackWrapper (/home/appl/kibana-6.4.0/node_modules/enhanced-resolve/lib/createInnerCallback.js:31:19)\n at next (/home/appl/kibana-6.4.0/node_modules/tapable/lib/Tapable.js:252:11)\n at innerCallback (/home/appl/kibana-6.4.0/node_modules/enhanced-resolve/lib/Resolver.js:144:11)\n at loggingCallbackWrapper (/home/appl/kibana-6.4.0/node_modules/enhanced-resolve/lib/createInnerCallback.js:31:19)\n at next (/home/appl/kibana-6.4.0/node_modules/tapable/lib/Tapable.js:249:35)\n at resolver.doResolve.createInnerCallback (/home/appl/kibana-6.4.0/node_modules/enhanced-resolve/lib/DescriptionFilePlugin.js:44:6)\n at loggingCallbackWrapper (/home/appl/kibana-6.4.0/node_modules/enhanced-resolve/lib/createInnerCallback.js:31:19)\n at afterInnerCallback (/home/appl/kibana-6.4.0/node_modules/enhanced-resolve/lib/Resolver.js:168:10)\n at loggingCallbackWrapper (/home/appl/kibana-6.4.0/node_modules/enhanced-resolve/lib/createInnerCallback.js:31:19)\n at next (/home/appl/kibana-6.4.0/node_modules/tapable/lib/Tapable.js:252:11)\n at innerCallback (/home/appl/kibana-6.4.0/node_modules/enhanced-resolve/lib/Resolver.js:144:11)\n at loggingCallbackWrapper (/home/appl/kibana-6.4.0/node_modules/enhanced-resolve/lib/createInnerCallback.js:31:19)\n at next (/home/appl/kibana-6.4.0/node_modules/tapable/lib/Tapable.js:249:35)\n at resolver.doResolve.createInnerCallback (/home/appl/kibana-6.4.0/node_modules/enhanced-resolve/lib/DescriptionFilePlugin.js:44:6)\u001b[39m\u001b[22m\n \n \u001b[1m\u001b[31mERROR in ../plugins/wazuh/public/templates/agents-prev/agents-prev.jade\n Module build failed: Error: ENOENT: no such file or directory, open '/home/appl/kibana-6.4.0/plugins/wazuh/public/templates/agents-prev/agents-prev.jade'\u001b[39m\u001b[22m\n \n \u001b[1m\u001b[31mERROR in ../plugins/wazuh/public/templates/agents/agents.jade\n Module build failed: Error: ENOENT: no such file or directory, open '/home/appl/kibana-6.4.0/plugins/wazuh/public/templates/agents/agents.jade'\u001b[39m\u001b[22m\n \n \u001b[1m\u001b[31mERROR in ../plugins/wazuh/public/templates/discover/discover.jade\n Module build failed: Error: ENOENT: no such file or directory, open '/home/appl/kibana-6.4.0/plugins/wazuh/public/templates/discover/discover.jade'\u001b[39m\u001b[22m\n \n \u001b[1m\u001b[31mERROR in ../plugins/wazuh/public/templates/management/management.jade\n Module build failed: Error: ENOENT: no such file or directory, open '/home/appl/kibana-6.4.0/plugins/wazuh/public/templates/management/management.jade'\u001b[39m\u001b[22m\n \n \u001b[1m\u001b[31mERROR in ../plugins/wazuh/public/templates/overview/overview.jade\n Module build failed: Error: ENOENT: no such file or directory, open '/home/appl/kibana-6.4.0/plugins/wazuh/public/templates/overview/overview.jade'\u001b[39m\u001b[22m\n \n \u001b[1m\u001b[31mERROR in ../plugins/wazuh/public/templates/settings/settings.jade\n Module build failed: Error: ENOENT: no such file or directory, open '/home/appl/kibana-6.4.0/plugins/wazuh/public/templates/settings/settings.jade'\u001b[39m\u001b[22m\n \n \u001b[1m\u001b[31mERROR in ../plugins/wazuh/node_modules/js2xmlparser/lib/main.js\n Module not found: Error: Can't resolve './options' in '/home/appl/kibana-6.4.0/plugins/wazuh/node_modules/js2xmlparser/lib'\u001b[39m\u001b[22m\n \n \u001b[1m\u001b[31mERROR in ../plugins/wazuh/public/services/routes.js\n Module not found: Error: Can't resolve './resolves' in '/home/appl/kibana-6.4.0/plugins/wazuh/public/services'\u001b[39m\u001b[22m\n \n \u001b[1m\u001b[31mERROR in ../plugins/wazuh/node_modules/js2xmlparser/lib/main.js\n Module not found: Error: Can't resolve './utils' in '/home/appl/kibana-6.4.0/plugins/wazuh/node_modules/js2xmlparser/lib'\u001b[39m\u001b[22m\n \n \u001b[1m\u001b[31mERROR in ../plugins/wazuh/public/factories/vis2png.js\n Module not found: Error: Can't resolve 'dom-to-image' in '/home/appl/kibana-6.4.0/plugins/wazuh/public/factories'\u001b[39m\u001b[22m\n \n \u001b[1m\u001b[31mERROR in ../plugins/wazuh/node_modules/js2xmlparser/lib/main.js\n Module not found: Error: Can't resolve 'xmlcreate' in '/home/appl/kibana-6.4.0/plugins/wazuh/node_modules/js2xmlparser/lib'\u001b[39m\u001b[22m\n","name":"Error","stack":"Error: Optimizations failure.\n 6604 modules\n \n \u001b[1m\u001b[31mERROR in ../plugins/wazuh/public/directives/wz-menu/wz-menu.less\n Module build failed: EntryModuleNotFoundError: Entry module not found: Error: Can't resolve '/home/appl/kibana-6.4.0/plugins/wazuh/public/directives/wz-menu/wz-menu.less' in '/home/appl/kibana-6.4.0/plugins/wazuh/public/directives/wz-menu'\n at moduleFactory.create (/home/appl/kibana-6.4.0/node_modules/webpack/lib/Compilation.js:404:30)\n at factory (/home/appl/kibana-6.4.0/node_modules/webpack/lib/NormalModuleFactory.js:235:20)\n at resolver (/home/appl/kibana-6.4.0/node_modules/webpack/lib/NormalModuleFactory.js:60:20)\n at asyncLib.parallel (/home/appl/kibana-6.4.0/node_modules/webpack/lib/NormalModuleFactory.js:127:20)\n at /home/appl/kibana-6.4.0/node_modules/async/dist/async.js:3874:9\n at /home/appl/kibana-6.4.0/node_modules/async/dist/async.js:473:16\n at iteratorCallback (/home/appl/kibana-6.4.0/node_modules/async/dist/async.js:1048:13)\n at /home/appl/kibana-6.4.0/node_modules/async/dist/async.js:958:16\n at /home/appl/kibana-6.4.0/node_modules/async/dist/async.js:3871:13\n at resolvers.normal.resolve (/home/appl/kibana-6.4.0/node_modules/webpack/lib/NormalModuleFactory.js:119:22)\n at onError (/home/appl/kibana-6.4.0/node_modules/enhanced-resolve/lib/Resolver.js:65:10)\n at loggingCallbackWrapper (/home/appl/kibana-6.4.0/node_modules/enhanced-resolve/lib/createInnerCallback.js:31:19)\n at runAfter (/home/appl/kibana-6.4.0/node_modules/enhanced-resolve/lib/Resolver.js:158:4)\n at innerCallback (/home/appl/kibana-6.4.0/node_modules/enhanced-resolve/lib/Resolver.js:146:3)\n at loggingCallbackWrapper (/home/appl/kibana-6.4.0/node_modules/enhanced-resolve/lib/createInnerCallback.js:31:19)\n at next (/home/appl/kibana-6.4.0/node_modules/tapable/lib/Tapable.js:252:11)\n at innerCallback (/home/appl/kibana-6.4.0/node_modules/enhanced-resolve/lib/Resolver.js:144:11)\n at loggingCallbackWrapper (/home/appl/kibana-6.4.0/node_modules/enhanced-resolve/lib/createInnerCallback.js:31:19)\n at next (/home/appl/kibana-6.4.0/node_modules/tapable/lib/Tapable.js:249:35)\n at resolver.doResolve.createInnerCallback (/home/appl/kibana-6.4.0/node_modules/enhanced-resolve/lib/DescriptionFilePlugin.js:44:6)\n at loggingCallbackWrapper (/home/appl/kibana-6.4.0/node_modules/enhanced-resolve/lib/createInnerCallback.js:31:19)\n at afterInnerCallback (/home/appl/kibana-6.4.0/node_modules/enhanced-resolve/lib/Resolver.js:168:10)\n at loggingCallbackWrapper (/home/appl/kibana-6.4.0/node_modules/enhanced-resolve/lib/createInnerCallback.js:31:19)\n at next (/home/appl/kibana-6.4.0/node_modules/tapable/lib/Tapable.js:252:11)\n at innerCallback (/home/appl/kibana-6.4.0/node_modules/enhanced-resolve/lib/Resolver.js:144:11)\n at loggingCallbackWrapper (/home/appl/kibana-6.4.0/node_modules/enhanced-resolve/lib/createInnerCallback.js:31:19)\n at next (/home/appl/kibana-6.4.0/node_modules/tapable/lib/Tapable.js:249:35)\n at resolver.doResolve.createInnerCallback (/home/appl/kibana-6.4.0/node_modules/enhanced-resolve/lib/DescriptionFilePlugin.js:44:6)\u001b[39m\u001b[22m\n \n \u001b[1m\u001b[31mERROR in ../plugins/wazuh/public/templates/agents-prev/agents-prev.jade\n Module build failed: Error: ENOENT: no such file or directory, open '/home/appl/kibana-6.4.0/plugins/wazuh/public/templates/agents-prev/agents-prev.jade'\u001b[39m\u001b[22m\n \n \u001b[1m\u001b[31mERROR in ../plugins/wazuh/public/templates/agents/agents.jade\n Module build failed: Error: ENOENT: no such file or directory, open '/home/appl/kibana-6.4.0/plugins/wazuh/public/templates/agents/agents.jade'\u001b[39m\u001b[22m\n \n \u001b[1m\u001b[31mERROR in ../plugins/wazuh/public/templates/discover/discover.jade\n Module build failed: Error: ENOENT: no such file or directory, open '/home/appl/kibana-6.4.0/plugins/wazuh/public/templates/discover/discover.jade'\u001b[39m\u001b[22m\n \n \u001b[1m\u001b[31mERROR in ../plugins/wazuh/public/templates/management/management.jade\n Module build failed: Error: ENOENT: no such file or directory, open '/home/appl/kibana-6.4.0/plugins/wazuh/public/templates/management/management.jade'\u001b[39m\u001b[22m\n \n \u001b[1m\u001b[31mERROR in ../plugins/wazuh/public/templates/overview/overview.jade\n Module build failed: Error: ENOENT: no such file or directory, open '/home/appl/kibana-6.4.0/plugins/wazuh/public/templates/overview/overview.jade'\u001b[39m\u001b[22m\n \n \u001b[1m\u001b[31mERROR in ../plugins/wazuh/public/templates/settings/settings.jade\n Module build failed: Error: ENOENT: no such file or directory, open '/home/appl/kibana-6.4.0/plugins/wazuh/public/templates/settings/settings.jade'\u001b[39m\u001b[22m\n \n \u001b[1m\u001b[31mERROR in ../plugins/wazuh/node_modules/js2xmlparser/lib/main.js\n Module not found: Error: Can't resolve './options' in '/home/appl/kibana-6.4.0/plugins/wazuh/node_modules/js2xmlparser/lib'\u001b[39m\u001b[22m\n \n \u001b[1m\u001b[31mERROR in ../plugins/wazuh/public/services/routes.js\n Module not found: Error: Can't resolve './resolves' in '/home/appl/kibana-6.4.0/plugins/wazuh/public/services'\u001b[39m\u001b[22m\n \n \u001b[1m\u001b[31mERROR in ../plugins/wazuh/node_modules/js2xmlparser/lib/main.js\n Module not found: Error: Can't resolve './utils' in '/home/appl/kibana-6.4.0/plugins/wazuh/node_modules/js2xmlparser/lib'\u001b[39m\u001b[22m\n \n \u001b[1m\u001b[31mERROR in ../plugins/wazuh/public/factories/vis2png.js\n Module not found: Error: Can't resolve 'dom-to-image' in '/home/appl/kibana-6.4.0/plugins/wazuh/public/factories'\u001b[39m\u001b[22m\n \n \u001b[1m\u001b[31mERROR in ../plugins/wazuh/node_modules/js2xmlparser/lib/main.js\n Module not found: Error: Can't resolve 'xmlcreate' in '/home/appl/kibana-6.4.0/plugins/wazuh/node_modules/js2xmlparser/lib'\u001b[39m\u001b[22m\n\n at FsOptimizer.failedStatsToError (/home/appl/kibana-6.4.0/src/optimize/base_optimizer.js:378:27)\n at compiler.run (/home/appl/kibana-6.4.0/src/optimize/fs_optimizer.js:47:26)\n at onCompiled (/home/appl/kibana-6.4.0/node_modules/webpack/lib/Compiler.js:241:12)\n at applyPluginsAsync.err (/home/appl/kibana-6.4.0/node_modules/webpack/lib/Compiler.js:514:14)\n at next (/home/appl/kibana-6.4.0/node_modules/tapable/lib/Tapable.js:202:11)\n at Compiler. (/home/appl/kibana-6.4.0/node_modules/webpack/lib/CachePlugin.js:78:5)\n at Compiler.applyPluginsAsyncSeries (/home/appl/kibana-6.4.0/node_modules/tapable/lib/Tapable.js:206:13)\n at compilation.seal.err (/home/appl/kibana-6.4.0/node_modules/webpack/lib/Compiler.js:511:11)\n at Compilation.applyPluginsAsyncSeries (/home/appl/kibana-6.4.0/node_modules/tapable/lib/Tapable.js:195:46)\n at self.applyPluginsAsync.err (/home/appl/kibana-6.4.0/node_modules/webpack/lib/Compilation.js:671:19)\n at Compilation.applyPluginsAsyncSeries (/home/appl/kibana-6.4.0/node_modules/tapable/lib/Tapable.js:195:46)\n at self.applyPluginsAsync.err (/home/appl/kibana-6.4.0/node_modules/webpack/lib/Compilation.js:662:11)\n at next (/home/appl/kibana-6.4.0/node_modules/tapable/lib/Tapable.js:202:11)\n at /home/appl/kibana-6.4.0/node_modules/uglifyjs-webpack-plugin/dist/index.js:282:11\n at step (/home/appl/kibana-6.4.0/node_modules/uglifyjs-webpack-plugin/dist/uglify/index.js:90:11)\n at done (/home/appl/kibana-6.4.0/node_modules/uglifyjs-webpack-plugin/dist/uglify/index.js:99:22)\n at /home/appl/kibana-6.4.0/node_modules/uglifyjs-webpack-plugin/dist/uglify/index.js:105:15\n at /home/appl/kibana-6.4.0/node_modules/worker-farm/lib/farm.js:191:19\n at _combinedTickCallback (internal/process/next_tick.js:131:7)\n at process._tickCallback (internal/process/next_tick.js:180:9)"},"message":"Optimizations failure.\n 6604 modules\n \n ERROR in ../plugins/wazuh/public/directives/wz-menu/wz-menu.less\n Module build failed: EntryModuleNotFoundError: Entry module not found: Error: Can't resolve '/home/appl/kibana-6.4.0/plugins/wazuh/public/directives/wz-menu/wz-menu.less' in '/home/appl/kibana-6.4.0/plugins/wazuh/public/directives/wz-menu'\n at moduleFactory.create (/home/appl/kibana-6.4.0/node_modules/webpack/lib/Compilation.js:404:30)\n at factory (/home/appl/kibana-6.4.0/node_modules/webpack/lib/NormalModuleFactory.js:235:20)\n at resolver (/home/appl/kibana-6.4.0/node_modules/webpack/lib/NormalModuleFactory.js:60:20)\n at asyncLib.parallel (/home/appl/kibana-6.4.0/node_modules/webpack/lib/NormalModuleFactory.js:127:20)\n at /home/appl/kibana-6.4.0/node_modules/async/dist/async.js:3874:9\n at /home/appl/kibana-6.4.0/node_modules/async/dist/async.js:473:16\n at iteratorCallback (/home/appl/kibana-6.4.0/node_modules/async/dist/async.js:1048:13)\n at /home/appl/kibana-6.4.0/node_modules/async/dist/async.js:958:16\n at /home/appl/kibana-6.4.0/node_modules/async/dist/async.js:3871:13\n at resolvers.normal.resolve (/home/appl/kibana-6.4.0/node_modules/webpack/lib/NormalModuleFactory.js:119:22)\n at onError (/home/appl/kibana-6.4.0/node_modules/enhanced-resolve/lib/Resolver.js:65:10)\n at loggingCallbackWrapper (/home/appl/kibana-6.4.0/node_modules/enhanced-resolve/lib/createInnerCallback.js:31:19)\n at runAfter (/home/appl/kibana-6.4.0/node_modules/enhanced-resolve/lib/Resolver.js:158:4)\n at innerCallback (/home/appl/kibana-6.4.0/node_modules/enhanced-resolve/lib/Resolver.js:146:3)\n at loggingCallbackWrapper (/home/appl/kibana-6.4.0/node_modules/enhanced-resolve/lib/createInnerCallback.js:31:19)\n at next (/home/appl/kibana-6.4.0/node_modules/tapable/lib/Tapable.js:252:11)\n at innerCallback (/home/appl/kibana-6.4.0/node_modules/enhanced-resolve/lib/Resolver.js:144:11)\n at loggingCallbackWrapper (/home/appl/kibana-6.4.0/node_modules/enhanced-resolve/lib/createInnerCallback.js:31:19)\n at next (/home/appl/kibana-6.4.0/node_modules/tapable/lib/Tapable.js:249:35)\n at resolver.doResolve.createInnerCallback (/home/appl/kibana-6.4.0/node_modules/enhanced-resolve/lib/DescriptionFilePlugin.js:44:6)\n at loggingCallbackWrapper (/home/appl/kibana-6.4.0/node_modules/enhanced-resolve/lib/createInnerCallback.js:31:19)\n at afterInnerCallback (/home/appl/kibana-6.4.0/node_modules/enhanced-resolve/lib/Resolver.js:168:10)\n at loggingCallbackWrapper (/home/appl/kibana-6.4.0/node_modules/enhanced-resolve/lib/createInnerCallback.js:31:19)\n at next (/home/appl/kibana-6.4.0/node_modules/tapable/lib/Tapable.js:252:11)\n at innerCallback (/home/appl/kibana-6.4.0/node_modules/enhanced-resolve/lib/Resolver.js:144:11)\n at loggingCallbackWrapper (/home/appl/kibana-6.4.0/node_modules/enhanced-resolve/lib/createInnerCallback.js:31:19)\n at next (/home/appl/kibana-6.4.0/node_modules/tapable/lib/Tapable.js:249:35)\n at resolver.doResolve.createInnerCallback (/home/appl/kibana-6.4.0/node_modules/enhanced-resolve/lib/DescriptionFilePlugin.js:44:6)\n \n ERROR in ../plugins/wazuh/public/templates/agents-prev/agents-prev.jade\n Module build failed: Error: ENOENT: no such file or directory, open '/home/appl/kibana-6.4.0/plugins/wazuh/public/templates/agents-prev/agents-prev.jade'\n \n ERROR in ../plugins/wazuh/public/templates/agents/agents.jade\n Module build failed: Error: ENOENT: no such file or directory, open '/home/appl/kibana-6.4.0/plugins/wazuh/public/templates/agents/agents.jade'\n \n ERROR in ../plugins/wazuh/public/templates/discover/discover.jade\n Module build failed: Error: ENOENT: no such file or directory, open '/home/appl/kibana-6.4.0/plugins/wazuh/public/templates/discover/discover.jade'\n \n ERROR in ../plugins/wazuh/public/templates/management/management.jade\n Module build failed: Error: ENOENT: no such file or directory, open '/home/appl/kibana-6.4.0/plugins/wazuh/public/templates/management/management.jade'\n \n ERROR in ../plugins/wazuh/public/templates/overview/overview.jade\n Module build failed: Error: ENOENT: no such file or directory, open '/home/appl/kibana-6.4.0/plugins/wazuh/public/templates/overview/overview.jade'\n \n ERROR in ../plugins/wazuh/public/templates/settings/settings.jade\n Module build failed: Error: ENOENT: no such file or directory, open '/home/appl/kibana-6.4.0/plugins/wazuh/public/templates/settings/settings.jade'\n \n ERROR in ../plugins/wazuh/node_modules/js2xmlparser/lib/main.js\n Module not found: Error: Can't resolve './options' in '/home/appl/kibana-6.4.0/plugins/wazuh/node_modules/js2xmlparser/lib'\n \n ERROR in ../plugins/wazuh/public/services/routes.js\n Module not found: Error: Can't resolve './resolves' in '/home/appl/kibana-6.4.0/plugins/wazuh/public/services'\n \n ERROR in ../plugins/wazuh/node_modules/js2xmlparser/lib/main.js\n Module not found: Error: Can't resolve './utils' in '/home/appl/kibana-6.4.0/plugins/wazuh/node_modules/js2xmlparser/lib'\n \n ERROR in ../plugins/wazuh/public/factories/vis2png.js\n Module not found: Error: Can't resolve 'dom-to-image' in '/home/appl/kibana-6.4.0/plugins/wazuh/public/factories'\n \n ERROR in ../plugins/wazuh/node_modules/js2xmlparser/lib/main.js\n Module not found: Error: Can't resolve 'xmlcreate' in '/home/appl/kibana-6.4.0/plugins/wazuh/node_modules/js2xmlparser/lib'\n"}

@jesusgn90
Copy link
Contributor

Ok @athanatos64 ,

Let's try to execute the next commands please:

Stop the service

systemctl stop kibana

Ignore any error in the next command

/usr/share/kibana/bin/kibana-plugin remove wazuh

Clear the bundles

rm -rf /usr/share/kibana/optimize/bundles

Correct permissions

chown -R kibana:kibana /usr/share/kibana/optimize
chown -R kibana:kibana /usr/share/kibana/plugins

Reinstall the plugin. Replace 3.6.1(I'm assuming that's your version) by your Wazuh version. Replace 6.4.0 by your Kibana version (it looks like that's your version).

sudo -u kibana /usr/share/kibana/bin/kibana-plugin install https://packages.wazuh.com/wazuhapp/wazuhapp-3.6.1_6.4.0.zip

Restart Kibana a wait some minutes:

systemctl restart kibana

I hope it helps. Regards!

@athanatos64
Copy link

@jesusgn90 Should I start kibana in verbosity mode or leave it shutdown for the time of the plugin installation?

@jesusgn90
Copy link
Contributor

Leave it off while you are installing the plugin. Also keep in mind that you must leave it off when using verbosity mode and you must not use verbosity mode if the service is running, use only one of them.

Regards

@athanatos64
Copy link

I cannot download directly due to self-signed certificate error (we are doing traffic inspection with SSL decryption). I re-downloaded the plugin. Here is the hash:
sha1sum wazuhapp-3.6.0_6.4.0.zip
724dc10a86b6181df6c64a5392109d3eec0007fe wazuhapp-3.6.0_6.4.0.zip

Can you confirm the hash please? I didn't find it available on the git repo.

Thanks!

@jesusgn90
Copy link
Contributor

jesusgn90 commented Sep 13, 2018

It looks good to me @athanatos64 :

sha1sum wazuhapp-3.6.0_6.4.0.zip 
724dc10a86b6181df6c64a5392109d3eec0007fe  wazuhapp-3.6.0_6.4.0.zip

Also, my suggestion is to upgrade your Wazuh environment to 3.6.1 because we have a known bug regarding to agent.name field in our alerts.

Regards

@jesusgn90 jesusgn90 assigned jesusgn90 and unassigned JuanjiJG Oct 10, 2018
@Mrfly-ca
Copy link

I am having the same issue running the install for wazuhapp-3.6.1_6.4.2.zip
Also added a comment to the thread #19678

All installed versions are as recommended by current installation guide:
https://documentation.wazuh.com/current/installation-guide/upgrading/latest_wazuh3_minor.html

@jesusgn90
Copy link
Contributor

Hi @Mrfly-ca ,

Try to execute the same steps as #664 (comment) but replacing the Wazuh app version in the installing step. Also, it would be nice if you add # export NODE_OPTIONS="--max-old-space-size=3072" before installing the plugin.

Let us know. Sorry about the inconvenience.

Regards!

@Mrfly-ca
Copy link

Thanks for the Reply,
I did check the permissions as per the comment #664 and everything was correct. I am also using "# export NODE_OPTIONS="--max-old-space-size=3072" as it is recommended in your installation guide. (Great document!)

I will try to "update" the permissions and try again as per the note the #664 comment.
TKS Again.

@jesusgn90
Copy link
Contributor

Ok @Mrfly-ca ,

Let us know once done. Also, keep in mind the RAM from your instance, a free -h could be useful before installing the Wazuh app. The app itself doesn't need a lot of RAM, but during the installation Kibana also optimizes other internal plugins and it has a memory peak at some time.

Regards

@Mrfly-ca
Copy link

The system has been running for awhile now, and I no-longer see the process getting CPU in TOP.
The system has 8G of memory form a fresh reboot. Log-stash and ElasticSearch are installed but I have stopped there daemons.

Currently the system is down to 423 mem free.
PS shows
15776 pts/0 S+ 0:00 22 0 239128 4112 0.0 sudo -u kibana /usr/share/kibana/bin/kibana-plugin install https://packages.wazuh.com/wazuhapp/wazuhapp-3.6.1_6.4.2.zip
15802 pts/0 Sl+ 0:18 7 0 1278852 98388 1.1 /usr/share/kibana/bin/../node/bin/node --no-warnings /usr/share/kibana/bin/../src/cli_plugin install https://packages.wazuh.com/wazuhapp/wazuhapp-3.6.1_6.4.2.zip
15852 pts/0 Sl+ 6:18 8 0 1814300 688132 8.2 /usr/share/kibana/node/bin/node /usr/share/kibana/src/cli --env.name=production --optimize.useBundleCache=false --server.autoListen=false --plugins.initialize=false --uiSettings

PS: I did install this on a test system in the same manner that only have 4g of memory.

Not sure how to diagnose anymore..

@jesusgn90
Copy link
Contributor

Yes, it's a common issue regardless of the amount of memory. In my opinion, it's installed successfully. Just press ctrl + c once you don't see any CPU activity using top and restart kibana (systemctl restart kibana) wait for about 30s and open a new window in your browser.

Sometimes it just doesn't exit the process but the plugin is properly installed. It's a Kibana related known bug, not the Wazuh app itself.

Also, the Kibana team is going to fully remove this "Optimizing" procedure in further releases, I hope it comes as soon as possible.

Regards

@Mrfly-ca
Copy link

I waited till the Node.js process stopped using a large amount of CPU.
Restarted Kibana and it is working correctly.
Thanks for your help.

@jesusgn90
Copy link
Contributor

Nice, happy to help!

@luckybool1020
Copy link

Hello @athanatos64,

You can start the Kibana binary in verbosity mode so you can check in real time what logs are being generated during plugin optimization process.

  1. Stop the Kibana service:
systemctl stop kibana
  1. Remove the optimization bundles:
rm -rf /usr/share/kibana/optimize/bundles
  1. Run the Kibana binary in foreground mode. The -c option is required to tell Kibana which configuration file to use:
/usr/share/kibana/bin/kibana --verbose -c /etc/kibana/kibana.yml
  1. At first, you'll se a message like this:
"message":"Optimizing and caching bundles for graph, monitoring, login, logout, ml, dashboardViewer, apm, wazuh, kibana, stateSessionStorageRedirect, status_page and timelion. This may take a few minutes"

You have to wait until you see this message:

"message":"Optimization of bundles for graph, monitoring, login, logout, ml, dashboardViewer, apm, wazuh, kibana, stateSessionStorageRedirect, status_page and timelion complete in 160.16 seconds"

This will tell us that the optimization process finished. Now you can close the Kibana binary execution using the Ctrl+C keyboard shortcut.

  1. Now you can restart the Kibana service:
systemctl restart kibana
  1. Wait some seconds and now try again to open the Kibana interface on an incognito window on your Internet browser.

Let us know if this helps.

Regards,
Juanjo

Screen Shot 2021-01-06 at 8 12 15 PM

Screen Shot 2021-01-06 at 8 11 18 PM

Follow your steps, but it doesn't work :(

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/bug Bug issue
Projects
None yet
Development

No branches or pull requests

7 participants