The guide assumes that you have an installed linux based server with the following packages installed and at least the versions given.
- nginx 1.4.x
- node 6.x
- supervisor 3.x
- Valid SSL certificates for you domain.
The document also assumes that you are logged in as the user deploy, if this is not the case, you need to change things (e.g. the supervisor run script etc.).
This document uses [] square brackets around the different variables in the configuration templates that needs to be exchanged with the actual values from other parts of the configuration (e.g. API keys).
Her is an explanation of the different key configuration variables.
- [server name]
- [client name]
- [SSL CERT]
- [SSL KEY]
- [CHANGE ME]
- [PASSWORD]
- [SEARCH API KEY]
- [SEARCH INDEX KEY]
- @TODO Document placeholders [...]
Things in boxes are commands that should be executed or configuration thats need in the files given.
Note: To install the newest version (development version that's not aways stable), you should checkout the development branches in the all the cloned repositories instead of the latest version tag.
The search node application is a general purpose application to provide a fast search engine through web-socket connections (using elasticsearch) and is designed to be used by other projects as well as aroskanalen. As a result of this, it will not follow the same versioning as the other parts of this installation. It's also why its UI is somewhat complex to use when setting up the right mappings to be used with aroskanalen.
Start by cloning the git repository for the search node and checkout the latest release tag (which can be found using git tag).
cd /home/www git clone [email protected]:search-node/search_node.git cd search_node git checkout [v1.x.x]
Search node, as the middleware, uses a plugin architecture that requires installation of libraries from the node package manager (npm). The application comes with an installation script to handle this, simply go to the root of the application and execute the script.
cd /home/www/search_node/ ./install.sh
We need to configure nginx to sit in front of the search node so it is accessible through normal web-ports and also proxy web-socket connections. So we add the upstream connection configuration in a nodejs configuration file for nginx.
sudo nano -w /etc/nginx/sites-available/nodejs
Append this upstream connection definition to the file.
upstream nodejs_search {
server 127.0.0.1:3010;
}
To access the search node UI and allow communication with the search node a virtual host configuration is needed. You need to change the [server name] with the actual name of the server.
sudo nano -w /etc/nginx/sites-available/search_[server name]
server {
listen 80;
server_name search-[server name];
rewrite ^ https://$server_name$request_uri? permanent;
access_log /var/log/nginx/search_access.log;
error_log /var/log/nginx/search_error.log;
}
# HTTPS server
#
server {
listen 443;
server_name search-[server name];
access_log /var/log/nginx/search_access.log;
error_log /var/log/nginx/search_error.log;
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_buffering off;
proxy_pass http://nodejs_search/;
proxy_redirect off;
}
location /socket.io/ {
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_pass http://nodejs_search;
}
ssl on;
ssl_certificate /etc/nginx/ssl/[SSL CERT].crt;
ssl_certificate_key /etc/nginx/ssl/[SSL KEY].key;
ssl_session_timeout 5m;
ssl_session_cache shared:SSL:10m;
# https://hynek.me/articles/hardening-your-web-servers-ssl-ciphers/
ssl_prefer_server_ciphers On;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS;
}
Enable the configuration by adding a symbolic links for the search node. The nodejs configuration file should be linked during configuration of the middleware. Restart nginx to enable the configuration.
cd /etc/nginx/sites-enabled/ sudo ln -s ../sites-available/nodejs sudo ln -s ../sites-available/search_[server name] sudo service nginx restart
Next the application needs to be configured by adding the following content to config.json. Remember to update the administration password and secret.
sudo nano -w /home/www/search_node/config.json
{
"port": 3010,
"secret": "[CHANGE ME]",
"admin": {
"username": "admin",
"password": "[PASSWORD]"
},
"log": {
"file": "messages.log",
"debug": false
},
"search": {
"hosts": [ "localhost:9200" ],
"mappings": "mappings.json"
},
"apikeys": "apikeys.json"
}
Before the application can be started the apikeys.json and mappings.json needs to exist and at least contain an empty JSON object ({}).
echo '{}' > /home/www/search_node/apikeys.json
echo '{}' > /home/www/search_node/mappings.json
The search node needs to be started at boot time which requires a Supervisor run script. Supervisor will also ensure that the node application is restarted, if an error happens and it stops unexpectedly.
sudo nano -w /etc/supervisor/conf.d/search_node.conf
Supervisor run script for the search node.
[program:search-node] command=node /home/www/search_node/app.js autostart=true autorestart=true environment=NODE_ENV=production stderr_logfile=/var/log/search-node.err.log stdout_logfile=/var/log/search-node.out.log user=deploy
sudo service supervisor restart
As mentioned the search node is not specially created for aroskanalen, so the mappings (configuration for elasticsearch) can be somewhat complex to setup in the UI. To get you started the mapping below can be used as a template for the configuration.
As we need the UI to complete the setup correctly the node application needs to have write access to the files.
cd /home/www/search_node/ chmod +w apikeys.json mappings.json
Now use the UI (https://search-[server name].aroskanalen.dk) and add a new api key. Then go to the mappings tabs in the UI and add a new empty mapping. Next edit the mappings file and add the fields, tag and dates section as in the template. This way you will get a new API key and search index key for each installation. Note that each installation of the admin application requires a new API key and search index.
nano -w /home/www/search_node/mappings.json
{
"5d437a016271077510c640e450bde9c3": {
"name": "demo",
"tag": "private",
"fields": [
{
"field": "title",
"type": "string",
"language": "da",
"country": "DK",
"default_analyzer": "string_index",
"sort": true,
"indexable": true
},
{
"field": "name",
"type": "string",
"language": "da",
"country": "DK",
"default_analyzer": "string_index",
"sort": true,
"indexable": true
}
],
"dates": [
"created_at",
"updated_at"
]
}
}
When you have update the mappings file go back into the UI and select the indexes that you need by edit the API key and select it/them in the edit window. Before a given index can be used you need to activate it in the indexes tab. So do that now.
@TODO: How to use the UI to add more configuration.
Search node used elasticsearch 1.5.x as its search engine and it needs to be installed as well.
First install java that is used to run elasticsearch.
sudo apt-get install openjdk-7-jre -y > /dev/null 2>&1
Download and install the engine.
sudo -i cd /root wget https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.1.deb dpkg -i elasticsearch-1.7.1.deb update-rc.d elasticsearch defaults 95 10
To enable ICU support (unicode) please install this plugin.
sudo /usr/share/elasticsearch/bin/plugin -install elasticsearch/elasticsearch-analysis-icu/2.5.0
For debuggin elasticsearch this small administration interface can come handy, but its optional.
/usr/share/elasticsearch/bin/plugin -install mobz/elasticsearch-head