Merge branch 'develop'

README.md

@@ -1,7 +1,9 @@
 #<a name="top"></a>Orion Context Broker
 
-[![license badge](https://img.shields.io/badge/license-AGPL-blue.svg)](https://opensource.org/licenses/AGPL-3.0)
-[![Documentation Status](https://readthedocs.org/projects/fiware-orion/badge/?version=latest)](http://fiware-orion.readthedocs.org/en/latest/?badge=latest)
+[![License badge](https://img.shields.io/badge/license-AGPL-blue.svg)](https://opensource.org/licenses/AGPL-3.0)
+[![Documentation badge](https://readthedocs.org/projects/fiware-orion/badge/?version=latest)](http://fiware-orion.readthedocs.org/en/latest/?badge=latest)
+[![Docker badge](https://img.shields.io/docker/pulls/fiware/orion.svg)](https://hub.docker.com/r/fiware/orion/)
+[![Support badge]( https://img.shields.io/badge/support-sof-yellowgreen.svg)](http://stackoverflow.com/questions/tagged/fiware-orion)
 
 * [Introduction](#introduction)
 * [GEi overall description](#gei-overall-description)

doc/apiary/v2/fiware-ngsiv2-cookbook.apib

@@ -450,7 +450,7 @@ This resource allows you to obtain a list of subresources published by the API:
 
     + Headers
 
-            Location: /v2/entities/Bcn-Welt
+            Location: /v2/entities/Bcn-Welt?type=Room
 
 
 ### Create a new entity. Error [POST /v2/entities]

doc/manuals/admin/build_source.md

@@ -7,10 +7,10 @@ Orion Context Broker reference distribution is CentOS 6.x. This doesn't mean tha
 The Orion Context Broker uses the following libraries as build dependencies:
 
 * boost: 1.41 (the one that comes in EPEL6 repository)
-* libmicrohttpd: 0.9.22 (the one that comes in EPEL6 repository)
+* libmicrohttpd: 0.9.48 (from source)
 * libcurl: 7.19.7
 * Mongo Driver: legacy-1.0.7 (from source)
-* rapidjson: 1.0.2
+* rapidjson: 1.0.2 (from source)
 * gtest (only for `make unit_test` building target): 1.5 (from sources)
 * gmock (only for `make unit_test` building target): 1.5 (from sources)
 
@@ -24,9 +24,9 @@ commands that require root privilege):
 
         sudo yum install make cmake gcc-c++ scons
 
-* Install the required libraries (except the mongo driver and gmock, described in following steps).
+* Install the required libraries (except what needs to be taken from source, described in following steps).
 
-        sudo yum install libmicrohttpd-devel boost-devel libcurl-devel
+        sudo yum install boost-devel libcurl-devel gnutls-devel libgcrypt-devel
 
 * Install the Mongo Driver from source:
 
@@ -42,6 +42,16 @@ commands that require root privilege):
         tar xfvz v1.0.2.tar.gz
         sudo mv rapidjson-1.0.2/include/rapidjson/ /usr/local/include
 
+* Install libmicrohttpd from sources (the `./configure` command below shows the recommended build configuration to get minimum library footprint, but if you are an advanced user, you can configure as you prefer)
+
+        wget http://ftp.gnu.org/gnu/libmicrohttpd/libmicrohttpd-0.9.48.tar.gz
+        tar xvf libmicrohttpd-0.9.48.tar.gz
+        cd libmicrohttpd-0.9.48
+        ./configure --disable-messages --disable-postprocessor --disable-dauth
+        make
+        sudo make install  # installation puts .h files in /usr/local/include and library in /usr/local/lib
+        sudo ldconfig      # just in case... it doesn't hurt :)
+
 * Install Google Test/Mock from sources (there are RPM pacakges for this, but they don't seem to be working with the current CMakeLists.txt configuration)
 
         wget http://googlemock.googlecode.com/files/gmock-1.5.0.tar.bz2

doc/manuals/admin/install.md

@@ -25,7 +25,7 @@ but it should work also in any later CentOS/RedHat 6.x version.
     * Note that the officially supported MongoDB version is 2.6. In the case of using MongoDB 3.0 with its new authentication mechanism
       (SCRAM_SHA1) you may need to compile from sources using special switches for the MongoDB driver. See [this issue](https://github.com/telefonicaid/fiware-orion/issues/1061) for details.
 * RPM dependencies (some of these packages could not be in the official CentOS/RedHat repository but in EPEL, in which case you have to configure EPEL repositories, see <http://fedoraproject.org/wiki/EPEL>):
-    * The contextBroker package (mandatory) depends on the following packages: boost-filesystem, boost-thread, libmicrohttpd, logrotate, libcurl and boost-regex.
+    * The contextBroker package (mandatory) depends on the following packages: boost-filesystem, boost-thread, gnutls, libgcrypt, logrotate and libcurl.
     * The contextBroker-test package (optional) depends on the following packages: python, python-flask, python-jinja2, curl, libxml2, libxslt, nc, mongo-10gen and contextBroker. The mongo-10gen dependency needs to configure MongoDB repository, check [this piece of documentation about that](http://docs.mongodb.org/manual/tutorial/install-mongodb-on-red-hat-centos-or-fedora-linux/).
 
 ## Installation

doc/manuals/admin/logs.md

@@ -161,7 +161,7 @@ that after fixing the problem with the JSON payload now has a new problem with t
 ## Summary traces
 
 You can enable log summary traces with the `-logSummary` [CLI parameter](cli.md) which value is the summary reporting period in seconds. For
-example `-logSummary 5` involves that summary traces will be print each 5 seconds.
+example `-logSummary 5` involves that summary traces will be print each 5 seconds (no matter which log level has been set with `-logLevel`).
 
 Four traces are printed each time, as follows (the lines have been abbreviated, omitting some fields, for the sake of clarity):
 

doc/manuals/deprecated.md

@@ -32,3 +32,32 @@ A list of deprecated features and the version in which they were deprecated foll
     the broker runs in "ConfMan mode".
 * Configuration Manager role (deprecated in 0.21.0, removed in 0.25.0)
 * Associations (deprecated in 0.21.0, removed in 0.25.0).
+
+## Using old Orion versions
+
+Although you are encouraged to use always the newest Orion version, take into account the following
+information in the case you want to use old versions:
+
+* Old RPMs (since Orion 0.1.1) are available at the [FIWARE yum repository](http://repositories.lab.fiware.org/repo/rpm/x86_64).
+* Code correponding to old releases (since Orion 0.8.1, the first one available as open source) is
+  available at the [Orion github repository](http://github.com/telefonicaid/fiware-orion). Each release number
+  (e.g. 0.23.0) has associated the following:
+  * A tag, e.g. `0.23.0`. It points to the base version.
+  * A release branch, `release/0.23.0`. The HEAD of this branch usually matches the aforementioned tag. However, if some
+    hotfixes were developed on the base version, this branch contains such hotfixes.
+* Documentation corresponding to old versions can be found:
+  * For 0.23.0 and before: documentation is available at FIWARE public wiki ([user manual](https://forge.fiware.org/plugins/mediawiki/wiki/fiware/index.php/Publish/Subscribe_Broker_-_Orion_Context_Broker_-_User_and_Programmers_Guide)
+    and [admin manual](https://forge.fiware.org/plugins/mediawiki/wiki/fiware/index.php/Publish/Subscribe_Broker_-_Orion_Context_Broker_-_Installation_and_Administration_Guide)).
+  * For 0.24.0 or newer: documentation is available at [readthedocs.org](https://fiware-orion.readthedocs.org).
+    Use the panel in the left bottom corner to navigate to the right version.
+* Docker images corresponding to Orion 0.24.0 and newer can be found at [Dockerhub](https://hub.docker.com/r/fiware/orion/tags/).
+
+The following table provides information about the last Orion version supporting currently removed features:
+
+| **Removed feature**                                                        | **Last Orion version supporting feature** | **That version release date**   |
+|----------------------------------------------------------------------------|-------------------------------------------|---------------------------------|
+| XML API                                                                    | Not yet removed (planned: 0.28.0)         | February 29st, 2016 (expected)     |
+| ONTIMEINTERVAL subscription                                                | Not yet removed (planned: 0.28.0)         | February 29st, 2016 (expected)     |
+| CLI `--silent`                                                             | Not yet removed (planned: 0.28.0)         | February 29st, 2016 (expected)     |
+| Configuration Manager role (including `-ngsi9`, `-fwdHost` and `-fwdPort`) | 0.24.0                                    | September 14th, 2015            ||
+| Associations                                                               | 0.24.0                                    | September 14th, 2015            |

doc/manuals/user/filtering.md

@@ -5,6 +5,7 @@
 * [No-Existence type filter](#no-existence-type-filter)
 * [Entity type filter](#entity-type-filter)
 * [Geo-location filter](#geo-location-filter)
+* [Geo-location filter NGSIv2](#geo-location-filter-ngsiv2)
 * [String query filter](#string-filter)
 
 ## Introduction
@@ -140,6 +141,13 @@ operation filter.
 
 [Top](#top)
 
+## Geo-location filter NGSIv2
+
+The scope corresponding to this type is "FIWARE::Location::NGSIv2". It is
+described in detail in [the following section](geolocation.md#geo-located-queries-ngsiv2).
+
+[Top](#top)
+
 ## String filter
 
 The scope corresponding to this type is "FIWARE::StringQuery".
@@ -151,7 +159,7 @@ The scope corresponding to this type is "FIWARE::StringQuery".
             "scopes": [
                 {
                     "type": "FIWARE::StringQuery",
-                    "value": "temperature<24;humidity==75..90;status=running"
+                    "value": "temperature<24;humidity==75..90;status==running"
                 }
             ]
         }
@@ -165,7 +173,7 @@ greater/less than, range or existence.
 There isn't any URL parameter correspondence for this filter in NGSI v1. In NGSI v2
 it corresponds to the `q` parameter:
 
-    curl 'localhost:1026/v2/entities?q=temperature<24;humidity==75..90;status=running'
+    curl 'localhost:1026/v2/entities?q=temperature<24;humidity==75..90;status==running'
 
 For a detailed syntax description of the `value` or `q` parameter, see [NGSIv2 specification
 document](http://telefonicaid.github.io/fiware-orion/api/v2/).

doc/manuals/user/forbidden_characters.md

@@ -46,6 +46,13 @@ attribute identifiers as part of the URL, their use should be avoided.
 The use of these characters is perfectly safe when only standard operations
 are involved, anyway. 
 
+### Exceptions
+
+There are some exception cases in which the above restrictions do not apply. In particular, in the following fields:
+
+* URL parameter `q` and the value of "FIWARE::StringQuery" scope allow the special characters needed by the Simple Query Language
+* URL parameter `georel` and `coords` and the corresponding fields in the "FIWARE::Location::NGSIv2" scope allow `;`
+
 ## Specific restrictions for ID fields
 
 NGSIv2 introduces syntax restrictions for ID fields (such as entity id/type, attribute name/type

doc/manuals/user/geolocation.md

@@ -464,3 +464,65 @@ be Alcobendas.
 }
 EOF
 ```
+
+## Geo-located queries NGSIv2
+
+The [NGSIv2 specification](http://telefonicaid.github.io/fiware-orion/api/v2/) defines a Geographical Query
+languague (based on `georel`, `geometry` and `coords` fields) that can be used also in NGSIv1 with the
+FIWARE::Location::NGSIv2 scope, e.g.:
+
+```
+(curl localhost:1026/v1/queryContext -s -S --header 'Content-Type: application/json' \
+    --header 'Accept: application/json' -d @- | python -mjson.tool) <<EOF
+{
+    "entities": [
+        {
+            "type": "City",
+            "isPattern": "true",
+            "id": ".*"
+        }
+    ],
+    "restriction": {
+      "scopes": [
+        {
+          "type" : "FIWARE::Location::NGSIv2",
+          "value" : {
+            "georel": [ "near", "minDistance:13500" ],
+            "geometry": "point",
+            "coords": [ [40.418889,-3.691944] ]
+          }
+        }
+      ]
+    }
+}
+EOF
+```
+
+```
+(curl localhost:1026/v1/queryContext -s -S --header 'Content-Type: application/json' \
+    --header 'Accept: application/json' -d @- | python -mjson.tool) <<EOF
+{
+    "entities": [
+        {
+            "type": "City",
+            "isPattern": "true",
+            "id": ".*"
+        }
+    ],
+    "restriction": {
+    "scopes": [
+        {
+          "type" : "FIWARE::Location::NGSIv2",
+          "value" : {
+            "georel": [ "equals" ],
+            "geometry": "polygon",
+            "coords": [ [0, 0], [24, 0], [0, 20], [0, 0] ]
+          }
+        }
+      ]
+    }
+}
+EOF
+```
+
+Please note that the `coords` field of the complex value of the scope needs to be put **after** the field `geometry`.

doc/manuals/user/pagination.md

@@ -28,10 +28,11 @@ allow pagination. The mechanism is based on three URI parameters:
     want to keep strict with NGSI, you can simply ignore the details
     parameter :)
 
-Result are returned ordered by increasing entity/registration creation
+By default, results are returned ordered by increasing entity/registration creation
 time. This is to ensure that if a new entity/registration is created
 while the client is going through all the results the new results are
-added at the end (thus avoiding duplication results).
+added at the end (thus avoiding duplication results). In the case of
+entities query, this can be changed with the [`orderBy` URL parameter](#ordering-results).
 
 Let’s illustrate with an example: a given client cannot process more
 than 100 results in a single response and the queryContext includes a
@@ -76,3 +77,20 @@ POST <orion_host>:1026/v1/queryContext?offset=1000&limit=100
     }
 }
 ```
+
+## Ordering results
+
+However, in the case of entities query, the `orderBy` URL parameter can be use to
+specify a different order. In particular, the value of `orderBy` is a comma-separated
+list of attributes and the results are ordered by the first attribute. On ties,
+the results are ordered by the second attribute and so on.
+A "!" before the attribute name means that the order is reversed. For example:
+
+    POST <orion_host>:1026/v1/queryContext?orderBy=temperature,!humidity
+
+orders first by temperature in ascending order, then by humidity in decreasing order
+in the case of temperature ties.
+
+Note that the special keywords `dateCreated` and `dateModified` can be used as
+elements in the `orderBy` comma-separated list (including the `!` syntax) to mean
+entity creation time and entity modification time respectively.

doc/manuals/user/v1_v2_coexistence.md

@@ -8,7 +8,7 @@ this is a big work and both API versions will be coexisting during some time.
 
 This document explains some consideration to take into account regarding such coexistence.
 
-# Native JSON types
+## Native JSON types
 
 NGSIv2 allows to create/update attributes (and metadata) whose values use JSON native 
 types (number, boolean, string, etc.). However, NGSIv1 uses a JSON parser that converts 
@@ -20,7 +20,7 @@ non-string JSON native types correctly. Thus, if you set `A=2` using NGSIv2 and
 attribute using NGSIv1 you will get `A=2`. Currently this work for attribute simple
 values, i.e. compound attribute values or metadata values always use string-based rendering.
 
-# Filtering
+## Filtering
 
 You can use the filtering capabilities developed for NGSIv2 (`GET /v2/entities?q=<query>`) also 
 in NGSIv1 using a Scope element in the payload of `POST /v1/queryContext`. See 
@@ -30,11 +30,24 @@ However, take into account that some of the filters (e. g. greater/less, range,
 for numeric values. Thus, in order to work properly, these filters (although using a 
 `POST /v1/queryContext`) needs that the attributes to which they refer were created using NGSIv2 operations.
 
-# Checking ID fields
+In addition, note that NGSIv2 geo-query filters can be used also in NGSIv1. See
+[the following section](geolocation.md#geo-located-queries-ngsiv2) for details
+
+## Checking ID fields
 
 NGSIv2 introduces syntax restrictions for ID fields (such as entity id/type, attribute name/type
 or metadata name/type) which are described in the "Field syntax restrictions" section in the
 [NGSIv2 specification](http://telefonicaid.github.io/fiware-orion/api/v2/). In order to
 keep backward compatibility, these restrictions are not used in the NGSIv1 API by default, but
 you can enable them using the `-strictNgsiv1Ids` [CLI parameter](../admin/cli.md).
 
+Related with this topic, note that NGSIv1 allows entities/attributes/metadatas without types
+and with types equal to the empty string (`""`). However, NGSIv2 ID fields (including types) have
+a minimum length of 1 character. Thus, entities created with NGSIv1 but rendered using NGSIv2 operation
+will automatically replace these cases with the string value `none` (which is the default type in NGSIv2).
+
+## `orderBy` parameter
+
+The `orderBy` parameter defined for NGSIv2 can be used also in NGSIv1 queryContext operation (see
+details in the [pagination documentation](pagination.md). However, note that the "geo:distance"
+order can be used only in NGSIv2.

docker/Dockerfile

@@ -18,21 +18,30 @@ RUN \
       bzip2 \
       cmake \
       epel-release \
+      gnutls-devel \
+      libgcrypt-devel \
       libcurl-devel \
-      libmicrohttpd-devel \
       make \
       nc \
       git \
       gcc-c++ \
       scons \
       tar \
       which && \
+    cd /opt && \
+    # Install libmicrohttpd from source
+    curl -kOL http://ftp.gnu.org/gnu/libmicrohttpd/libmicrohttpd-0.9.48.tar.gz && \
+    tar xvf libmicrohttpd-0.9.48.tar.gz && \
+    cd libmicrohttpd-0.9.48 && \
+    ./configure --disable-messages --disable-postprocessor --disable-dauth && \
+    make && \
+    make install && \
+    ldconfig && \
     # Install mongodb driver from source
     curl -kOL https://github.com/mongodb/mongo-cxx-driver/archive/legacy-1.0.7.tar.gz && \
     tar xfz legacy-1.0.7.tar.gz && \
     cd mongo-cxx-driver-legacy-1.0.7 && \
     scons install --prefix=/usr/local && \
-    cd /opt && \
     # Install rapidjson from source
     curl -kOL https://github.com/miloyip/rapidjson/archive/v1.0.2.tar.gz && \
     tar xfz v1.0.2.tar.gz && \
@@ -62,7 +71,10 @@ RUN \
     cd /opt && \
     if [ ${CLEAN_DEV_TOOLS} -eq 0 ] ; then yum clean all && exit 0 ; fi && \
     # cleanup sources, dev tools, locales and yum cache to reduce the final image size
-    rm -rf /opt/legacy-1.0.7.tar.gz \
+    rm -rf /opt/libmicrohttpd-0.9.48.tar.gz \
+           /usr/local/include/microhttpd.h \
+           /usr/local/lib/libmicrohttpd.* \
+           /opt/legacy-1.0.7.tar.gz \
            /opt/mongo-cxx-driver-legacy-1.0.7 \
            /usr/local/include/mongo \
            /usr/local/lib/libmongoclient.a \
@@ -84,7 +96,7 @@ RUN \
         cmake libarchive \
         gcc-c++ cloog-ppl cpp gcc glibc-devel glibc-headers \
         kernel-headers libgomp libstdc++-devel mpfr ppl \
-        scons libmicrohttpd-devel boost-devel libcurl-devel \
+        scons boost-devel libcurl-devel gnutls-devel libgcrypt-devel \
         clang llvm llvm-libs \
         CUnit-devel CUnit \
         autoconf automake m4 libidn-devel \