Updating entity attributes fails when mongo key becomes too long

[olivierdeckers]

The following call creates an entity in a service path:

POST /v2/entities/test
Content-Type: application/json
Fiware-Service: default
Fiware-ServicePath: /tester1testcom/e2dbaecbcccccccccccccccccccccccc

{
    "type": "Patient",
    "key": "value"
}

If the call is sent again, this should update the attribute 'key'. However, Sending the call a second time returns HTTP statuscode 472, with a mongodb error of a duplicate key insertion.

Also, when using PUT calls to update attributes in this entity, orion reports the entity cannot be found.
The entity was inserted successfully, because a GET call to v2/entities/test returns the correct data.

This leads me to believe the POST/PUT code uses another query to test for the existance of the entity. I assume it fails to find this entity and proceeds by running an insert query, which fails.

By trial and error, I discovered that this behaviour occurs as soon as the mongo _id property has a certain length. In the example call, if we remove one character from either the service path, the entity id or the entity type, orion behaves as expected.

Our service path respects the restrictions of a maximum of 50 characters per level and a maximum of 10 levels.

Also, the mongo maximum index key length is 1024 bytes, which is well above the length of the _id value. Since the entities are successfully inserted into the mongodb database, I don't think the issue is located somewhere in the orion entity update logic rather than in mongodb

This was tested on orion v0.26.1 and mongo v2.6.11

[abbasahmad]

Hello,

Just wanted to confirm and give more details about this error on how to reproduce :

First TRY (UPDATE With APPEND)

POST :

http://IP:PORT/v1/updateContext

HEADERS :

• Content-type: application/json
• Accept: application/json
• Fiware-Service: default
• Fiware-ServicePath: /tester1testcom/e2dbaecbcccccccccccccccccccccccc

PAYLOAD :

{
    "contextElements": [
        {
            "type": "Sensor",
            "isPattern": "false",
            "id": "Room1",
            "attributes": [
            {
                "name": "temperature",
                "type": "float",
                "value": "23"
            },
            {
                "name": "pressure",
                "type": "integer",
                "value": "720"
            }
            ]
        }
    ],
    "updateAction": "APPEND"
}

Response

{
  "contextResponses": [
    {
      "contextElement": {
        "type": "Sensor",
        "isPattern": "false",
        "id": "Room1",
        "attributes": [
          {
            "name": "temperature",
            "type": "float",
            "value": ""
          },
          {
            "name": "pressure",
            "type": "integer",
            "value": ""
          }
        ]
      },
      "statusCode": {
        "code": "200",
        "reasonPhrase": "OK"
      }
    }
  ]
}

Second try

Do the Same POST With the SAME headers and payload
Here is the response :

Response

{
  "contextResponses" : [
    {
      "contextElement" : {
        "type" : "Sensor",
        "isPattern" : "false",
        "id" : "Room1",
        "attributes" : [
          {
            "name" : "temperature",
            "type" : "float",
            "value" : ""
          },
          {
            "name" : "pressure",
            "type" : "integer",
            "value" : ""
          }
        ]
      },
      "statusCode" : {
        "code" : "472",
        "reasonPhrase" : "request parameter is invalid/not allowed",
        "details" : "Database Error (collection: orion-default.entities - insert(): { _id: { id: \"Room1\", type: \"Sensor\", servicePath: \"/tester1testcom/e2dbaecbcccccccccccccccccccccccc\" }, attrNames: [ \"temperature\", \"pressure\" ], attrs: { temperature: { type: \"float\", creDate: 1454686805, modDate: 1454686805, value: \"23\" }, pressure: { type: \"integer\", creDate: 1454686805, modDate: 1454686805, value: \"720\" } }, creDate: 1454686805, modDate: 1454686805 } 
- exception: OperationException: { index: 0, code: 11000, errmsg: \"\", op: { _id: { id: \"Room1\", type: \"Sensor\", servicePath: \"/tester1testcom/e2dbaecbcccccccccccccccccccccccc\" }, attrNames: [ \"temperature\", \"pressure\" ], attrs: { temperature: { type: \"float\", creDate: 1454686805, modDate: 1454686805, value: \"23\" }, pressure: { type: \"integer\", creDate: 1454686805, modDate: 1454686805, value: \"720\" } }, creDate: 1454686805, modDate: 1454686805 } })"
      }
    }
  ]
}

I tried to reduce the Header : Fiware-ServicePath: /tester1testcom/e2dbaecbcccccccccccccccccccccccc to this value : Fiware-ServicePath: /tester1testcom/e2dbaecbccccccc and the Multiple append works with no problem.

Hope it helps you correcting the problem.

Abbas AHMAD
Easy Global Market

[fgalan]

Thank you for the feedback and the detailed report! Our QA team will have a look to it.

[iariasleon]

Hi @olivierdeckers,

We saw the problem in 0,26.1 version, was reported in #1423 and was fixed in in 0.27.0. But this request has changed in 0.27.0 version, I recomend to update.

In 0.27.0:
POST v2/entities/<id> can not to create a new entity
idandtype` have a max length allowed (256) and only plain ascii are allowed, so it is impossible to arrives to the mongo maximum index key length (1024 bytes), See "Field syntax restrictions" in http://telefonicaid.github.io/fiware-orion/api/v2/

And the json format will be:
Ex:

POST /v2/entities/test
{
    "key": {
         "type": "Patient",
        "value": "value"
    }
}

Or

POST /v2/entities/test?options=keyValues
{
    "key":  "value"
}

I have created an issue to v1 error, Thanks, @abbasahmad

[iariasleon]

I close, if you have some comments, reopen this issue again

[fgalan]

The fix in PR #1780 could be related with the solution to this issue.