Tor blutmagie #139

srilumpa · 2017-11-15T11:53:37Z

As for PR #138, this PR aims to offer a solution to issue #45 with an analyzer extracting data from torstatus.blutmagie.de to check if an artifact is linked to a TOR node. The artifact can be an IP address, a FQDN or a domain.

I documented both of the classes but if you need a more advance documentation, please tell me. Also, I hope i coped with your coding standards. If not tell me so I can fix it.

As for #138, there is a caching system in place.

… number

3c7 · 2017-12-15T12:29:48Z

Thanks for the fast response. Caching works great.

Python3 compatibility
Caching
Shell test

(TorBlutmagie) ➜  TorBlutmagie git:(TorBlutmagie) ✗ ./tor_blutmagie_analyzer.py <<< '{
"dataType": "ip",
"data": "134.119.3.2"
}' | jq
{
  "success": true,
  "summary": {
    "taxonomies": [
      {
        "level": "suspicious",
        "namespace": "TorBlutmagie",
        "predicate": "Node",
        "value": "1 node"
      }
    ]
  },
  "artifacts": [
    {
      "type": "fqdn",
      "value": "j72505.servers.jiffybox.net"
    },
    {
      "type": "ip",
      "value": "134.119.3.2"
    }
  ],
  "full": {
    "nodes": [
      {
        "hostname": "j72505.servers.jiffybox.net",
        "name": "Zwiebel2",
        "country_code": "DE",
        "ip": "134.119.3.2",
        "as_name": "HOSTEUROPE-AS- DE",
        "as_number": "20773"
      }
    ]
  }
}
(TorBlutmagie) ➜  TorBlutmagie git:(TorBlutmagie) ✗ ./tor_blutmagie_analyzer.py <<< '{
"dataType": "fqdn",
"data": "j72505.servers.jiffybox.net"
}' | jq
{
  "success": true,
  "summary": {
    "taxonomies": [
      {
        "level": "suspicious",
        "namespace": "TorBlutmagie",
        "predicate": "Node",
        "value": "1 node"
      }
    ]
  },
  "artifacts": [
    {
      "type": "fqdn",
      "value": "j72505.servers.jiffybox.net"
    },
    {
      "type": "ip",
      "value": "134.119.3.2"
    }
  ],
  "full": {
    "nodes": [
      {
        "hostname": "j72505.servers.jiffybox.net",
        "name": "Zwiebel2",
        "country_code": "DE",
        "ip": "134.119.3.2",
        "as_name": "HOSTEUROPE-AS- DE",
        "as_number": "20773"
      }
    ]
  }
}
(TorBlutmagie) ➜  TorBlutmagie git:(TorBlutmagie) ✗ ./tor_blutmagie_analyzer.py <<< '{
"dataType": "domain",
"data": "jiffybox.net" 
}' | jq
{
  "success": true,
  "summary": {
    "taxonomies": [
      {
        "level": "suspicious",
        "namespace": "TorBlutmagie",
        "predicate": "Node",
        "value": "7 nodes"
      }
    ]
  },
  "artifacts": [
    {
      "type": "fqdn",
      "value": "j98727.servers.jiffybox.net"
    },
    {
      "type": "ip",
      "value": "93.180.156.84"
    },
    {
      "type": "fqdn",
      "value": "j60204.servers.jiffybox.net"
    },
    {
      "type": "ip",
      "value": "37.200.99.251"
    },
    {
      "type": "fqdn",
      "value": "j17531.servers.jiffybox.net"
    },
    {
      "type": "ip",
      "value": "46.252.26.2"
    },
    {
      "type": "fqdn",
      "value": "j54490.servers.jiffybox.net"
    },
    {
      "type": "ip",
      "value": "37.200.98.5"
    },
    {
      "type": "fqdn",
      "value": "j184462.servers.jiffybox.net"
    },
    {
      "type": "ip",
      "value": "134.119.36.135"
    },
    {
      "type": "fqdn",
      "value": "j32098.servers.jiffybox.net"
    },
    {
      "type": "ip",
      "value": "37.200.98.117"
    },
    {
      "type": "fqdn",
      "value": "j72505.servers.jiffybox.net"
    },
    {
      "type": "ip",
      "value": "134.119.3.2"
    }
  ],
  "full": {
    "nodes": [
      {
        "hostname": "j98727.servers.jiffybox.net",
        "name": "BARACUDA",
        "country_code": "DE",
        "ip": "93.180.156.84",
        "as_name": "HOSTEUROPE-AS- DE",
        "as_number": "20773"
      },
      {
        "hostname": "j60204.servers.jiffybox.net",
        "name": "JPsi2",
        "country_code": "DE",
        "ip": "37.200.99.251",
        "as_name": "HOSTEUROPE-AS- DE",
        "as_number": "20773"
      },
      {
        "hostname": "j17531.servers.jiffybox.net",
        "name": "marlen1",
        "country_code": "DE",
        "ip": "46.252.26.2",
        "as_name": "HOSTEUROPE-AS- DE",
        "as_number": "20773"
      },
      {
        "hostname": "j54490.servers.jiffybox.net",
        "name": "torpidsDEdomainf",
        "country_code": "DE",
        "ip": "37.200.98.5",
        "as_name": "HOSTEUROPE-AS- DE",
        "as_number": "20773"
      },
      {
        "hostname": "j184462.servers.jiffybox.net",
        "name": "torpidsDEdomainf2",
        "country_code": "DE",
        "ip": "134.119.36.135",
        "as_name": "HOSTEUROPE-AS- DE",
        "as_number": "20773"
      },
      {
        "hostname": "j32098.servers.jiffybox.net",
        "name": "Unnamed",
        "country_code": "DE",
        "ip": "37.200.98.117",
        "as_name": "HOSTEUROPE-AS- DE",
        "as_number": "20773"
      },
      {
        "hostname": "j72505.servers.jiffybox.net",
        "name": "Zwiebel2",
        "country_code": "DE",
        "ip": "134.119.3.2",
        "as_name": "HOSTEUROPE-AS- DE",
        "as_number": "20773"
      }
    ]
  }
}

TheHive Integration Test (Reports etc.)

3c7

Beside the requested change, everything works perfect. Also, it could be considered to include #!/usr/bin/env python3 instead of #!/usr/bin/env python in tor_blutmagie_analyzer.py, because of the python2/python3 encoding issues that will pop up, if people run that analyzer using python2.

3c7 · 2017-12-18T07:46:53Z

thehive-templates/TorBlutmagie_1_0/long.html

+        {{n.name}}
+      </div>
+      <div class="panel-body">
+        <dl class="dl-horizontal">


Please switch the <dt> and <dd> here:

3c7 · 2017-12-18T07:54:57Z

thehive-templates/TorBlutmagie_1_0/long.html

+      <div class="panel-body">
+        <dl class="dl-horizontal">
+          <dd>Address</dd>
+          <dt>{{n.hostname | fang}} ({{n.ip_address | fang}})</dt>


Here, it has to be n.ip.

3c7 · 2017-12-18T07:55:28Z

thehive-templates/TorBlutmagie_1_0/long.html

+        </dl>
+        <span>
+          <i class="fa fa-search"></i>
+          <a ng-href="http://torstatus.blutmagie.de/cgi-bin/whois.pl?ip={{n.ip_address}}" target="_blank">WHOIS</a>


This has to be n.ip, also.

srilumpa · 2017-12-18T08:49:54Z

Here are the fixes. I am so sorry for those basic errors.

3c7

Thank you very much for the contribution.

srilumpa and others added 5 commits October 27, 2017 12:42

Add TorBlutmagie analyzer

7c282d2

Add reports for TheHive usage

3c4f747

Properly escape artifacts in report

6dc740d

Add documentation to class

f307340

Merge branch 'hotfix/1.7.1'

105fea2

saadkadhi added scope:analyzer Issue is analyzer related category:feature-request Issue is related to a feature request status:pr-submitted labels Dec 13, 2017

jeromeleonard added this to the 1.8.0 milestone Dec 14, 2017

srilumpa added 5 commits December 14, 2017 16:16

Merge branch 'master' into TorBlutmagie

f6813d2

Remove dependency with pyfscache to ensure Python3 compatibility

98ef989

Fix taxonomies building when no result returned from analyzer

f97f984

Set up cache key to be a private class variable instead of some magic…

72d1872

… number

Fix getting data when cache is deactivated

0d86579

Respect contribution policy

06330ca

3c7 suggested changes Dec 18, 2017

View reviewed changes

srilumpa added 2 commits December 18, 2017 09:42

Force usage of python3 to execute analyzer

c4470c7

Fix long TheHive report

572c996

3c7 approved these changes Dec 18, 2017

View reviewed changes

nadouani changed the base branch from master to develop December 18, 2017 10:22

nadouani merged commit cf66ed2 into TheHive-Project:develop Dec 18, 2017

srilumpa deleted the TorBlutmagie branch February 22, 2018 16:33

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Tor blutmagie #139

Tor blutmagie #139

srilumpa commented Nov 15, 2017

3c7 commented Dec 15, 2017

3c7 left a comment

3c7 Dec 18, 2017

3c7 Dec 18, 2017

3c7 Dec 18, 2017

srilumpa commented Dec 18, 2017

3c7 left a comment

Tor blutmagie #139

Tor blutmagie #139

Conversation

srilumpa commented Nov 15, 2017

3c7 commented Dec 15, 2017

3c7 left a comment

Choose a reason for hiding this comment

3c7 Dec 18, 2017

Choose a reason for hiding this comment

3c7 Dec 18, 2017

Choose a reason for hiding this comment

3c7 Dec 18, 2017

Choose a reason for hiding this comment

srilumpa commented Dec 18, 2017

3c7 left a comment

Choose a reason for hiding this comment