[METRON-403] Bro elasticsearch bulk index item fails when DNS response includes CNAME

• Type: Bug
• Status: Done
• Priority: Minor
• Resolution: Done
• Affects Version/s: 0.2.2BETA
• Fix Version/s: 0.3.0
• Labels: Bro Indexing newbie
• Environment: 7 node Ambari 2.2 managinf HDP 2.4; 4 Metron nodes; Followed wiki "Metron Installation on an Ambari-Managed Cluster"
• Assignee: Nick Allen
• Reporter: Andrew McKinney
• Created: 02/Sep/16 11:09
• Updated: 11/Nov/16 15:03
• Resolved: 21/Oct/16 15:06

Description

Querying CNAME records returns nested answers, e.g. fedora.aau.at.

;; ANSWER SECTION:
fedora.aau.at.          239     IN      CNAME   www-rpm.aau.at.
www-rpm.aau.at.         149     IN      A       143.205.180.155

This seems to get past the BasicBroParser but when it it comes to indexing, the bro es template expects type ip, not nested string/ip.

 {  
         "TTLs":[  
            445.0,
            414.0
         ],
         "bro_timestamp":"1.472812583319753E9",
         "ip_dst_port":53,
         "threatinteljoinbolt:joiner:ts":"1472812589689",
         "rejected":false,
         "answers":[  
            "www-rpm.aau.at",
            "143.205.180.155"
         ],
         "enrichmentsplitterbolt:splitter:begin:ts":"1472812589689",
         "enrichmentjoinbolt:joiner:ts":"1472812589689",
         "trans_id":802,
         "adapter:geoadapter:begin:ts":"1472812589689",
         "uid":"C6jPJB1uNqfcJmUPMd",
         "protocol":"dns",
         "source:type":"bro",
         "adapter:threatinteladapter:end:ts":"1472812589689",
         "original_string":"DNS | AA:false TTLs:[445.0,414.0] id.orig_p:47902 rejected:false id.resp_p:53 query:fedora.aau.at answers:[\"www-rpm.aau.at\",\"143.205.180.155\"] trans_id:802 rcode:0 rcode_name:NOERROR TC:false RA:true uid:C6jPJB1uNqfcJmUPMd RD:false proto:udp id.orig_h:10.150.194.160 Z:0 ts:1.472812583319753E9 id.resp_h:10.150.194.5",
         "ip_dst_addr":"10.150.194.5",
         "adapter:hostfromjsonlistadapter:end:ts":"1472812589689",
         "Z":0,
         "adapter:geoadapter:end:ts":"1472812589689",
         "ip_src_addr":"10.150.194.160",
         "threatintelsplitterbolt:splitter:end:ts":"1472812589689",
         "timestamp":1472812583319,
         "AA":false,
         "enrichmentsplitterbolt:splitter:end:ts":"1472812589689",
         "query":"fedora.aau.at",
         "rcode":0,
         "adapter:hostfromjsonlistadapter:begin:ts":"1472812589689",
         "rcode_name":"NOERROR",
         "TC":false,
         "RA":true,
         "RD":false,
         "ip_src_port":47902,
         "proto":"udp",
         "threatintelsplitterbolt:splitter:begin:ts":"1472812589689",
         "adapter:threatinteladapter:begin:ts":"1472812589689"
      }

throws

nested:IllegalArgumentException[  
   failed to parse ip   [  
      www-rpm.aau.at
   ],
   not a valid ip address
];

from bro_index.template

{
   "answers": {
          "type": "ip"
        },

Issue Links

• relates to: METRON-293 indexingBolt errors out for bro logs having IPV6 address or FQDNs
• links to: GitHub Pull Request #305