Uploaded image for project: 'Metron'
  1. Metron
  2. METRON-1321

Metaalert Threat Score Type Does Not Match Sensor Indices

    Details

    • Type: Bug
    • Status: Done
    • Priority: Major
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: 0.4.2
    • Labels:
      None

      Description

      Open up the Alerts UI, then sort by threat triage score.

      The following request is sent.

      {
        "indices": [],
        "facetFields": [
          "source:type",
          "ip_src_addr",
          "ip_dst_addr",
          "host",
          "enrichments:geo:ip_dst_addr:country"
        ],
        "query": "*",
        "from": 0,
        "size": 25,
        "sort": [
          {
            "field": "threat:triage:score",
            "sortOrder": "asc"
          }
        ]
      }
      

      The following error is logged from the API side.

      17/11/20 16:55:40 ERROR dao.ElasticsearchColumnMetadataDao: Field type mismatch: snort_index_2017.11.20.16.threat:triage:score has type float while metaalert_index.threat:triage:score has type double.  Defaulting type to other.
      17/11/20 16:55:40 ERROR dao.ElasticsearchColumnMetadataDao: Field type mismatch: bro_index_2017.11.20.16.id has type string while snort_index_2017.11.20.16.id has type integer.  Defaulting type to other.
      17/11/20 16:55:40 ERROR dao.ElasticsearchRequestSubmitter: Failed to execute search; error='NotSerializableExceptionWrapper: class_cast_exception: java.lang.Double cannot be cast to java.lang.Float', search='{"from":0,"size":25,"query":{"constant_score":{"filter":{"bool":{"must":[{"bool":{"should":[{"query_string":{"query":"*"}},{"nested":{"query":{"query_string":{"query":"*"}},"path":"alert"}}]}},{"bool":{"should":[{"term":{"status":"active"}},{"bool":{"must_not":{"exists":{"field":"status"}}}}]}}],"must_not":{"exists":{"field":"metaalerts"}}}}}},"_source":{"includes":[],"excludes":[]},"sort":[{"threat:triage:score":{"order":"desc","missing":"_last","unmapped_type":"other"}}],"track_scores":true,"aggregations":{"source:type_count":{"terms":{"field":"source:type"}},"ip_src_addr_count":{"terms":{"field":"ip_src_addr"}},"ip_dst_addr_count":{"terms":{"field":"ip_dst_addr"}},"host_count":{"terms":{"field":"host"}},"enrichments:geo:ip_dst_addr:country_count":{"terms":{"field":"enrichments:geo:ip_dst_addr:country"}}}}'
      Failed to execute phase [query], [reduce] ; shardFailures {[0KqVPgyOT2KKCjTKZYIl3Q][bro_index_2017.11.20.16][0]: RemoteTransportException[[node1][192.168.66.121:9300][indices:data/read/search[phase/query]]]; nested: SearchParseException[failed to parse search source [{"from":0,"size":25,"query":{"constant_score":{"filter":{"bool":{"must":[{"bool":{"should":[{"query_string":{"query":"*"}},{"nested":{"query":{"query_string":{"query":"*"}},"path":"alert"}}]}},{"bool":{"should":[{"term":{"status":"active"}},{"bool":{"must_not":{"exists":{"field":"status"}}}}]}}],"must_not":{"exists":{"field":"metaalerts"}}}}}},"_source":{"includes":[],"excludes":[]},"sort":[{"threat:triage:score":{"order":"desc","missing":"_last","unmapped_type":"other"}}],"track_scores":true,"aggregations":{"source:type_count":{"terms":{"field":"source:type"}},"ip_src_addr_count":{"terms":{"field":"ip_src_addr"}},"ip_dst_addr_count":{"terms":{"field":"ip_dst_addr"}},"host_count":{"terms":{"field":"host"}},"enrichments:geo:ip_dst_addr:country_count":{"terms":{"field":"enrichments:geo:ip_dst_addr:country"}}}}]]; nested: IllegalArgumentException[No mapper found for type [other]]; }
              at org.elasticsearch.action.search.AbstractSearchAsyncAction.onFirstPhaseResult(AbstractSearchAsyncAction.java:176)
              at org.elasticsearch.action.search.AbstractSearchAsyncAction$1.onResponse(AbstractSearchAsyncAction.java:147)
              at org.elasticsearch.action.search.AbstractSearchAsyncAction$1.onResponse(AbstractSearchAsyncAction.java:144)
              at org.elasticsearch.action.ActionListenerResponseHandler.handleResponse(ActionListenerResponseHandler.java:41)
              at org.elasticsearch.transport.TransportService$DirectResponseChannel.processResponse(TransportService.java:819)
              at org.elasticsearch.transport.TransportService$DirectResponseChannel.sendResponse(TransportService.java:803)
              at org.elasticsearch.transport.TransportService$DirectResponseChannel.sendResponse(TransportService.java:793)
              at org.elasticsearch.transport.DelegatingTransportChannel.sendResponse(DelegatingTransportChannel.java:58)
              at org.elasticsearch.transport.RequestHandlerRegistry$TransportChannelWrapper.sendResponse(RequestHandlerRegistry.java:134)
              at org.elasticsearch.search.action.SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(SearchServiceTransportAction.java:369)
              at org.elasticsearch.search.action.SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(SearchServiceTransportAction.java:365)
              at org.elasticsearch.transport.TransportRequestHandler.messageReceived(TransportRequestHandler.java:33)
              at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:75)
              at org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:376)
              at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
              at java.lang.Thread.run(Thread.java:745)
      Caused by: NotSerializableExceptionWrapper[class_cast_exception: java.lang.Double cannot be cast to java.lang.Float]
              at java.lang.Float.compareTo(Float.java:49)
              at org.apache.lucene.search.FieldComparator.compareValues(FieldComparator.java:132)
              at org.apache.lucene.search.TopDocs$MergeSortQueue.lessThan(TopDocs.java:170)
              at org.apache.lucene.search.TopDocs$MergeSortQueue.lessThan(TopDocs.java:119)
              at org.apache.lucene.util.PriorityQueue.upHeap(PriorityQueue.java:263)
              at org.apache.lucene.util.PriorityQueue.add(PriorityQueue.java:140)
              at org.apache.lucene.search.TopDocs.mergeAux(TopDocs.java:255)
              at org.apache.lucene.search.TopDocs.merge(TopDocs.java:232)
              at org.elasticsearch.search.controller.SearchPhaseController.sortDocs(SearchPhaseController.java:253)
              at org.elasticsearch.action.search.SearchQueryThenFetchAsyncAction.moveToSecondPhase(SearchQueryThenFetchAsyncAction.java:72)
              at org.elasticsearch.action.search.AbstractSearchAsyncAction.innerMoveToSecondPhase(AbstractSearchAsyncAction.java:374)
              at org.elasticsearch.action.search.AbstractSearchAsyncAction.onFirstPhaseResult(AbstractSearchAsyncAction.java:171)
              ... 17 more
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                nickwallen Nick Allen
                Reporter:
                nickwallen Nick Allen
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: