Uploaded image for project: 'Infrastructure'
  1. Infrastructure
  2. INFRA-19000

machines.html issues

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Not A Problem
    • Fix Version/s: None
    • Component/s: Monitoring
    • Labels:
      None
    • Project:
      Infrastructure

      Description

      There are some problems with the data displayed on the machines.html page

      The "Changes noticed" listing currently shows the epoch; this is not very useful. Also the node name is not shown.

      Fixes have been committed to the source in SVN, but these have not appeared on the live site.

      Another issue is that the change timestamp is not correct; it can be a week later than the actual time.

      For example:
      Key changed at 1567386004. Was d0:89:7a:7f:d2:81:6b:f1:39:8b:69:25:f6:cb:b5:d2, is now 4a:f2:81:2f:b0:d2:97:47:30:ad:f4:17:07:e6:2f:80

      1567386004 = 2019-09-02 01:00:04

      However the latest appearance of the old hash was
      1566779500000 = 2019/08/26 00:31:40
      and the earliest timestamp for the new hash is
      1566812289000 = 2019/08/26 09:38:09

      This suggests that the change timestamp should be 1566812289000.

      I think the reason for this is two-fold:
      - the code assumes the md5 hashes are listed in chronological order within the results. However they are listed most recent first in the tests I have done. Since the code only uses the last entry it uses the oldest value.
      - the code grabs data for the last week. This means that the old hashes will appear in the data for a week. Once the old hash has not been used for a week, it will disappear from the results and the code will pick up the only entry, i.e. the new hash.

      It's not clear why the data is collected for the last week, as the script is run hourly. Perhaps that's done to avoid having to expire entries by checking the last seen date? Seems wasteful if so. Reducing the time window would reduce the overlap when old and new hashes are both present.

      It's not clear whether ES guarantees the order of entries in the bucket for the current query. Is it always most recent first?
      If not, maybe some kind of sort can be applied?
      Otherwise, the code could perhaps check all entries to decide if there has been a change. Could be tricky if there have been two changes in the analysis window.

      Alternatively, if the data can be queried for the most recent entry for each host (within the time window), that would simplify checking for changes at the cost of needing to expire older entries.

      Also the changes appear for hosts that are not listed in the main table.
      That is confusing.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              sebb Sebb
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:
                Review Date: