Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Not a Problem
    • Affects Version/s: 0.2.0
    • Fix Version/s: 0.5.0
    • Component/s: Data Processors
    • Labels:
      None

      Description

      There's some code in MetricsDataLoader for doing unit conversions. This should be cleaned up, moved to Demux, and documented.

        Activity

        Hide
        Jiaqi Tan added a comment -

        Although this is sar-specific (or iostat) the Demux processor should read the column names, parse them for units, and use that information as hints for converting the numerical into the base units for the final information that's written to HDFS.

        Show
        Jiaqi Tan added a comment - Although this is sar-specific (or iostat) the Demux processor should read the column names, parse them for units, and use that information as hints for converting the numerical into the base units for the final information that's written to HDFS.
        Hide
        Ari Rabkin added a comment -

        I actually think we're better off hand-coding the conversions we want. LIke kb/sec *1000--> bytes/sec . I think trying to do it automagically won't work well, since there's not just unit conversions, but also different abbreviations.

        Show
        Ari Rabkin added a comment - I actually think we're better off hand-coding the conversions we want. LIke kb/sec *1000--> bytes/sec . I think trying to do it automagically won't work well, since there's not just unit conversions, but also different abbreviations.
        Hide
        Jiaqi Tan added a comment -

        Sure, I meant using the column headings' listed units to guide, perhaps, an if-else or switch to do the correct conversions for some degree of automation.

        Show
        Jiaqi Tan added a comment - Sure, I meant using the column headings' listed units to guide, perhaps, an if-else or switch to do the correct conversions for some degree of automation.
        Hide
        Ari Rabkin added a comment -

        I assume this is actually not going to be fixed in 0.2. Can I tag it as 0.3? It seems like it should be a priority.

        Show
        Ari Rabkin added a comment - I assume this is actually not going to be fixed in 0.2. Can I tag it as 0.3? It seems like it should be a priority.
        Hide
        Eric Yang added a comment -

        +1 for 0.3.

        Show
        Eric Yang added a comment - +1 for 0.3.
        Hide
        Eric Yang added a comment -

        This is no longer an issue. Demux parsers have been updated to use bytes for all data sources.

        Show
        Eric Yang added a comment - This is no longer an issue. Demux parsers have been updated to use bytes for all data sources.

          People

          • Assignee:
            Unassigned
            Reporter:
            Ari Rabkin
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development