Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-828

LoggerSink representation of the event's body isn't too useful

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • NG alpha 1
    • 1.2.0
    • Sinks+Sources
    • None

    Description

      LoggerSink logs entries to console that looks like this:
      Event: { headers:{} body:[B@5c1ae90c }

      ...where the body is just "getClass().getName() + "@" + Integer.toHexString(hashCode())". The "getClass().getName() will always resolve to [B.

      The issue seems to be how can we represent a SimpleEvent's payload as a String, when the payload is some arbitrary byte array... the array's bytes could represent encoded ascii chars, encoded UTF-8 chars, or binary data such as an encrypted payload. If we default to ASCII translation for everything, then the resulting String won't be useful for binary payloads since not all 256 possible bytes have equivalent printable ASCII chars. Here's one idea:

      For each event body, we can print up to the first 16 bytes in hex format. If there are >16 bytes, then print a "..." suffix at the end. The output would look similar to what you get with unix "hexdump -C". Here's what a sample output from LoggerSink would look like:
      Event: { headers:{} body: 00000000 54 68 65 20 71 75 69 63 6B 20 62 72 6F 77 6E 20 |The quick brown | ... }
      ...where both the hex and the ascii are displayed for the first 16 chars.

      Is it the most useful representation of the body? Probably not. Is it as least more useful than printing "[B@" + Integer.toHexString(hashCode())"? I think so.

      The commons io lib has a useful HexDump.dump cmd we can leverage.

      Thoughts?

      Attachments

        1. FLUME-828-0.patch
          3 kB
          Brock Noland
        2. FLUME-828-1.patch
          4 kB
          Brock Noland
        3. FLUME-828-2.patch
          4 kB
          Brock Noland
        4. FLUME-828-3.patch
          4 kB
          Brock Noland

        Issue Links

          Activity

            People

              brocknoland Brock Noland
              will@cloudera.com Will McQueen
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: