Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-1945

HBase Serializer allow key from regular expression group

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.3.1
    • Fix Version/s: 1.4.0
    • Component/s: None
    • Labels:
      None
    • Release Note:
      Hide
      Allow users to chose a regular expression group as hbase row key. To use this feature users need to:

      1. Specify the row key index.
      2. Specify ROW_KEY as the column name for the corresponding index

      For example:
      a1.sinks.k1.serializer.rowKeyIndex = 1
      a1.sinks.k1.serializer.colNames = host,ROW_KEY,timestamp
      Show
      Allow users to chose a regular expression group as hbase row key. To use this feature users need to: 1. Specify the row key index. 2. Specify ROW_KEY as the column name for the corresponding index For example: a1.sinks.k1.serializer.rowKeyIndex = 1 a1.sinks.k1.serializer.colNames = host,ROW_KEY,timestamp

      Description

      Back when we created the the HBase serializer interface we decided that most people would have to implement their own since they knew their data and generically mapping an event to columns and keys was likely going to be difficult. I have observed that something like the following: https://github.com/brockn/flume-hbase-serializer

      would be useful to users. Basically it's the regex serializer where a single value from the regex match can be used as the key. It might be possible to modify the current regex serializer to behave in this way so as to limit the number of serializers.

        Attachments

        1. FLUME-1945.patch
          5 kB
          Sravya Tirukkovalur
        2. FLUME-1945.patch
          8 kB
          Sravya Tirukkovalur

          Issue Links

            Activity

              People

              • Assignee:
                sravya Sravya Tirukkovalur
                Reporter:
                brocknoland Brock Noland
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: