Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-1945

HBase Serializer allow key from regular expression group

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.3.1
    • 1.4.0
    • None
    • None
    • Hide
      Allow users to chose a regular expression group as hbase row key. To use this feature users need to:

      1. Specify the row key index.
      2. Specify ROW_KEY as the column name for the corresponding index

      For example:
      a1.sinks.k1.serializer.rowKeyIndex = 1
      a1.sinks.k1.serializer.colNames = host,ROW_KEY,timestamp
      Show
      Allow users to chose a regular expression group as hbase row key. To use this feature users need to: 1. Specify the row key index. 2. Specify ROW_KEY as the column name for the corresponding index For example: a1.sinks.k1.serializer.rowKeyIndex = 1 a1.sinks.k1.serializer.colNames = host,ROW_KEY,timestamp

    Description

      Back when we created the the HBase serializer interface we decided that most people would have to implement their own since they knew their data and generically mapping an event to columns and keys was likely going to be difficult. I have observed that something like the following: https://github.com/brockn/flume-hbase-serializer

      would be useful to users. Basically it's the regex serializer where a single value from the regex match can be used as the key. It might be possible to modify the current regex serializer to behave in this way so as to limit the number of serializers.

      Attachments

        1. FLUME-1945.patch
          8 kB
          Sravya Tirukkovalur
        2. FLUME-1945.patch
          5 kB
          Sravya Tirukkovalur

        Issue Links

          Activity

            People

              sravya Sravya Tirukkovalur
              brocknoland Brock Noland
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: