Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-1678

Incorrect documentation for HBase sink

    Details

    • Type: Documentation
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: v1.2.0
    • Fix Version/s: v1.5.0
    • Component/s: Docs
    • Labels:

      Description

      Description of sink type for HBase sink in documentation is wrong. It should be org.apache.flume.sink.hbase.HBaseSink instead of org.apache.flume.sink.HBaseSink.

      Also, example configuration is missing serializer and column attributes to make it functional.

      Here are the properties I had to add to make it work:

      # HBase
      agent.sinks.hbase-sink-1.channel = mem-channel-2
      agent.sinks.hbase-sink-1.type = org.apache.flume.sink.hbase.HBaseSink
      agent.sinks.hbase-sink-1.table = flume_data
      agent.sinks.hbase-sink-1.columnFamily = events
      agent.sinks.hbase-sink-1.batchSize = 100
      
      # Missing properties in example
      agent.sinks.hbase-sink-1.column = value
      agent.sinks.hbase-sink-1.serializer = org.apache.flume.sink.hbase.SimpleHbaseEventSerializer
      agent.sinks.hbase-sink-1.serializer.payloadColumn = payload
      

        Activity

        Hide
        paliwalashish Ashish Paliwal added a comment -

        As of 1.4 docs, the configuration uses short notation like

        a1.channels = c1
        a1.sinks = k1
        a1.sinks.k1.type = hbase
        a1.sinks.k1.table = foo_table
        a1.sinks.k1.columnFamily = bar_cf
        a1.sinks.k1.serializer = org.apache.flume.sink.hbase.RegexHbaseEventSerializer
        a1.sinks.k1.channel = c1

        The type is internally resolved to FQCN using class org.apache.flume.conf.sink.SinkType.

        Just checked in 1.2.0 docs, and this seems to correct, snippet from 1.2.0 doc

        agent_foo.channels = memoryChannel-1
        agent_foo.sinks = hbaseSink-1
        agent_foo.sinks.hbaseSink-1.type = org.apache.flume.sink.hbase.HBaseSink
        agent_foo.sinks.hbaseSink-1.table = foo_table
        agent_foo.sinks.hbaseSink-1.columnFamily = bar_cf
        agent_foo.sinks.hbaseSink-1.serializer = org.apache.flume.sink.hbase.RegexHbaseEventSerializer
        agent_foo.sinks.hbaseSink-1.channels = memoryChannel-1

        Is there anything that need to be addressed?

        Show
        paliwalashish Ashish Paliwal added a comment - As of 1.4 docs, the configuration uses short notation like a1.channels = c1 a1.sinks = k1 a1.sinks.k1.type = hbase a1.sinks.k1.table = foo_table a1.sinks.k1.columnFamily = bar_cf a1.sinks.k1.serializer = org.apache.flume.sink.hbase.RegexHbaseEventSerializer a1.sinks.k1.channel = c1 The type is internally resolved to FQCN using class org.apache.flume.conf.sink.SinkType. Just checked in 1.2.0 docs, and this seems to correct, snippet from 1.2.0 doc agent_foo.channels = memoryChannel-1 agent_foo.sinks = hbaseSink-1 agent_foo.sinks.hbaseSink-1.type = org.apache.flume.sink.hbase.HBaseSink agent_foo.sinks.hbaseSink-1.table = foo_table agent_foo.sinks.hbaseSink-1.columnFamily = bar_cf agent_foo.sinks.hbaseSink-1.serializer = org.apache.flume.sink.hbase.RegexHbaseEventSerializer agent_foo.sinks.hbaseSink-1.channels = memoryChannel-1 Is there anything that need to be addressed?
        Hide
        paliwalashish Ashish Paliwal added a comment -

        Fixed as part of earlier releases. Marking as resolved

        Show
        paliwalashish Ashish Paliwal added a comment - Fixed as part of earlier releases. Marking as resolved

          People

          • Assignee:
            paliwalashish Ashish Paliwal
            Reporter:
            samir.eljazovic Samir Eljazovic
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development