Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-270

[HBase] Build a Lucene index on an HBase table

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • None
    • None
    • None
    • None

    Description

      This patch provides a Reducer class and other related classes which help to build a Lucene index on an HBase table. The index build part is similar to that of Nutch.

      • Each row is modeled as a Lucene document: row key is indexed in its untokenized form, column name-value pairs are Lucene field name-value pairs.
      • IndexConf is used to configure various Lucene parameters, specify whether to optimize an index and which columns to index and/or store, in tokenized or untokenized form, etc.
      • The number of reduce tasks decides the number of indexes (partitions). The index(es) is stored in the output path of job configuration.
      • The index build process is done in the reduce phase. Users can use the map phase to join rows from different tables or to pre-parse/analyze column content, etc.
      • A junit test is added to test the build of an index on an HBase table with an identity mapper. It also serves as an example on how to use the new classes.
      • BuildTableIndex is provided to help building an index on an HBase table. It should be moved to examples package if HBase decides to have one.

      This patch requires the inclusion of the Lucene library.

      Attachments

        1. build_table_index.patch
          38 kB
          Ning Li
        2. build_table_index.take2.again.patch
          37 kB
          Ning Li
        3. build_table_index.take2.patch
          37 kB
          Ning Li
        4. build_table_index.take3.patch
          39 kB
          Ning Li
        5. build_table_index.take4.patch
          44 kB
          Michael Stack
        6. build_table_index.take5.patch
          35 kB
          Michael Stack
        7. build_table_index.take6.patch
          39 kB
          Michael Stack
        8. build_table_index.take7.patch
          69 kB
          Michael Stack
        9. build_table_index.take8.patch
          69 kB
          Michael Stack

        Activity

          People

            Unassigned Unassigned
            ningli Ning Li
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: