HBase
  1. HBase
  2. HBASE-77

Streaming: Clients should be able to open an OutputStream for putting a value into a table

    Details

    • Type: Improvement Improvement
    • Status: Open
    • Priority: Minor Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Client
    • Labels:
      None

      Description

      > Right now the only interface for insertion is via a byte[], which severely limits the flexibility of the system. Added 2007-10-08 by stuhood.

      Tell us more Stu why you need this feature? (Because you want to write BLOBs of multiple Gigabytes into an hbase cell?)

      Added as part of migrating new feature requests from the obsoleted http://wiki.apache.org/hadoop/Hbase/HbaseFeatureRequests

        Activity

        Hide
        stack added a comment -

        I'm ambigious about closing this issue. On the one hand it seems impossible to do given current state of things. On other hand, seems like an ideal we should be heading for. I come down more on side of leaving it open because this issue will come up again IMO.

        Show
        stack added a comment - I'm ambigious about closing this issue. On the one hand it seems impossible to do given current state of things. On other hand, seems like an ideal we should be heading for. I come down more on side of leaving it open because this issue will come up again IMO.
        Hide
        Bryan Duxbury added a comment -

        Should we resolve this issue won't fix? It seems like a bad match for HBase.

        Show
        Bryan Duxbury added a comment - Should we resolve this issue won't fix? It seems like a bad match for HBase.
        Hide
        Bryan Duxbury added a comment -

        I'm not sure that streaming multi-gigabyte blobs into HBase really makes any sense. If you're going to have a lot of multi-gigabyte blobs, HDFS is probably a better fit for you, and just store the metadata in HBase.

        However, I guess I could see situations where you might want to stream something like a few megabytes rather than keep it all in memory. However, as I understand it, the current RPC system probably wouldn't support this very well, so it would bear some scrutiny.

        Show
        Bryan Duxbury added a comment - I'm not sure that streaming multi-gigabyte blobs into HBase really makes any sense. If you're going to have a lot of multi-gigabyte blobs, HDFS is probably a better fit for you, and just store the metadata in HBase. However, I guess I could see situations where you might want to stream something like a few megabytes rather than keep it all in memory. However, as I understand it, the current RPC system probably wouldn't support this very well, so it would bear some scrutiny.
        Hide
        Stu Hood added a comment -

        Because you want to write BLOBs of multiple Gigabytes into an hbase cell?

        Yes, that would be one usecase, but the BLOB wouldn't even need to be that large: even for streams in the hundreds of megabytes, it would be impractical to read them fully.

        For the perennial example of a web or document crawler, it makes more sense to open a stream to a document you are fetching, and then pass the stream to HBase, rather than buffering it into memory first.

        Show
        Stu Hood added a comment - Because you want to write BLOBs of multiple Gigabytes into an hbase cell? Yes, that would be one usecase, but the BLOB wouldn't even need to be that large: even for streams in the hundreds of megabytes, it would be impractical to read them fully. For the perennial example of a web or document crawler, it makes more sense to open a stream to a document you are fetching, and then pass the stream to HBase, rather than buffering it into memory first.

          People

          • Assignee:
            Unassigned
            Reporter:
            stack
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:

              Development