Details

    • Type: Sub-task Sub-task
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Won't Fix
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Activity

      Hide
      ryan rawson added a comment -

      this is pretty intrusive (invalidates all older data stores) and I'm not sure its completely necessary to achieve replication.

      there is also the question of, how do we do multi-master replication.

      Show
      ryan rawson added a comment - this is pretty intrusive (invalidates all older data stores) and I'm not sure its completely necessary to achieve replication. there is also the question of, how do we do multi-master replication.
      Hide
      Andrew Purtell added a comment -

      The main problem I had with not doing it this way is getting to column family metadata from various places in the code, for example HLog.completeCacheFlush(). This was because I was trying to tag HLogKeys with scope as they are written, and thus far scope information is kept in HCD. However it doesn't have to be done that way. A replicator could look up what to do for each KV according to information in the key. No HLogKey or KV would be tagged with any scoping information. However byte[] compares for every KV would be more overhead than modifying the write path to carry scope information along.

      Other options?

      Show
      Andrew Purtell added a comment - The main problem I had with not doing it this way is getting to column family metadata from various places in the code, for example HLog.completeCacheFlush(). This was because I was trying to tag HLogKeys with scope as they are written, and thus far scope information is kept in HCD. However it doesn't have to be done that way. A replicator could look up what to do for each KV according to information in the key. No HLogKey or KV would be tagged with any scoping information. However byte[] compares for every KV would be more overhead than modifying the write path to carry scope information along. Other options?
      Hide
      ryan rawson added a comment -

      i think the data should go into only HLog and not actually enter KeyValue or the HFile stores. Then we can either ship entire log block segments or do 1 key at a time (preferred).

      Show
      ryan rawson added a comment - i think the data should go into only HLog and not actually enter KeyValue or the HFile stores. Then we can either ship entire log block segments or do 1 key at a time (preferred).
      Hide
      Andrew Purtell added a comment -

      i think the data should go into only HLog and not actually enter KeyValue or the HFile stores

      That was my strategy – put it in HLogKey only and do it one key at a time.

      I'll go back and try it again.

      Show
      Andrew Purtell added a comment - i think the data should go into only HLog and not actually enter KeyValue or the HFile stores That was my strategy – put it in HLogKey only and do it one key at a time. I'll go back and try it again.
      Hide
      Jean-Daniel Cryans added a comment -

      Scoping the KVs is very intrusive, instead that will be done in HLogKey in HBASE-1728.

      Show
      Jean-Daniel Cryans added a comment - Scoping the KVs is very intrusive, instead that will be done in HLogKey in HBASE-1728 .

        People

        • Assignee:
          Unassigned
          Reporter:
          Andrew Purtell
        • Votes:
          0 Vote for this issue
          Watchers:
          0 Start watching this issue

          Dates

          • Created:
            Updated:
            Resolved:

            Development