Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.90.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Incompatible change, Reviewed
    • Release Note:
      Removed deprecated API calls and internal references to family:qualifier notation.

      Description

      Remove all the deprecated stuff in client and mapred.

      1. HBASE-1822-v6.patch
        894 kB
        Jonathan Gray
      2. HBASE-1822-v5.patch
        869 kB
        Jonathan Gray
      3. HBASE-1822-v3.patch
        879 kB
        Jonathan Gray
      4. HBASE-1822-v2.patch
        872 kB
        Jonathan Gray
      5. HBASE-1822-v1.patch
        592 kB
        Jonathan Gray

        Issue Links

          Activity

          Hide
          Jean-Daniel Cryans added a comment -

          And remove the nineteen migration package.

          Show
          Jean-Daniel Cryans added a comment - And remove the nineteen migration package.
          Hide
          Jonathan Gray added a comment -

          First go.

          REST, Transactional, and Stargate still have errors. Didn't want to touch those until we decided which would be removed for 0.21 or how that is going to work.

          Otherwise all tests compile, have not run them all. Removed old api tests as well.

          Only test that needs to be reworked (using a lot of HSK for some reason, though should be KV) is o.a.h.h.regionserver.TestStoreFile.

          Whee that was fun.

          Show
          Jonathan Gray added a comment - First go. REST, Transactional, and Stargate still have errors. Didn't want to touch those until we decided which would be removed for 0.21 or how that is going to work. Otherwise all tests compile, have not run them all. Removed old api tests as well. Only test that needs to be reworked (using a lot of HSK for some reason, though should be KV) is o.a.h.h.regionserver.TestStoreFile. Whee that was fun.
          Hide
          Andrew Purtell added a comment -

          What are the errors for Stargate?

          Show
          Andrew Purtell added a comment - What are the errors for Stargate?
          Hide
          Jonathan Gray added a comment -

          Easy to fix error in Stargate.

          But it brings up a question. There are uses all over the place of family:qualifier treated together in a single byte[], often times it's byte [][] columns.

          I took those APIs out of Scan/Get because they are old-API and there has been confusion from new users (email on the list today that was at least partially misusing it).

          Do we want to retain something in Scan/Get, rework consumers who are using byte [][] columns, or just deal with iterating and disassembling it (that's what I've done so far).

          Show
          Jonathan Gray added a comment - Easy to fix error in Stargate. But it brings up a question. There are uses all over the place of family:qualifier treated together in a single byte[], often times it's byte [][] columns. I took those APIs out of Scan/Get because they are old-API and there has been confusion from new users (email on the list today that was at least partially misusing it). Do we want to retain something in Scan/Get, rework consumers who are using byte [][] columns, or just deal with iterating and disassembling it (that's what I've done so far).
          Hide
          Andrew Purtell added a comment -

          I don't think Scan or Get methods which take old-style byte[][] arguments are necessary, but KeyValue.parseColumn is useful for at least Stargate. It's convenient to use column:qualifier syntactic sugar in RESTful resource specifications.

          Show
          Andrew Purtell added a comment - I don't think Scan or Get methods which take old-style byte[][] arguments are necessary, but KeyValue.parseColumn is useful for at least Stargate. It's convenient to use column:qualifier syntactic sugar in RESTful resource specifications.
          Hide
          Jonathan Gray added a comment -

          That's exactly what I did. I'm using KV.parseColumn internally.

          Show
          Jonathan Gray added a comment - That's exactly what I did. I'm using KV.parseColumn internally.
          Hide
          stack added a comment -

          I'm good w/ axing the band-aid methods that took old style family:qualifier.

          Show
          stack added a comment - I'm good w/ axing the band-aid methods that took old style family:qualifier.
          Hide
          Jonathan Gray added a comment -

          Indexed stuff uses fam:col everywhere, often times just fam:... I've fixed much of it but it's a pain and I'm worried I might break it.

          @Clint Do you want to try to clean it up, or shall I proceed?

          Show
          Jonathan Gray added a comment - Indexed stuff uses fam:col everywhere, often times just fam:... I've fixed much of it but it's a pain and I'm worried I might break it. @Clint Do you want to try to clean it up, or shall I proceed?
          Hide
          Jonathan Gray added a comment -

          Everything compiles with this patch except for o.a.h.h.regionserver.TestStoreFile which contains a bunch of odd usage of new HStoreKey(midkey). An attempt at moving it to KVs resulted in the tests not passing.

          Removes REST in favor of stargate contrib.

          Hacky fixes on stargate, indexed, and thrift until the authors go back through and rework their internal APIs to be more friendly to the current API (though this might not be necessary, up to the authors).

          Need to run more tests, though I ran many as I went.

          Show
          Jonathan Gray added a comment - Everything compiles with this patch except for o.a.h.h.regionserver.TestStoreFile which contains a bunch of odd usage of new HStoreKey(midkey). An attempt at moving it to KVs resulted in the tests not passing. Removes REST in favor of stargate contrib. Hacky fixes on stargate, indexed, and thrift until the authors go back through and rework their internal APIs to be more friendly to the current API (though this might not be necessary, up to the authors). Need to run more tests, though I ran many as I went.
          Hide
          stack added a comment -

          I'd say just commit once all tests pass. Remove TestStoreFile tests that do HSK... Remove whole test if you can't move it over.

          This is a big change. Get it in now while we are being fast and loose with TRUNK.

          Show
          stack added a comment - I'd say just commit once all tests pass. Remove TestStoreFile tests that do HSK... Remove whole test if you can't move it over. This is a big change. Get it in now while we are being fast and loose with TRUNK.
          Hide
          Jonathan Gray added a comment -

          Fixed TestStoreFile, passes.

          Running more tests now.

          Show
          Jonathan Gray added a comment - Fixed TestStoreFile, passes. Running more tests now.
          Hide
          Jonathan Gray added a comment -

          Another version of the patch. Lots of tests pass, need to run more though.

          Show
          Jonathan Gray added a comment - Another version of the patch. Lots of tests pass, need to run more though.
          Hide
          Jonathan Gray added a comment -

          I believe all tests are passing with this patch. Doing one final run and then will commit.

          Show
          Jonathan Gray added a comment - I believe all tests are passing with this patch. Doing one final run and then will commit.
          Hide
          Jonathan Gray added a comment -

          This has been committed to trunk. (25,000 line patch, eek)

          It completely removes the old API and is an attempt at removing all remaining references to family:qualifier notation. It drops the mapred and rest packages completely, in favor of the new mapreduce package and stargate. Things like HStoreKey, RowResult, and BatchUpdate are gone.

          The only remaining calls related to fam:qf notation are KeyValue.makeColumn(fam,qf) and KeyValue.parseColumn(col) which takes a byte[] and splits it on the :. If it's just family it returns a single dimension only with the family in it, no :. This switched a bit as to how it worked, and it's used heavily in my adapting Thrift, Stargate, and Transactional/Indexed to work and pass their tests after ripping out old API calls they were using.

          Those same contrib modules (should we move thrift to contrib?) really need a reworking of some sort (stargate is probably fine, except some nastiness in tests), as they are heavily tied to the old API and their new API compatibility was just a bolt on.

          So this may have destabilized trunk a bit. Please file issues if you run into problems.

          All unit tests pass.

          Show
          Jonathan Gray added a comment - This has been committed to trunk. (25,000 line patch, eek) It completely removes the old API and is an attempt at removing all remaining references to family:qualifier notation. It drops the mapred and rest packages completely, in favor of the new mapreduce package and stargate. Things like HStoreKey, RowResult, and BatchUpdate are gone. The only remaining calls related to fam:qf notation are KeyValue.makeColumn(fam,qf) and KeyValue.parseColumn(col) which takes a byte[] and splits it on the :. If it's just family it returns a single dimension only with the family in it, no :. This switched a bit as to how it worked, and it's used heavily in my adapting Thrift, Stargate, and Transactional/Indexed to work and pass their tests after ripping out old API calls they were using. Those same contrib modules (should we move thrift to contrib?) really need a reworking of some sort (stargate is probably fine, except some nastiness in tests), as they are heavily tied to the old API and their new API compatibility was just a bolt on. So this may have destabilized trunk a bit. Please file issues if you run into problems. All unit tests pass.

            People

            • Assignee:
              Jonathan Gray
              Reporter:
              Jean-Daniel Cryans
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development