HBase
  1. HBase
  2. HBASE-2681

Graceful fallback to NONE when the native compression algo is not found

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Won't Fix
    • Affects Version/s: None
    • Fix Version/s: 0.90.0
    • Component/s: io
    • Labels:
      None

      Description

      stack's word: "Fellas have complained about the way broke lzo manifests itself. HBase will actually take on writes. Its only when it goes to flush that it drops the edits and in a way that is essentially hidden to the client - exceptions are thrown in the regionserver log. So, i'd say, make another issue if you don't mind but its not for you to fix, not unless you are inclined. It'd be about better user experience around choosing a compression that is not supported or not properly installed."

      1. HBASE-2681.patch
        5 kB
        Michele Catasta

        Issue Links

          Activity

          Hide
          Michele Catasta added a comment -

          Extracted a method that tries to instantiate a native codec and, if not found, just writes a WARN in the logs instead of throwing a RuntimeException. Trivial test included.

          Show
          Michele Catasta added a comment - Extracted a method that tries to instantiate a native codec and, if not found, just writes a WARN in the logs instead of throwing a RuntimeException. Trivial test included.
          Hide
          Dave Latham added a comment -

          It's definitely not good to fail silently at a later time.

          However, I'd prefer to see it fail loudly and immediately if I try to use a form of compression that is not available rather than a quiet fallback to no compression that I would likely not notice.

          Show
          Dave Latham added a comment - It's definitely not good to fail silently at a later time. However, I'd prefer to see it fail loudly and immediately if I try to use a form of compression that is not available rather than a quiet fallback to no compression that I would likely not notice.
          Hide
          stack added a comment -

          @Dave Currently, hbase runs and failure happens when we go to flush. Failure is 'loud' if you are looking at logs but if not looking in logs, its not plain what is wrong. You are suggesting Dave that rather than just proceed, that instead we fail even earlier? Like when you try to create a table or alter a schema to add a compression that does not exist, that it fail at this time? Doing this latter is probalby the right thing to do but its kinda tough to do in that you need to ensure the compression is available on all members of the cluster inline w/ the setting of schema. We don't have a mechanism to do this. What do we do also when a new server is added to the cluster only the admins forgot to add compression libs.... should this new node fail loudly or just, log and keep going?

          Show
          stack added a comment - @Dave Currently, hbase runs and failure happens when we go to flush. Failure is 'loud' if you are looking at logs but if not looking in logs, its not plain what is wrong. You are suggesting Dave that rather than just proceed, that instead we fail even earlier? Like when you try to create a table or alter a schema to add a compression that does not exist, that it fail at this time? Doing this latter is probalby the right thing to do but its kinda tough to do in that you need to ensure the compression is available on all members of the cluster inline w/ the setting of schema. We don't have a mechanism to do this. What do we do also when a new server is added to the cluster only the admins forgot to add compression libs.... should this new node fail loudly or just, log and keep going?
          Hide
          ryan rawson added a comment -

          During a flush/hfile write it is easy to fall back to "NONE", but when you try to read a HFile (which was written with a specific codec) and that codec fails to load then there is no good graceful failure while also returning data.

          Show
          ryan rawson added a comment - During a flush/hfile write it is easy to fall back to "NONE", but when you try to read a HFile (which was written with a specific codec) and that codec fails to load then there is no good graceful failure while also returning data.
          Hide
          Dave Latham added a comment -

          I agree it's not easy as it is right now.

          Would it make sense to have a set of expected, "supported" compression schemes in the settings. On regionserver startup, they check to see if the supported algorithms are available, possibly report that to master or just fail to load. Then when creating / altering a table immediately complain if the compression isn't in the supported set?

          Show
          Dave Latham added a comment - I agree it's not easy as it is right now. Would it make sense to have a set of expected, "supported" compression schemes in the settings. On regionserver startup, they check to see if the supported algorithms are available, possibly report that to master or just fail to load. Then when creating / altering a table immediately complain if the compression isn't in the supported set?
          Hide
          Andrew Purtell added a comment -

          I think it makes sense for the region server to refuse to open a region if the schema indicates to use an unavailable codec.

          Show
          Andrew Purtell added a comment - I think it makes sense for the region server to refuse to open a region if the schema indicates to use an unavailable codec.
          Hide
          ryan rawson added a comment -

          A region may contain multiple files with multiple codecs. Thus even
          if the schema says "GZ" the files may be "LZO". Thus the regionserver
          must have all the codecs applicable.

          But it's a little more insidious, in our case there are 2 failure
          modes with LZO:

          • missing class exceptions
          • missing native library exceptions

          Testing for all applicable codecs, including an end to end functional
          test would probably be necessary to detect both these cases.

          Show
          ryan rawson added a comment - A region may contain multiple files with multiple codecs. Thus even if the schema says "GZ" the files may be "LZO". Thus the regionserver must have all the codecs applicable. But it's a little more insidious, in our case there are 2 failure modes with LZO: missing class exceptions missing native library exceptions Testing for all applicable codecs, including an end to end functional test would probably be necessary to detect both these cases.
          Hide
          stack added a comment -

          I think we should now close this issue and do HBASE-2514 instead. What do folks think?

          Show
          stack added a comment - I think we should now close this issue and do HBASE-2514 instead. What do folks think?
          Hide
          stack added a comment -

          We'll go the HBASE-2514 route instead of not loading region if compression not available.

          We can't go to NONE if region was written w/ a compression lib – we won't be able to open the files if absent compression lib.

          Show
          stack added a comment - We'll go the HBASE-2514 route instead of not loading region if compression not available. We can't go to NONE if region was written w/ a compression lib – we won't be able to open the files if absent compression lib.

            People

            • Assignee:
              Unassigned
              Reporter:
              Michele Catasta
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development