Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-22103

HDFS-13209 in Hadoop 3.3.0 breaks asyncwal

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.0.0-alpha-1, 2.3.0, 2.2.5
    • Component/s: hadoop3, wal
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      HDFS-13209 added an additional parameter to DistributedFileSystem.create and broke asyncwal.

      2019-03-25 12:19:21,061 ERROR [Listener at localhost/54758] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(562): Couldn't properly initialize access to HDFS internals. Please update your WAL Provider to not make use of the 'asyncfs' provider. See HBASE-16110 for more information.
      java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.protocol.ClientProtocol.create(java.lang.String, org.apache.hadoop.fs.permission.FsPermission, java.lang.String, org.apache.hadoop.io.EnumSetWritable, boolean, short, long, [Lorg.apache.hadoop.crypto.CryptoProtocolVersion;)
              at java.lang.Class.getMethod(Class.java:1786)
              at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createFileCreator2(FanOutOneBlockAsyncDFSOutputHelper.java:513)
              at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createFileCreator(FanOutOneBlockAsyncDFSOutputHelper.java:530)
              at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.<clinit>(FanOutOneBlockAsyncDFSOutputHelper.java:557)
              at org.apache.hadoop.hbase.io.asyncfs.AsyncFSOutputHelper.createOutput(AsyncFSOutputHelper.java:51)
              at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.initOutput(AsyncProtobufLogWriter.java:169)
              at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(AbstractProtobufLogWriter.java:166)
              at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(AsyncFSWALProvider.java:105)
              at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createAsyncWriter(AsyncFSWAL.java:663)
              at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:669)
              at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:126)
              at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:813)
              at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:519)
              at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.init(AbstractFSWAL.java:460)
              at org.apache.hadoop.hbase.regionserver.wal.TestAsyncFSWAL.newWAL(TestAsyncFSWAL.java:72)
              at org.apache.hadoop.hbase.regionserver.wal.AbstractTestFSWAL.testWALComparator(AbstractTestFSWAL.java:194)
      

      Credit: this bug was found by Gabor Bota

        Attachments

        1. HBASE-22103.master.001.patch
          2 kB
          Wei-Chiu Chuang

          Issue Links

          There are no Sub-Tasks for this issue.

            Activity

              People

              • Assignee:
                weichiu Wei-Chiu Chuang
                Reporter:
                weichiu Wei-Chiu Chuang
              • Votes:
                0 Vote for this issue
                Watchers:
                11 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: