Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-7240 Scaling HDFS
  3. HDFS-12941

Ozone: ConfServlet does not trim values during the description parsing

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: HDFS-7240
    • Fix Version/s: HDFS-7240
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      In ozone-default.xml an unnecessary space is added to the ozone.open.key.expire.threshold key name.

      It causes a NPE in ConfServlet:

      <pre>    Server Error</pre></p><h3>Caused by:</h3><pre>java.lang.NullPointerException
      	at org.apache.hadoop.conf.ConfServlet.lambda$processConfigTagRequest$0(ConfServlet.java:147)
      	at java.util.Iterator.forEachRemaining(Iterator.java:116)
      	at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
      	at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:580)
      	at org.apache.hadoop.conf.ConfServlet.processConfigTagRequest(ConfServlet.java:143)
      	at org.apache.hadoop.conf.ConfServlet.doGet(ConfServlet.java:101)
      	at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
      	at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
      	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:848)
      	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1772)
      	at org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:110)
      	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
      	at org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1578)
      	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
      	at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
      	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
      	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
      	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
      	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
      	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
      	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
      	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
      	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
      	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
      	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
      	at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
      	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
      	at org.eclipse.jetty.server.Server.handle(Server.java:534)
      	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
      	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
      	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
      	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
      	at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
      	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
      	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
      	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
      	at java.lang.Thread.run(Thread.java:748)
      

      The problem is here:

       Properties properties = config.getAllPropertiesByTags(tagList);
            if (propertyMap == null) {
              loadDescriptions();
            }
      
            List<OzoneConfiguration.Property> filteredProperties = new ArrayList<>();
      
            properties.stringPropertyNames().stream().forEach(key -> {
              if (config.get(key) != null) {
                propertyMap.get(key).setValue(config.get(key));
                filteredProperties.add(propertyMap.get(key));
              }
            });
      

      We iterate over the keys from the loaded configuration (which is trimmed by the config.getAllPropertiesByTags) and try to find the description, but in the key->description map the key was not trimmed.

        Attachments

          Activity

            People

            • Assignee:
              elek Marton Elek
              Reporter:
              elek Marton Elek
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: