Uploaded image for project: 'Accumulo'
  1. Accumulo
  2. ACCUMULO-4505

Shell still reads accumulo-site.xml when using Zookeeper CLI options

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 1.6.6, 1.7.2, 1.8.0
    • Fix Version/s: 1.7.3, 1.8.1, 2.0.0
    • Component/s: shell
    • Labels:
      None

      Description

      The Shell will try to read Zookeeper values from accumulo-site.xml whether it has to or not. Using the Zookeeper CLI options (-z or -zi and zh), the user provides enough information that the shell doesn't need to read the values from accumulo-site.xml. This can be reproduced easily by starting accumulo and then running these commands (from accumulo home):
      chmod 000 ./conf/accumulo-site.xml
      ./bin/accumulo shell -u root -z instance17 localhost:2181

      This will produce a FileNotFoundException (Permission denied), with different stacktrace depending on the version.
      1.6.6 Stacktrace:

      2016-10-28 14:49:57,584 [conf.Configuration] FATAL: error parsing conf accumulo-site.xml
      java.io.FileNotFoundException: /local/mpmill4/uno/install/accumulo-1.6.6/conf/accumulo-site.xml (Permission denied)
      	at java.io.FileInputStream.open0(Native Method)
      	at java.io.FileInputStream.open(FileInputStream.java:195)
      	at java.io.FileInputStream.<init>(FileInputStream.java:138)
      	at java.io.FileInputStream.<init>(FileInputStream.java:93)
      	at sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection.java:90)
      	at sun.net.www.protocol.file.FileURLConnection.getInputStream(FileURLConnection.java:188)
      	at java.net.URL.openStream(URL.java:1045)
      	at org.apache.hadoop.conf.Configuration.parse(Configuration.java:2468)
      	at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2539)
      	at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2502)
      	at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2405)
      	at org.apache.hadoop.conf.Configuration.iterator(Configuration.java:2451)
      	at org.apache.accumulo.core.conf.SiteConfiguration.getProperties(SiteConfiguration.java:101)
      	at org.apache.accumulo.core.conf.AccumuloConfiguration.iterator(AccumuloConfiguration.java:69)
      	at org.apache.accumulo.core.conf.ConfigSanityCheck.validate(ConfigSanityCheck.java:42)
      	at org.apache.accumulo.core.conf.SiteConfiguration.getInstance(SiteConfiguration.java:49)
      	at org.apache.accumulo.core.util.shell.ShellOptionsJC.getClientConfiguration(ShellOptionsJC.java:294)
      	at org.apache.accumulo.core.util.shell.Shell.setInstance(Shell.java:430)
      	at org.apache.accumulo.core.util.shell.Shell.config(Shell.java:287)
      	at org.apache.accumulo.core.util.shell.Shell.main(Shell.java:544)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at org.apache.accumulo.start.Main$1.run(Main.java:141)
      	at java.lang.Thread.run(Thread.java:745)
      Thread "shell" died Unable to load client config from null
      java.lang.IllegalArgumentException: Unable to load client config from null
      	at org.apache.accumulo.core.util.shell.Shell.setInstance(Shell.java:432)
      	at org.apache.accumulo.core.util.shell.Shell.config(Shell.java:287)
      	at org.apache.accumulo.core.util.shell.Shell.main(Shell.java:544)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at org.apache.accumulo.start.Main$1.run(Main.java:141)
      	at java.lang.Thread.run(Thread.java:745)
      Caused by: java.lang.RuntimeException: java.io.FileNotFoundException: /local/mpmill4/uno/install/accumulo-1.6.6/conf/accumulo-site.xml (Permission denied)
      	at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2639)
      	at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2502)
      	at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2405)
      	at org.apache.hadoop.conf.Configuration.iterator(Configuration.java:2451)
      	at org.apache.accumulo.core.conf.SiteConfiguration.getProperties(SiteConfiguration.java:101)
      	at org.apache.accumulo.core.conf.AccumuloConfiguration.iterator(AccumuloConfiguration.java:69)
      	at org.apache.accumulo.core.conf.ConfigSanityCheck.validate(ConfigSanityCheck.java:42)
      	at org.apache.accumulo.core.conf.SiteConfiguration.getInstance(SiteConfiguration.java:49)
      	at org.apache.accumulo.core.util.shell.ShellOptionsJC.getClientConfiguration(ShellOptionsJC.java:294)
      	at org.apache.accumulo.core.util.shell.Shell.setInstance(Shell.java:430)
      	... 8 more
      Caused by: java.io.FileNotFoundException: /local/mpmill4/uno/install/accumulo-1.6.6/conf/accumulo-site.xml (Permission denied)
      	at java.io.FileInputStream.open0(Native Method)
      	at java.io.FileInputStream.open(FileInputStream.java:195)
      	at java.io.FileInputStream.<init>(FileInputStream.java:138)
      	at java.io.FileInputStream.<init>(FileInputStream.java:93)
      	at sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection.java:90)
      	at sun.net.www.protocol.file.FileURLConnection.getInputStream(FileURLConnection.java:188)
      	at java.net.URL.openStream(URL.java:1045)
      	at org.apache.hadoop.conf.Configuration.parse(Configuration.java:2468)
      	at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2539)
      	... 17 more
      
      1.7.3 & 1.8.0 stacktrace:
      2016-10-28 15:01:22,862 [conf.Configuration] FATAL: error parsing conf accumulo-site.xml
      java.io.FileNotFoundException: /local/mpmill4/uno/install/accumulo-1.7.3-SNAPSHOT/conf/accumulo-site.xml (Permission denied)
      	at java.io.FileInputStream.open0(Native Method)
      	at java.io.FileInputStream.open(FileInputStream.java:195)
      	at java.io.FileInputStream.<init>(FileInputStream.java:138)
      	at java.io.FileInputStream.<init>(FileInputStream.java:93)
      	at sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection.java:90)
      	at sun.net.www.protocol.file.FileURLConnection.getInputStream(FileURLConnection.java:188)
      	at java.net.URL.openStream(URL.java:1045)
      	at org.apache.hadoop.conf.Configuration.parse(Configuration.java:2468)
      	at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2539)
      	at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2502)
      	at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2405)
      	at org.apache.hadoop.conf.Configuration.size(Configuration.java:2427)
      	at org.apache.accumulo.core.conf.SiteConfiguration.<init>(SiteConfiguration.java:61)
      	at org.apache.accumulo.core.conf.SiteConfiguration.getInstance(SiteConfiguration.java:87)
      	at org.apache.accumulo.core.conf.SiteConfiguration.getInstance(SiteConfiguration.java:94)
      	at org.apache.accumulo.server.zookeeper.ZooReaderWriter.getInstance(ZooReaderWriter.java:36)
      	at org.apache.accumulo.server.init.Initialize.<clinit>(Initialize.java:130)
      	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
      	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
      	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
      	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
      	at java.lang.Class.newInstance(Class.java:442)
      	at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
      	at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
      	at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
      	at org.apache.accumulo.start.Main.checkDuplicates(Main.java:223)
      	at org.apache.accumulo.start.Main.getExecutables(Main.java:215)
      	at org.apache.accumulo.start.Main.main(Main.java:78)
      2016-10-28 15:01:22,864 [start.Main] ERROR: Uncaught exception
      java.util.ServiceConfigurationError: org.apache.accumulo.start.spi.KeywordExecutable: Provider org.apache.accumulo.server.init.Initialize could not be instantiated
      	at java.util.ServiceLoader.fail(ServiceLoader.java:232)
      	at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
      	at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
      	at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
      	at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
      	at org.apache.accumulo.start.Main.checkDuplicates(Main.java:223)
      	at org.apache.accumulo.start.Main.getExecutables(Main.java:215)
      	at org.apache.accumulo.start.Main.main(Main.java:78)
      Caused by: java.lang.ExceptionInInitializerError
      	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
      	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
      	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
      	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
      	at java.lang.Class.newInstance(Class.java:442)
      	at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
      	... 5 more
      Caused by: java.lang.RuntimeException: java.io.FileNotFoundException: /local/mpmill4/uno/install/accumulo-1.7.3-SNAPSHOT/conf/accumulo-site.xml (Permission denied)
      	at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2639)
      	at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2502)
      	at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2405)
      	at org.apache.hadoop.conf.Configuration.size(Configuration.java:2427)
      	at org.apache.accumulo.core.conf.SiteConfiguration.<init>(SiteConfiguration.java:61)
      	at org.apache.accumulo.core.conf.SiteConfiguration.getInstance(SiteConfiguration.java:87)
      	at org.apache.accumulo.core.conf.SiteConfiguration.getInstance(SiteConfiguration.java:94)
      	at org.apache.accumulo.server.zookeeper.ZooReaderWriter.getInstance(ZooReaderWriter.java:36)
      	at org.apache.accumulo.server.init.Initialize.<clinit>(Initialize.java:130)
      	... 11 more
      Caused by: java.io.FileNotFoundException: /local/mpmill4/uno/install/accumulo-1.7.3-SNAPSHOT/conf/accumulo-site.xml (Permission denied)
      	at java.io.FileInputStream.open0(Native Method)
      	at java.io.FileInputStream.open(FileInputStream.java:195)
      	at java.io.FileInputStream.<init>(FileInputStream.java:138)
      	at java.io.FileInputStream.<init>(FileInputStream.java:93)
      	at sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection.java:90)
      	at sun.net.www.protocol.file.FileURLConnection.getInputStream(FileURLConnection.java:188)
      	at java.net.URL.openStream(URL.java:1045)
      	at org.apache.hadoop.conf.Configuration.parse(Configuration.java:2468)
      	at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2539)
      	... 19 more
      

        Issue Links

          Activity

          Hide
          ctubbsii Christopher Tubbs added a comment -

          It looks like this was broken by ACCUMULO-3779, which caused it to always construct a SiteConfiguration object, which was merged with the ClientConfiguration.

          Really, we should never create a SiteConfiguration if the ClientConfiguration is enough to function. Reading from the SiteConfiguration is provided only for backwards compatibility, but the shell should be runnable, without any expectation to have access to the server-side configuration. That's the whole point of the ClientConfiguration and the zookeeper options on the command-line.

          There may be other places in the shell which try to read/construct a SiteConfiguration object.

          Show
          ctubbsii Christopher Tubbs added a comment - It looks like this was broken by ACCUMULO-3779 , which caused it to always construct a SiteConfiguration object, which was merged with the ClientConfiguration. Really, we should never create a SiteConfiguration if the ClientConfiguration is enough to function. Reading from the SiteConfiguration is provided only for backwards compatibility, but the shell should be runnable, without any expectation to have access to the server-side configuration. That's the whole point of the ClientConfiguration and the zookeeper options on the command-line. There may be other places in the shell which try to read/construct a SiteConfiguration object.
          Hide
          ctubbsii Christopher Tubbs added a comment -

          Looks like there were two similar changes... one in Shell.java and one in ShellOptionsJC.java which cause a dependency on the site configuration file by constructing SiteConfiguration when it shouldn't be used.

          Show
          ctubbsii Christopher Tubbs added a comment - Looks like there were two similar changes... one in Shell.java and one in ShellOptionsJC.java which cause a dependency on the site configuration file by constructing SiteConfiguration when it shouldn't be used.
          Hide
          milleruntime Michael Miller added a comment -

          Problem still occurs when using "-zi" and "-zh" parameters.

          Show
          milleruntime Michael Miller added a comment - Problem still occurs when using "-zi" and "-zh" parameters.
          Hide
          ctubbsii Christopher Tubbs added a comment -

          Thanks for fixing this! The backport to 1.6.6 in the Fedora packaging is working well.

          Show
          ctubbsii Christopher Tubbs added a comment - Thanks for fixing this! The backport to 1.6.6 in the Fedora packaging is working well.
          Hide
          milleruntime Michael Miller added a comment -

          Good to hear! I wasn't sure if modifying the patch file was appropriate but it was easier than creating a second patch file from a diff of the patched code.

          Show
          milleruntime Michael Miller added a comment - Good to hear! I wasn't sure if modifying the patch file was appropriate but it was easier than creating a second patch file from a diff of the patched code.

            People

            • Assignee:
              milleruntime Michael Miller
              Reporter:
              milleruntime Michael Miller
            • Votes:
              1 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 1h 40m
                1h 40m

                  Development