Uploaded image for project: 'Slider'
  1. Slider
  2. SLIDER-1192

Slider Agent tarball should be uploaded to HDFS to avoid client side dependency

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Blocker
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: Slider 1.0.0
    • Component/s: None
    • Labels:
      None

      Description

      Trying to use Slider client from a Hive process that is shipped with slider client JAR

       
      Caused by: java.io.FileNotFoundException: File /grid/5/sershe/tez-autobuild/dist/hive/lib/slider-agent.tar.gz does not exist
         at org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:606)
         at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:819)
         at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:596)
         at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:421)
         at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:337)
         at org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1965)
         at org.apache.slider.providers.ProviderUtils.addAgentTar(ProviderUtils.java:128)
         at org.apache.slider.providers.agent.AgentClientProvider.prepareAMAndConfigForLaunch(AgentClientProvider.java:270)

      I can see the gz file in the slider lib directory. However, using slider client one should not even be required to have slider on the box (Hive already has the jars); nor is there a good way to find where it is, if installed (given that jar is shipped and we are not aware of the slider installation).

      The gz file should be included with client jars (perhaps as a resource?)

      Update: setting slider.libdir and/or SLIDER_HOME actually doesn't work at all.
      The code in addAgentTar basically looks for it in the directory of the client jar, no option to override it at all.
      That makes slider client absolutely unusable

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                gsaha Gour Saha
                Reporter:
                sershe Sergey Shelukhin
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated: