Flume
  1. Flume
  2. FLUME-2881

Windows Launch Script fails in plugins dir code

    Details

    • Flags:
      Patch

      Description

      Running flume-ng.cmd results in the attached error from the Windows command line.

      The problem seems to originate in flume-ng.ps1, line 323 where the plugins are added to the class path. Adding together directory information does not seem to be supported on windows 7 or 8. I was able to fix the problem by separating out the two plugin directories in the attached patch.

      1. fix_windows_launch.patch
        1 kB
        Jonathan Smith
      2. op-addition-not-found.log
        1 kB
        Jonathan Smith

        Activity

        Hide
        Jonathan Smith added a comment -

        Added windows powershell error log from running flume-ng.ps1

        Show
        Jonathan Smith added a comment - Added windows powershell error log from running flume-ng.ps1
        Hide
        Jonathan Smith added a comment -

        Added patch

        Show
        Jonathan Smith added a comment - Added patch
        Hide
        Jonathan Smith added a comment -

        This patch adds plugins in flume/plugins.d//lib and flume/plugins.d//libext in such a way that it will not add the DirectoryInfo objects generated by Get-ChildItem, which does not work properly on Windows 7 and 8.

        Show
        Jonathan Smith added a comment - This patch adds plugins in flume/plugins.d/ /lib and flume/plugins.d/ /libext in such a way that it will not add the DirectoryInfo objects generated by Get-ChildItem, which does not work properly on Windows 7 and 8.
        Hide
        Hari Shreedharan added a comment -

        Roshan Naik - Could you please take a look?

        Show
        Hari Shreedharan added a comment - Roshan Naik - Could you please take a look?
        Hide
        Roshan Naik added a comment -

        sure.. will take a look by tomorrow.

        Show
        Roshan Naik added a comment - sure.. will take a look by tomorrow.
        Hide
        Roshan Naik added a comment -

        Jonathan Smith i am able to run the script just fine without the need for this fix. what version of powershell are you using ?
        i used 4.0

        PS C:\Users\Administrator\Downloads\apache-flume-1.6.0-bin> $PSVersionTable
        
        Name                           Value
        ----                           -----
        PSVersion                      4.0
        WSManStackVersion              3.0
        SerializationVersion           1.1.0.1
        CLRVersion                     4.0.30319.42000
        BuildVersion                   6.3.9600.17400
        PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0}
        PSRemotingProtocolVersion      2.2
        
        Show
        Roshan Naik added a comment - Jonathan Smith i am able to run the script just fine without the need for this fix. what version of powershell are you using ? i used 4.0 PS C:\Users\Administrator\Downloads\apache-flume-1.6.0-bin> $PSVersionTable Name Value ---- ----- PSVersion 4.0 WSManStackVersion 3.0 SerializationVersion 1.1.0.1 CLRVersion 4.0.30319.42000 BuildVersion 6.3.9600.17400 PSCompatibleVersions {1.0, 2.0, 3.0, 4.0} PSRemotingProtocolVersion 2.2
        Hide
        Jonathan Smith added a comment - - edited

        Thanks for testing Roshan Naik.

        The only difference is my CLRVersion: 4.0.30319.34014 on Windows 8.1 Pro N

        I've done some additional testing and it appears the script fails when there are plugins included, and works fine when there are no JAR files in the plugins.d directory. Try placing some additional JARs in plugins.d/jsmith/lib/ in the package folder and the error appears.

        I downloaded the binary flume package from here: http://www.apache.org/dyn/closer.lua/flume/1.6.0/apache-flume-1.6.0-bin.tar.gz and used that for testing.

        Show
        Jonathan Smith added a comment - - edited Thanks for testing Roshan Naik . The only difference is my CLRVersion: 4.0.30319.34014 on Windows 8.1 Pro N I've done some additional testing and it appears the script fails when there are plugins included , and works fine when there are no JAR files in the plugins.d directory. Try placing some additional JARs in plugins.d/jsmith/lib/ in the package folder and the error appears. I downloaded the binary flume package from here: http://www.apache.org/dyn/closer.lua/flume/1.6.0/apache-flume-1.6.0-bin.tar.gz and used that for testing.
        Hide
        Roshan Naik added a comment -

        oh i see. i did add some plugin dirs but it didn't seem to make a difference. let me try adding some jars there and check.

        Show
        Roshan Naik added a comment - oh i see. i did add some plugin dirs but it didn't seem to make a difference. let me try adding some jars there and check.
        Hide
        Roshan Naik added a comment -

        Added some jars in plugins.d/1/lib/x.jar plugins.d/1/lib/y.jar and plugins.d/1/libext/x.jar but it didnt make a difference.

        Anyway I test that your fixes to the script work on my setup .. and since other may also experience the same issue that you have noticed,...

        I am +1 on this and will commit it shortly.

        Thanks for the patch Jonathan Smith

        Show
        Roshan Naik added a comment - Added some jars in plugins.d/1/lib/x.jar plugins.d/1/lib/y.jar and plugins.d/1/libext/x.jar but it didnt make a difference. Anyway I test that your fixes to the script work on my setup .. and since other may also experience the same issue that you have noticed,... I am +1 on this and will commit it shortly. Thanks for the patch Jonathan Smith
        Hide
        ASF subversion and git services added a comment -

        Commit de6ecf48568761de829c8d8caf9bc41f9bf9e64c in flume's branch refs/heads/trunk from Roshan Naik
        [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=de6ecf4 ]

        FLUME-2881. Windows Launch Script fails in plugins dir code

        (Jonathan Smith via Roshan Naik)

        Show
        ASF subversion and git services added a comment - Commit de6ecf48568761de829c8d8caf9bc41f9bf9e64c in flume's branch refs/heads/trunk from Roshan Naik [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=de6ecf4 ] FLUME-2881 . Windows Launch Script fails in plugins dir code (Jonathan Smith via Roshan Naik)
        Hide
        ASF subversion and git services added a comment -

        Commit bd49bc32852aed588b8284fa37f2357dd453bceb in flume's branch refs/heads/flume-1.7 from Roshan Naik
        [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=bd49bc3 ]

        FLUME-2881. Windows Launch Script fails in plugins dir code

        (Jonathan Smith via Roshan Naik)

        Show
        ASF subversion and git services added a comment - Commit bd49bc32852aed588b8284fa37f2357dd453bceb in flume's branch refs/heads/flume-1.7 from Roshan Naik [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=bd49bc3 ] FLUME-2881 . Windows Launch Script fails in plugins dir code (Jonathan Smith via Roshan Naik)
        Hide
        Roshan Naik added a comment -

        committed.
        Also added Jonathan Smith to contributors list.

        Show
        Roshan Naik added a comment - committed. Also added Jonathan Smith to contributors list.
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in Flume-trunk-hbase-1 #149 (See https://builds.apache.org/job/Flume-trunk-hbase-1/149/)
        FLUME-2881. Windows Launch Script fails in plugins dir code (roshan: http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=de6ecf48568761de829c8d8caf9bc41f9bf9e64c)

        • bin/flume-ng.ps1
        Show
        Hudson added a comment - SUCCESS: Integrated in Flume-trunk-hbase-1 #149 (See https://builds.apache.org/job/Flume-trunk-hbase-1/149/ ) FLUME-2881 . Windows Launch Script fails in plugins dir code (roshan: http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=de6ecf48568761de829c8d8caf9bc41f9bf9e64c ) bin/flume-ng.ps1
        Hide
        Jonathan Smith added a comment -

        Thanks Roshan Naik!

        Show
        Jonathan Smith added a comment - Thanks Roshan Naik !

          People

          • Assignee:
            Jonathan Smith
            Reporter:
            Jonathan Smith
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development