Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-11985 Improve Solaris support in Hadoop
  3. HADOOP-7824

NativeIO.java flags and identifiers must be set correctly for each platform, not hardcoded to their Linux values

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.20.204.0, 0.20.205.0, 1.0.3, 0.23.0, 2.0.0-alpha, 3.0.0-alpha1
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: native
    • Labels:
    • Environment:

      Mac OS X, Linux, Solaris, Windows, ...

    • Target Version/s:
    • Tags:
      osx, native

      Description

      NativeIO.java flags and identifiers must be set correctly for each platform, not hardcoded to their Linux values. Constants like O_CREAT, O_EXCL, etc. have different values on OS X and many other operating systems.

      1. HADOOP-7824.004.patch
        31 kB
        Martin Walsh
      2. HADOOP-7824.003.patch
        31 kB
        Martin Walsh
      3. HADOOP-7824.002.patch
        32 kB
        Martin Walsh
      4. HADOOP-7824.001.patch
        32 kB
        Martin Walsh
      5. hadoop-7824.txt
        18 kB
        Todd Lipcon
      6. HADOOP-7824.patch
        4 kB
        Allen Wittenauer
      7. HADOOP-7824.patch
        1 kB
        Dmytro Shteflyuk

        Issue Links

          Activity

          Hide
          kpumuk Dmytro Shteflyuk added a comment -

          This patch intended to demonstrate how what constants on OS X should look like. It breaks hadoop for systems other then OS X, so it is not intended to be used as is for the Hadoop project.

          Show
          kpumuk Dmytro Shteflyuk added a comment - This patch intended to demonstrate how what constants on OS X should look like. It breaks hadoop for systems other then OS X, so it is not intended to be used as is for the Hadoop project.
          Hide
          kpumuk Dmytro Shteflyuk added a comment -

          This patch intended to demonstrate how what constants on OS X should look like. It breaks hadoop for systems other then OS X, so it is not intended to be used as is for the Hadoop project.

          Show
          kpumuk Dmytro Shteflyuk added a comment - This patch intended to demonstrate how what constants on OS X should look like. It breaks hadoop for systems other then OS X, so it is not intended to be used as is for the Hadoop project.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12503759/HADOOP-7824.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/376//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12503759/HADOOP-7824.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/376//console This message is automatically generated.
          Hide
          tlipcon Todd Lipcon added a comment -

          cancelling patch - I'm not actively working on this but when I get some free time I will try to circle back. If anyone else wants to steal from me, feel free.

          Show
          tlipcon Todd Lipcon added a comment - cancelling patch - I'm not actively working on this but when I get some free time I will try to circle back. If anyone else wants to steal from me, feel free.
          Hide
          aw Allen Wittenauer added a comment -

          Changing this to reflect reality. NativeIO.java's definitions mismatch RHEL 6 as well, which means that even on Linux this is untrustworthy.

          Show
          aw Allen Wittenauer added a comment - Changing this to reflect reality. NativeIO.java's definitions mismatch RHEL 6 as well, which means that even on Linux this is untrustworthy.
          Hide
          aw Allen Wittenauer added a comment -

          This patch generates a bit field translator based upon what Hadoop thinks is true in NativeIO.java vs. what fcntl.h and friends know is real.

          This isn't fully cooked into the build process yet, however.

          Show
          aw Allen Wittenauer added a comment - This patch generates a bit field translator based upon what Hadoop thinks is true in NativeIO.java vs. what fcntl.h and friends know is real. This isn't fully cooked into the build process yet, however.
          Hide
          tlipcon Todd Lipcon added a comment -

          Instead of generating code at compile time like this, can we do better by creating enums on the Java side, and then translating from the Java enum to the C constants in the JNI layer? See how Errno.java translates to its C counterpart.

          Show
          tlipcon Todd Lipcon added a comment - Instead of generating code at compile time like this, can we do better by creating enums on the Java side, and then translating from the Java enum to the C constants in the JNI layer? See how Errno.java translates to its C counterpart.
          Hide
          aw Allen Wittenauer added a comment -

          I suspect that's going to be a lot slower. The C compiler should be able to optimize the heck out of pre-generated code rather than doing it on the fly. Given that the whole reason why this code exists is speed....

          Show
          aw Allen Wittenauer added a comment - I suspect that's going to be a lot slower. The C compiler should be able to optimize the heck out of pre-generated code rather than doing it on the fly. Given that the whole reason why this code exists is speed....
          Hide
          aw Allen Wittenauer added a comment -

          (well, that and the fact that Java actively works against you if you want to use OS resources)

          Show
          aw Allen Wittenauer added a comment - (well, that and the fact that Java actively works against you if you want to use OS resources)
          Hide
          tlipcon Todd Lipcon added a comment -

          How does something like this look?

          Do we need to do the translation for the stat constants, too? Or are those standardized in POSIX?

          Show
          tlipcon Todd Lipcon added a comment - How does something like this look? Do we need to do the translation for the stat constants, too? Or are those standardized in POSIX?
          Hide
          cmccabe Colin P. McCabe added a comment -

          I think we should just do the translation in the JNI code.

          In general, people expect the jar files to be portable, meaning you can copy them from one PC to another without suffering disaster. This will not be true if we start injecting native constants into the Java code. In contrast, libhadoop.so is designed to contain the operating-system and architecture-specific parts of the code.

          Also, do we really want to depend on Perl for our build process? I realize there's already a changes2html.pl script, but in general Perl seems like another build dependency we'd be better off without. Installing Perl on some platforms will definitely be a hassle-- like Windows. We also don't document our Perl dependency anywhere right now.

          I suspect that [translating the constants] is going to be a lot slower.

          I seriously doubt that the performance difference between translating the constants and passing them straight through will even be measurable. If you like, I can do some before and after benchmarks with constant translation and without. Keep in mind, this is a couple of if statements every time we open a file. Considering the other performance issues we have to tackle, this is less than noise-- on par with changing constants because you believe comparisons with 0 are faster, etc.

          Show
          cmccabe Colin P. McCabe added a comment - I think we should just do the translation in the JNI code. In general, people expect the jar files to be portable, meaning you can copy them from one PC to another without suffering disaster. This will not be true if we start injecting native constants into the Java code. In contrast, libhadoop.so is designed to contain the operating-system and architecture-specific parts of the code. Also, do we really want to depend on Perl for our build process? I realize there's already a changes2html.pl script, but in general Perl seems like another build dependency we'd be better off without. Installing Perl on some platforms will definitely be a hassle-- like Windows. We also don't document our Perl dependency anywhere right now. I suspect that [translating the constants] is going to be a lot slower. I seriously doubt that the performance difference between translating the constants and passing them straight through will even be measurable. If you like, I can do some before and after benchmarks with constant translation and without. Keep in mind, this is a couple of if statements every time we open a file. Considering the other performance issues we have to tackle, this is less than noise-- on par with changing constants because you believe comparisons with 0 are faster, etc.
          Hide
          mattf Matt Foley added a comment -

          Changed Target Version to 1.3.0 upon release of 1.2.0. Please change to 1.2.1 if you intend to submit a fix for branch-1.2.

          Show
          mattf Matt Foley added a comment - Changed Target Version to 1.3.0 upon release of 1.2.0. Please change to 1.2.1 if you intend to submit a fix for branch-1.2.
          Hide
          alanbur Alan Burlison added a comment -

          I've prototyped an alternate fix for this, which is to remove 'final' from the public static members used to hold the various O_ constants defined in NativeIO.java but otherwise to leave them alone, then use a JNI_OnLoad hook to update the values of the public static members to the correct platform-specific values. Looking at the classfile disassembly it will be slightly slower than at present - the existing public static final members are inlined to either sipush or ldc of the appropriate constant values whereas if the final modifier is removed (needed to prevent the inlining) then getstatic is used to fetch the value. However it should still be significantly faster than the string-based approach suggested in the latest patch attached to this bug. If this approach is acceptable please let me know and I'll get a patch sorted out.

          I'm specifically interested in fixing this for Solaris but it should work on all platforms.

          Show
          alanbur Alan Burlison added a comment - I've prototyped an alternate fix for this, which is to remove 'final' from the public static members used to hold the various O_ constants defined in NativeIO.java but otherwise to leave them alone, then use a JNI_OnLoad hook to update the values of the public static members to the correct platform-specific values. Looking at the classfile disassembly it will be slightly slower than at present - the existing public static final members are inlined to either sipush or ldc of the appropriate constant values whereas if the final modifier is removed (needed to prevent the inlining) then getstatic is used to fetch the value. However it should still be significantly faster than the string-based approach suggested in the latest patch attached to this bug. If this approach is acceptable please let me know and I'll get a patch sorted out. I'm specifically interested in fixing this for Solaris but it should work on all platforms.
          Hide
          cmccabe Colin P. McCabe added a comment -

          Hi Alan Burlison, I like this approach. Best of both worlds. Just one comment: would you consider doing import static NativeIO.Constants.O_WRONLY at the top of these files, to avoid having to type out NativeIO.Constants.O_WRONLY, etc. later on?

          19	#define _GNU_SOURCE
          

          Shouldn't be needed, as CMakeLists.txt will set this.

          73	  if (const_name == NULL) return 0; // JVM throws Exception
          

          Should use braces.

          +1 pending those fixes and pending jenkins

          Show
          cmccabe Colin P. McCabe added a comment - Hi Alan Burlison , I like this approach. Best of both worlds. Just one comment: would you consider doing import static NativeIO.Constants.O_WRONLY at the top of these files, to avoid having to type out NativeIO.Constants.O_WRONLY , etc. later on? 19 #define _GNU_SOURCE Shouldn't be needed, as CMakeLists.txt will set this. 73 if (const_name == NULL) return 0; // JVM throws Exception Should use braces. +1 pending those fixes and pending jenkins
          Hide
          alanburlison Alan Burlison added a comment -

          [Switching to new work JIRA account] Sure, I'll submit a patch with the suggested approach above and the extra cleanup you requested.

          I've just submitted the necessary paperwork to get myself added to Oracle's Apache contributor agreement for Hadoop, that process should take a week or so, after that I'll be able to post a patch.

          Show
          alanburlison Alan Burlison added a comment - [Switching to new work JIRA account] Sure, I'll submit a patch with the suggested approach above and the extra cleanup you requested. I've just submitted the necessary paperwork to get myself added to Oracle's Apache contributor agreement for Hadoop, that process should take a week or so, after that I'll be able to post a patch.
          Hide
          cmccabe Colin P. McCabe added a comment -

          Thanks, Alan Burlison.

          Show
          cmccabe Colin P. McCabe added a comment - Thanks, Alan Burlison .
          Hide
          martinw Martin Walsh added a comment -

          Changes introduce checkstyle issues. This is because for the JNI solution to work the constants we modify the value for must have the "final" modifier removed. Checkstyle then thinks this is a standard variable and applies a different ruleset to it.

          Show
          martinw Martin Walsh added a comment - Changes introduce checkstyle issues. This is because for the JNI solution to work the constants we modify the value for must have the "final" modifier removed. Checkstyle then thinks this is a standard variable and applies a different ruleset to it.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 pre-patch 21m 22s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 2 new or modified test files.
          +1 javac 8m 22s There were no new javac warning messages.
          +1 javadoc 10m 23s There were no new javadoc warning messages.
          +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 2m 8s The applied patch generated 67 new checkstyle issues (total was 81, now 145).
          +1 whitespace 0m 2s The patch has no lines that end in whitespace.
          +1 install 1m 23s mvn install still works.
          +1 eclipse:eclipse 0m 35s The patch built with eclipse:eclipse.
          +1 findbugs 5m 30s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 common tests 22m 46s Tests passed in hadoop-common.
          +1 mapreduce tests 0m 19s Tests passed in hadoop-mapreduce-client-shuffle.
          -1 hdfs tests 161m 5s Tests failed in hadoop-hdfs.
              235m 16s  



          Reason Tests
          Failed unit tests hadoop.hdfs.TestDistributedFileSystem
            hadoop.hdfs.TestAppendSnapshotTruncate
            hadoop.hdfs.qjournal.client.TestQuorumJournalManager



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12746361/HADOOP-7824.001.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 29cf887b
          Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HADOOP-Build/7313/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/7313/artifact/patchprocess/diffcheckstylehadoop-common.txt
          hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7313/artifact/patchprocess/testrun_hadoop-common.txt
          hadoop-mapreduce-client-shuffle test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7313/artifact/patchprocess/testrun_hadoop-mapreduce-client-shuffle.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7313/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7313/testReport/
          Java 1.7.0_55
          uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7313/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 21m 22s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 2 new or modified test files. +1 javac 8m 22s There were no new javac warning messages. +1 javadoc 10m 23s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 2m 8s The applied patch generated 67 new checkstyle issues (total was 81, now 145). +1 whitespace 0m 2s The patch has no lines that end in whitespace. +1 install 1m 23s mvn install still works. +1 eclipse:eclipse 0m 35s The patch built with eclipse:eclipse. +1 findbugs 5m 30s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 common tests 22m 46s Tests passed in hadoop-common. +1 mapreduce tests 0m 19s Tests passed in hadoop-mapreduce-client-shuffle. -1 hdfs tests 161m 5s Tests failed in hadoop-hdfs.     235m 16s   Reason Tests Failed unit tests hadoop.hdfs.TestDistributedFileSystem   hadoop.hdfs.TestAppendSnapshotTruncate   hadoop.hdfs.qjournal.client.TestQuorumJournalManager Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12746361/HADOOP-7824.001.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 29cf887b Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HADOOP-Build/7313/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/7313/artifact/patchprocess/diffcheckstylehadoop-common.txt hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7313/artifact/patchprocess/testrun_hadoop-common.txt hadoop-mapreduce-client-shuffle test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7313/artifact/patchprocess/testrun_hadoop-mapreduce-client-shuffle.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7313/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7313/testReport/ Java 1.7.0_55 uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7313/console This message was automatically generated.
          Hide
          cmccabe Colin P. McCabe added a comment -

          Thanks for posting this, Martin Walsh. It looks like a big improvement.

          Rather than having stuff like this:

          64	typedef enum {
          65	  CT_BOOL = 1,
          66	  CT_INT,
          67	  CT_LONG,
          68	} const_type;
          69	
          70	struct field {
          71	  char *name;
          72	  const_type type;
          73	  union {
          74	    jboolean bool_val;
          75	    jint int_val;
          76	    jlong long_val;
          77	  } value;
          78	};
          

          Why not just have three functions:

          jthrowable setIntConstant(const char *name, jint val);
          jthrowable setLongConstant(const char *name, jlong val);
          jthrowable setBooleanConstant(const char *name, jboolean val);
          

          Then you could have:

          jthr = setIntConstant("O_RDONLY", O_RDONLY);
          if (jthr != null) {
            return;
          }
          jthr = setBooleanConstant("fadvisePossible", HAVE_POSIX_FADVISE);
          if (jthr != null) {
            return;
          }
          ...
          

          This would be a lot simpler than coming up with all those structures.

          Show
          cmccabe Colin P. McCabe added a comment - Thanks for posting this, Martin Walsh . It looks like a big improvement. Rather than having stuff like this: 64 typedef enum { 65 CT_BOOL = 1, 66 CT_INT, 67 CT_LONG, 68 } const_type; 69 70 struct field { 71 char *name; 72 const_type type; 73 union { 74 jboolean bool_val; 75 jint int_val; 76 jlong long_val; 77 } value; 78 }; Why not just have three functions: jthrowable setIntConstant( const char *name, jint val); jthrowable setLongConstant( const char *name, jlong val); jthrowable setBooleanConstant( const char *name, jboolean val); Then you could have: jthr = setIntConstant( "O_RDONLY" , O_RDONLY); if (jthr != null ) { return ; } jthr = setBooleanConstant( "fadvisePossible" , HAVE_POSIX_FADVISE); if (jthr != null ) { return ; } ... This would be a lot simpler than coming up with all those structures.
          Hide
          martinw Martin Walsh added a comment -

          Thanks for the comments Colin P. McCabe. I added the new structures to make it simple to maintain the list of constants being initialised as they would all be in the list at the top of the file. I would argue that modifying that list of consts at the top of the file is simpler and less error prone then modifying the consts_init() function, which should not need to be touched. Additionally changing it as you suggest would bulk up the consts_init() function considerably. However if still prefer me to change it as you suggest, I will do so.

          Show
          martinw Martin Walsh added a comment - Thanks for the comments Colin P. McCabe . I added the new structures to make it simple to maintain the list of constants being initialised as they would all be in the list at the top of the file. I would argue that modifying that list of consts at the top of the file is simpler and less error prone then modifying the consts_init() function, which should not need to be touched. Additionally changing it as you suggest would bulk up the consts_init() function considerably. However if still prefer me to change it as you suggest, I will do so.
          Hide
          alanburlison Alan Burlison added a comment -

          I suggested to Martin that an array of structs would be easiest way of doing this as it avoids lots of cut+paste boilerplate - doing it as explicit code would be 280 lines rather than the 100 or so the current mechanism uses. Plus having it table-driven means it's only a small step to make this code reusable by any Hadoop component that needs to do the same thing.

          Show
          alanburlison Alan Burlison added a comment - I suggested to Martin that an array of structs would be easiest way of doing this as it avoids lots of cut+paste boilerplate - doing it as explicit code would be 280 lines rather than the 100 or so the current mechanism uses. Plus having it table-driven means it's only a small step to make this code reusable by any Hadoop component that needs to do the same thing.
          Hide
          martinw Martin Walsh added a comment -

          Removed fix for bug HADOOP-12184 which was accidentally included.

          Show
          martinw Martin Walsh added a comment - Removed fix for bug HADOOP-12184 which was accidentally included.
          Hide
          cmccabe Colin P. McCabe added a comment -

          Hmm. I don't think there is a lot of "cut and paste boilerplate" associated with doing it as function calls. The only thing I can think of is the "if (failed)..." check after each function call. Personally, I prefer seeing that, though, since it makes the control flow clearer to me. And surely using function calls is just as reusable by other Hadoop components as using a table-driven approach? Code being at the top of the file versus the middle of the file seems like an odd argument to make. I don't think it matters where in the file the code is.

          Mainly, what I don't like about the table-driven approach is that it involves a clunky "parallel type system" (CT_BOOL, CT_INT, CT_LONG) and a set of somewhat complex macros. I also find it confusing to have a bunch of types that start with "const" like const_type, const_signature, etc.... but are not const in the C sense of the word! I think this code may depend on C99 as well... did C89 support

          { .bool_val=C }

          and so forth?

          It seems much simpler to just call a function to do what you want. If you really find it cumbersome to have an "if" statement after each function call, how about having something like this:

          jthrowable jthr = NULL;
          setIntConstant("O_RDONLY", O_RDONLY, &jthr);
          setBooleanConstant("fadvisePossible", HAVE_POSIX_FADVISE, &jthr);
          ...
          if (jthr) {
            (*env)->Throw(env, jthr);
          }
          

          And have some code in the functions which just returns and does nothing if jthr is set.

          Show
          cmccabe Colin P. McCabe added a comment - Hmm. I don't think there is a lot of "cut and paste boilerplate" associated with doing it as function calls. The only thing I can think of is the "if (failed)..." check after each function call. Personally, I prefer seeing that, though, since it makes the control flow clearer to me. And surely using function calls is just as reusable by other Hadoop components as using a table-driven approach? Code being at the top of the file versus the middle of the file seems like an odd argument to make. I don't think it matters where in the file the code is. Mainly, what I don't like about the table-driven approach is that it involves a clunky "parallel type system" (CT_BOOL, CT_INT, CT_LONG) and a set of somewhat complex macros. I also find it confusing to have a bunch of types that start with "const" like const_type , const_signature , etc.... but are not const in the C sense of the word! I think this code may depend on C99 as well... did C89 support { .bool_val=C } and so forth? It seems much simpler to just call a function to do what you want. If you really find it cumbersome to have an "if" statement after each function call, how about having something like this: jthrowable jthr = NULL; setIntConstant( "O_RDONLY" , O_RDONLY, &jthr); setBooleanConstant( "fadvisePossible" , HAVE_POSIX_FADVISE, &jthr); ... if (jthr) { (*env)->Throw(env, jthr); } And have some code in the functions which just returns and does nothing if jthr is set.
          Hide
          alanburlison Alan Burlison added a comment -

          I don't have particularly strong preferences either way, it's purely down to personal preference, functionally the two are equivalent.

          Show
          alanburlison Alan Burlison added a comment - I don't have particularly strong preferences either way, it's purely down to personal preference, functionally the two are equivalent.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 19m 39s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 2 new or modified test files.
          +1 javac 7m 41s There were no new javac warning messages.
          +1 javadoc 9m 43s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 2m 1s The applied patch generated 67 new checkstyle issues (total was 81, now 145).
          +1 whitespace 0m 2s The patch has no lines that end in whitespace.
          +1 install 1m 22s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 5m 2s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          -1 common tests 22m 2s Tests failed in hadoop-common.
          +1 mapreduce tests 0m 19s Tests passed in hadoop-mapreduce-client-shuffle.
          -1 hdfs tests 160m 53s Tests failed in hadoop-hdfs.
              230m 35s  



          Reason Tests
          Failed unit tests hadoop.security.token.delegation.web.TestWebDelegationToken
            hadoop.hdfs.TestDistributedFileSystem
            hadoop.hdfs.server.namenode.ha.TestStandbyIsHot



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12746537/HADOOP-7824.002.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / e202efa
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/7335/artifact/patchprocess/diffcheckstylehadoop-common.txt
          hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7335/artifact/patchprocess/testrun_hadoop-common.txt
          hadoop-mapreduce-client-shuffle test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7335/artifact/patchprocess/testrun_hadoop-mapreduce-client-shuffle.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7335/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7335/testReport/
          Java 1.7.0_55
          uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7335/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 19m 39s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 2 new or modified test files. +1 javac 7m 41s There were no new javac warning messages. +1 javadoc 9m 43s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 2m 1s The applied patch generated 67 new checkstyle issues (total was 81, now 145). +1 whitespace 0m 2s The patch has no lines that end in whitespace. +1 install 1m 22s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 5m 2s The patch does not introduce any new Findbugs (version 3.0.0) warnings. -1 common tests 22m 2s Tests failed in hadoop-common. +1 mapreduce tests 0m 19s Tests passed in hadoop-mapreduce-client-shuffle. -1 hdfs tests 160m 53s Tests failed in hadoop-hdfs.     230m 35s   Reason Tests Failed unit tests hadoop.security.token.delegation.web.TestWebDelegationToken   hadoop.hdfs.TestDistributedFileSystem   hadoop.hdfs.server.namenode.ha.TestStandbyIsHot Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12746537/HADOOP-7824.002.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / e202efa checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/7335/artifact/patchprocess/diffcheckstylehadoop-common.txt hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7335/artifact/patchprocess/testrun_hadoop-common.txt hadoop-mapreduce-client-shuffle test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7335/artifact/patchprocess/testrun_hadoop-mapreduce-client-shuffle.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7335/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7335/testReport/ Java 1.7.0_55 uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7335/console This message was automatically generated.
          Hide
          martinw Martin Walsh added a comment -

          So what is the preference then. Leave the patch as is or re-work it to remove the table and set the constants directly in a function as suggested above. As mentioned above, both methods are functionally equivalent.

          Show
          martinw Martin Walsh added a comment - So what is the preference then. Leave the patch as is or re-work it to remove the table and set the constants directly in a function as suggested above. As mentioned above, both methods are functionally equivalent.
          Hide
          cmccabe Colin P. McCabe added a comment -

          Hmm. I think it would be easier to read if we set the values directly from a function, if you guys don't have a strong preference.

          Show
          cmccabe Colin P. McCabe added a comment - Hmm. I think it would be easier to read if we set the values directly from a function, if you guys don't have a strong preference.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 20m 4s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 2 new or modified test files.
          +1 javac 7m 43s There were no new javac warning messages.
          +1 javadoc 9m 44s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 57s The applied patch generated 67 new checkstyle issues (total was 81, now 145).
          +1 whitespace 0m 1s The patch has no lines that end in whitespace.
          +1 install 1m 22s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 5m 4s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 common tests 22m 25s Tests passed in hadoop-common.
          +1 mapreduce tests 0m 20s Tests passed in hadoop-mapreduce-client-shuffle.
          -1 hdfs tests 161m 57s Tests failed in hadoop-hdfs.
              232m 27s  



          Reason Tests
          Failed unit tests hadoop.hdfs.server.namenode.ha.TestStandbyIsHot



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12747773/HADOOP-7824.003.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 6374ee0
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/7359/artifact/patchprocess/diffcheckstylehadoop-common.txt
          hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7359/artifact/patchprocess/testrun_hadoop-common.txt
          hadoop-mapreduce-client-shuffle test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7359/artifact/patchprocess/testrun_hadoop-mapreduce-client-shuffle.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7359/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7359/testReport/
          Java 1.7.0_55
          uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7359/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 20m 4s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 2 new or modified test files. +1 javac 7m 43s There were no new javac warning messages. +1 javadoc 9m 44s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 57s The applied patch generated 67 new checkstyle issues (total was 81, now 145). +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 22s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 5m 4s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 common tests 22m 25s Tests passed in hadoop-common. +1 mapreduce tests 0m 20s Tests passed in hadoop-mapreduce-client-shuffle. -1 hdfs tests 161m 57s Tests failed in hadoop-hdfs.     232m 27s   Reason Tests Failed unit tests hadoop.hdfs.server.namenode.ha.TestStandbyIsHot Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12747773/HADOOP-7824.003.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 6374ee0 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/7359/artifact/patchprocess/diffcheckstylehadoop-common.txt hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7359/artifact/patchprocess/testrun_hadoop-common.txt hadoop-mapreduce-client-shuffle test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7359/artifact/patchprocess/testrun_hadoop-mapreduce-client-shuffle.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7359/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7359/testReport/ Java 1.7.0_55 uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7359/console This message was automatically generated.
          Hide
          cmccabe Colin P. McCabe added a comment -

          Thanks, Martin Walsh! It looks really good.

          Just a few last nits:

          • SET_INT_CONST: can we change this to SET_INT_OR_RETURN?
          • setBooleanConstant / setIntConstant / etc. : can we change these to setStaticBoolean, setStaticInt, etc.?

          Two reasons:

          • "Setting a constant" is just weird... by definition, constants can't be set in C, so it feels confusing. These things are really static fields in the Java class, not constants at all. (They're not Java constants-- those can't be modified.)
          • If a macro performs flow control, like returning from a function, it should include that in the name to avoid confusion.

          Avoid whitespace changes. For example, we don't need this:

          -  int fd;  
          +  int fd;
          

          The git history is much nicer when changes are minimal. Related: do we even need to change ReadaheadPool.java? It seems to just be adding a static import.

          +1 once those changes are addressed and Jenkins runs again. Thanks for working on this.

          Show
          cmccabe Colin P. McCabe added a comment - Thanks, Martin Walsh ! It looks really good. Just a few last nits: SET_INT_CONST : can we change this to SET_INT_OR_RETURN ? setBooleanConstant / setIntConstant / etc. : can we change these to setStaticBoolean , setStaticInt , etc.? Two reasons: "Setting a constant" is just weird... by definition, constants can't be set in C, so it feels confusing. These things are really static fields in the Java class, not constants at all. (They're not Java constants-- those can't be modified.) If a macro performs flow control, like returning from a function, it should include that in the name to avoid confusion. Avoid whitespace changes. For example, we don't need this: - int fd; + int fd; The git history is much nicer when changes are minimal. Related: do we even need to change ReadaheadPool.java ? It seems to just be adding a static import. +1 once those changes are addressed and Jenkins runs again. Thanks for working on this.
          Hide
          martinw Martin Walsh added a comment -

          Addressed the nits as recommended. Just re-spinning locally to make sure all looks good.

          I changed ReadaheadPool.java and a number of other files to allow us to utilise the static import, as mentioned in the comments above. e.g. in ReadaheadPool.java:

          • fd, off, len, NativeIO.POSIX.POSIX_FADV_WILLNEED);
            + fd, off, len, POSIX_FADV_WILLNEED);
          Show
          martinw Martin Walsh added a comment - Addressed the nits as recommended. Just re-spinning locally to make sure all looks good. I changed ReadaheadPool.java and a number of other files to allow us to utilise the static import, as mentioned in the comments above. e.g. in ReadaheadPool.java: fd, off, len, NativeIO.POSIX.POSIX_FADV_WILLNEED); + fd, off, len, POSIX_FADV_WILLNEED);
          Hide
          cmccabe Colin P. McCabe added a comment -

          I changed ReadaheadPool.java and a number of other files to allow us to utilise the static import, as mentioned in the comments above. e.g. in ReadaheadPool.java:

          OK, fair enough.

          Thanks, Martin. +1 pending Jenkins.

          Show
          cmccabe Colin P. McCabe added a comment - I changed ReadaheadPool.java and a number of other files to allow us to utilise the static import, as mentioned in the comments above. e.g. in ReadaheadPool.java: OK, fair enough. Thanks, Martin. +1 pending Jenkins.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 19m 32s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 2 new or modified test files.
          +1 javac 7m 41s There were no new javac warning messages.
          +1 javadoc 9m 43s There were no new javadoc warning messages.
          +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 55s The applied patch generated 67 new checkstyle issues (total was 81, now 145).
          +1 whitespace 0m 1s The patch has no lines that end in whitespace.
          +1 install 1m 20s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 5m 3s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 common tests 22m 20s Tests passed in hadoop-common.
          +1 mapreduce tests 0m 20s Tests passed in hadoop-mapreduce-client-shuffle.
          -1 hdfs tests 161m 29s Tests failed in hadoop-hdfs.
              231m 15s  



          Reason Tests
          Failed unit tests hadoop.hdfs.server.namenode.ha.TestStandbyIsHot



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12748198/HADOOP-7824.004.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 93d50b7
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/7385/artifact/patchprocess/diffcheckstylehadoop-common.txt
          hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7385/artifact/patchprocess/testrun_hadoop-common.txt
          hadoop-mapreduce-client-shuffle test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7385/artifact/patchprocess/testrun_hadoop-mapreduce-client-shuffle.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7385/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7385/testReport/
          Java 1.7.0_55
          uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7385/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 19m 32s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 2 new or modified test files. +1 javac 7m 41s There were no new javac warning messages. +1 javadoc 9m 43s There were no new javadoc warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 55s The applied patch generated 67 new checkstyle issues (total was 81, now 145). +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 20s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 5m 3s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 common tests 22m 20s Tests passed in hadoop-common. +1 mapreduce tests 0m 20s Tests passed in hadoop-mapreduce-client-shuffle. -1 hdfs tests 161m 29s Tests failed in hadoop-hdfs.     231m 15s   Reason Tests Failed unit tests hadoop.hdfs.server.namenode.ha.TestStandbyIsHot Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12748198/HADOOP-7824.004.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 93d50b7 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/7385/artifact/patchprocess/diffcheckstylehadoop-common.txt hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7385/artifact/patchprocess/testrun_hadoop-common.txt hadoop-mapreduce-client-shuffle test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7385/artifact/patchprocess/testrun_hadoop-mapreduce-client-shuffle.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7385/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7385/testReport/ Java 1.7.0_55 uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7385/console This message was automatically generated.
          Hide
          cmccabe Colin P. McCabe added a comment -

          Test failure is unrelated. Checkstyle issues are not relevant. Committed to 2.8. Thanks, Martin!

          Show
          cmccabe Colin P. McCabe added a comment - Test failure is unrelated. Checkstyle issues are not relevant. Committed to 2.8. Thanks, Martin!
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8253 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8253/)
          HADOOP-7824. NativeIO.java flags and identifiers must be set correctly for each platform, not hardcoded to their Linux values (Martin Walsh via Colin P. McCabe) (cmccabe: rev 21d10ccc6e463cf250414264c78acb4a6e7c83e3)

          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/FadvisedFileRegion.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/nativeio/TestNativeIO.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
          • hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/nativeio/NativeIO.c
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestCachingStrategy.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/ReadaheadPool.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/FadvisedChunkedFile.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/nativeio/NativeIO.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8253 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8253/ ) HADOOP-7824 . NativeIO.java flags and identifiers must be set correctly for each platform, not hardcoded to their Linux values (Martin Walsh via Colin P. McCabe) (cmccabe: rev 21d10ccc6e463cf250414264c78acb4a6e7c83e3) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/FadvisedFileRegion.java hadoop-common-project/hadoop-common/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/nativeio/TestNativeIO.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/nativeio/NativeIO.c hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestCachingStrategy.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/ReadaheadPool.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/FadvisedChunkedFile.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/nativeio/NativeIO.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #1004 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1004/)
          HADOOP-7824. NativeIO.java flags and identifiers must be set correctly for each platform, not hardcoded to their Linux values (Martin Walsh via Colin P. McCabe) (cmccabe: rev 21d10ccc6e463cf250414264c78acb4a6e7c83e3)

          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/FadvisedChunkedFile.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/nativeio/TestNativeIO.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/ReadaheadPool.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
          • hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/nativeio/NativeIO.c
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/FadvisedFileRegion.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/nativeio/NativeIO.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestCachingStrategy.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #1004 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1004/ ) HADOOP-7824 . NativeIO.java flags and identifiers must be set correctly for each platform, not hardcoded to their Linux values (Martin Walsh via Colin P. McCabe) (cmccabe: rev 21d10ccc6e463cf250414264c78acb4a6e7c83e3) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/FadvisedChunkedFile.java hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/nativeio/TestNativeIO.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/ReadaheadPool.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/nativeio/NativeIO.c hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/FadvisedFileRegion.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/nativeio/NativeIO.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestCachingStrategy.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #274 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/274/)
          HADOOP-7824. NativeIO.java flags and identifiers must be set correctly for each platform, not hardcoded to their Linux values (Martin Walsh via Colin P. McCabe) (cmccabe: rev 21d10ccc6e463cf250414264c78acb4a6e7c83e3)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/nativeio/TestNativeIO.java
          • hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/nativeio/NativeIO.c
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/ReadaheadPool.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestCachingStrategy.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/nativeio/NativeIO.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/FadvisedChunkedFile.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/FadvisedFileRegion.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #274 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/274/ ) HADOOP-7824 . NativeIO.java flags and identifiers must be set correctly for each platform, not hardcoded to their Linux values (Martin Walsh via Colin P. McCabe) (cmccabe: rev 21d10ccc6e463cf250414264c78acb4a6e7c83e3) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/nativeio/TestNativeIO.java hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/nativeio/NativeIO.c hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/ReadaheadPool.java hadoop-common-project/hadoop-common/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestCachingStrategy.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/nativeio/NativeIO.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/FadvisedChunkedFile.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/FadvisedFileRegion.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2220 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2220/)
          HADOOP-7824. NativeIO.java flags and identifiers must be set correctly for each platform, not hardcoded to their Linux values (Martin Walsh via Colin P. McCabe) (cmccabe: rev 21d10ccc6e463cf250414264c78acb4a6e7c83e3)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/nativeio/TestNativeIO.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/ReadaheadPool.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/FadvisedFileRegion.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/nativeio/NativeIO.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/FadvisedChunkedFile.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestCachingStrategy.java
          • hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/nativeio/NativeIO.c
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2220 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2220/ ) HADOOP-7824 . NativeIO.java flags and identifiers must be set correctly for each platform, not hardcoded to their Linux values (Martin Walsh via Colin P. McCabe) (cmccabe: rev 21d10ccc6e463cf250414264c78acb4a6e7c83e3) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/nativeio/TestNativeIO.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/ReadaheadPool.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/FadvisedFileRegion.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/nativeio/NativeIO.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/FadvisedChunkedFile.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestCachingStrategy.java hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/nativeio/NativeIO.c
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2201 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2201/)
          HADOOP-7824. NativeIO.java flags and identifiers must be set correctly for each platform, not hardcoded to their Linux values (Martin Walsh via Colin P. McCabe) (cmccabe: rev 21d10ccc6e463cf250414264c78acb4a6e7c83e3)

          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/FadvisedChunkedFile.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/ReadaheadPool.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestCachingStrategy.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/FadvisedFileRegion.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/nativeio/TestNativeIO.java
          • hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/nativeio/NativeIO.c
          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/nativeio/NativeIO.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2201 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2201/ ) HADOOP-7824 . NativeIO.java flags and identifiers must be set correctly for each platform, not hardcoded to their Linux values (Martin Walsh via Colin P. McCabe) (cmccabe: rev 21d10ccc6e463cf250414264c78acb4a6e7c83e3) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/FadvisedChunkedFile.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/ReadaheadPool.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestCachingStrategy.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/FadvisedFileRegion.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/nativeio/TestNativeIO.java hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/nativeio/NativeIO.c hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/nativeio/NativeIO.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #263 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/263/)
          HADOOP-7824. NativeIO.java flags and identifiers must be set correctly for each platform, not hardcoded to their Linux values (Martin Walsh via Colin P. McCabe) (cmccabe: rev 21d10ccc6e463cf250414264c78acb4a6e7c83e3)

          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/nativeio/TestNativeIO.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/FadvisedChunkedFile.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/ReadaheadPool.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/FadvisedFileRegion.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestCachingStrategy.java
          • hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/nativeio/NativeIO.c
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/nativeio/NativeIO.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #263 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/263/ ) HADOOP-7824 . NativeIO.java flags and identifiers must be set correctly for each platform, not hardcoded to their Linux values (Martin Walsh via Colin P. McCabe) (cmccabe: rev 21d10ccc6e463cf250414264c78acb4a6e7c83e3) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/nativeio/TestNativeIO.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/FadvisedChunkedFile.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/ReadaheadPool.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/FadvisedFileRegion.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java hadoop-common-project/hadoop-common/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestCachingStrategy.java hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/nativeio/NativeIO.c hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/nativeio/NativeIO.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #271 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/271/)
          HADOOP-7824. NativeIO.java flags and identifiers must be set correctly for each platform, not hardcoded to their Linux values (Martin Walsh via Colin P. McCabe) (cmccabe: rev 21d10ccc6e463cf250414264c78acb4a6e7c83e3)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/nativeio/NativeIO.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/ReadaheadPool.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestCachingStrategy.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/FadvisedChunkedFile.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java
          • hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/nativeio/NativeIO.c
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/FadvisedFileRegion.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/nativeio/TestNativeIO.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #271 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/271/ ) HADOOP-7824 . NativeIO.java flags and identifiers must be set correctly for each platform, not hardcoded to their Linux values (Martin Walsh via Colin P. McCabe) (cmccabe: rev 21d10ccc6e463cf250414264c78acb4a6e7c83e3) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/nativeio/NativeIO.java hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/ReadaheadPool.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestCachingStrategy.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/FadvisedChunkedFile.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/nativeio/NativeIO.c hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/FadvisedFileRegion.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/nativeio/TestNativeIO.java
          Hide
          vinayrpet Vinayakumar B added a comment -

          After this, I started facing compile errors in NativeIO.c in windows, similar to below,

            src\org\apache\hadoop\io\nativeio\NativeIO.c(145): error C2065: 'O_RDONLY' : undeclared identifier [C:\work\hadoop\mai
          n\hadoop-common-project\hadoop-common\src\main\native\native.vcxproj]

          I am not JNI expert, but seeing the code change I understood that, from JNI trying to set the value of the constant in the Java class. And this constant name must be passed to SET_INT_OR_RETURN. So enclosing all these constants names in quotes solved the compilation error for me. Just followed similar to setStaticBoolean(env, clazz, "fadvisePossible", JNI_TRUE);

          Please someone with JNI expertise confirm whether this changes are correct?

          If so, I will raise a Jira and provide the patch to fix the compilation.

          Show
          vinayrpet Vinayakumar B added a comment - After this, I started facing compile errors in NativeIO.c in windows, similar to below, src\org\apache\hadoop\io\nativeio\NativeIO.c(145): error C2065: 'O_RDONLY' : undeclared identifier [C:\work\hadoop\mai n\hadoop-common-project\hadoop-common\src\main\native\native.vcxproj] I am not JNI expert, but seeing the code change I understood that, from JNI trying to set the value of the constant in the Java class. And this constant name must be passed to SET_INT_OR_RETURN . So enclosing all these constants names in quotes solved the compilation error for me. Just followed similar to setStaticBoolean(env, clazz, "fadvisePossible", JNI_TRUE); Please someone with JNI expertise confirm whether this changes are correct? If so, I will raise a Jira and provide the patch to fix the compilation.
          Hide
          martinw Martin Walsh added a comment -

          Hi. Sorry about this. What I suspect actually needs to happen is

          
          

          #ifdef UNIX
          consts_init()

          { ... } #endif {/code}
          Show
          martinw Martin Walsh added a comment - Hi. Sorry about this. What I suspect actually needs to happen is #ifdef UNIX consts_init() { ... } #endif {/code}
          Hide
          vinayrpet Vinayakumar B added a comment -

          Current code is already in #ifdef,

          #ifdef UNIX
            consts_init(env);
            PASS_EXCEPTIONS_GOTO(env, error);
          #endif
          Show
          vinayrpet Vinayakumar B added a comment - Current code is already in #ifdef, #ifdef UNIX consts_init(env); PASS_EXCEPTIONS_GOTO(env, error); #endif
          Hide
          vinayrpet Vinayakumar B added a comment -

          Seems like, #ifdef UNIX check needs to be added inside consts_init(..). This worked for me.

          Show
          vinayrpet Vinayakumar B added a comment - Seems like, #ifdef UNIX check needs to be added inside consts_init(..) . This worked for me.
          Hide
          vinayrpet Vinayakumar B added a comment -

          Raised HADOOP-12302 and provided a patch to fix the same. Please review.

          Show
          vinayrpet Vinayakumar B added a comment - Raised HADOOP-12302 and provided a patch to fix the same. Please review.
          Hide
          martinw Martin Walsh added a comment -

          Sorry, I was not clear. I meant you also need to #ifdef the whole function declaration so its not looking for those constants in windows.

          #ifdef UNIX
          /**
           * Initialises a list of java constants that are platform specific.
           * These are only initialized in UNIX.
           * Any exceptions that occur will be dealt at the level above.
          **/
          static void consts_init(JNIEnv *env) {
            jclass clazz = (*env)->FindClass(env, NATIVE_IO_POSIX_CLASS);
            if (clazz == NULL) {
              return; // exception has been raised
            }
            SET_INT_OR_RETURN(env, clazz, O_RDONLY);
            SET_INT_OR_RETURN(env, clazz, O_WRONLY);
          ...
          }
          #endif
          
          Show
          martinw Martin Walsh added a comment - Sorry, I was not clear. I meant you also need to #ifdef the whole function declaration so its not looking for those constants in windows. #ifdef UNIX /** * Initialises a list of java constants that are platform specific. * These are only initialized in UNIX. * Any exceptions that occur will be dealt at the level above. **/ static void consts_init(JNIEnv *env) { jclass clazz = (*env)->FindClass(env, NATIVE_IO_POSIX_CLASS); if (clazz == NULL) { return ; // exception has been raised } SET_INT_OR_RETURN(env, clazz, O_RDONLY); SET_INT_OR_RETURN(env, clazz, O_WRONLY); ... } #endif
          Hide
          vinayrpet Vinayakumar B added a comment -

          Sorry, I was not clear. I meant you also need to #ifdef the whole function declaration so its not looking for those constants in windows.

          Okay, thanks. I have updated the patch accordingly in HADOOP-12302

          Show
          vinayrpet Vinayakumar B added a comment - Sorry, I was not clear. I meant you also need to #ifdef the whole function declaration so its not looking for those constants in windows. Okay, thanks. I have updated the patch accordingly in HADOOP-12302
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8264 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8264/)
          HADOOP-12302. Fix native compilation on Windows after HADOOP-7824 (Vinayakumar B via Colin P. McCabe) (cmccabe: rev 4ab49a436522f87cd462a9eae20fe10fe5c28fb2)

          • hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/nativeio/NativeIO.c
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8264 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8264/ ) HADOOP-12302 . Fix native compilation on Windows after HADOOP-7824 (Vinayakumar B via Colin P. McCabe) (cmccabe: rev 4ab49a436522f87cd462a9eae20fe10fe5c28fb2) hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/nativeio/NativeIO.c hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Yarn-trunk #1009 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1009/)
          HADOOP-12302. Fix native compilation on Windows after HADOOP-7824 (Vinayakumar B via Colin P. McCabe) (cmccabe: rev 4ab49a436522f87cd462a9eae20fe10fe5c28fb2)

          • hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/nativeio/NativeIO.c
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #1009 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1009/ ) HADOOP-12302 . Fix native compilation on Windows after HADOOP-7824 (Vinayakumar B via Colin P. McCabe) (cmccabe: rev 4ab49a436522f87cd462a9eae20fe10fe5c28fb2) hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/nativeio/NativeIO.c hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #279 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/279/)
          HADOOP-12302. Fix native compilation on Windows after HADOOP-7824 (Vinayakumar B via Colin P. McCabe) (cmccabe: rev 4ab49a436522f87cd462a9eae20fe10fe5c28fb2)

          • hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/nativeio/NativeIO.c
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #279 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/279/ ) HADOOP-12302 . Fix native compilation on Windows after HADOOP-7824 (Vinayakumar B via Colin P. McCabe) (cmccabe: rev 4ab49a436522f87cd462a9eae20fe10fe5c28fb2) hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/nativeio/NativeIO.c hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2206 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2206/)
          HADOOP-12302. Fix native compilation on Windows after HADOOP-7824 (Vinayakumar B via Colin P. McCabe) (cmccabe: rev 4ab49a436522f87cd462a9eae20fe10fe5c28fb2)

          • hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/nativeio/NativeIO.c
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2206 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2206/ ) HADOOP-12302 . Fix native compilation on Windows after HADOOP-7824 (Vinayakumar B via Colin P. McCabe) (cmccabe: rev 4ab49a436522f87cd462a9eae20fe10fe5c28fb2) hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/nativeio/NativeIO.c hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #268 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/268/)
          HADOOP-12302. Fix native compilation on Windows after HADOOP-7824 (Vinayakumar B via Colin P. McCabe) (cmccabe: rev 4ab49a436522f87cd462a9eae20fe10fe5c28fb2)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/nativeio/NativeIO.c
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #268 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/268/ ) HADOOP-12302 . Fix native compilation on Windows after HADOOP-7824 (Vinayakumar B via Colin P. McCabe) (cmccabe: rev 4ab49a436522f87cd462a9eae20fe10fe5c28fb2) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/nativeio/NativeIO.c
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #276 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/276/)
          HADOOP-12302. Fix native compilation on Windows after HADOOP-7824 (Vinayakumar B via Colin P. McCabe) (cmccabe: rev 4ab49a436522f87cd462a9eae20fe10fe5c28fb2)

          • hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/nativeio/NativeIO.c
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #276 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/276/ ) HADOOP-12302 . Fix native compilation on Windows after HADOOP-7824 (Vinayakumar B via Colin P. McCabe) (cmccabe: rev 4ab49a436522f87cd462a9eae20fe10fe5c28fb2) hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/nativeio/NativeIO.c hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2225 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2225/)
          HADOOP-12302. Fix native compilation on Windows after HADOOP-7824 (Vinayakumar B via Colin P. McCabe) (cmccabe: rev 4ab49a436522f87cd462a9eae20fe10fe5c28fb2)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/nativeio/NativeIO.c
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2225 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2225/ ) HADOOP-12302 . Fix native compilation on Windows after HADOOP-7824 (Vinayakumar B via Colin P. McCabe) (cmccabe: rev 4ab49a436522f87cd462a9eae20fe10fe5c28fb2) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/nativeio/NativeIO.c

            People

            • Assignee:
              martinw Martin Walsh
              Reporter:
              kpumuk Dmytro Shteflyuk
            • Votes:
              0 Vote for this issue
              Watchers:
              15 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development