|
[
Permlink
| « Hide
]
Owen O'Malley added a comment - 03/Dec/07 11:29 PM
Has anyone tried out the java 6 from http://landonf.bikemonkey.org/code/macosx/
 ?
To be a little more direct.... Lack of Java 6 support on MacOS has been cited in the past as a reason to hold back on requiring Java 6 for Hadoop. If you develop on the Mac, you should try out Owen's suggested open source Java 6 implementation with your IDE and post your feedback here. That java6 requires agreement to the java research license, which is unacceptable for commercial use.
To quote from the bikemonkey link:
So, while IANAL, it appears this can only be used under JRL. But JRL only permits Research Use, defining it as:
So I suspect that running hadoop client libraries at a commercial search engine does not fall under Research Use. Using File.getFreeSpace() and File.getUsableSpace() instead of 'df -k' is a part of this issue. Refer
Because java 6 (compiler + runtime) is not available under acceptable license on all platforms, I think references to functions like File.getFreeSpace should be done in a way that is backwards compatible with java 5. I suggest you put all the references to java 6 functions in an optional jar, with fallback methods for java 5 users.
If Java 6 is not yet available to our Mac-based developers, then we should probably delay implementing this to a subsequent release.
Do we believe that the licensing situation is likely to change in the near future? If not, we should perhaps do what Michael suggests. I certainly agree that we should strongly encourage Java 6, but I'm not yet sure that we must require it. The primary Java6 feature that's been identified as useful is File.getFreeSpace(), but this will not totally replace df, since Java still doesn't give the mount point, and we'll still need to invoke lots of other shell commands. So, assuming
This didn't pan out for 0.16.0. Assigning to 0.17.0 for another crack at it.
> Do we believe that the licensing situation is likely to change in the near future?
I think we just have to wait for Apple to do an "official" 1.6 release for OSX. There was a lot of speculation that they would release shortly after OSX10.5 came out, but then they didn't. Not an apple fanboy, but I think java6 is officially released : http://docs.info.apple.com/article.html?artnum=307403
I'm strongly +1 for java 6 in 0.17. Should we force Java 6 in 0.17? If so we should mark this blocker ! > Should we force Java 6 in 0.17?
I think it's too late for 0.17, since that's already been branched, and we should no longer make incompatible changes. java 6 is available on 10.5, but not 10.4 (which I still use).... Is there any particular reason we want to force java 6? File.getFreeSpace doesn't seem like a good reason if there is such a simple workaround.
This patch updates Hadoop to require Java 1.6.
This may fail, if Hudson uses Java 1.5.
Hudson indeed uses Java 1.5 since it cannot compile hadoop now. See http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2522/artifact/trunk/current/trunkJavacWarnings.txt
I'm not sure I understand the point of this patch: are there any actual dependencies on java6 we want to introduce? Users can certainly run hadoop in java6 vms if they wish, but making it impossible for mac 10.4 desktop users to use hadoop is a big problem for me.
-1
I don't see a need for this. Looks like
The patch was provided by me and it had the necessary jars (Commons net & Apache Mina FTP server) created after compiling the relevant classes under Java 1.6 on my local linux machine. While creation of commons net jars on local machine was unnecessary, I had to create Mina FTP server jars locally as it does not have an official release yet.
I have fixed problem by picking up commons net jar from the download site and re-compiling Mina FTP server classes under Java 1.5. The jars have been packaged and attached on both My sincere apologies for the inconvenience caused to everybody. It should be fairly trivial to run File.getFreeSpace() using introspection; what is equally interesting is the java6 script engine and built in javascript interpreter; you could add support for mappers and reducers written in JS or another supported language. I would strongly argue against anyone using JAXB or the JAX-WS runtime built into Java6, as the exact version varies from point release of the JVM.. java 6 U4 and later provides a different JAXB runtime, just to break everyone's code. This effectively means 'dont use JAXB or JAX-WS'. There is no urgency to this, but it would be nice not to delay it.
Hadoop performs better on Java 6, so we want to encourage folks to use Java 6. The vast majority of Hadoop contributors use Java 6 when developing, and hence do run into (usually minor) back-compatibility problems, like the one we've just seen with On the other hand, we don't want to leave contributors behind. It appears that Java 6 is still not available on PowerPC-based Macs. Sigh. Michael, is the G4 your primary development platform? Java 6 appears to not be available on OSX 10.4, either. I would be willing to use this as an excuse to upgrade my dev machine to 10.5, though. Have you tried with IcedTea ? It's the JDK I use to build and run Hadoop on my Ubuntu/x86 system. It seems that it have been ported to MacOS 10.4 [1], and that it can be built on PowerPC [2], but I'm unsure about the combination of the two.
[1] http://landonf.bikemonkey.org/static/soylatte/ [2] http://langel.wordpress.com/2008/01/03/icedtea-15-released-with-ppc-support/ Doug, should we mark this for 0.19?
> Doug, should we mark this for 0.19?
Sure. Folks won't like it then either, but we should probably revisit it periodically. Moving to 0.19 release to continue discussion in the context of that release.
Should we commit this now, early in the 0.19 cycle, so that we can address things like
Would it be possible to move all the servers to java6, but keep the DFSClient on java 5?
> Would it be possible to move all the servers to java6, but keep the DFSClient on java 5?
Maintaining that would be more painful than the benefits it would offer. We should wholly switch to Java 6 or wholly stay in Java 5. +1 to the move, with some caveats
Stuff built with Java6 doesn't necessarily load in Java5; I've encountered problems with libraries I built myself not loading in Java5. So keeping some stuff java5 and java6 isn't going to work reliably. Best to make the jump and spell it out in advance "18.x is the last java5 version". Re: hudson
+1 to moving forward to java 6.
Here's a new version of the patch, updated for changes to trunk. I also fixed things so that forrest (which requires Java5) will now work only if -Djava5.home is specified, and updated the test-patch.sh script accordingly.
Nigel, if you can please switch Hudson to:
Then I will submit this patch and I think it will pass Hudson's tests and we can commit it without breaking Hudson. Updated Hudson. Trying patch.
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12384854/HADOOP-2325.patch against trunk revision 673517. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 1 new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2792/testReport/ This message is automatically generated. The problem was that Java 5 was not found, which caused some targets to fail. Nigel, can you re-check the value of -Djava5.home? Thanks!
Doug, the problem is that the patch testing process can't test these changes to the patch testing process itself. Hudson calls src/test/bin/test-patch.sh before it has been patched.
I think I've got all the Hudson pieces in place for this patch. Please commit it on Monday and I'll monitor the first few builds. Integrated in Hadoop-trunk #581 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/581/
) |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
 |
|
Improvement
Closed
Major
Require Java 6
