[HADOOP-8562] Enhancements to support Hadoop on Windows Server and Windows Azure environments - ASF JIRA

XML

Word

Printable

JSON

Details

Type: New Feature
Status: Closed
Priority: Major
Resolution: Fixed
Affects Version/s: 3.0.0-alpha1
Fix Version/s: 2.1.0-beta
Component/s: None
Labels:
None

Hadoop Flags:

Reviewed
Release Note:
This umbrella jira makes enhancements to support Hadoop natively on Windows Server and Windows Azure environments.
Tags:
Windows

Description

This JIRA tracks the work that needs to be done on trunk to enable Hadoop to run on Windows Server and Azure environments. This incorporates porting relevant work from the similar effort on branch 1 tracked via ~~HADOOP-8079~~.

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending

branch-2.merge.patch
23/May/13 17:49
640 kB
Arpit Agarwal
branch-2.merge.patch
23/May/13 17:18
640 kB
Arpit Agarwal
branch-trunk-win.min.patch
01/Mar/13 22:13
340 kB
Suresh Srinivas
branch-trunk-win.min-notest.patch
01/Mar/13 22:11
225 kB
Suresh Srinivas
branch-trunk-win.patch
06/Mar/13 04:07
642 kB
Suresh Srinivas
branch-trunk-win.patch
06/Mar/13 02:17
593 kB
Suresh Srinivas
branch-trunk-win.patch
05/Mar/13 07:11
593 kB
Suresh Srinivas
branch-trunk-win.patch
07/Feb/13 23:32
561 kB
Suresh Srinivas
branch-trunk-win.patch
29/Jan/13 22:01
565 kB
Suresh Srinivas
branch-trunk-win.patch
24/Jan/13 22:53
543 kB
Suresh Srinivas
branch-trunk-win.patch
03/Jan/13 21:43
557 kB
Suresh Srinivas
branch-trunk-win.patch
17/Dec/12 18:15
544 kB
Suresh Srinivas
branch-trunk-win.patch
15/Dec/12 16:44
635 kB
Suresh Srinivas
branch-trunk-win.patch
14/Dec/12 04:49
609 kB
Suresh Srinivas
branch-trunk-win.patch
13/Dec/12 19:39
738 kB
Suresh Srinivas
branch-trunk-win-min.patch
28/Feb/13 02:11
60 kB
Suresh Srinivas
test-untar.tar
17/Dec/12 07:16
20 kB
Chris Nauroth
test-untar.tgz
17/Dec/12 07:16
2 kB
Chris Nauroth

Issue Links

breaks

HADOOP-9397 Incremental dist tar build fails

Closed

HADOOP-9481 Broken conditional logic with HADOOP_SNAPPY_LIBRARY

Closed

incorporates

HADOOP-8980 TestRPC and TestSaslRPC fail on Windows

Open

HADOOP-9133 Windows tarball build fails when enlistment root path is long

Open

HADOOP-8953 Shell PathData parsing failures on Windows

Resolved

HADOOP-8954 "stat" executable not found on Windows

Resolved

HADOOP-8955 "chmod" executable not found on Windows

Resolved

HADOOP-8956 FileSystem.primitiveMkdir failures on Windows cause multiple test suites to fail

Resolved

HADOOP-8959 TestUserGroupInformation fails on Windows due to "id" executable not found

Resolved

HADOOP-8960 TestMetricsServlet fails on Windows

Resolved

HADOOP-8961 GenericOptionsParser URI parsing failure on Windows

Resolved

HADOOP-8974 TestDFVariations fails on Windows

Resolved

HADOOP-8975 TestFileContextResolveAfs fails on Windows

Resolved

HADOOP-8976 TestFSMainOperationsLocalFileSystem fails on Windows

Resolved

HADOOP-8977 multiple FsShell test failures on Windows

Resolved

HADOOP-8978 TestTrash fails on Windows

Resolved

HADOOP-8979 TestHttpServer fails on Windows

Resolved

HADOOP-9005 Merge hadoop cmd line scripts from branch-1-win

Resolved

HADOOP-9008 Building hadoop tarball fails on Windows

Resolved

HADOOP-9011 saveVersion.py does not include branch in version annotation

Resolved

HADOOP-9110 winutils ls off-by-one error indexing MONTHS array can cause access violation

Resolved

HADOOP-9144 FindBugs reports new warnings in branch-trunk-win

Resolved

HADOOP-9146 failure to set sticky bit regression on branch-trunk-win

Resolved

HADOOP-9266 correct javac, findbugs, and release audit warnings on branch-trunk-win

Resolved

HADOOP-9309 test failures on Windows due to UnsatisfiedLinkError in NativeCodeLoader#buildSupportsSnappy

Resolved

HADOOP-9313 Remove spurious mkdir from hadoop-config.cmd

Resolved

HADOOP-9368 Add timeouts to new tests in branch-trunk-win

Resolved

HDFS-4241 DataNode unit test failures on Windows

Resolved

HADOOP-9687 branch-1-win TestJobTrackerQuiescence and TestFileLengthOnClusterRestart failed caused by incorrect DFS path construction on Windows

Resolved

HADOOP-9759 Add support for NativeCodeLoader#getLibraryName on Windows

Closed

HADOOP-8948 TestFileUtil.testGetDU fails on Windows due to incorrect assumption of line separator

Closed

HADOOP-8957 AbstractFileSystem#IsValidName should be overridden for embedded file systems like ViewFs

Closed

HADOOP-8958 ViewFs:Non absolute mount name failures when running multiple tests on Windows

Closed

HADOOP-8973 DiskChecker cannot reliably detect an inaccessible disk on Windows with NTFS ACLs

Closed

HADOOP-8981 TestMetricsSystemImpl fails on Windows

Closed

HADOOP-8982 TestSocketIOWithTimeout fails on Windows

Closed

HADOOP-9043 disallow in winutils creating symlinks with forwards slashes

Closed

HADOOP-9131 TestLocalFileSystem#testListStatusWithColons cannot run on Windows

Closed

HADOOP-9264 port change to use Java untar API on Windows from branch-1-win to trunk

Closed

HADOOP-9278 HarFileSystem may leak file handle

Closed

HADOOP-9290 Some tests cannot load native library

Closed

HADOOP-9294 GetGroupsTestBase fails on Windows

Closed

HADOOP-9364 PathData#expandAsGlob does not return correct results for absolute paths on Windows

Closed

HADOOP-9365 TestHAZKUtil fails on Windows

Closed

HADOOP-9372 Fix bad timeout annotations on tests

Closed

HADOOP-9437 TestNativeIO#testRenameTo fails on Windows due to assumption that POSIX errno is embedded in NativeIOException

Closed

HADOOP-9443 Port winutils static code analysis change to trunk

Closed

HADOOP-9488 FileUtil#createJarWithClassPath only substitutes environment variables from current process environment/does not support overriding when launching new process

Closed

HADOOP-9500 TestUserGroupInformation#testGetServerSideGroups fails on Windows due to failure to find winutils.exe

Closed

HADOOP-9524 Fix ShellCommandFencer to work on Windows

Closed

HADOOP-9526 TestShellCommandFencer and TestShell fail on Windows

Closed

HADOOP-9527 Add symlink support to LocalFileSystem on Windows

Closed

HADOOP-9553 TestAuthenticationToken fails on Windows

Closed

HADOOP-10030 FsShell -put/copyFromLocal should support Windows local path

Closed

HADOOP-9656 Gridmix unit tests fail on Windows and Linux

Closed

HADOOP-9681 FileUtil.unTarUsingJava() should close the InputStream upon finishing

Closed

HADOOP-9422 HADOOP_HOME should not be required to be set to be able to launch commands using hadoop.util.Shell

Patch Available

HADOOP-9018 Reject invalid Windows URIs

Resolved

HADOOP-9056 Build native library on Windows

Resolved

HADOOP-9081 Port TestWinUtils from branch-1-win to branch-trunk-win

Resolved

HADOOP-9347 add instructions to BUILDING.txt describing how to build on Windows

Resolved

HADOOP-9356 remove remaining references to cygwin/cygpath from scripts

Resolved

HDFS-3602 Enhancements to HDFS for Windows Server and Windows Azure development and runtime environments

Resolved

MAPREDUCE-4401 Enhancements to MapReduce for Windows Server and Windows Azure development and runtime environments

Resolved

YARN-191 Enhancements to YARN for Windows Server and Windows Azure development and runtime environments

Resolved

HADOOP-9348 address TODO in winutils to add more command line usage and examples

Resolved

HADOOP-9354 Windows native project files missing license headers

Resolved

HADOOP-9483 winutils support for readlink command

Closed

HDFS-4554 TestLocalDirAllocator and TestDiskError need to be reviewed for Windows compatibility and either re-enabled or re-documented.

Open

HADOOP-8945 winutils: merge codebase from branch-1-win to branch-trunk-win

Resolved

HADOOP-8946 winutils: compile codebase during Maven build on branch-trunk-win

Resolved

HADOOP-8947 Merge FileUtil and Shell changes from branch-1-win to branch-trunk-win to enable initial test pass

Resolved

HADOOP-8949 FileUtil.CygPathCommand is dead code on branch-1-win and branch-trunk-win

Resolved

is related to

HADOOP-9600 In Windows: Hadoop fails to run when JAVA_HOME has spaces in it

Open

HADOOP-9433 TestLocalFileSystem#testHasFileDescriptor leaks file handle

Resolved

HADOOP-9431 TestSecurityUtil#testLocalHostNameForNullOrWild on systems where hostname contains capital letters

Resolved

HADOOP-9555 HA functionality that uses ZooKeeper may experience inadvertent TCP RST and miss session expiration event due to bug in client connection management

Closed

HADOOP-9556 disable HA tests on Windows that fail due to ZooKeeper client connection management bug

Closed

HADOOP-9632 TestShellCommandFencer will fail if there is a 'host' machine in the network

Closed

relates to

HADOOP-16045 Don't run TestDU on Windows

Resolved

HADOOP-9376 TestProxyUserFromEnv fails on a Windows domain joined machine

Closed

HADOOP-9388 TestFsShellCopy fails on Windows

Closed

HADOOP-9413 Introduce common utils for File#setReadable/Writable/Executable and File#canRead/Write/Execute that work cross-platform

Closed

HADOOP-9490 LocalFileSystem#reportChecksumFailure not closing the checksum file handle before rename

Closed

HADOOP-9599 hadoop-config.cmd doesn't set JAVA_LIBRARY_PATH correctly

Closed

HADOOP-9678 TestRPC#testStopsAllThreads intermittently fails on Windows

Closed

HDFS-4572 Fix TestJournal failures on Windows

Closed

HDFS-4584 Fix TestNNWithQJM failures on Windows

Closed

HDFS-4615 Fix TestDFSShell failures on Windows

Closed

HADOOP-9373 Merge CHANGES.branch-trunk-win.txt to CHANGES.txt

Closed

HADOOP-9387 TestDFVariations fails on Windows after the merge

Closed

HADOOP-14309 Add PowerShell NodeFencer

Resolved

HADOOP-9525 Add tests that validate winutils chmod behavior on folders

Closed

supercedes

HADOOP-8621 FileUtil.symLink fails if spaces in path

Resolved

(68 incorporates, 6 is related to, 14 relates to, 1 supercedes)

Activity

People

Assignee:: Bikas Saha

Reporter:: Bikas Saha

Votes:: 0 Vote for this issue

Watchers:: 20 Start watching this issue

Dates

Created:: 05/Jul/12 17:57

Updated:: 12/Jan/19 00:21

Resolved:: 23/May/13 20:42