Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-13560

Insufficient system resources exist to complete the requested service for some tests on Windows

    Details

    • Type: Test
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.10.0, 3.2.0, 3.1.1, 2.9.2, 3.0.3
    • Component/s: None
    • Labels:
    • Hadoop Flags:
      Reviewed

      Description

      On Windows, there are 30 tests in HDFS component giving error like the following:
      [ERROR] Tests run: 7, Failures: 0, Errors: 7, Skipped: 0, Time elapsed: 50.149 s <<< FAILURE! - in org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistFiles
      [ERROR] testDisableLazyPersistFileScrubber(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistFiles) Time elapsed: 16.513 s <<< ERROR!
      1450: Insufficient system resources exist to complete the requested service.

      at org.apache.hadoop.io.nativeio.NativeIO$Windows.extendWorkingSetSize(Native Method)
      at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:1339)
      at org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:495)
      at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:2695)
      at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:2598)
      at org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:1554)
      at org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:904)
      at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:514)
      at org.apache.hadoop.hdfs.MiniDFSCluster$Builder.build(MiniDFSCluster.java:473)
      at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.LazyPersistTestCase.startUpCluster(LazyPersistTestCase.java:316)
      at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.LazyPersistTestCase$ClusterWithRamDiskBuilder.build(LazyPersistTestCase.java:415)
      at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistFiles.testDisableLazyPersistFileScrubber(TestLazyPersistFiles.java:128)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:498)
      at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
      at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
      at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
      at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
      at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
      at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)

      The involved tests are

      TestLazyPersistFiles,TestLazyPersistPolicy,TestLazyPersistReplicaRecovery,TestLazyPersistLockedMemory#testWritePipelineFailure,TestLazyPersistLockedMemory#testShortBlockFinalized,TestLazyPersistReplicaPlacement#testRamDiskNotChosenByDefault,TestLazyPersistReplicaPlacement#testFallbackToDisk,TestLazyPersistReplicaPlacement#testPlacementOnSizeLimitedRamDisk,TestLazyPersistReplicaPlacement#testPlacementOnRamDisk,TestLazyWriter#testDfsUsageCreateDelete,TestLazyWriter#testDeleteAfterPersist,TestLazyWriter#testDeleteBeforePersist,TestLazyWriter#testLazyPersistBlocksAreSaved,TestDirectoryScanner#testDeleteBlockOnTransientStorage,TestDirectoryScanner#testRetainBlockOnPersistentStorage,TestDirectoryScanner#testExceptionHandlingWhileDirectoryScan,TestDirectoryScanner#testDirectoryScanner,TestDirectoryScanner#testThrottling,TestDirectoryScanner#testDirectoryScannerInFederatedCluster,TestNameNodeMXBean#testNameNodeMXBeanInfo

      [ERROR] Errors:
      [ERROR] TestDirectoryScanner.testDeleteBlockOnTransientStorage:385 ╗ NativeIO Insuffic...
      [ERROR] TestDirectoryScanner.testDirectoryScanner:426->runTest:431 ╗ NativeIO Insuffic...
      [ERROR] TestDirectoryScanner.testDirectoryScannerInFederatedCluster:1026 ╗ NativeIO In...
      [ERROR] TestDirectoryScanner.testExceptionHandlingWhileDirectoryScan:982 ╗ NativeIO In...
      [ERROR] TestDirectoryScanner.testRetainBlockOnPersistentStorage:344 ╗ NativeIO Insuffi...
      [ERROR] TestDirectoryScanner.testThrottling:583 ╗ NativeIO Insufficient system resourc...
      [ERROR] TestLazyPersistFiles.testAppendIsDenied:51->LazyPersistTestCase.startUpCluster:316 ╗ NativeIO
      [ERROR] TestLazyPersistFiles.testConcurrentRead:186->LazyPersistTestCase.startUpCluster:316 ╗ NativeIO
      [ERROR] TestLazyPersistFiles.testConcurrentWrites:237->LazyPersistTestCase.startUpCluster:316 ╗ NativeIO
      [ERROR] TestLazyPersistFiles.testCorruptFilesAreDiscarded:94->LazyPersistTestCase.startUpCluster:316 ╗ NativeIO
      [ERROR] TestLazyPersistFiles.testDisableLazyPersistFileScrubber:128->LazyPersistTestCase.startUpCluster:316 ╗ NativeIO
      [ERROR] TestLazyPersistFiles.testFileShouldNotDiscardedIfNNRestarted:157->LazyPersistTestCase.startUpCluster:316 ╗ NativeIO
      [ERROR] TestLazyPersistFiles.testTruncateIsDenied:72->LazyPersistTestCase.startUpCluster:316 ╗ NativeIO
      [ERROR] TestLazyPersistLockedMemory.testShortBlockFinalized:134->LazyPersistTestCase.startUpCluster:316 ╗ NativeIO
      [ERROR] TestLazyPersistLockedMemory.testWritePipelineFailure:154->LazyPersistTestCase.startUpCluster:316 ╗ NativeIO
      [ERROR] TestLazyPersistPolicy.testPolicyNotSetByDefault:37->LazyPersistTestCase.startUpCluster:316 ╗ NativeIO
      [ERROR] TestLazyPersistPolicy.testPolicyPersistenceInEditLog:62->LazyPersistTestCase.startUpCluster:316 ╗ NativeIO
      [ERROR] TestLazyPersistPolicy.testPolicyPersistenceInFsImage:76->LazyPersistTestCase.startUpCluster:316 ╗ NativeIO
      [ERROR] TestLazyPersistPolicy.testPolicyPropagation:50->LazyPersistTestCase.startUpCluster:316 ╗ NativeIO
      [ERROR] TestLazyPersistReplicaPlacement.testFallbackToDisk:72->LazyPersistTestCase.startUpCluster:316 ╗ NativeIO
      [ERROR] TestLazyPersistReplicaPlacement.testPlacementOnRamDisk:41->LazyPersistTestCase.startUpCluster:316 ╗ NativeIO
      [ERROR] TestLazyPersistReplicaPlacement.testPlacementOnSizeLimitedRamDisk:52->LazyPersistTestCase.startUpCluster:316 ╗ NativeIO
      [ERROR] TestLazyPersistReplicaPlacement.testRamDiskNotChosenByDefault:163->LazyPersistTestCase.startUpCluster:316 ╗ NativeIO
      [ERROR] TestLazyPersistReplicaRecovery.testDnRestartWithSavedReplicas:36->LazyPersistTestCase.startUpCluster:316 ╗ NativeIO
      [ERROR] TestLazyPersistReplicaRecovery.testDnRestartWithUnsavedReplicas:61->LazyPersistTestCase.startUpCluster:316 ╗ NativeIO
      [ERROR] TestLazyWriter.testDeleteAfterPersist:211->LazyPersistTestCase.startUpCluster:316 ╗ NativeIO
      [ERROR] TestLazyWriter.testDeleteBeforePersist:184->LazyPersistTestCase.startUpCluster:316 ╗ NativeIO
      [ERROR] TestLazyWriter.testDfsUsageCreateDelete:236->LazyPersistTestCase.startUpCluster:316 ╗ NativeIO
      [ERROR] TestLazyWriter.testLazyPersistBlocksAreSaved:43->LazyPersistTestCase.startUpCluster:316 ╗ NativeIO
      [ERROR] TestNameNodeMXBean.testNameNodeMXBeanInfo:99 ╗ NativeIO Insufficient system re...
      [INFO]
      [ERROR] Tests run: 30, Failures: 0, Errors: 30, Skipped: 0

        Attachments

        1. HDFS-13560.004.patch
          5 kB
          Anbang Hu
        2. HDFS-13560.003.patch
          5 kB
          Anbang Hu
        3. HDFS-13560.002.patch
          5 kB
          Anbang Hu
        4. HDFS-13560.001.patch
          6 kB
          Anbang Hu
        5. HDFS-13560.000.patch
          5 kB
          Anbang Hu

          Issue Links

            Activity

              People

              • Assignee:
                huanbang1993 Anbang Hu
                Reporter:
                huanbang1993 Anbang Hu
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: