Uploaded image for project: 'Geode'
  1. Geode
  2. GEODE-10 HDFS Integration
  3. GEODE-315

Hadoop integration tests fail under Windows

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Won't Fix
    • 1.0.0-incubating
    • None
    • hdfs
    • Windows

    Description

      The complete list of tests failing on Windows because of Hadoop:

      HDFSConfigJUnitTest. testCacheXMLParsing
      HDFSConfigJUnitTest. testHDFSStoreCreation
      HDFSConfigJUnitTest. testHdfsStoreConfFullParsing
      HDFSEntriesSetJUnitTest. testDuplicateIterator
      HDFSEntriesSetJUnitTest. testEitherOrIterator
      HDFSEntriesSetJUnitTest. testEmptyIterator
      HDFSEntriesSetJUnitTest. testHdfsOnlyIterator
      HDFSEntriesSetJUnitTest. testQueueOnlyIterator
      HDFSEntriesSetJUnitTest. testUnevenIterator
      HdfsStoreMutatorJUnitTest. testMutatorInitialState
      HdfsStoreMutatorJUnitTest. testMutatorReturnsUpdatedValues
      CardinalityEstimatorJUnitTest. testCardinalityAfterMajorCompaction
      CardinalityEstimatorJUnitTest. testCardinalityAfterMinorCompaction
      CardinalityEstimatorJUnitTest. testCardinalityAfterRestart
      CardinalityEstimatorJUnitTest. testMultipleHoplogCardinality
      CardinalityEstimatorJUnitTest. testSingleHoplogCardinality
      CardinalityEstimatorJUnitTest. testSingleHoplogCardinalityWithDuplicates
      HDFSCompactionManagerJUnitTest. testAlterAutoMajorCompaction
      HDFSCompactionManagerJUnitTest. testAlterAutoMinorCompaction
      HDFSCompactionManagerJUnitTest. testAlterMajorThreadsIncrease
      HDFSCompactionManagerJUnitTest. testForceCompaction
      HDFSCompactionManagerJUnitTest. testForceCompactionWithAutoDisabled
      HDFSRegionDirectorJUnitTest. testCompactionEvents
      HDFSRegionDirectorJUnitTest. testDirector
      HDFSStatsJUnitTest. testBloomStats
      HDFSStatsJUnitTest. testInactiveFileStats
      HDFSStatsJUnitTest. testReadStats
      HDFSStatsJUnitTest. testRegionBucketShareStats
      HDFSStatsJUnitTest. testScanStats
      HDFSStatsJUnitTest. testStoreUsageStats
      HDFSStatsJUnitTest. testWriteStats
      HDFSUnsortedHoplogOrganizerJUnitTest. testAlterRollOverInterval
      HDFSUnsortedHoplogOrganizerJUnitTest. testClear
      HDFSUnsortedHoplogOrganizerJUnitTest. testFlush
      HDFSUnsortedHoplogOrganizerJUnitTest. testSequenceFileScan
      HdfsSortedOplogOrganizerJUnitTest. testAlterPurgeInterval
      HdfsSortedOplogOrganizerJUnitTest. testClear
      HdfsSortedOplogOrganizerJUnitTest. testConcurrentReadInactiveClose
      HdfsSortedOplogOrganizerJUnitTest. testCorruptHfileBucketFail
      HdfsSortedOplogOrganizerJUnitTest. testExpireFilterRetartAfterClear
      HdfsSortedOplogOrganizerJUnitTest. testExpiredFilterAtStartup
      HdfsSortedOplogOrganizerJUnitTest. testExpiredHoplogCleanup
      HdfsSortedOplogOrganizerJUnitTest. testExpiryMarkerIdentification
      HdfsSortedOplogOrganizerJUnitTest. testExposeCleanupInterval
      HdfsSortedOplogOrganizerJUnitTest. testFlush
      HdfsSortedOplogOrganizerJUnitTest. testHoplogIdentification
      HdfsSortedOplogOrganizerJUnitTest. testInUseExpiredHoplogCleanup
      HdfsSortedOplogOrganizerJUnitTest. testMakeLegitimate
      HdfsSortedOplogOrganizerJUnitTest. testMaxOpenReaders
      HdfsSortedOplogOrganizerJUnitTest. testNoFileJanitor
      HdfsSortedOplogOrganizerJUnitTest. testOneHoplogMajorCompaction
      HdfsSortedOplogOrganizerJUnitTest. testRead
      HdfsSortedOplogOrganizerJUnitTest. testReopen
      HdfsSortedOplogOrganizerJUnitTest. testSeqInitialization
      HfileSortedOplogJUnitTest. testDeletion
      HfileSortedOplogJUnitTest. testDuplicateKeys
      HfileSortedOplogJUnitTest. testEmptyFileIterator
      HfileSortedOplogJUnitTest. testFromExclusiveIterator
      HfileSortedOplogJUnitTest. testFromInclusiveIterator
      HfileSortedOplogJUnitTest. testFromToIterator
      HfileSortedOplogJUnitTest. testHoplogReader
      HfileSortedOplogJUnitTest. testHoplogWriter
      HfileSortedOplogJUnitTest. testIterator
      HfileSortedOplogJUnitTest. testNewScannerDetechAndUseRecycledFs
      HfileSortedOplogJUnitTest. testOffsetBasedScan
      HfileSortedOplogJUnitTest. testOffsetScanBeyondFileSize
      HfileSortedOplogJUnitTest. testReaderDetectAndUseRecycledFs
      HfileSortedOplogJUnitTest. testSingleKVIterator
      HfileSortedOplogJUnitTest. testToExclusiveIterator
      HfileSortedOplogJUnitTest. testToInclusiveIterator
      HfileSortedOplogJUnitTest. testZeroValueOffsetScan
      SortedOplogListIterJUnitTest. testMergedIterator
      SortedOplogListIterJUnitTest. testOneIterDuplicateKey
      SortedOplogListIterJUnitTest. testOneIterOneKey
      SortedOplogListIterJUnitTest. testTwoIterDiffKey
      SortedOplogListIterJUnitTest. testTwoIterSameKey
      TieredCompactionJUnitTest. testIterativeMinorCompaction
      TieredCompactionJUnitTest. testMajorCompactionSetsSeqNum
      TieredCompactionJUnitTest. testMajorCompactionWithDelete
      TieredCompactionJUnitTest. testMinorCTargetsIgnoreMajorC
      TieredCompactionJUnitTest. testMinorCompaction
      TieredCompactionJUnitTest. testMinorCompactionLessTargetsStatsUpdate
      TieredCompactionJUnitTest. testMinorCompactionTargetsOptimizer
      TieredCompactionJUnitTest. testMinorWatchesMajorsSeqNum
      TieredCompactionJUnitTest. testSuspendMinorByMajor
      TieredCompactionJUnitTest. testTargetOverlap
      TieredCompactionJUnitTest. testTimeBoundedSuspend
      HDFSSplitIteratorJUnitTest. test1Hop1BlockIter
      HDFSSplitIteratorJUnitTest. test1Hop1BlockIterSkipDeletedHoplogs
      HDFSSplitIteratorJUnitTest. test1HopNBlockIter
      HDFSSplitIteratorJUnitTest. testMRLikeNHopIter
      HDFSSplitIteratorJUnitTest. testMRLikeNHopIterSkipDeletedHoplogs
      HDFSSplitIteratorJUnitTest. testNHoplogNBlockIter
      HoplogUtilJUnitTest. testCheckpointSelection
      HoplogUtilJUnitTest. testExcludeSoonCleanedHoplogs
      HoplogUtilJUnitTest. testHoplogListingEmptyBucket
      HoplogUtilJUnitTest. testHoplogListingInvalidBucket
      HoplogUtilJUnitTest. testHoplogListingInvalidFiles
      HoplogUtilJUnitTest. testHoplogListingMixFileTypes
      HoplogUtilJUnitTest. testHoplogListingMultiBucket
      HoplogUtilJUnitTest. testTimeRange
      HDFSQueueRegionOperationsJUnitTest. test010PUTDMLSupport
      HDFSQueueRegionOperationsJUnitTest. test020GetOperationalData
      HDFSQueueRegionOperationsJUnitTest. test030RemoveOperationalData
      HDFSQueueRegionOperationsJUnitTest. test050LRURegionAttributesForPR
      HDFSQueueRegionOperationsJUnitTest. test060LRURegionAttributesForRegionShortcutPR
      HDFSQueueRegionOperationsJUnitTest. test070SizeEstimate
      HDFSQueueRegionOperationsJUnitTest. test080PutGet
      HDFSQueueRegionOperationsJUnitTest. test090Delete
      HDFSQueueRegionOperationsJUnitTest. test100Invalidate
      HDFSQueueRegionOperationsJUnitTest. test110Size
      HDFSQueueRegionOperationsJUnitTest. test120KeyIterator
      HDFSQueueRegionOperationsJUnitTest. test130EntriesIterator
      HDFSQueueRegionOperationsJUnitTest. test140ContainsKey
      HDFSQueueRegionOperationsJUnitTest. test150ContainsValue
      HDFSQueueRegionOperationsJUnitTest. test160DestroyRegion
      HDFSQueueRegionOperationsJUnitTest. test170PutIfAbsent
      HDFSQueueRegionOperationsJUnitTest. test180Replace
      HDFSQueueRegionOperationsJUnitTest. test190ReplaceKVV
      HDFSQueueRegionOperationsJUnitTest. test200Accuracy
      HDFSRegionOperationsJUnitTest. test010PUTDMLSupport
      HDFSRegionOperationsJUnitTest. test020GetOperationalData
      HDFSRegionOperationsJUnitTest. test030RemoveOperationalData
      HDFSRegionOperationsJUnitTest. test050LRURegionAttributesForPR
      HDFSRegionOperationsJUnitTest. test060LRURegionAttributesForRegionShortcutPR
      HDFSRegionOperationsJUnitTest. test070SizeEstimate
      HDFSRegionOperationsJUnitTest. test080PutGet
      HDFSRegionOperationsJUnitTest. test090Delete
      HDFSRegionOperationsJUnitTest. test100Invalidate
      HDFSRegionOperationsJUnitTest. test110Size
      HDFSRegionOperationsJUnitTest. test120KeyIterator
      HDFSRegionOperationsJUnitTest. test130EntriesIterator
      HDFSRegionOperationsJUnitTest. test140ContainsKey
      HDFSRegionOperationsJUnitTest. test150ContainsValue
      HDFSRegionOperationsJUnitTest. test160DestroyRegion
      HDFSRegionOperationsJUnitTest. test170PutIfAbsent
      HDFSRegionOperationsJUnitTest. test180Replace
      HDFSRegionOperationsJUnitTest. test190ReplaceKVV
      HDFSRegionOperationsJUnitTest. test200Accuracy
      HDFSRegionOperationsOffHeapJUnitTest. test010PUTDMLSupport
      HDFSRegionOperationsOffHeapJUnitTest. test020GetOperationalData
      HDFSRegionOperationsOffHeapJUnitTest. test030RemoveOperationalData
      HDFSRegionOperationsOffHeapJUnitTest. test050LRURegionAttributesForPR
      HDFSRegionOperationsOffHeapJUnitTest. test060LRURegionAttributesForRegionShortcutPR
      HDFSRegionOperationsOffHeapJUnitTest. test070SizeEstimate
      HDFSRegionOperationsOffHeapJUnitTest. test080PutGet
      HDFSRegionOperationsOffHeapJUnitTest. test090Delete
      HDFSRegionOperationsOffHeapJUnitTest. test100Invalidate
      HDFSRegionOperationsOffHeapJUnitTest. test110Size
      HDFSRegionOperationsOffHeapJUnitTest. test120KeyIterator
      HDFSRegionOperationsOffHeapJUnitTest. test130EntriesIterator
      HDFSRegionOperationsOffHeapJUnitTest. test140ContainsKey
      HDFSRegionOperationsOffHeapJUnitTest. test150ContainsValue
      HDFSRegionOperationsOffHeapJUnitTest. test160DestroyRegion
      HDFSRegionOperationsOffHeapJUnitTest. test170PutIfAbsent
      HDFSRegionOperationsOffHeapJUnitTest. test180Replace
      HDFSRegionOperationsOffHeapJUnitTest. test190ReplaceKVV
      HDFSRegionOperationsOffHeapJUnitTest. test200Accuracy
      HDFSRegionMBeanAttributeJUnitTest. testStoreUsageStats

      The failure stack trace looks like:

      java.lang.NullPointerException
  at java.lang.ProcessBuilder.start(ProcessBuilder.java:1012)
  at org.apache.hadoop.util.Shell.runCommand(Shell.java:445)
  at org.apache.hadoop.util.Shell.run(Shell.java:418)
  at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:650)
  at org.apache.hadoop.util.Shell.execCommand(Shell.java:739)
  at org.apache.hadoop.util.Shell.execCommand(Shell.java:722)
  at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:633)
  at org.apache.hadoop.fs.FilterFileSystem.setPermission(FilterFileSystem.java:467)
  at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:456)
  at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:424)
  at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:906)
  at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:887)
  at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:784)
  at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:773)
  at com.gemstone.gemfire.cache.hdfs.internal.hoplog.mapreduce.HoplogUtil.exposeCleanupIntervalMillis(HoplogUtil.java:226)
  at com.gemstone.gemfire.cache.hdfs.internal.HDFSStoreImpl.<init>(HDFSStoreImpl.java:135)
  at com.gemstone.gemfire.cache.hdfs.internal.HDFSStoreFactoryImpl.create(HDFSStoreFactoryImpl.java:53)
  at com.gemstone.gemfire.internal.cache.xmlcache.CacheCreation.create(CacheCreation.java:510)
  at com.gemstone.gemfire.internal.cache.xmlcache.CacheXmlParser.create(CacheXmlParser.java:347)
  at com.gemstone.gemfire.internal.cache.GemFireCacheImpl.loadCacheXml(GemFireCacheImpl.java:4301)
  at com.gemstone.gemfire.cache.hdfs.internal.HDFSConfigJUnitTest.testCacheXMLParsing(HDFSConfigJUnitTest.java:180)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:497)
  at junit.framework.TestCase.runTest(TestCase.java:176)
  at junit.framework.TestCase.runBare(TestCase.java:141)
  at junit.framework.TestResult$1.protect(TestResult.java:122)
  at junit.framework.TestResult.runProtected(TestResult.java:142)
  at junit.framework.TestResult.run(TestResult.java:125)
  at junit.framework.TestCase.run(TestCase.java:129)
  at junit.framework.TestSuite.runTest(TestSuite.java:252)
  at junit.framework.TestSuite.run(TestSuite.java:247)
  at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:86)
  at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecuter.runTestClass(JUnitTestClassExecuter.java:86)
  at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecuter.execute(JUnitTestClassExecuter.java:49)
  at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassProcessor.processTestClass(JUnitTestClassProcessor.java:64)
  at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:50)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:497)
  at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
  at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
  at org.gradle.messaging.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:32)
  at org.gradle.messaging.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93)
  at com.sun.proxy.$Proxy2.processTestClass(Unknown Source)
  at org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:106)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:497)
  at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
  at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
  at org.gradle.messaging.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:360)
  at org.gradle.internal.concurrent.DefaultExecutorFactory$StoppableExecutorImpl$1.run(DefaultExecutorFactory.java:64)
  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  at java.lang.Thread.run(Thread.java:745)

      Attachments

        Activity

          People

            Unassigned Unassigned
            klund Kirk Lund
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: