XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.2.2, 3.3.1, 3.4.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Replacing Usage of guava.Supplier<> are in Unit tests GenereicTestUtils.waitFor() in Hadoop project.

      • To make things more convenient for reviewers, I decided:
        • Not to replace Object instantiation by lambda expressions because this will increase the patch size significantly and require code adjustments to pass the checkstyle scripts.
        • Not to refactor the imports because this will make reading the patch more difficult.
      • Merge should be done to the following branches: trunk, branch-3.3, branch-3.2, branch-3.1

      The task is straightforward because java.util.Supplier has the same API as guava.Supplier<> and the vast majority of usage comes from Test-units.
      Therefore, we need only to do the following a "one-line" change in all 147 files.

       
      -import com.google.common.base.Supplier;
      +import java.util.function.Supplier;
      

      The code change needs to be applied to the following list of files:

       
      Targets 
          Occurrences of 'com.google.common.base.Supplier' in project with mask '*.java' 
      Found Occurrences (146 usages found) 
          org.apache.hadoop.conf (1 usage found) 
              TestReconfiguration.java (1 usage found) 
                  21 import com.google.common.base.Supplier; 
          org.apache.hadoop.crypto.key.kms.server (1 usage found) 
              TestKMS.java (1 usage found) 
                  20 import com.google.common.base.Supplier; 
          org.apache.hadoop.fs (2 usages found) 
              FCStatisticsBaseTest.java (1 usage found) 
                  40 import com.google.common.base.Supplier; 
              TestEnhancedByteBufferAccess.java (1 usage found) 
                  75 import com.google.common.base.Supplier; 
          org.apache.hadoop.fs.viewfs (1 usage found) 
              TestViewFileSystemWithTruncate.java (1 usage found) 
                  23 import com.google.common.base.Supplier; 
          org.apache.hadoop.ha (1 usage found) 
              TestZKFailoverController.java (1 usage found) 
                  25 import com.google.common.base.Supplier; 
          org.apache.hadoop.hdfs (20 usages found) 
              DFSTestUtil.java (1 usage found) 
                  79 import com.google.common.base.Supplier; 
              MiniDFSCluster.java (1 usage found) 
                  78 import com.google.common.base.Supplier; 
              TestBalancerBandwidth.java (1 usage found) 
                  29 import com.google.common.base.Supplier; 
              TestClientProtocolForPipelineRecovery.java (1 usage found) 
                  30 import com.google.common.base.Supplier; 
              TestDatanodeRegistration.java (1 usage found) 
                  44 import com.google.common.base.Supplier; 
              TestDataTransferKeepalive.java (1 usage found) 
                  47 import com.google.common.base.Supplier; 
              TestDeadNodeDetection.java (1 usage found) 
                  20 import com.google.common.base.Supplier; 
              TestDecommission.java (1 usage found) 
                  41 import com.google.common.base.Supplier; 
              TestDFSShell.java (1 usage found) 
                  37 import com.google.common.base.Supplier; 
              TestEncryptedTransfer.java (1 usage found) 
                  35 import com.google.common.base.Supplier; 
              TestEncryptionZonesWithKMS.java (1 usage found) 
                  22 import com.google.common.base.Supplier; 
              TestFileCorruption.java (1 usage found) 
                  21 import com.google.common.base.Supplier; 
              TestLeaseRecovery2.java (1 usage found) 
                  32 import com.google.common.base.Supplier; 
              TestLeaseRecoveryStriped.java (1 usage found) 
                  21 import com.google.common.base.Supplier; 
              TestMaintenanceState.java (1 usage found) 
                  63 import com.google.common.base.Supplier; 
              TestPread.java (1 usage found) 
                  61 import com.google.common.base.Supplier; 
              TestQuota.java (1 usage found) 
                  39 import com.google.common.base.Supplier; 
              TestReplaceDatanodeOnFailure.java (1 usage found) 
                  20 import com.google.common.base.Supplier; 
              TestReplication.java (1 usage found) 
                  27 import com.google.common.base.Supplier; 
              TestSafeMode.java (1 usage found) 
                  62 import com.google.common.base.Supplier; 
          org.apache.hadoop.hdfs.client.impl (2 usages found) 
              TestBlockReaderLocalMetrics.java (1 usage found) 
                  20 import com.google.common.base.Supplier; 
              TestLeaseRenewer.java (1 usage found) 
                  20 import com.google.common.base.Supplier; 
          org.apache.hadoop.hdfs.qjournal (1 usage found) 
              MiniJournalCluster.java (1 usage found) 
                  31 import com.google.common.base.Supplier; 
          org.apache.hadoop.hdfs.qjournal.client (1 usage found) 
              TestIPCLoggerChannel.java (1 usage found) 
                  43 import com.google.common.base.Supplier; 
          org.apache.hadoop.hdfs.qjournal.server (1 usage found) 
              TestJournalNodeSync.java (1 usage found) 
                  20 import com.google.common.base.Supplier; 
          org.apache.hadoop.hdfs.server.blockmanagement (7 usages found) 
              TestBlockManagerSafeMode.java (1 usage found) 
                  20 import com.google.common.base.Supplier; 
              TestBlockReportRateLimiting.java (1 usage found) 
                  25 import com.google.common.base.Supplier; 
              TestNameNodePrunesMissingStorages.java (1 usage found) 
                  21 import com.google.common.base.Supplier; 
              TestPendingInvalidateBlock.java (1 usage found) 
                  43 import com.google.common.base.Supplier; 
              TestPendingReconstruction.java (1 usage found) 
                  34 import com.google.common.base.Supplier; 
              TestRBWBlockInvalidation.java (1 usage found) 
                  49 import com.google.common.base.Supplier; 
              TestSlowDiskTracker.java (1 usage found) 
                  48 import com.google.common.base.Supplier; 
          org.apache.hadoop.hdfs.server.datanode (13 usages found) 
              DataNodeTestUtils.java (1 usage found) 
                  40 import com.google.common.base.Supplier; 
              TestBlockRecovery.java (1 usage found) 
                  120 import com.google.common.base.Supplier; 
              TestBlockScanner.java (1 usage found) 
                  43 import com.google.common.base.Supplier; 
              TestBPOfferService.java (1 usage found) 
                  92 import com.google.common.base.Supplier; 
              TestCorruptMetadataFile.java (1 usage found) 
                  20 import com.google.common.base.Supplier; 
              TestDataNodeLifeline.java (1 usage found) 
                  74 import com.google.common.base.Supplier; 
              TestDataNodeMetrics.java (1 usage found) 
                  37 import com.google.common.base.Supplier; 
              TestDataNodeMetricsLogger.java (1 usage found) 
                  57 import com.google.common.base.Supplier; 
              TestDataNodeMultipleRegistrations.java (1 usage found) 
                  33 import com.google.common.base.Supplier; 
              TestDataNodeMXBean.java (1 usage found) 
                  31 import com.google.common.base.Supplier; 
              TestDatanodeProtocolRetryPolicy.java (1 usage found) 
                  32 import com.google.common.base.Supplier; 
              TestDataNodeVolumeFailure.java (1 usage found) 
                  94 import com.google.common.base.Supplier; 
              TestDiskError.java (1 usage found) 
                  31 import com.google.common.base.Supplier; 
          org.apache.hadoop.hdfs.server.datanode.checker (1 usage found) 
              TestThrottledAsyncChecker.java (1 usage found) 
                  21 import com.google.common.base.Supplier; 
          org.apache.hadoop.hdfs.server.datanode.fsdataset.impl (7 usages found) 
              TestCacheByPmemMappableBlockLoader.java (1 usage found) 
                  65 import com.google.common.base.Supplier; 
              TestFsDatasetCache.java (1 usage found) 
                  94 import com.google.common.base.Supplier; 
              TestFsDatasetImpl.java (1 usage found) 
                  20 import com.google.common.base.Supplier; 
              TestFsVolumeList.java (1 usage found) 
                  20 import com.google.common.base.Supplier; 
              TestLazyPersistLockedMemory.java (1 usage found) 
                  22 import com.google.common.base.Supplier; 
              TestPmemCacheRecovery.java (1 usage found) 
                  65 import com.google.common.base.Supplier; 
              TestSpaceReservation.java (1 usage found) 
                  21 import com.google.common.base.Supplier; 
          org.apache.hadoop.hdfs.server.datanode.metrics (1 usage found) 
              TestDataNodeOutlierDetectionViaMetrics.java (1 usage found) 
                  21 import com.google.common.base.Supplier; 
          org.apache.hadoop.hdfs.server.datanode.web.webhdfs (1 usage found) 
              TestDataNodeUGIProvider.java (1 usage found) 
                  54 import com.google.common.base.Supplier; 
          org.apache.hadoop.hdfs.server.diskbalancer (2 usages found) 
              TestDiskBalancer.java (1 usage found) 
                  20 import com.google.common.base.Supplier; 
              TestDiskBalancerWithMockMover.java (1 usage found) 
                  23 import com.google.common.base.Supplier; 
          org.apache.hadoop.hdfs.server.federation (1 usage found) 
              FederationTestUtils.java (1 usage found) 
                  95 import com.google.common.base.Supplier; 
          org.apache.hadoop.hdfs.server.federation.router (4 usages found) 
              TestRouterAdminCLI.java (1 usage found) 
                  67 import com.google.common.base.Supplier; 
              TestRouterQuota.java (1 usage found) 
                  77 import com.google.common.base.Supplier; 
              TestRouterRpc.java (1 usage found) 
                  131 import com.google.common.base.Supplier; 
              TestRouterRPCClientRetries.java (1 usage found) 
                  57 import com.google.common.base.Supplier; 
          org.apache.hadoop.hdfs.server.mover (1 usage found) 
              TestMover.java (1 usage found) 
                  98 import com.google.common.base.Supplier; 
          org.apache.hadoop.hdfs.server.namenode (17 usages found) 
              TestAddStripedBlockInFBR.java (1 usage found) 
                  43 import com.google.common.base.Supplier; 
              TestBackupNode.java (1 usage found) 
                  59 import com.google.common.base.Supplier; 
              TestCacheDirectives.java (1 usage found) 
                  99 import com.google.common.base.Supplier; 
              TestCheckpoint.java (1 usage found) 
                  98 import com.google.common.base.Supplier; 
              TestDeadDatanode.java (1 usage found) 
                  20 import com.google.common.base.Supplier; 
              TestEditLogAutoroll.java (1 usage found) 
                  49 import com.google.common.base.Supplier; 
              TestEditLogRace.java (1 usage found) 
                  41 import com.google.common.base.Supplier; 
              TestFsck.java (1 usage found) 
                  62 import com.google.common.base.Supplier; 
              TestFSNamesystemLock.java (1 usage found) 
                  21 import com.google.common.base.Supplier; 
              TestMetaSave.java (1 usage found) 
                  33 import com.google.common.base.Supplier; 
              TestNameNodeMetadataConsistency.java (1 usage found) 
                  33 import com.google.common.base.Supplier; 
              TestNameNodeMetricsLogger.java (1 usage found) 
                  21 import com.google.common.base.Supplier; 
              TestNameNodeMXBean.java (1 usage found) 
                  21 import com.google.common.base.Supplier; 
              TestNameNodeStatusMXBean.java (1 usage found) 
                  20 import com.google.common.base.Supplier; 
              TestPersistentStoragePolicySatisfier.java (1 usage found) 
                  40 import com.google.common.base.Supplier; 
              TestReencryption.java (1 usage found) 
                  34 import com.google.common.base.Supplier; 
              TestUpgradeDomainBlockPlacementPolicy.java (1 usage found) 
                  52 import com.google.common.base.Supplier; 
          org.apache.hadoop.hdfs.server.namenode.ha (11 usages found) 
              HATestUtil.java (1 usage found) 
                  64 import com.google.common.base.Supplier; 
              TestBootstrapStandby.java (1 usage found) 
                  30 import com.google.common.base.Supplier; 
              TestDNFencing.java (1 usage found) 
                  32 import com.google.common.base.Supplier; 
              TestDNFencingWithReplication.java (1 usage found) 
                  39 import com.google.common.base.Supplier; 
              TestEditLogTailer.java (1 usage found) 
                  65 import com.google.common.base.Supplier; 
              TestHASafeMode.java (1 usage found) 
                  75 import com.google.common.base.Supplier; 
              TestPendingCorruptDnMessages.java (1 usage found) 
                  44 import com.google.common.base.Supplier; 
              TestPipelinesFailover.java (1 usage found) 
                  65 import com.google.common.base.Supplier; 
              TestStandbyCheckpoints.java (1 usage found) 
                  20 import com.google.common.base.Supplier; 
              TestStandbyInProgressTail.java (1 usage found) 
                  52 import com.google.common.base.Supplier; 
              TestStandbyIsHot.java (1 usage found) 
                  46 import com.google.common.base.Supplier; 
          org.apache.hadoop.hdfs.server.namenode.snapshot (1 usage found) 
              TestRandomOpsWithSnapshots.java (1 usage found) 
                  20 import com.google.common.base.Supplier; 
          org.apache.hadoop.hdfs.server.namenode.sps (1 usage found) 
              TestStoragePolicySatisfierWithStripedFile.java (1 usage found) 
                  55 import com.google.common.base.Supplier; 
          org.apache.hadoop.hdfs.server.sps (1 usage found) 
              TestExternalStoragePolicySatisfier.java (1 usage found) 
                  101 import com.google.common.base.Supplier; 
          org.apache.hadoop.hdfs.shortcircuit (1 usage found) 
              TestShortCircuitCache.java (1 usage found) 
                  93 import com.google.common.base.Supplier; 
          org.apache.hadoop.hdfs.tools (2 usages found) 
              TestDFSAdmin.java (1 usage found) 
                  28 import com.google.common.base.Supplier; 
              TestDFSZKFailoverController.java (1 usage found) 
                  58 import com.google.common.base.Supplier; 
          org.apache.hadoop.http (1 usage found) 
              TestSSLHttpServerConfigs.java (1 usage found) 
                  21 import com.google.common.base.Supplier; 
          org.apache.hadoop.ipc (1 usage found) 
              TestIPC.java (1 usage found) 
                  101 import com.google.common.base.Supplier; 
          org.apache.hadoop.mapred (2 usages found) 
              TestTaskAttemptListenerImpl.java (1 usage found) 
                  20 import com.google.common.base.Supplier; 
              UtilsForTests.java (1 usage found) 
                  64 import com.google.common.base.Supplier; 
          org.apache.hadoop.mapreduce.v2.app (4 usages found) 
              TestFetchFailure.java (1 usage found) 
                  29 import com.google.common.base.Supplier; 
              TestMRApp.java (1 usage found) 
                  31 import com.google.common.base.Supplier; 
              TestRecovery.java (1 usage found) 
                  31 import com.google.common.base.Supplier; 
              TestTaskHeartbeatHandler.java (1 usage found) 
                  28 import com.google.common.base.Supplier; 
          org.apache.hadoop.mapreduce.v2.app.rm (1 usage found) 
              TestRMContainerAllocator.java (1 usage found) 
                  156 import com.google.common.base.Supplier; 
          org.apache.hadoop.mapreduce.v2.hs (1 usage found) 
              TestJHSDelegationTokenSecretManager.java (1 usage found) 
                  30 import com.google.common.base.Supplier; 
          org.apache.hadoop.metrics2.impl (1 usage found) 
              TestMetricsSystemImpl.java (1 usage found) 
                  42 import com.google.common.base.Supplier; 
          org.apache.hadoop.metrics2.lib (1 usage found) 
              TestMutableRollingAverages.java (1 usage found) 
                  20 import com.google.common.base.Supplier; 
          org.apache.hadoop.security (1 usage found) 
              TestGroupsCaching.java (1 usage found) 
                  37 import com.google.common.base.Supplier; 
          org.apache.hadoop.security.ssl (1 usage found) 
              TestReloadingX509TrustManager.java (1 usage found) 
                  24 import com.google.common.base.Supplier; 
          org.apache.hadoop.security.token.delegation (1 usage found) 
              TestZKDelegationTokenSecretManager.java (1 usage found) 
                  27 import com.google.common.base.Supplier; 
          org.apache.hadoop.test (2 usages found) 
              GenericTestUtils.java (1 usage found) 
                  64 import com.google.common.base.Supplier; 
              TestGenericTestUtils.java (1 usage found) 
                  26 import com.google.common.base.Supplier; 
          org.apache.hadoop.tracing (1 usage found) 
              SetSpanReceiver.java (1 usage found) 
                  20 import com.google.common.base.Supplier; 
          org.apache.hadoop.util (1 usage found) 
              TestShell.java (1 usage found) 
                  20 import com.google.common.base.Supplier; 
          org.apache.hadoop.yarn.applications.distributedshell (1 usage found) 
              TestDistributedShell.java (1 usage found) 
                  43 import com.google.common.base.Supplier; 
          org.apache.hadoop.yarn.client (1 usage found) 
              TestRMFailover.java (1 usage found) 
                  64 import com.google.common.base.Supplier; 
          org.apache.hadoop.yarn.client.api.impl (1 usage found) 
              TestYarnClientWithReservation.java (1 usage found) 
                  20 import com.google.common.base.Supplier; 
          org.apache.hadoop.yarn.server.nodemanager.containermanager (1 usage found) 
              TestContainerManager.java (1 usage found) 
                  51 import com.google.common.base.Supplier; 
          org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher (1 usage found) 
              TestContainerLaunch.java (1 usage found) 
                  57 import com.google.common.base.Supplier; 
          org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer (1 usage found) 
              TestContainerLocalizer.java (1 usage found) 
                  97 import com.google.common.base.Supplier; 
          org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation (1 usage found) 
              TestLogAggregationService.java (1 usage found) 
                  150 import com.google.common.base.Supplier; 
          org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor (1 usage found) 
              TestContainersMonitor.java (1 usage found) 
                  40 import com.google.common.base.Supplier; 
          org.apache.hadoop.yarn.server.nodemanager.logaggregation.tracker (1 usage found) 
              TestNMLogAggregationStatusTracker.java (1 usage found) 
                  24 import com.google.common.base.Supplier; 
          org.apache.hadoop.yarn.server.resourcemanager (6 usages found) 
              TestApplicationMasterLauncher.java (1 usage found) 
                  95 import com.google.common.base.Supplier; 
              TestLeaderElectorService.java (1 usage found) 
                  21 import com.google.common.base.Supplier; 
              TestRM.java (1 usage found) 
                  21 import com.google.common.base.Supplier; 
              TestRMHA.java (1 usage found) 
                  21 import com.google.common.base.Supplier; 
              TestRMRestart.java (1 usage found) 
                  137 import com.google.common.base.Supplier; 
              TestWorkPreservingRMRestart.java (1 usage found) 
                  21 import com.google.common.base.Supplier; 
          org.apache.hadoop.yarn.server.resourcemanager.recovery (1 usage found) 
              TestZKRMStateStore.java (1 usage found) 
                  75 import com.google.common.base.Supplier; 
          org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity (1 usage found) 
              TestCapacityScheduler.java (1 usage found) 
                  192 import com.google.common.base.Supplier; 
          org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair (1 usage found)
              TestContinuousScheduling.java (1 usage found) 
                  21 import com.google.common.base.Supplier; 
          org.apache.hadoop.yarn.server.resourcemanager.security (2 usages found) 
              TestDelegationTokenRenewer.java (1 usage found) 
                  117 import com.google.common.base.Supplier; 
              TestRMDelegationTokens.java (1 usage found) 
                  29 import com.google.common.base.Supplier; 
          org.apache.hadoop.yarn.server.router.webapp (1 usage found) 
              TestRouterWebServicesREST.java (1 usage found) 
                  135 import com.google.common.base.Supplier; 
          org.apache.hadoop.yarn.server.webproxy.amfilter (1 usage found) 
              TestAmFilter.java (1 usage found) 
                  53 import com.google.common.base.Supplier; 
          org.apache.hadoop.yarn.service (1 usage found) 
              MockServiceAM.java (1 usage found) 
                  21 import com.google.common.base.Supplier; 
          org.apache.hadoop.yarn.sls.nodemanager (1 usage found) 
              TestNMSimulator.java (1 usage found) 
                  20 import com.google.common.base.Supplier; 
      
      

        Attachments

        1. HADOOP-17100.001.patch
          118 kB
          Ahmed Hussein
        2. HADOOP-17100.002.patch
          26 kB
          Ahmed Hussein
        3. HADOOP-17100.003.patch
          27 kB
          Ahmed Hussein
        4. HADOOP-17100.006.patch
          120 kB
          Ahmed Hussein
        5. HADOOP-17100.branch-3.1.006.patch
          115 kB
          Ahmed Hussein
        6. HADOOP-17100.branch-3.2.006.patch
          117 kB
          Ahmed Hussein
        7. HADOOP-17100.branch-3.3.006.patch
          119 kB
          Ahmed Hussein

          Issue Links

            Activity

              People

              • Assignee:
                ahussein Ahmed Hussein
                Reporter:
                ahussein Ahmed Hussein
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: