Hadoop Common
  1. Hadoop Common
  2. HADOOP-6099

Allow configuring the IPC module to send pings

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.21.0
    • Component/s: ipc
    • Labels:
      None

      Description

      The IPC Client sets a socketTimeout for the time period specified by the pingInterval and then sends a ping every pingInterval. This means that if a RPC server does not respond to a RPC client, then the RPC client blocks forever. This is a problem for applications that wants to switch quickly from one un-responsive HDFS cluster to a good one.

      1. ipcPing.txt
        2 kB
        dhruba borthakur
      2. ipcPing2.txt
        3 kB
        dhruba borthakur

        Issue Links

          Activity

          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk #22 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk/22/)
          . The RPC module can be configured to not send period pings.
          The default behaviour remains unchanged. (dhruba)

          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk #22 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk/22/ ) . The RPC module can be configured to not send period pings. The default behaviour remains unchanged. (dhruba)
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12412894/ipcPing2.txt
          against trunk revision 792812.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          -1 patch. The patch command could not apply the patch.

          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/562/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12412894/ipcPing2.txt against trunk revision 792812. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/562/console This message is automatically generated.
          Hide
          dhruba borthakur added a comment -

          I just committed this.

          Show
          dhruba borthakur added a comment - I just committed this.
          Hide
          dhruba borthakur added a comment -

          Hudson is stuck, so I ran the tests myself:

          run-test-core:
          [mkdir] Created dir: /mnt/vol/devrs004.snc1/dhruba/raincommon/build/test/data
          [mkdir] Created dir: /mnt/vol/devrs004.snc1/dhruba/raincommon/build/test/logs
          [copy] Copying 1 file to /mnt/vol/devrs004.snc1/dhruba/raincommon/build/test/extraconf
          [junit] Running org.apache.hadoop.cli.TestCLI
          [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.095 sec
          [junit] Running org.apache.hadoop.conf.TestConfiguration
          [junit] Tests run: 12, Failures: 0, Errors: 0, Time elapsed: 0.364 sec
          [junit] Running org.apache.hadoop.conf.TestConfigurationSubclass
          [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.192 sec
          [junit] Running org.apache.hadoop.conf.TestGetInstances
          [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.146 sec
          [junit] Running org.apache.hadoop.filecache.TestDistributedCache
          [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.39 sec
          [junit] Running org.apache.hadoop.fs.TestBlockLocation
          [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.095 sec
          [junit] Running org.apache.hadoop.fs.TestChecksumFileSystem
          [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.263 sec
          [junit] Running org.apache.hadoop.fs.TestDFVariations
          [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.192 sec
          [junit] Running org.apache.hadoop.fs.TestDU
          [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 5.125 sec
          [junit] Running org.apache.hadoop.fs.TestGetFileBlockLocations
          [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.45 sec
          [junit] Running org.apache.hadoop.fs.TestGlobExpander
          [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.018 sec
          [junit] Running org.apache.hadoop.fs.TestLocalDirAllocator
          [junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed: 0.158 sec
          [junit] Running org.apache.hadoop.fs.TestLocalFileSystem
          [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 0.51 sec
          [junit] Running org.apache.hadoop.fs.TestLocalFileSystemPermission
          [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.415 sec
          [junit] Running org.apache.hadoop.fs.TestPath
          [junit] Tests run: 8, Failures: 0, Errors: 0, Time elapsed: 0.03 sec
          [junit] Running org.apache.hadoop.fs.TestTrash
          [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.511 sec
          [junit] Running org.apache.hadoop.fs.TestTruncatedInputBug
          [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.256 sec
          [junit] Running org.apache.hadoop.fs.kfs.TestKosmosFileSystem
          [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.348 sec
          [junit] Running org.apache.hadoop.fs.permission.TestFsPermission
          [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 0.036 sec
          [junit] Running org.apache.hadoop.fs.s3.TestINode
          [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.019 sec
          [junit] Running org.apache.hadoop.fs.s3.TestInMemoryS3FileSystemContract
          [junit] Tests run: 29, Failures: 0, Errors: 0, Time elapsed: 0.621 sec
          [junit] Running org.apache.hadoop.fs.s3.TestS3Credentials
          [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.096 sec
          [junit] Running org.apache.hadoop.fs.s3.TestS3FileSystem
          [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.112 sec
          [junit] Running org.apache.hadoop.fs.s3native.TestInMemoryNativeS3FileSystemContract
          [junit] Tests run: 35, Failures: 0, Errors: 0, Time elapsed: 0.655 sec
          [junit] Running org.apache.hadoop.http.TestGlobalFilter
          [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.704 sec
          [junit] Running org.apache.hadoop.http.TestServletFilter
          [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.729 sec
          [junit] Running org.apache.hadoop.io.TestArrayFile
          [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 3.057 sec
          [junit] Running org.apache.hadoop.io.TestArrayWritable
          [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.099 sec
          [junit] Running org.apache.hadoop.io.TestBloomMapFile
          [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.358 sec
          [junit] Running org.apache.hadoop.io.TestBytesWritable
          [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 0.081 sec
          [junit] Running org.apache.hadoop.io.TestDefaultStringifier
          [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 0.157 sec
          [junit] Running org.apache.hadoop.io.TestEnumSetWritable
          [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.093 sec
          [junit] Running org.apache.hadoop.io.TestGenericWritable
          [junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed: 0.24 sec
          [junit] Running org.apache.hadoop.io.TestMD5Hash
          [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.03 sec
          [junit] Running org.apache.hadoop.io.TestMapFile
          [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.31 sec
          [junit] Running org.apache.hadoop.io.TestMapWritable
          [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.121 sec
          [junit] Running org.apache.hadoop.io.TestSequenceFileSerialization
          [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.312 sec
          [junit] Running org.apache.hadoop.io.TestSetFile
          [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 2.978 sec
          [junit] Running org.apache.hadoop.io.TestSortedMapWritable
          [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.107 sec
          [junit] Running org.apache.hadoop.io.TestText
          [junit] Tests run: 9, Failures: 0, Errors: 0, Time elapsed: 0.569 sec
          [junit] Running org.apache.hadoop.io.TestTextNonUTF8
          [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.019 sec
          [junit] Running org.apache.hadoop.io.TestUTF8
          [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.097 sec
          [junit] Running org.apache.hadoop.io.TestVersionedWritable
          [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.028 sec
          [junit] Running org.apache.hadoop.io.TestWritable
          [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.027 sec
          [junit] Running org.apache.hadoop.io.TestWritableName
          [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 0.098 sec
          [junit] Running org.apache.hadoop.io.TestWritableUtils
          [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.016 sec
          [junit] Running org.apache.hadoop.io.compress.TestCodec
          [junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed: 8.667 sec
          [junit] Running org.apache.hadoop.io.compress.TestCodecFactory
          [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.205 sec
          [junit] Running org.apache.hadoop.io.file.tfile.TestTFile
          [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 1.074 sec
          [junit] Running org.apache.hadoop.io.file.tfile.TestTFileByteArrays
          [junit] Tests run: 25, Failures: 0, Errors: 0, Time elapsed: 2.378 sec
          [junit] Running org.apache.hadoop.io.file.tfile.TestTFileComparators
          [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.261 sec
          [junit] Running org.apache.hadoop.io.file.tfile.TestTFileJClassComparatorByteArrays
          [junit] Tests run: 25, Failures: 0, Errors: 0, Time elapsed: 2.327 sec
          [junit] Running org.apache.hadoop.io.file.tfile.TestTFileLzoCodecsByteArrays
          [junit] Tests run: 25, Failures: 0, Errors: 0, Time elapsed: 0.069 sec
          [junit] Running org.apache.hadoop.io.file.tfile.TestTFileLzoCodecsStreams
          [junit] Tests run: 19, Failures: 0, Errors: 0, Time elapsed: 0.08 sec
          [junit] Running org.apache.hadoop.io.file.tfile.TestTFileNoneCodecsByteArrays
          [junit] Tests run: 25, Failures: 0, Errors: 0, Time elapsed: 1.154 sec
          [junit] Running org.apache.hadoop.io.file.tfile.TestTFileNoneCodecsJClassComparatorByteArrays
          [junit] Tests run: 25, Failures: 0, Errors: 0, Time elapsed: 1.137 sec
          [junit] Running org.apache.hadoop.io.file.tfile.TestTFileNoneCodecsStreams
          [junit] Tests run: 19, Failures: 0, Errors: 0, Time elapsed: 1.36 sec
          [junit] Running org.apache.hadoop.io.file.tfile.TestTFileSeek
          [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 3.536 sec
          [junit] Running org.apache.hadoop.io.file.tfile.TestTFileSeqFileComparison
          [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 10.179 sec
          [junit] Running org.apache.hadoop.io.file.tfile.TestTFileSplit
          [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 4.519 sec
          [junit] Running org.apache.hadoop.io.file.tfile.TestTFileStreams
          [junit] Tests run: 19, Failures: 0, Errors: 0, Time elapsed: 1.355 sec
          [junit] Running org.apache.hadoop.io.file.tfile.TestTFileUnsortedByteArrays
          [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 0.366 sec
          [junit] Running org.apache.hadoop.io.file.tfile.TestVLong
          [junit] Tests run: 9, Failures: 0, Errors: 0, Time elapsed: 1.749 sec
          [junit] Running org.apache.hadoop.io.retry.TestRetryProxy
          [junit] Tests run: 9, Failures: 0, Errors: 0, Time elapsed: 0.18 sec
          [junit] Running org.apache.hadoop.io.serializer.TestWritableSerialization
          [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.038 sec
          [junit] Running org.apache.hadoop.ipc.TestIPC
          [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 12.743 sec
          [junit] Running org.apache.hadoop.ipc.TestIPCServerResponder
          [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 11.831 sec
          [junit] Running org.apache.hadoop.ipc.TestRPC
          [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 32.536 sec
          [junit] Running org.apache.hadoop.log.TestLogLevel
          [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.675 sec
          [junit] Running org.apache.hadoop.metrics.TestMetricsServlet
          [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.048 sec
          [junit] Running org.apache.hadoop.metrics.spi.TestOutputRecord
          [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.017 sec
          [junit] Running org.apache.hadoop.net.TestDNS
          [junit] Tests run: 7, Failures: 0, Errors: 0, Time elapsed: 0.156 sec
          [junit] Running org.apache.hadoop.net.TestScriptBasedMapping
          [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.388 sec
          [junit] Running org.apache.hadoop.net.TestSocketIOWithTimeout
          [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 3.032 sec
          [junit] Running org.apache.hadoop.record.TestBuffer
          [junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed: 0.018 sec
          [junit] Running org.apache.hadoop.record.TestRecordIO
          [junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed: 0.073 sec
          [junit] Running org.apache.hadoop.record.TestRecordVersioning
          [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.052 sec
          [junit] Running org.apache.hadoop.security.TestAccessControlList
          [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.019 sec
          [junit] Running org.apache.hadoop.security.TestAccessToken
          [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.54 sec
          [junit] Running org.apache.hadoop.security.TestUnixUserGroupInformation
          [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 0.279 sec
          [junit] Running org.apache.hadoop.security.authorize.TestConfiguredPolicy
          [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.18 sec
          [junit] Running org.apache.hadoop.util.TestCyclicIteration
          [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.021 sec
          [junit] Running org.apache.hadoop.util.TestGenericsUtil
          [junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed: 0.116 sec
          [junit] Running org.apache.hadoop.util.TestIndexedSort
          [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.716 sec
          [junit] Running org.apache.hadoop.util.TestProcfsBasedProcessTree
          [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 7.54 sec
          [junit] Running org.apache.hadoop.util.TestShell
          [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 4.13 sec
          [junit] Running org.apache.hadoop.util.TestStringUtils
          [junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed: 0.025 sec

          [exec] +1 @author. The patch does not contain any @author tags.
          [exec]
          [exec] +1 tests included. The patch appears to include 3 new or modified tests.
          [exec]
          [exec] +1 javadoc. The javadoc tool did not generate any warning messages.
          [exec]
          [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
          [exec]
          [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
          [exec]
          [exec] -1 release audit. The applied patch generated 265 release audit warnings (more than the trunk's current 263 warnings

          Show
          dhruba borthakur added a comment - Hudson is stuck, so I ran the tests myself: run-test-core: [mkdir] Created dir: /mnt/vol/devrs004.snc1/dhruba/raincommon/build/test/data [mkdir] Created dir: /mnt/vol/devrs004.snc1/dhruba/raincommon/build/test/logs [copy] Copying 1 file to /mnt/vol/devrs004.snc1/dhruba/raincommon/build/test/extraconf [junit] Running org.apache.hadoop.cli.TestCLI [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.095 sec [junit] Running org.apache.hadoop.conf.TestConfiguration [junit] Tests run: 12, Failures: 0, Errors: 0, Time elapsed: 0.364 sec [junit] Running org.apache.hadoop.conf.TestConfigurationSubclass [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.192 sec [junit] Running org.apache.hadoop.conf.TestGetInstances [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.146 sec [junit] Running org.apache.hadoop.filecache.TestDistributedCache [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.39 sec [junit] Running org.apache.hadoop.fs.TestBlockLocation [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.095 sec [junit] Running org.apache.hadoop.fs.TestChecksumFileSystem [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.263 sec [junit] Running org.apache.hadoop.fs.TestDFVariations [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.192 sec [junit] Running org.apache.hadoop.fs.TestDU [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 5.125 sec [junit] Running org.apache.hadoop.fs.TestGetFileBlockLocations [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.45 sec [junit] Running org.apache.hadoop.fs.TestGlobExpander [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.018 sec [junit] Running org.apache.hadoop.fs.TestLocalDirAllocator [junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed: 0.158 sec [junit] Running org.apache.hadoop.fs.TestLocalFileSystem [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 0.51 sec [junit] Running org.apache.hadoop.fs.TestLocalFileSystemPermission [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.415 sec [junit] Running org.apache.hadoop.fs.TestPath [junit] Tests run: 8, Failures: 0, Errors: 0, Time elapsed: 0.03 sec [junit] Running org.apache.hadoop.fs.TestTrash [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.511 sec [junit] Running org.apache.hadoop.fs.TestTruncatedInputBug [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.256 sec [junit] Running org.apache.hadoop.fs.kfs.TestKosmosFileSystem [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.348 sec [junit] Running org.apache.hadoop.fs.permission.TestFsPermission [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 0.036 sec [junit] Running org.apache.hadoop.fs.s3.TestINode [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.019 sec [junit] Running org.apache.hadoop.fs.s3.TestInMemoryS3FileSystemContract [junit] Tests run: 29, Failures: 0, Errors: 0, Time elapsed: 0.621 sec [junit] Running org.apache.hadoop.fs.s3.TestS3Credentials [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.096 sec [junit] Running org.apache.hadoop.fs.s3.TestS3FileSystem [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.112 sec [junit] Running org.apache.hadoop.fs.s3native.TestInMemoryNativeS3FileSystemContract [junit] Tests run: 35, Failures: 0, Errors: 0, Time elapsed: 0.655 sec [junit] Running org.apache.hadoop.http.TestGlobalFilter [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.704 sec [junit] Running org.apache.hadoop.http.TestServletFilter [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.729 sec [junit] Running org.apache.hadoop.io.TestArrayFile [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 3.057 sec [junit] Running org.apache.hadoop.io.TestArrayWritable [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.099 sec [junit] Running org.apache.hadoop.io.TestBloomMapFile [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.358 sec [junit] Running org.apache.hadoop.io.TestBytesWritable [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 0.081 sec [junit] Running org.apache.hadoop.io.TestDefaultStringifier [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 0.157 sec [junit] Running org.apache.hadoop.io.TestEnumSetWritable [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.093 sec [junit] Running org.apache.hadoop.io.TestGenericWritable [junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed: 0.24 sec [junit] Running org.apache.hadoop.io.TestMD5Hash [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.03 sec [junit] Running org.apache.hadoop.io.TestMapFile [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.31 sec [junit] Running org.apache.hadoop.io.TestMapWritable [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.121 sec [junit] Running org.apache.hadoop.io.TestSequenceFileSerialization [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.312 sec [junit] Running org.apache.hadoop.io.TestSetFile [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 2.978 sec [junit] Running org.apache.hadoop.io.TestSortedMapWritable [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.107 sec [junit] Running org.apache.hadoop.io.TestText [junit] Tests run: 9, Failures: 0, Errors: 0, Time elapsed: 0.569 sec [junit] Running org.apache.hadoop.io.TestTextNonUTF8 [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.019 sec [junit] Running org.apache.hadoop.io.TestUTF8 [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.097 sec [junit] Running org.apache.hadoop.io.TestVersionedWritable [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.028 sec [junit] Running org.apache.hadoop.io.TestWritable [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.027 sec [junit] Running org.apache.hadoop.io.TestWritableName [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 0.098 sec [junit] Running org.apache.hadoop.io.TestWritableUtils [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.016 sec [junit] Running org.apache.hadoop.io.compress.TestCodec [junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed: 8.667 sec [junit] Running org.apache.hadoop.io.compress.TestCodecFactory [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.205 sec [junit] Running org.apache.hadoop.io.file.tfile.TestTFile [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 1.074 sec [junit] Running org.apache.hadoop.io.file.tfile.TestTFileByteArrays [junit] Tests run: 25, Failures: 0, Errors: 0, Time elapsed: 2.378 sec [junit] Running org.apache.hadoop.io.file.tfile.TestTFileComparators [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.261 sec [junit] Running org.apache.hadoop.io.file.tfile.TestTFileJClassComparatorByteArrays [junit] Tests run: 25, Failures: 0, Errors: 0, Time elapsed: 2.327 sec [junit] Running org.apache.hadoop.io.file.tfile.TestTFileLzoCodecsByteArrays [junit] Tests run: 25, Failures: 0, Errors: 0, Time elapsed: 0.069 sec [junit] Running org.apache.hadoop.io.file.tfile.TestTFileLzoCodecsStreams [junit] Tests run: 19, Failures: 0, Errors: 0, Time elapsed: 0.08 sec [junit] Running org.apache.hadoop.io.file.tfile.TestTFileNoneCodecsByteArrays [junit] Tests run: 25, Failures: 0, Errors: 0, Time elapsed: 1.154 sec [junit] Running org.apache.hadoop.io.file.tfile.TestTFileNoneCodecsJClassComparatorByteArrays [junit] Tests run: 25, Failures: 0, Errors: 0, Time elapsed: 1.137 sec [junit] Running org.apache.hadoop.io.file.tfile.TestTFileNoneCodecsStreams [junit] Tests run: 19, Failures: 0, Errors: 0, Time elapsed: 1.36 sec [junit] Running org.apache.hadoop.io.file.tfile.TestTFileSeek [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 3.536 sec [junit] Running org.apache.hadoop.io.file.tfile.TestTFileSeqFileComparison [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 10.179 sec [junit] Running org.apache.hadoop.io.file.tfile.TestTFileSplit [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 4.519 sec [junit] Running org.apache.hadoop.io.file.tfile.TestTFileStreams [junit] Tests run: 19, Failures: 0, Errors: 0, Time elapsed: 1.355 sec [junit] Running org.apache.hadoop.io.file.tfile.TestTFileUnsortedByteArrays [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 0.366 sec [junit] Running org.apache.hadoop.io.file.tfile.TestVLong [junit] Tests run: 9, Failures: 0, Errors: 0, Time elapsed: 1.749 sec [junit] Running org.apache.hadoop.io.retry.TestRetryProxy [junit] Tests run: 9, Failures: 0, Errors: 0, Time elapsed: 0.18 sec [junit] Running org.apache.hadoop.io.serializer.TestWritableSerialization [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.038 sec [junit] Running org.apache.hadoop.ipc.TestIPC [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 12.743 sec [junit] Running org.apache.hadoop.ipc.TestIPCServerResponder [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 11.831 sec [junit] Running org.apache.hadoop.ipc.TestRPC [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 32.536 sec [junit] Running org.apache.hadoop.log.TestLogLevel [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.675 sec [junit] Running org.apache.hadoop.metrics.TestMetricsServlet [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.048 sec [junit] Running org.apache.hadoop.metrics.spi.TestOutputRecord [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.017 sec [junit] Running org.apache.hadoop.net.TestDNS [junit] Tests run: 7, Failures: 0, Errors: 0, Time elapsed: 0.156 sec [junit] Running org.apache.hadoop.net.TestScriptBasedMapping [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.388 sec [junit] Running org.apache.hadoop.net.TestSocketIOWithTimeout [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 3.032 sec [junit] Running org.apache.hadoop.record.TestBuffer [junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed: 0.018 sec [junit] Running org.apache.hadoop.record.TestRecordIO [junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed: 0.073 sec [junit] Running org.apache.hadoop.record.TestRecordVersioning [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.052 sec [junit] Running org.apache.hadoop.security.TestAccessControlList [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.019 sec [junit] Running org.apache.hadoop.security.TestAccessToken [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.54 sec [junit] Running org.apache.hadoop.security.TestUnixUserGroupInformation [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 0.279 sec [junit] Running org.apache.hadoop.security.authorize.TestConfiguredPolicy [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.18 sec [junit] Running org.apache.hadoop.util.TestCyclicIteration [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.021 sec [junit] Running org.apache.hadoop.util.TestGenericsUtil [junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed: 0.116 sec [junit] Running org.apache.hadoop.util.TestIndexedSort [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.716 sec [junit] Running org.apache.hadoop.util.TestProcfsBasedProcessTree [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 7.54 sec [junit] Running org.apache.hadoop.util.TestShell [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 4.13 sec [junit] Running org.apache.hadoop.util.TestStringUtils [junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed: 0.025 sec [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to include 3 new or modified tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings. [exec] [exec] -1 release audit. The applied patch generated 265 release audit warnings (more than the trunk's current 263 warnings
          Hide
          dhruba borthakur added a comment -

          Retry test

          Show
          dhruba borthakur added a comment - Retry test
          Hide
          dhruba borthakur added a comment -

          The core test failed beucase of this reason

          Warning: bad ps syntax, perhaps a bogus '-'? See http://procps.sf.net/faq.html
          [exec] /usr/bin/kill -9 22462
          [exec] /usr/bin/xargs: /usr/bin/kill: No such file or directory
          [exec] /home/hudson/tools/ant/latest/bin/ant -Dversion=791937_HADOOP-6099_PATCH-12412894 -DHadoopPatchProcess= -Dtest.junit.output.format=xml -Dtest.output=yes -Dcompile.c+=yes -Dforrest.home=/home/nigel/tools/forrest/latest -Djava5.home=/home/hudson/tools/java/latest1.5 create-c+-configure test-core
          [exec] Buildfile: build.xml
          [exec]
          [exec] BUILD FAILED
          [exec] Target "create-c++-configure" d

          Is this a problem with the HadoopQA framework?

          Show
          dhruba borthakur added a comment - The core test failed beucase of this reason Warning: bad ps syntax, perhaps a bogus '-'? See http://procps.sf.net/faq.html [exec] /usr/bin/kill -9 22462 [exec] /usr/bin/xargs: /usr/bin/kill: No such file or directory [exec] /home/hudson/tools/ant/latest/bin/ant -Dversion=791937_ HADOOP-6099 _PATCH-12412894 -DHadoopPatchProcess= -Dtest.junit.output.format=xml -Dtest.output=yes -Dcompile.c+ =yes -Dforrest.home=/home/nigel/tools/forrest/latest -Djava5.home=/home/hudson/tools/java/latest1.5 create-c +-configure test-core [exec] Buildfile: build.xml [exec] [exec] BUILD FAILED [exec] Target "create-c++-configure" d Is this a problem with the HadoopQA framework?
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12412894/ipcPing2.txt
          against trunk revision 791937.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          -1 release audit. The applied patch generated 260 release audit warnings (more than the trunk's current 258 warnings).

          -1 core tests. The patch failed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/558/testReport/
          Release audit warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/558/artifact/trunk/current/releaseAuditDiffWarnings.txt
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/558/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/558/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/558/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12412894/ipcPing2.txt against trunk revision 791937. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. -1 release audit. The applied patch generated 260 release audit warnings (more than the trunk's current 258 warnings). -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/558/testReport/ Release audit warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/558/artifact/trunk/current/releaseAuditDiffWarnings.txt Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/558/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/558/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/558/console This message is automatically generated.
          Hide
          dhruba borthakur added a comment -

          Thanks for the review Konstantin. I also attached a unit test.

          Show
          dhruba borthakur added a comment - Thanks for the review Konstantin. I also attached a unit test.
          Hide
          Konstantin Shvachko added a comment -

          Yes I think this is fine.
          +1

          Show
          Konstantin Shvachko added a comment - Yes I think this is fine. +1
          Hide
          dhruba borthakur added a comment -

          @Konstantin: your assumption is right. is this ok with you?

          Show
          dhruba borthakur added a comment - @Konstantin: your assumption is right. is this ok with you?
          Hide
          Konstantin Shvachko added a comment -

          I see. So you want to be able to set
          ipc.client.ping = false - no ping, and at the same time
          ipc.ping.interval = 20, which is now treated not as ping interva because there are no pingsl, but as a client timeout.

          Show
          Konstantin Shvachko added a comment - I see. So you want to be able to set ipc.client.ping = false - no ping, and at the same time ipc.ping.interval = 20, which is now treated not as ping interva because there are no pingsl, but as a client timeout.
          Hide
          dhruba borthakur added a comment -

          Konstantin commented on this issue via HDFS-278:

          "why do you need additional configuration parameter ipc.client.ping?
          Can you just use set ipc.ping.interval = -1 and treat it as no ping?
          Then by default since ipc.ping.interval parameter is unofficial it will still equal 1 min,
          and if you want to turn pings off you will need to specify it. Everything looks consistent with your plan.
          I mean less parameters is better, right?"

          I agree that your suggestion is simpler. But the reason I need another parameter is because I would like the dfsclient to abort if the server is unresponsive for only 20 seconds or so. Your approach does not let me change the default timeout value of 1 minute. Thats the reason I need a separate config parameter. Does it make sense?

          Show
          dhruba borthakur added a comment - Konstantin commented on this issue via HDFS-278 : "why do you need additional configuration parameter ipc.client.ping? Can you just use set ipc.ping.interval = -1 and treat it as no ping? Then by default since ipc.ping.interval parameter is unofficial it will still equal 1 min, and if you want to turn pings off you will need to specify it. Everything looks consistent with your plan. I mean less parameters is better, right?" I agree that your suggestion is simpler. But the reason I need another parameter is because I would like the dfsclient to abort if the server is unresponsive for only 20 seconds or so. Your approach does not let me change the default timeout value of 1 minute. Thats the reason I need a separate config parameter. Does it make sense?
          Hide
          dhruba borthakur added a comment -

          If ipc.client.ping is set to false, then the ipc Client does not send periodic pings to IPC server. The default is set to true to maintain compatibility with pre-existing semantics of sending periodic pings.

          Show
          dhruba borthakur added a comment - If ipc.client.ping is set to false, then the ipc Client does not send periodic pings to IPC server. The default is set to true to maintain compatibility with pre-existing semantics of sending periodic pings.

            People

            • Assignee:
              dhruba borthakur
              Reporter:
              dhruba borthakur
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development