Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Cannot Reproduce
    • Affects Version/s: 1.4.4
    • Fix Version/s: None
    • Component/s: None
    • Labels:
    • Environment:

      1.4.5-SNAPSHOT, sha: 0f23245
      cdh 4.5.0

      Description

      In the "Table Problems" page of the monitor, errors started with:
      java.io.IOException: java.io.FileNotFoundException: File /accumulo/recovery/48bad96b-4e57-4989-80b8-4844268e931a.recovered does not exist.

      Then there are a bunch (50+) errors that look like:
      java.io.IOException: Unable to find recovery files for extent 1;68c059;68b05af logEntry: 1;68c059; 48bad96b-4e57-4989-80b8-4844268e931a (1539)
      but with different extents and log entries (same uuid, which makes sense).

      Meanwhile, on the "Recent Logs" page there's a bunch of 10.0.0.1:10011 reports assignment failed for tablet 1;68c059;68b05af. The extents all match up with the other page, pairwise.

      About 25 minutes later, there is

      Unable to cleanly close logger 10.0.0.1:11224: org.apache.thrift.transport.TTransportException: java.io.IOException: Broken pipe
      Unable to connect to 10.0.0.1:11224: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused
      unable to get tablet server status 10.0.0.1:10011[143d763f42201b5] java.net.ConnectException: Connection refused
      

      After that I get the same error from ACCUMULO-2150.

        Activity

        Mike Drob created issue -
        Mike Drob made changes -
        Field Original Value New Value
        Description In the "Table Problems" page of the monitor, errors started with: {{java.io.IOException: java.io.FileNotFoundException: File /accumulo/recovery/48bad96b-4e57-4989-80b8-4844268e931a.recovered does not exist.}}
        Then there are a bunch (50+) errors that look like: {{java.io.IOException: Unable to find recovery files for extent 1;68c059;68b05af logEntry: 1;68c059; 48bad96b-4e57-4989-80b8-4844268e931a (1539)}} but with different extents and log entries (same uuid, which makes sense).

        Meanwhile, on the "Recent Logs" page there's a bunch of {{10.0.0.1:10011 reports assignment failed for tablet 1;68c059;68b05af}}. The extents all match up with the other page, pairwise.

        About 25 minutes later, there is
        {{Unable to cleanly close logger 10.0.0.1:11224: org.apache.thrift.transport.TTransportException: java.io.IOException: Broken pipe}}
        {{Unable to connect to 10.0.0.1:11224: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused}}
        {{unable to get tablet server status 10.0.0.1:10011[143d763f42201b5] java.net.ConnectException: Connection refused}}

        After which I get the same error from ACCUMULO-2150.
        In the "Table Problems" page of the monitor, errors started with:
        {{java.io.IOException: java.io.FileNotFoundException: File /accumulo/recovery/48bad96b-4e57-4989-80b8-4844268e931a.recovered does not exist.}}

        Then there are a bunch (50+) errors that look like:
        {{java.io.IOException: Unable to find recovery files for extent 1;68c059;68b05af logEntry: 1;68c059; 48bad96b-4e57-4989-80b8-4844268e931a (1539)}}
        but with different extents and log entries (same uuid, which makes sense).

        Meanwhile, on the "Recent Logs" page there's a bunch of {{10.0.0.1:10011 reports assignment failed for tablet 1;68c059;68b05af}}. The extents all match up with the other page, pairwise.

        About 25 minutes later, there is
        {noformat}
        Unable to cleanly close logger 10.0.0.1:11224: org.apache.thrift.transport.TTransportException: java.io.IOException: Broken pipe
        Unable to connect to 10.0.0.1:11224: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused
        unable to get tablet server status 10.0.0.1:10011[143d763f42201b5] java.net.ConnectException: Connection refused
        {noformat}

        After that I get the same error from ACCUMULO-2150.
        Mike Drob made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Won't Fix [ 2 ]
        Mike Drob made changes -
        Resolution Won't Fix [ 2 ]
        Status Resolved [ 5 ] Reopened [ 4 ]
        Mike Drob made changes -
        Status Reopened [ 4 ] Resolved [ 5 ]
        Resolution Cannot Reproduce [ 5 ]

          People

          • Assignee:
            Unassigned
            Reporter:
            Mike Drob
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development