[CASSANDRA-5129] newly bootstrapping nodes hang indefinitely in STATUS:BOOT while JOINING cluster

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: 1.2.2

Environment

Ubuntu 12.04

• Severity: Normal
• Assignee: Brandon Williams Assign to me
• Reporter: Michael Kjellman

Authors

Brandon Williams

Reviewers

Michael Kjellman

• Created: 08/Jan/13 06:27
• Updated: 16/Apr/19 09:32
• Resolved: 14/Feb/13 05:48

Description

bootstrapping a new node causes it to hang indefinitely in STATUS:BOOT

Nodes streaming to the new node report

Mode: NORMAL
 Nothing streaming to /10.8.30.16
Not receiving any streams.
Pool Name                    Active   Pending      Completed
Commands                        n/a         0        1843990
Responses                       n/a         2         661750

the node being streamed to stuck in the JOINING state reports:

Mode: JOINING
Not sending any streams.
 Nothing streaming from /10.8.30.103
 Nothing streaming from /10.8.30.102
Pool Name                    Active   Pending      Completed
Commands                        n/a         0             10
Responses                       n/a         0         613577

it appears that the nodes in the "nothing streaming" state never sends a "finished streaming" to the joining node.

no exceptions are thrown during the streaming on either node while the node is in this state.

/10.8.30.16
  NET_VERSION:6
  RELEASE_VERSION:1.2.0
  STATUS:BOOT,127605887595351923798765477786913079289
  RACK:RAC1
  RPC_ADDRESS:0.0.0.0
  DC:DC1
  SCHEMA:5cd8420d-ce3c-3625-8293-67558a24816b
  HOST_ID:e20817ce-7454-4dc4-a1c6-b1dec35c4491
  LOAD:1.11824041581E11

Issue Links

• is blocked by: CASSANDRA-5244 Compactions don't work while node is bootstrapping