Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Fix Version/s: 1.1.1
    • Component/s: None
    • Labels:
      None

      Description

      just a thought.. the ownership info currently just look at the token and calculate the % between nodes. It would be nice if it could do more, such as discriminate nodes of each DC, replica set, etc.

      ticket is open for suggestion...

      1. 0001-CASSANDRA-3412.patch
        10 kB
        Vijay
      2. 0001-CASSANDRA-3412-v2.patch
        10 kB
        Vijay
      3. 0001-CASSANDRA-3412-v3.patch
        10 kB
        Vijay

        Issue Links

          Activity

          Hide
          Jonathan Ellis added a comment -

          In particular the current design ignores all but the partitioner-defined replica, which usually results in nonsense when NTS is used w/ multiple DC.

          Show
          Jonathan Ellis added a comment - In particular the current design ignores all but the partitioner-defined replica, which usually results in nonsense when NTS is used w/ multiple DC.
          Hide
          Jackson Chung added a comment -

          eg:

          xx.10.1.2 Up Normal 33.66 KB 10.00% 0
          xx.20.2.4 Up Normal 285.98 KB 0.00% 1
          xx.10.1.3 Up Normal 228.76 KB 10.00% 17014118346046923173168730371588410572
          xx.20.2.5 Up Normal 486.17 KB 0.00% 17014118346046923173168730371588410573
          xx.10.1.4 Up Normal 407.44 KB 10.00% 34028236692093846346337460743176821145
          xx.20.2.6 Up Normal 204.85 KB 0.00% 34028236692093846346337460743176821146
          xx.10.1.5 Up Normal 111.07 KB 10.00% 51042355038140769519506191114765231718
          xx.20.2.7 Up Normal 135.98 KB 0.00% 51042355038140769519506191114765231719
          xx.10.1.6 Up Normal 33.67 KB 10.00% 68056473384187692692674921486353642291
          xx.20.2.8 Up Normal 170.09 KB 0.00% 68056473384187692692674921486353642292
          xx.10.1.7 Up Normal 188.81 KB 10.00% 85070591730234615865843651857942052864
          xx.20.2.9 Up Normal 298.02 KB 0.00% 85070591730234615865843651857942052865
          xx.10.1.8 Up Normal 202.61 KB 10.00% 102084710076281539039012382229530463436
          xx.20.2.10 Up Normal 382.01 KB 0.00% 102084710076281539039012382229530463437
          xx.10.1.9 Up Normal 220.16 KB 10.00% 119098828422328462212181112601118874009
          xx.20.2.11 Up Normal 403.32 KB 0.00% 119098828422328462212181112601118874010
          xx.10.1.10 Up Normal 229.49 KB 10.00% 136112946768375385385349842972707284582
          xx.20.2.12 Up Normal 400.71 KB 0.00% 136112946768375385385349842972707284583
          xx.10.1.11 Up Normal 29.29 KB 10.00% 153127065114422308558518573344295695155
          xx.20.2.13 Up Normal 635.95 KB 0.00% 153127065114422308558518573344295695156

          In the above, RackInferringSnitch is used, 10 is DC1 and 20 is DC2 . If not paying close attention, it would seems nodes in the .20 owns nothing:

          xx.20.2.4 Up Normal 285.98 KB 0.00% 1
          xx.20.2.5 Up Normal 486.17 KB 0.00% 17014118346046923173168730371588410573
          xx.20.2.6 Up Normal 204.85 KB 0.00% 34028236692093846346337460743176821146
          xx.20.2.7 Up Normal 135.98 KB 0.00% 51042355038140769519506191114765231719
          xx.20.2.8 Up Normal 170.09 KB 0.00% 68056473384187692692674921486353642292
          xx.20.2.9 Up Normal 298.02 KB 0.00% 85070591730234615865843651857942052865
          xx.20.2.10 Up Normal 382.01 KB 0.00% 102084710076281539039012382229530463437
          xx.20.2.11 Up Normal 403.32 KB 0.00% 119098828422328462212181112601118874010
          xx.20.2.12 Up Normal 400.71 KB 0.00% 136112946768375385385349842972707284583
          xx.20.2.13 Up Normal 635.95 KB 0.00% 153127065114422308558518573344295695156

          Show
          Jackson Chung added a comment - eg: xx.10.1.2 Up Normal 33.66 KB 10.00% 0 xx.20.2.4 Up Normal 285.98 KB 0.00% 1 xx.10.1.3 Up Normal 228.76 KB 10.00% 17014118346046923173168730371588410572 xx.20.2.5 Up Normal 486.17 KB 0.00% 17014118346046923173168730371588410573 xx.10.1.4 Up Normal 407.44 KB 10.00% 34028236692093846346337460743176821145 xx.20.2.6 Up Normal 204.85 KB 0.00% 34028236692093846346337460743176821146 xx.10.1.5 Up Normal 111.07 KB 10.00% 51042355038140769519506191114765231718 xx.20.2.7 Up Normal 135.98 KB 0.00% 51042355038140769519506191114765231719 xx.10.1.6 Up Normal 33.67 KB 10.00% 68056473384187692692674921486353642291 xx.20.2.8 Up Normal 170.09 KB 0.00% 68056473384187692692674921486353642292 xx.10.1.7 Up Normal 188.81 KB 10.00% 85070591730234615865843651857942052864 xx.20.2.9 Up Normal 298.02 KB 0.00% 85070591730234615865843651857942052865 xx.10.1.8 Up Normal 202.61 KB 10.00% 102084710076281539039012382229530463436 xx.20.2.10 Up Normal 382.01 KB 0.00% 102084710076281539039012382229530463437 xx.10.1.9 Up Normal 220.16 KB 10.00% 119098828422328462212181112601118874009 xx.20.2.11 Up Normal 403.32 KB 0.00% 119098828422328462212181112601118874010 xx.10.1.10 Up Normal 229.49 KB 10.00% 136112946768375385385349842972707284582 xx.20.2.12 Up Normal 400.71 KB 0.00% 136112946768375385385349842972707284583 xx.10.1.11 Up Normal 29.29 KB 10.00% 153127065114422308558518573344295695155 xx.20.2.13 Up Normal 635.95 KB 0.00% 153127065114422308558518573344295695156 In the above, RackInferringSnitch is used, 10 is DC1 and 20 is DC2 . If not paying close attention, it would seems nodes in the .20 owns nothing: xx.20.2.4 Up Normal 285.98 KB 0.00% 1 xx.20.2.5 Up Normal 486.17 KB 0.00% 17014118346046923173168730371588410573 xx.20.2.6 Up Normal 204.85 KB 0.00% 34028236692093846346337460743176821146 xx.20.2.7 Up Normal 135.98 KB 0.00% 51042355038140769519506191114765231719 xx.20.2.8 Up Normal 170.09 KB 0.00% 68056473384187692692674921486353642292 xx.20.2.9 Up Normal 298.02 KB 0.00% 85070591730234615865843651857942052865 xx.20.2.10 Up Normal 382.01 KB 0.00% 102084710076281539039012382229530463437 xx.20.2.11 Up Normal 403.32 KB 0.00% 119098828422328462212181112601118874010 xx.20.2.12 Up Normal 400.71 KB 0.00% 136112946768375385385349842972707284583 xx.20.2.13 Up Normal 635.95 KB 0.00% 153127065114422308558518573344295695156
          Hide
          Jonathan Ellis added a comment -

          Vijay, do you have time to take a stab at this?

          Show
          Jonathan Ellis added a comment - Vijay, do you have time to take a stab at this?
          Hide
          Vijay added a comment -

          Will do!

          Jackson, It is fairly trivial to change the own's.... The problem with this last time i looked at this, was that if we show % in the ring and if we show % per DC it will add up to be more than 100% and hence will cause some confusions for the staters (I wish it was color coded or something like that)... What do you think? Will it make sense to rename OWNS to OWNS-PER-DC or a better name and do the above?

          Show
          Vijay added a comment - Will do! Jackson, It is fairly trivial to change the own's.... The problem with this last time i looked at this, was that if we show % in the ring and if we show % per DC it will add up to be more than 100% and hence will cause some confusions for the staters (I wish it was color coded or something like that)... What do you think? Will it make sense to rename OWNS to OWNS-PER-DC or a better name and do the above?
          Hide
          Peter Schuller added a comment -

          Our internal tool (external, in python, based on describe_ring) simply uses describe_ring and looks at each range and their responsible nodes and just adds them all up. The ownership we report for a node is the total amount of ringspace (regardless of primary/secondary/dc/etc concerns) that the node has, compared to the overall total.

          It ends up giving you the real number while completely blackboxing "why" we got there - whether it be due to rack awareness (CASSANDRA-3810) or DC:s.

          FWIW, here is the code for that. It's not self-contained and won't run, but it's an FYI. The topology_xref is just post-processing the describe ring results to yield the map of range -> nodes_responsible.

          def cmd_effective_ownership(opts, args):
              """                                                                                                                                                                                                                                                                        
              Print effective ownership of nodes in a cluster.                                                                                                                                                                                                                           
                                                                                                                                                                                                                                                                                         
              Effective ownership means the actual amount of the ring for which                                                                                                                                                                                                          
              it has data, whether or not it is because it is the primary or                                                                                                                                                                                                             
              secondary (etc) owner of the ring segment. This is essentially the                                                                                                                                                                                                         
              ownership you would want "nodetool ring" to print but doesn't.                                                                                                                                                                                                             
              """
              if not args and not opts.all:
                  return
          
              node_ranges, range_nodes = topology_xref(describe_ring(*((opts,) + split_hostport(seed(opts, 'localhost') if opts.all else args[0]))))
          
              if opts.all:
                  args = node_ranges.keys()
          
              # acrobatics to handle wrap-around                                                                                                                                                                                                                                         
              max_token = 0
              min_token = 2**127
              for r in range_nodes.keys():
                  if r[0] < min_token:
                      min_token = r[0]
                  if r[1] > max_token:
                      max_token = r[1]
          
              def ownership(start_token, end_token):
                  start_token, end_token = int(start_token), int(end_token)
                  if end_token < start_token:
                      # wrap-around                                                                                                                                                                                                                                                      
                      return end_token + (2**127 - start_token)
                  else:
                      return end_token - start_token
          
              toprint = [] # list of (owned, ranges), later to be sorted                                                                                                                                                                                                                 
              for node in (hostnames.normalize_hostname(arg) for arg in args):
                  if not node in node_ranges:
                      raise cmdline.UserError('node %s not in ring' % (node,))
                  ranges = node_ranges[node]
                  owned = reduce(lambda a, b: a + b, [ownership(r[0], r[1]) for r in ranges], 0)
                  toprint.append((owned, node, ranges))
          
              toprint = sorted(toprint, reverse=True)
              for owned, node, ranges in toprint:
                  print '%s %f%%' % (node, float(owned) / 2**127 * 100.0)
                  if opts.print_ranges:
                      for r in sorted(ranges):
                          print '  %s - %s' % (r[0], r[1])
          
          Show
          Peter Schuller added a comment - Our internal tool (external, in python, based on describe_ring) simply uses describe_ring and looks at each range and their responsible nodes and just adds them all up. The ownership we report for a node is the total amount of ringspace (regardless of primary/secondary/dc/etc concerns) that the node has, compared to the overall total. It ends up giving you the real number while completely blackboxing "why" we got there - whether it be due to rack awareness ( CASSANDRA-3810 ) or DC:s. FWIW, here is the code for that. It's not self-contained and won't run, but it's an FYI. The topology_xref is just post-processing the describe ring results to yield the map of range -> nodes_responsible. def cmd_effective_ownership(opts, args): """ Print effective ownership of nodes in a cluster. Effective ownership means the actual amount of the ring for which it has data, whether or not it is because it is the primary or secondary (etc) owner of the ring segment. This is essentially the ownership you would want "nodetool ring" to print but doesn't. """ if not args and not opts.all: return node_ranges, range_nodes = topology_xref(describe_ring(*((opts,) + split_hostport(seed(opts, 'localhost') if opts.all else args[0])))) if opts.all: args = node_ranges.keys() # acrobatics to handle wrap-around max_token = 0 min_token = 2**127 for r in range_nodes.keys(): if r[0] < min_token: min_token = r[0] if r[1] > max_token: max_token = r[1] def ownership(start_token, end_token): start_token, end_token = int (start_token), int (end_token) if end_token < start_token: # wrap-around return end_token + (2**127 - start_token) else : return end_token - start_token toprint = [] # list of (owned, ranges), later to be sorted for node in (hostnames.normalize_hostname(arg) for arg in args): if not node in node_ranges: raise cmdline.UserError('node %s not in ring' % (node,)) ranges = node_ranges[node] owned = reduce(lambda a, b: a + b, [ownership(r[0], r[1]) for r in ranges], 0) toprint.append((owned, node, ranges)) toprint = sorted(toprint, reverse=True) for owned, node, ranges in toprint: print '%s %f%%' % (node, float (owned) / 2**127 * 100.0) if opts.print_ranges: for r in sorted(ranges): print ' %s - %s' % (r[0], r[1])
          Hide
          Vijay added a comment -

          Makes sense, but currently ring is a cluster wide command, are you proposing it to be keyspace level one? if i read the code right it also factors in the replication

          Show
          Vijay added a comment - Makes sense, but currently ring is a cluster wide command, are you proposing it to be keyspace level one? if i read the code right it also factors in the replication
          Hide
          Peter Schuller added a comment -

          Yes, our tool auto-detects the keyspace if possible (if there is just a single non-system keyspace) and requires a --keyspace otherwise.

          If rack awareness is not enabled or the ring spaced in such a way that it doesn't matter (again CASSANDRA-3810) there would be no need to be per-ks and just looking at data centers would be correct and thus keyspace/RF knowledge would not be necessary.

          Maybe it could look at all keyspaces and pick the one with the highest RF, if one doesn't choose a keyspace?

          Show
          Peter Schuller added a comment - Yes, our tool auto-detects the keyspace if possible (if there is just a single non-system keyspace) and requires a --keyspace otherwise. If rack awareness is not enabled or the ring spaced in such a way that it doesn't matter (again CASSANDRA-3810 ) there would be no need to be per-ks and just looking at data centers would be correct and thus keyspace/RF knowledge would not be necessary. Maybe it could look at all keyspaces and pick the one with the highest RF, if one doesn't choose a keyspace?
          Hide
          Peter Schuller added a comment -

          Technically speaking, even making assumptions about DC is "incorrect" unless those assumptions are tied to the choice of replication strategy. My point here, is that a solution which is strictly correct and works with any replication strategy would either have to be strategy specific, or blackbox the details (such as our tool does). So I would suggest that it's actually okay to try to generalize it and pick the widest keyspace if not otherwise picked; it seems more "future proof" to assume the RF varies by KS, than to make assumptions about the replication strategy. Especially if the alternative doesn't get us the "true" ownership and only gets us the per-DC ring information.

          Show
          Peter Schuller added a comment - Technically speaking, even making assumptions about DC is "incorrect" unless those assumptions are tied to the choice of replication strategy. My point here, is that a solution which is strictly correct and works with any replication strategy would either have to be strategy specific, or blackbox the details (such as our tool does). So I would suggest that it's actually okay to try to generalize it and pick the widest keyspace if not otherwise picked; it seems more "future proof" to assume the RF varies by KS, than to make assumptions about the replication strategy. Especially if the alternative doesn't get us the "true" ownership and only gets us the per-DC ring information.
          Hide
          Vijay added a comment -

          We also have something similar to this internally, but if we make it generic we will almost never know which keyspace is the right one to show or what user wants to be as default... if there are clusters with 10's of keyspace's (which is a norm) and not all of them are connected to all the DC's and if we pick the wrong one we will still be showing the user like own's => 0... we can print the keyspace name, but it might not help a guy who operates this cluster who might not know about the the use case.
          What do u think? in either ways it is better than what we have now

          Show
          Vijay added a comment - We also have something similar to this internally, but if we make it generic we will almost never know which keyspace is the right one to show or what user wants to be as default... if there are clusters with 10's of keyspace's (which is a norm) and not all of them are connected to all the DC's and if we pick the wrong one we will still be showing the user like own's => 0... we can print the keyspace name, but it might not help a guy who operates this cluster who might not know about the the use case. What do u think? in either ways it is better than what we have now
          Hide
          Peter Schuller added a comment -

          You're right, I forgot that the RF is per DC, so there is no guarantee that you can pick the "keyspace with the highest replication factor [in each dc]". One could auto-select ks if it turns out that there is such a keyspace (that has highest RF in each DC), but that's a pretty specific case and the code to do that is probably non-trivial since the replication options are part of the replication strategy.

          How about this for a compromise:

          • If there is only a single non-system keyspace, use that (and do the describe ring type of dance)
          • If there are multiple keyspaces, leave the percentage undefined unless the user selects keyspace

          The main problem I think is that it's not clear how to make the user realize why he's not getting ownership information.

          Another possibility:

          • Make nodetool ring never print ownership.
          • Add nodetool ownership (or similar) to print ownership, but in this case it either always requires a ks, or requires a ks if > 1 non-system one.

          It's a bigger change to 'nodetool ring', but on the other hand the breakage will be obvious to people. And when specifically using 'ownership' you are perhaps more likely to read the docs or accept that you need to select a key space.

          Not sure...

          Show
          Peter Schuller added a comment - You're right, I forgot that the RF is per DC, so there is no guarantee that you can pick the "keyspace with the highest replication factor [in each dc] ". One could auto-select ks if it turns out that there is such a keyspace (that has highest RF in each DC), but that's a pretty specific case and the code to do that is probably non-trivial since the replication options are part of the replication strategy. How about this for a compromise: If there is only a single non-system keyspace, use that (and do the describe ring type of dance) If there are multiple keyspaces, leave the percentage undefined unless the user selects keyspace The main problem I think is that it's not clear how to make the user realize why he's not getting ownership information. Another possibility: Make nodetool ring never print ownership. Add nodetool ownership (or similar) to print ownership, but in this case it either always requires a ks, or requires a ks if > 1 non-system one. It's a bigger change to 'nodetool ring', but on the other hand the breakage will be obvious to people. And when specifically using 'ownership' you are perhaps more likely to read the docs or accept that you need to select a key space. Not sure...
          Hide
          Jonathan Ellis added a comment -

          If there are multiple keyspaces, leave the percentage undefined unless the user selects keyspace

          I think we can do a little better than that – we only have to leave ownership undefined if the KS actually have different replication settings.

          The main problem I think is that it's not clear how to make the user realize why he's not getting ownership information.

          Shrug, I don't have a problem with just printing out a line explaining why, at the bottom.

          Show
          Jonathan Ellis added a comment - If there are multiple keyspaces, leave the percentage undefined unless the user selects keyspace I think we can do a little better than that – we only have to leave ownership undefined if the KS actually have different replication settings. The main problem I think is that it's not clear how to make the user realize why he's not getting ownership information. Shrug, I don't have a problem with just printing out a line explaining why, at the bottom.
          Hide
          Peter Schuller added a comment -

          I think we can do a little better than that – we only have to leave ownership undefined if the KS actually have different replication settings.

          Agreed.

          Shrug, I don't have a problem with just printing out a line explaining why, at the bottom.

          The reason I didn't suggest it is that it might break scripts. On the one hand, you normally shouldn't use human readable tool output for important scripting, but on the other hand we don't offer a real alternative so I have to assume that's what people do (as I have done myself).

          Anyways, I'm not too fussed personally. If complaints come in I'd rather just provide something intended for scripting than be forever locked into not changing output.

          Show
          Peter Schuller added a comment - I think we can do a little better than that – we only have to leave ownership undefined if the KS actually have different replication settings. Agreed. Shrug, I don't have a problem with just printing out a line explaining why, at the bottom. The reason I didn't suggest it is that it might break scripts. On the one hand, you normally shouldn't use human readable tool output for important scripting, but on the other hand we don't offer a real alternative so I have to assume that's what people do (as I have done myself). Anyways, I'm not too fussed personally. If complaints come in I'd rather just provide something intended for scripting than be forever locked into not changing output.
          Hide
          Vijay added a comment - - edited

          I tried to make this generic enough and attached is a simple patch to display what we discussed. If we cannot figure out a keyspace to display the default is the same as today.

          Default:
          Warning: Output contains ownership information which does not include replication factor.
          Warning: Use nodetool ring <keyspace> to specify a keyspace.
          Address DC Rack Status State Load Owns Token
          141784319550391026443072753096942836216
          107.21.183.168 us-east 1c Up Normal 38.57 KB 27.78% 18904575940052136859076367081351254013
          79.125.30.58 eu-west 1c Up Normal 36.44 KB 22.22% 56713727820156410577229101239000783353
          50.16.117.152 us-east 1c Up Normal 52.03 KB 11.11% 75618303760208547436305468319979289255
          50.19.163.142 us-east 1c Up Normal 51.59 KB 33.33% 132332031580364958013534569558607324497
          46.51.157.33 eu-west 1c Up Normal 31.64 KB 5.56% 141784319550391026443072753096942836216

          Effective nt ring: ('org.apache.cassandra.locator.NetworkTopologyStrategy' and strategy_options=

          {us-east:2,eu-west:1}

          )
          [vijay_tcasstest@vijay_tcass-i-a6643ac3 ~]$ nt ring
          Address DC Rack Status State Load Effective-Owership Token
          141784319550391026443072753096942836216
          107.21.183.168 us-east 1c Up Normal 27.23 KB 66.67% 18904575940052136859076367081351254013
          79.125.30.58 eu-west 1c Up Normal 31.51 KB 50.00% 56713727820156410577229101239000783353
          50.16.117.152 us-east 1c Up Normal 47.1 KB 66.67% 75618303760208547436305468319979289255
          50.19.163.142 us-east 1c Up Normal 42.52 KB 66.67% 132332031580364958013534569558607324497
          46.51.157.33 eu-west 1c Up Normal 36.32 KB 50.00% 141784319550391026443072753096942836216

          Show
          Vijay added a comment - - edited I tried to make this generic enough and attached is a simple patch to display what we discussed. If we cannot figure out a keyspace to display the default is the same as today. Default: Warning: Output contains ownership information which does not include replication factor. Warning: Use nodetool ring <keyspace> to specify a keyspace. Address DC Rack Status State Load Owns Token 141784319550391026443072753096942836216 107.21.183.168 us-east 1c Up Normal 38.57 KB 27.78% 18904575940052136859076367081351254013 79.125.30.58 eu-west 1c Up Normal 36.44 KB 22.22% 56713727820156410577229101239000783353 50.16.117.152 us-east 1c Up Normal 52.03 KB 11.11% 75618303760208547436305468319979289255 50.19.163.142 us-east 1c Up Normal 51.59 KB 33.33% 132332031580364958013534569558607324497 46.51.157.33 eu-west 1c Up Normal 31.64 KB 5.56% 141784319550391026443072753096942836216 Effective nt ring: ('org.apache.cassandra.locator.NetworkTopologyStrategy' and strategy_options= {us-east:2,eu-west:1} ) [vijay_tcasstest@vijay_tcass-i-a6643ac3 ~] $ nt ring Address DC Rack Status State Load Effective-Owership Token 141784319550391026443072753096942836216 107.21.183.168 us-east 1c Up Normal 27.23 KB 66.67% 18904575940052136859076367081351254013 79.125.30.58 eu-west 1c Up Normal 31.51 KB 50.00% 56713727820156410577229101239000783353 50.16.117.152 us-east 1c Up Normal 47.1 KB 66.67% 75618303760208547436305468319979289255 50.19.163.142 us-east 1c Up Normal 42.52 KB 66.67% 132332031580364958013534569558607324497 46.51.157.33 eu-west 1c Up Normal 36.32 KB 50.00% 141784319550391026443072753096942836216
          Hide
          Brandon Williams added a comment -

          When I have multiple keyspaces (one simple, one NTS) and I specify the NTS keyspace, it still gives me the warning and tells me to specify a keyspace, even though I did.

          Show
          Brandon Williams added a comment - When I have multiple keyspaces (one simple, one NTS) and I specify the NTS keyspace, it still gives me the warning and tells me to specify a keyspace, even though I did.
          Hide
          Vijay added a comment -

          Hi Brandon, it works fine for me, Not sure what is missing.

          [vijay_tcasstest@vijay_tcass-i-a6643ac3 ~]$ nt ring Keyspace2
          Address DC Rack Status State Load Effective-Owership Token
          141784319550391026443072753096942836216
          107.21.183.168 us-east 1c Up Normal 31.91 KB 33.33% 18904575940052136859076367081351254013
          79.125.30.58 eu-west 1c Up Normal 31.51 KB 50.00% 56713727820156410577229101239000783353
          50.16.117.152 us-east 1c Up Normal 51.78 KB 33.33% 75618303760208547436305468319979289255
          50.19.163.142 us-east 1c Up Normal 47.11 KB 44.44% 132332031580364958013534569558607324497
          46.51.157.33 eu-west 1c Up Normal 27.37 KB 38.89% 141784319550391026443072753096942836216
          [vijay_tcasstest@vijay_tcass-i-a6643ac3 ~]$ nt ring Keyspace3
          Address DC Rack Status State Load Effective-Owership Token
          141784319550391026443072753096942836216
          107.21.183.168 us-east 1c Up Normal 31.91 KB 66.67% 18904575940052136859076367081351254013
          79.125.30.58 eu-west 1c Up Normal 31.51 KB 50.00% 56713727820156410577229101239000783353
          50.16.117.152 us-east 1c Up Normal 51.78 KB 66.67% 75618303760208547436305468319979289255
          50.19.163.142 us-east 1c Up Normal 47.11 KB 66.67% 132332031580364958013534569558607324497
          46.51.157.33 eu-west 1c Up Normal 27.37 KB 50.00% 141784319550391026443072753096942836216
          [vijay_tcasstest@vijay_tcass-i-a6643ac3 ~]$ nt ring
          Warning: Output contains ownership information which does not include replication factor.
          Warning: Use nodetool ring <keyspace> to specify a keyspace.
          Address DC Rack Status State Load Owns Token
          141784319550391026443072753096942836216
          107.21.183.168 us-east 1c Up Normal 31.91 KB 27.78% 18904575940052136859076367081351254013
          79.125.30.58 eu-west 1c Up Normal 31.51 KB 22.22% 56713727820156410577229101239000783353
          50.16.117.152 us-east 1c Up Normal 51.78 KB 11.11% 75618303760208547436305468319979289255
          50.19.163.142 us-east 1c Up Normal 47.11 KB 33.33% 132332031580364958013534569558607324497
          46.51.157.33 eu-west 1c Up Normal 27.37 KB 5.56% 141784319550391026443072753096942836216
          [vijay_tcasstest@vijay_tcass-i-a6643ac3 ~]$ echo "show schema;"> schema; cli -f schema
          Connected to: "vijay_tcass" on 0.0.0.0/7102
          create keyspace Keyspace2
          with placement_strategy = 'SimpleStrategy'
          and strategy_options =

          {replication_factor : 2}

          and durable_writes = true;

          use Keyspace2;

          create keyspace Keyspace3
          with placement_strategy = 'NetworkTopologyStrategy'
          and strategy_options =

          {eu-west : 1, us-east : 2}

          and durable_writes = true;

          Show
          Vijay added a comment - Hi Brandon, it works fine for me, Not sure what is missing. [vijay_tcasstest@vijay_tcass-i-a6643ac3 ~] $ nt ring Keyspace2 Address DC Rack Status State Load Effective-Owership Token 141784319550391026443072753096942836216 107.21.183.168 us-east 1c Up Normal 31.91 KB 33.33% 18904575940052136859076367081351254013 79.125.30.58 eu-west 1c Up Normal 31.51 KB 50.00% 56713727820156410577229101239000783353 50.16.117.152 us-east 1c Up Normal 51.78 KB 33.33% 75618303760208547436305468319979289255 50.19.163.142 us-east 1c Up Normal 47.11 KB 44.44% 132332031580364958013534569558607324497 46.51.157.33 eu-west 1c Up Normal 27.37 KB 38.89% 141784319550391026443072753096942836216 [vijay_tcasstest@vijay_tcass-i-a6643ac3 ~] $ nt ring Keyspace3 Address DC Rack Status State Load Effective-Owership Token 141784319550391026443072753096942836216 107.21.183.168 us-east 1c Up Normal 31.91 KB 66.67% 18904575940052136859076367081351254013 79.125.30.58 eu-west 1c Up Normal 31.51 KB 50.00% 56713727820156410577229101239000783353 50.16.117.152 us-east 1c Up Normal 51.78 KB 66.67% 75618303760208547436305468319979289255 50.19.163.142 us-east 1c Up Normal 47.11 KB 66.67% 132332031580364958013534569558607324497 46.51.157.33 eu-west 1c Up Normal 27.37 KB 50.00% 141784319550391026443072753096942836216 [vijay_tcasstest@vijay_tcass-i-a6643ac3 ~] $ nt ring Warning: Output contains ownership information which does not include replication factor. Warning: Use nodetool ring <keyspace> to specify a keyspace. Address DC Rack Status State Load Owns Token 141784319550391026443072753096942836216 107.21.183.168 us-east 1c Up Normal 31.91 KB 27.78% 18904575940052136859076367081351254013 79.125.30.58 eu-west 1c Up Normal 31.51 KB 22.22% 56713727820156410577229101239000783353 50.16.117.152 us-east 1c Up Normal 51.78 KB 11.11% 75618303760208547436305468319979289255 50.19.163.142 us-east 1c Up Normal 47.11 KB 33.33% 132332031580364958013534569558607324497 46.51.157.33 eu-west 1c Up Normal 27.37 KB 5.56% 141784319550391026443072753096942836216 [vijay_tcasstest@vijay_tcass-i-a6643ac3 ~] $ echo "show schema;"> schema; cli -f schema Connected to: "vijay_tcass" on 0.0.0.0/7102 create keyspace Keyspace2 with placement_strategy = 'SimpleStrategy' and strategy_options = {replication_factor : 2} and durable_writes = true; use Keyspace2; create keyspace Keyspace3 with placement_strategy = 'NetworkTopologyStrategy' and strategy_options = {eu-west : 1, us-east : 2} and durable_writes = true;
          Hide
          Brandon Williams added a comment -

          Here are some simple steps to repro:

          • run stress as a cheap way to create a simple ks
          • create keyspace nts with placement_strategy 'org.apache.cassandra.locator.NetworkTopologyStrategy' and strategy_options= {DC1:2, DC2:2}

            ;

          Show
          Brandon Williams added a comment - Here are some simple steps to repro: run stress as a cheap way to create a simple ks create keyspace nts with placement_strategy 'org.apache.cassandra.locator.NetworkTopologyStrategy' and strategy_options= {DC1:2, DC2:2} ;
          Hide
          Vijay added a comment -

          Ahaaa.... I was not handling the case where DC1 and DC2 wasnt valid DC's... Attached patch has the fix.

          [vijay_tcasstest@vijay_tcass-i-a6643ac3 ~]$ nt ring Keyspace1
          Address DC Rack Status State Load Effective-Owership Token
          141784319550391026443072753096942836216
          79.125.30.58 eu-west 1c Up Normal 57.55 KB 21.36% 7980510910520838461985007190626882496
          107.21.183.168 us-east 1c Up Normal 79.97 KB 6.42% 18904575940052136859076367081351254013
          46.137.134.6 eu-west 1c Up Normal 13.49 KB 22.22% 56713727820156410577229101239000783353
          50.16.117.152 us-east 1c Up Normal 79.78 KB 11.11% 75618303760208547436305468319979289255
          50.19.163.142 us-east 1c Up Normal 119.48 KB 33.33% 132332031580364958013534569558607324497
          46.51.157.33 eu-west 1c Down Normal 27.37 KB 5.56% 141784319550391026443072753096942836216
          [vijay_tcasstest@vijay_tcass-i-a6643ac3 ~]$ nt ring
          Warning: Output contains ownership information which does not include replication factor.
          Why? Non System keyspaces doesnt have the same the same topology
          Warning: Use nodetool ring <keyspace> to specify a keyspace.
          Address DC Rack Status State Load Owns Token
          141784319550391026443072753096942836216
          79.125.30.58 eu-west 1c Up Normal 57.55 KB 21.36% 7980510910520838461985007190626882496
          107.21.183.168 us-east 1c Up Normal 79.97 KB 6.42% 18904575940052136859076367081351254013
          46.137.134.6 eu-west 1c Up Normal 13.49 KB 22.22% 56713727820156410577229101239000783353
          50.16.117.152 us-east 1c Up Normal 79.78 KB 11.11% 75618303760208547436305468319979289255
          50.19.163.142 us-east 1c Up Normal 119.48 KB 33.33% 132332031580364958013534569558607324497
          46.51.157.33 eu-west 1c Down Normal 27.37 KB 5.56% 141784319550391026443072753096942836216
          [vijay_tcasstest@vijay_tcass-i-a6643ac3 ~]$ nt ring nts
          Address DC Rack Status State Load Effective-Owership Token
          141784319550391026443072753096942836216
          79.125.30.58 eu-west 1c Up Normal 57.55 KB 0.00% 7980510910520838461985007190626882496
          107.21.183.168 us-east 1c Up Normal 79.97 KB 0.00% 18904575940052136859076367081351254013
          46.137.134.6 eu-west 1c Up Normal 13.49 KB 0.00% 56713727820156410577229101239000783353
          50.16.117.152 us-east 1c Up Normal 79.78 KB 0.00% 75618303760208547436305468319979289255
          50.19.163.142 us-east 1c Up Normal 119.48 KB 0.00% 132332031580364958013534569558607324497
          46.51.157.33 eu-west 1c Down Normal 27.37 KB 0.00% 141784319550391026443072753096942836216
          [vijay_tcasstest@vijay_tcass-i-a6643ac3 ~]$

          Show
          Vijay added a comment - Ahaaa.... I was not handling the case where DC1 and DC2 wasnt valid DC's... Attached patch has the fix. [vijay_tcasstest@vijay_tcass-i-a6643ac3 ~] $ nt ring Keyspace1 Address DC Rack Status State Load Effective-Owership Token 141784319550391026443072753096942836216 79.125.30.58 eu-west 1c Up Normal 57.55 KB 21.36% 7980510910520838461985007190626882496 107.21.183.168 us-east 1c Up Normal 79.97 KB 6.42% 18904575940052136859076367081351254013 46.137.134.6 eu-west 1c Up Normal 13.49 KB 22.22% 56713727820156410577229101239000783353 50.16.117.152 us-east 1c Up Normal 79.78 KB 11.11% 75618303760208547436305468319979289255 50.19.163.142 us-east 1c Up Normal 119.48 KB 33.33% 132332031580364958013534569558607324497 46.51.157.33 eu-west 1c Down Normal 27.37 KB 5.56% 141784319550391026443072753096942836216 [vijay_tcasstest@vijay_tcass-i-a6643ac3 ~] $ nt ring Warning: Output contains ownership information which does not include replication factor. Why? Non System keyspaces doesnt have the same the same topology Warning: Use nodetool ring <keyspace> to specify a keyspace. Address DC Rack Status State Load Owns Token 141784319550391026443072753096942836216 79.125.30.58 eu-west 1c Up Normal 57.55 KB 21.36% 7980510910520838461985007190626882496 107.21.183.168 us-east 1c Up Normal 79.97 KB 6.42% 18904575940052136859076367081351254013 46.137.134.6 eu-west 1c Up Normal 13.49 KB 22.22% 56713727820156410577229101239000783353 50.16.117.152 us-east 1c Up Normal 79.78 KB 11.11% 75618303760208547436305468319979289255 50.19.163.142 us-east 1c Up Normal 119.48 KB 33.33% 132332031580364958013534569558607324497 46.51.157.33 eu-west 1c Down Normal 27.37 KB 5.56% 141784319550391026443072753096942836216 [vijay_tcasstest@vijay_tcass-i-a6643ac3 ~] $ nt ring nts Address DC Rack Status State Load Effective-Owership Token 141784319550391026443072753096942836216 79.125.30.58 eu-west 1c Up Normal 57.55 KB 0.00% 7980510910520838461985007190626882496 107.21.183.168 us-east 1c Up Normal 79.97 KB 0.00% 18904575940052136859076367081351254013 46.137.134.6 eu-west 1c Up Normal 13.49 KB 0.00% 56713727820156410577229101239000783353 50.16.117.152 us-east 1c Up Normal 79.78 KB 0.00% 75618303760208547436305468319979289255 50.19.163.142 us-east 1c Up Normal 119.48 KB 0.00% 132332031580364958013534569558607324497 46.51.157.33 eu-west 1c Down Normal 27.37 KB 0.00% 141784319550391026443072753096942836216 [vijay_tcasstest@vijay_tcass-i-a6643ac3 ~] $
          Hide
          Brandon Williams added a comment -

          +1, but let's consolidate the 'warning' output lines to:

          Note: Ownership information does not include topology, please specify a keyspace

          Show
          Brandon Williams added a comment - +1, but let's consolidate the 'warning' output lines to: Note: Ownership information does not include topology, please specify a keyspace
          Hide
          Vijay added a comment -

          Attached patch with the fix for warning message, I have also added a note to changes.txt to inform the users. Will commit in few min, Thanks!

          Show
          Vijay added a comment - Attached patch with the fix for warning message, I have also added a note to changes.txt to inform the users. Will commit in few min, Thanks!
          Hide
          Vijay added a comment -

          Committed to trunk, Thanks!

          Show
          Vijay added a comment - Committed to trunk, Thanks!

            People

            • Assignee:
              Vijay
              Reporter:
              Jackson Chung
              Reviewer:
              Brandon Williams
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development