[SOLR-12978] Autoscaling Suggester tries to test metrics for dead nodes

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 7.6, 8.0
• Component/s: AutoScaling, SolrCloud
• Assignee: Noble Paul
• Reporter: Shalin Shekhar Mangar
• Created: 08/Nov/18 22:41
• Updated: 08/Jun/19 15:02
• Resolved: 02/Jan/19 06:58

Description

Suggester tries to test clauses in the applyRules phase for each row regardless of whether the row is live or not. When the node is not live and there are no metrics fetched, testing the clause causes an NPE.

20586 WARN  (AutoscalingActionExecutor-37-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Exception executing actions
org.apache.solr.cloud.autoscaling.TriggerActionException: Error processing action for trigger event: {
  "id":"21d1e96fd8737T4ighk35ce6gv7f6h5zbndib4n",
  "source":"node_lost_trigger",
  "eventTime":594967172843319,
  "eventType":"NODELOST",
  "properties":{
    "eventTimes":[594967172843319],
    "preferredOperation":"movereplica",
    "_enqueue_time_":594968181417909,
    "nodeNames":["127.0.0.1:63255_solr"]}}
	at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$419(ScheduledTriggers.java:311) [java/:?]
	at org.apache.solr.cloud.autoscaling.ScheduledTriggers$$Lambda$498/1669229711.run(Unknown Source) [java/:?]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_51]
	at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_51]
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$328(ExecutorUtil.java:209) [java/:?]
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$10/1568754952.run(Unknown Source) [java/:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_51]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_51]
	at java.lang.Thread.run(Thread.java:745) [?:1.8.0_51]
Caused by: org.apache.solr.common.SolrException: Unexpected exception while processing event: {
  "id":"21d1e96fd8737T4ighk35ce6gv7f6h5zbndib4n",
  "source":"node_lost_trigger",
  "eventTime":594967172843319,
  "eventType":"NODELOST",
  "properties":{
    "eventTimes":[594967172843319],
    "preferredOperation":"movereplica",
    "_enqueue_time_":594968181417909,
    "nodeNames":["127.0.0.1:63255_solr"]}}
	at org.apache.solr.cloud.autoscaling.ComputePlanAction.process(ComputePlanAction.java:160) ~[java/:?]
	at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$419(ScheduledTriggers.java:308) ~[java/:?]
	... 8 more
Caused by: java.lang.NullPointerException
	at org.apache.solr.client.solrj.cloud.autoscaling.RangeVal.match(RangeVal.java:34) ~[java/:?]
	at org.apache.solr.client.solrj.cloud.autoscaling.Operand$2.match(Operand.java:43) ~[java/:?]
	at org.apache.solr.client.solrj.cloud.autoscaling.Variable.match(Variable.java:46) ~[java/:?]
	at org.apache.solr.client.solrj.cloud.autoscaling.Variable$Type.match(Variable.java:358) ~[java/:?]
	at org.apache.solr.client.solrj.cloud.autoscaling.Condition.isPass(Condition.java:71) ~[java/:?]
	at org.apache.solr.client.solrj.cloud.autoscaling.Condition.isPass(Condition.java:76) ~[java/:?]
	at org.apache.solr.client.solrj.cloud.autoscaling.Clause.test(Clause.java:531) ~[java/:?]
	at org.apache.solr.client.solrj.cloud.autoscaling.Policy$Session.applyRules(Policy.java:635) ~[java/:?]
	at org.apache.solr.client.solrj.cloud.autoscaling.Suggester.getSuggestion(Suggester.java:185) ~[java/:?]
	at org.apache.solr.cloud.autoscaling.ComputePlanAction.process(ComputePlanAction.java:114) ~[java/:?]
	at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$419(ScheduledTriggers.java:308) ~[java/:?]
	... 8 more

Issue Links

• blocks: SOLR-12845 Add a default cluster policy