Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-13383

auto-scaling not working in solr7.4 - autoaddreplica

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Blocker
    • Resolution: Invalid
    • 7.4
    • 7.4.1
    • config-api
    • Production

    Description

      We're able to create new server via auto-scaling for our solr cluster 7.4 version. But the newly created server is not adding in our solr cluster automatically. Is there any settings or configurations we need to add in order to add the replica automatically in cluster for any collections.

      commands used:

      curl "http://localhost:8983/solr/admin/collections?action=CREATE&name=my_collection_3&numShards=1&replicationFactor=3&autoAddReplicas=true"
      

      Currently 3 nodes are in our cluster and i'm trying to add 4th nod but its not getting added in solr 7.4 version. The same scenario is working fine in solr 7.5 version.

      scaling policy used:

      1)
      curl http://localhost:8983/solr/admin/autoscaling -H 'Content-type:application/json' -d '{
       "set-cluster-policy" : [{
       "replica" : "1",
       "shard" : "#EACH",
       "node" : "#ANY",
       }]
      
      }'
      2)###Node Added Trigger
      curl http://localhost:8983/solr/admin/autoscaling -H 'Content-type:application/json' -d '{
      "set-trigger": {
      "name" : "node_added_trigger",
      "event" : "nodeAdded",
      "waitFor" : "5s",
      "preferredOperation": "ADDREPLICA",
      "enabled" : true,
      "actions" : [
      {
      "name" : "compute_plan",
      "class": "solr.ComputePlanAction"
      },
      {
      "name" : "execute_plan",
      "class": "solr.ExecutePlanAction"
      }
      ]
      }
      }'
      
      3)###Node Lost Trigger
      curl http://localhost:8983/solr/admin/autoscaling -H 'Content-type:application/json' -d '{
      "set-trigger": {
      "name" : "node_lost_trigger",
      "event" : "nodeLost",
      "waitFor" : "5s",
      "preferredOperation": "DELETENODE",
      "enabled" : true,
      "actions" : [
      {
      "name" : "compute_plan",
      "class": "solr.ComputePlanAction"
      },
      {
      "name" : "execute_plan",
      "class": "solr.ExecutePlanAction"
      }
      ]
      }
      }'
      

      Note the same policy(2,3) not working in 7.4

      Errors:

      [Mon Apr 08 11:52:00 UTC root@hawkeye-common ~]# curl http://localhost:8983/solr/admin/autoscaling -H 'Content-type:application/json' -d '{
      >  "set-trigger": {
      >   "name" : "node_added_trigger",
      >   "event" : "nodeAdded",
      >   "waitFor" : "5s",
      >   "preferredOperation": "ADDREPLICA",
      >   "enabled" : true,
      >   "actions" : [
      >    {
      >     "name" : "compute_plan",
      >     "class": "solr.ComputePlanAction"
      >    },
      >    {
      >     "name" : "execute_plan",
      >     "class": "solr.ExecutePlanAction"
      >    }
      >   ]
      >  }
      > }'
      {
        "responseHeader":{
          "status":400,
          "QTime":5},
        "result":"failure",
        "WARNING":"This response format is experimental.  It is likely to change in the future.",
        "error":{
          "metadata":[
            "error-class","org.apache.solr.api.ApiBag$ExceptionWithErrObject",
            "root-error-class","org.apache.solr.api.ApiBag$ExceptionWithErrObject"],
          "details":[{
              "set-trigger":{
                "name":"node_added_trigger",
                "event":"nodeAdded",
                "waitFor":"5s",
                "preferredOperation":"ADDREPLICA",
                "enabled":true,
                "actions":[{
                    "name":"compute_plan",
                    "class":"solr.ComputePlanAction"},
                  {
                    "name":"execute_plan",
                    "class":"solr.ExecutePlanAction"}]},
              "errorMessages":["Error validating trigger config node_added_trigger: TriggerValidationException{name=node_added_trigger, details='{preferredOperation=unknown property}'}"]}],
          "msg":"Error in command payload",
          "code":400}}
      [Mon Apr 08 11:52:09 UTC root@hawkeye-common ~]#
      [Mon Apr 08 11:52:16 UTC root@hawkeye-common ~]# curl http://localhost:8983/solr/admin/autoscaling -H 'Content-type:application/json' -d '{
      >  "set-trigger": {
      >   "name" : "node_lost_trigger",
      >   "event" : "nodeLost",
      >   "waitFor" : "5s",
      >   "preferredOperation": "DELETENODE",
      >   "enabled" : true,
      >   "actions" : [
      >    {
      >     "name" : "compute_plan",
      >     "class": "solr.ComputePlanAction"
      >    },
      >    {
      >     "name" : "execute_plan",
      >     "class": "solr.ExecutePlanAction"
      >    }
      >   ]
      >  }
      > }'
      {
        "responseHeader":{
          "status":400,
          "QTime":1},
        "result":"failure",
        "WARNING":"This response format is experimental.  It is likely to change in the future.",
        "error":{
          "metadata":[
            "error-class","org.apache.solr.api.ApiBag$ExceptionWithErrObject",
            "root-error-class","org.apache.solr.api.ApiBag$ExceptionWithErrObject"],
          "details":[{
              "set-trigger":{
                "name":"node_lost_trigger",
                "event":"nodeLost",
                "waitFor":"5s",
                "preferredOperation":"DELETENODE",
                "enabled":true,
                "actions":[{
                    "name":"compute_plan",
                    "class":"solr.ComputePlanAction"},
                  {
                    "name":"execute_plan",
                    "class":"solr.ExecutePlanAction"}]},
              "errorMessages":["Error validating trigger config node_lost_trigger: TriggerValidationException{name=node_lost_trigger, details='{preferredOperation=unknown property}'}"]}],
          "msg":"Error in command payload",
          "code":400}}
      

      We're able to create new server via auto-scaling for our solr cluster 7.4 version. But the newly created server is not adding in our solr cluster automatically. Is there any settings or configurations we need to add in order to add the replica automatically in cluster for any collections.

      commands used: curl "http://localhost:8983/solr/admin/collections?action=CREATE&name=my_collection_3&numShards=1&replicationFactor=3&autoAddReplicas=true"

      Currently 3 nodes are in our cluster and i'm trying to add 4th nod but its not getting added in solr 7.4 version. The same scenario is working fine in solr 7.5 version.

      scaling policy used:

      1)
      curl http://localhost:8983/solr/admin/autoscaling -H 'Content-type:application/json' -d '{
      "set-cluster-policy" : [

      { "replica" : "1", "shard" : "#EACH", "node" : "#ANY", }

      ]
      }'

      2)###Node Added Trigger
      curl http://localhost:8983/solr/admin/autoscaling -H 'Content-type:application/json' -d '{
      "set-trigger": {
      "name" : "node_added_trigger",
      "event" : "nodeAdded",
      "waitFor" : "5s",
      "preferredOperation": "ADDREPLICA",
      "enabled" : true,
      "actions" : [

      { "name" : "compute_plan", "class": "solr.ComputePlanAction" }

      ,

      { "name" : "execute_plan", "class": "solr.ExecutePlanAction" }

      ]
      }
      }'

      3)###Node Lost Trigger
      curl http://localhost:8983/solr/admin/autoscaling -H 'Content-type:application/json' -d '{
      "set-trigger": {
      "name" : "node_lost_trigger",
      "event" : "nodeLost",
      "waitFor" : "5s",
      "preferredOperation": "DELETENODE",
      "enabled" : true,
      "actions" : [

      { "name" : "compute_plan", "class": "solr.ComputePlanAction" }

      ,

      { "name" : "execute_plan", "class": "solr.ExecutePlanAction" }

      ]
      }
      }'
      Note the same policy(2,3) not working in 7.4

      Errors:

      
      [Mon Apr 08 11:52:00 UTC root@hawkeye-common ~]# curl http://localhost:8983/solr/admin/autoscaling -H 'Content-type:application/json' -d '{
      >  "set-trigger": {
      >   "name" : "node_added_trigger",
      >   "event" : "nodeAdded",
      >   "waitFor" : "5s",
      >   "preferredOperation": "ADDREPLICA",
      >   "enabled" : true,
      >   "actions" : [
      >    {
      >     "name" : "compute_plan",
      >     "class": "solr.ComputePlanAction"
      >    },
      >    {
      >     "name" : "execute_plan",
      >     "class": "solr.ExecutePlanAction"
      >    }
      >   ]
      >  }
      > }'
      {
        "responseHeader":{
          "status":400,
          "QTime":5},
        "result":"failure",
        "WARNING":"This response format is experimental.  It is likely to change in the future.",
        "error":{
          "metadata":[
            "error-class","org.apache.solr.api.ApiBag$ExceptionWithErrObject",
            "root-error-class","org.apache.solr.api.ApiBag$ExceptionWithErrObject"],
          "details":[{
              "set-trigger":{
                "name":"node_added_trigger",
                "event":"nodeAdded",
                "waitFor":"5s",
                "preferredOperation":"ADDREPLICA",
                "enabled":true,
                "actions":[{
                    "name":"compute_plan",
                    "class":"solr.ComputePlanAction"},
                  {
                    "name":"execute_plan",
                    "class":"solr.ExecutePlanAction"}]},
              "errorMessages":["Error validating trigger config node_added_trigger: TriggerValidationException{name=node_added_trigger, details='{preferredOperation=unknown property}'}"]}],
          "msg":"Error in command payload",
          "code":400}}
      [Mon Apr 08 11:52:09 UTC root@hawkeye-common ~]#
      [Mon Apr 08 11:52:16 UTC root@hawkeye-common ~]# curl http://localhost:8983/solr/admin/autoscaling -H 'Content-type:application/json' -d '{
      >  "set-trigger": {
      >   "name" : "node_lost_trigger",
      >   "event" : "nodeLost",
      >   "waitFor" : "5s",
      >   "preferredOperation": "DELETENODE",
      >   "enabled" : true,
      >   "actions" : [
      >    {
      >     "name" : "compute_plan",
      >     "class": "solr.ComputePlanAction"
      >    },
      >    {
      >     "name" : "execute_plan",
      >     "class": "solr.ExecutePlanAction"
      >    }
      >   ]
      >  }
      > }'
      {
        "responseHeader":{
          "status":400,
          "QTime":1},
        "result":"failure",
        "WARNING":"This response format is experimental.  It is likely to change in the future.",
        "error":{
          "metadata":[
            "error-class","org.apache.solr.api.ApiBag$ExceptionWithErrObject",
            "root-error-class","org.apache.solr.api.ApiBag$ExceptionWithErrObject"],
          "details":[{
              "set-trigger":{
                "name":"node_lost_trigger",
                "event":"nodeLost",
                "waitFor":"5s",
                "preferredOperation":"DELETENODE",
                "enabled":true,
                "actions":[{
                    "name":"compute_plan",
                    "class":"solr.ComputePlanAction"},
                  {
                    "name":"execute_plan",
                    "class":"solr.ExecutePlanAction"}]},
              "errorMessages":["Error validating trigger config node_lost_trigger: TriggerValidationException{name=node_lost_trigger, details='{preferredOperation=unknown property}'}"]}],
          "msg":"Error in command payload",
          "code":400}}
      
      

      if any new sever added into that cluster the data(collection) must be added as a replica in that cluster.[AddReplica Automatically added in version 7.5]

      https://i.stack.imgur.com/DgEbL.png

      Kindly help me to resolve this case.

      I need a exact trigger which will perform a AutoAddReplica Action

      Attachments

        1. 3collections.PNG
          17 kB
          AntonyJohnson

        Activity

          People

            Unassigned Unassigned
            antojohn AntonyJohnson
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: