Solr
  1. Solr
  2. SOLR-3460

Improve cmd line config bootstrap tool.

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.0-BETA, 6.0
    • Component/s: SolrCloud
    • Labels:
      None

      Description

      Improve cmd line tool for bootstrapping config sets. Rather than take a config set name and directory, make it work like -Dboostrap_conf=true and read solr.xml to find config sets. Config sets will be named after the collection and auto linked to the identically named collection.

      1. SOLR-3460.patch
        42 kB
        Mark Miller
      2. SOLR-3460.patch
        13 kB
        Mark Miller

        Issue Links

          Activity

          Hide
          Mark Miller added a comment -

          first pass

          Show
          Mark Miller added a comment - first pass
          Hide
          Mark Miller added a comment -

          Okay, I'll commit this shortly.

          Show
          Mark Miller added a comment - Okay, I'll commit this shortly.
          Hide
          Mark Miller added a comment -

          Committed - I'll add some doc to the wiki.

          Show
          Mark Miller added a comment - Committed - I'll add some doc to the wiki.
          Hide
          Mark Miller added a comment -

          Still need to add to the wiki.

          Show
          Mark Miller added a comment - Still need to add to the wiki.
          Hide
          Mark Miller added a comment -

          This little tool was original pretty modest in scope - it's clear we need more though, so I've already started on working on a 'next gen' version of this thing. Instead of a simple bootstrap cmd util, it will be our simple ZooKeeper CLI tool that can do a variety of things.

          It will initially allow you to:
          bootstrap config based on solr.xml and local setup
          bootstrap a config dir to a named config set
          reverse the previous two (eg go from zk to local file system with config files)
          link a specific collection to a config set name (even before startup up your cluster)
          clear ZooKeeper (for starting fresh)

          This should make all of these tasks easily scriptable.

          Show
          Mark Miller added a comment - This little tool was original pretty modest in scope - it's clear we need more though, so I've already started on working on a 'next gen' version of this thing. Instead of a simple bootstrap cmd util, it will be our simple ZooKeeper CLI tool that can do a variety of things. It will initially allow you to: bootstrap config based on solr.xml and local setup bootstrap a config dir to a named config set reverse the previous two (eg go from zk to local file system with config files) link a specific collection to a config set name (even before startup up your cluster) clear ZooKeeper (for starting fresh) This should make all of these tasks easily scriptable.
          Hide
          Mark Miller added a comment -

          Here is a patch of my current work. A few tweaks to go. Also need to implement the zk -> filesystem reverse feature still. Close though - and some tests.

          Includes SOLR-3609 (the new scripts count on it).
          Also includes SOLR-3612.

          Show
          Mark Miller added a comment - Here is a patch of my current work. A few tweaks to go. Also need to implement the zk -> filesystem reverse feature still. Close though - and some tests. Includes SOLR-3609 (the new scripts count on it). Also includes SOLR-3612 .
          Hide
          Mark Miller added a comment -

          I'm ready to commit soon. I just added the download config to dir option. Don't have a reverse for the full solr.xml bootstrap, but then can come later as a new feature when I get the time.

          I'll commit SOLR-3609 with this.

          Show
          Mark Miller added a comment - I'm ready to commit soon. I just added the download config to dir option. Don't have a reverse for the full solr.xml bootstrap, but then can come later as a new feature when I get the time. I'll commit SOLR-3609 with this.
          Hide
          Mark Miller added a comment - - edited

          I've added example/cloud-scripts

          In that is zkcli.bat and zkcli.sh - both of which allow you to pass cmd line params.

          usage: ZkCLI
           -c,--collection <arg>   for linkconfig: name of the collection
           -cmd <arg>              cmd to run: bootstrap, upconfig, downconfig, linkconfig,
                                   makepath, clear
           -d,--confdir <arg>      for upconfig: a directory of configuration files
           -h,--help               bring up this help page
           -n,--confname <arg>     for upconfig, linkconfig: name of the config set
           -r,--runzk <arg>        run zk internally by passing the solr run port -
                                   only for clusters on one machine (tests, dev)
           -s,--solrhome <arg>     for bootstrap, runzk: solrhome location
           -z,--zkhost <arg>       ZooKeeper host address
          
          Show
          Mark Miller added a comment - - edited I've added example/cloud-scripts In that is zkcli.bat and zkcli.sh - both of which allow you to pass cmd line params. usage: ZkCLI -c,--collection <arg> for linkconfig: name of the collection -cmd <arg> cmd to run: bootstrap, upconfig, downconfig, linkconfig, makepath, clear -d,--confdir <arg> for upconfig: a directory of configuration files -h,--help bring up this help page -n,--confname <arg> for upconfig, linkconfig: name of the config set -r,--runzk <arg> run zk internally by passing the solr run port - only for clusters on one machine (tests, dev) -s,--solrhome <arg> for bootstrap, runzk: solrhome location -z,--zkhost <arg> ZooKeeper host address
          Hide
          Mark Miller added a comment -

          Added a makepath cmd as well - chime in if you have any useful ideas we should add. I'm going to commit this first iteration shortly.

          Part of this change allows you to setup collection to config links before solr even starts up in cloud mode - if on startup, a collection node with a given config already set is seen, it is simply used as is.

          This should provide the flexibility of one of the more complicated cases that came up on the mailing list a couple months back.

          Show
          Mark Miller added a comment - Added a makepath cmd as well - chime in if you have any useful ideas we should add. I'm going to commit this first iteration shortly. Part of this change allows you to setup collection to config links before solr even starts up in cloud mode - if on startup, a collection node with a given config already set is seen, it is simply used as is. This should provide the flexibility of one of the more complicated cases that came up on the mailing list a couple months back.
          Hide
          Hoss Man added a comment -

          bulk fixing the version info for 4.0-ALPHA and 4.0 all affected issues have "hoss20120711-bulk-40-change" in comment

          Show
          Hoss Man added a comment - bulk fixing the version info for 4.0-ALPHA and 4.0 all affected issues have "hoss20120711-bulk-40-change" in comment
          Hide
          Mark Miller added a comment -

          This is in.

          Show
          Mark Miller added a comment - This is in.

            People

            • Assignee:
              Mark Miller
              Reporter:
              Mark Miller
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development