Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-15960

Unified use of system properties and environment variables

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Resolved
    • None
    • 9.5
    • None
    • None

    Description

      We have a lot of boiler-plate code in Solr related to resolving configuration from config-files, system properties and environment variables.

      The main pattern so far has been to load a config from an xml file, which uses system property variables like ${myVar}. All the environment variables that we expose in solr.in.sh are converted to system.properties in bin/solr and inside of Solr we only care about sys.props. This has served us quite well, but is also has certain disadvantages:

      • Naming mismatches. You have one config name in the xml file, one as system property and yet another for environment variable.
      • Duplicate code to deal with type conversion, and converting comma separated lists from env.var into Java lists
      • Every new config option needs to touch bin/solr, bin/solr.cmd and often more.

      In the world of containers and k8s, we want to configure almost every aspect of an app using environment variables. It is sometimes also more secure than passing sys.props on the cmdline since they won't show up in a "ps".

      So this is a proposal to unify all Solr's configs in a more structured way

      • Make naming a convention. All env.variable should be uppercase with format SOLR_X_Y and all sys.propos should be lowercase with the format solr.x.y. Perhaps solr.camelCase should map to SOLR_CAMEL_CASE, or we discourage camel case in favour of dots.
      • Add a central ConfigResolver class to Solr that can answer e.g. getInt("solr.my.number") and it would return either prop solr.my.number or SOLR_MY_NUMBER. Similar for String, bool etc, and with fallback-values
      • List support, e.g. getListOfStrings("solr.modules") and it would return a List<String> from either solr.modules or SOLR_MODULES, supporting comma-separated, custom separator and why not also json list format ["foo","bar"]?

      A pitfall of using environment variables directly is testing, since env.vars are immutable. I suggest we solve this by reading all SOLR_* env.variables on startup and inserting them into a static, mutable map somewhere which is the single source of truth for env.vars. Then we can ban the use of System.getenv().

      Attachments

        Issue Links

          Activity

            People

              janhoy Jan Høydahl
              janhoy Jan Høydahl
              Votes:
              1 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 10h 20m
                  10h 20m