Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-10352

Low entropy warning in bin/solr script

    Details

    • Type: Improvement
    • Status: Reopened
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 7.0
    • Component/s: SolrCLI
    • Security Level: Public (Default Security Level. Issues are Public)
    • Labels:
      None

      Description

      We should add a warning in the startup script for Linux, if the output of the following is below a certain threshold (maybe 300?). The warning could indicate that features like UUIDField, SSL etc. might not work properly (or be slow). As a hint, we could then suggest the user to configure a non blocking SecureRandom (SOLR-10338) or install rng-tools, haveged etc.

      cat /proc/sys/kernel/random/entropy_avail

      Original discussion:
      https://issues.apache.org/jira/browse/SOLR-10338?focusedCommentId=15938904&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15938904

      1. SOLR-10352.patch
        1 kB
        Esther Quansah

        Activity

        Hide
        mihaly.toth Mihaly Toth added a comment -

        I dont want to hijack the original intention of this issue. So let me know if this is not the point ..

        I think it is a valid point that entropy might run out also in production not just in test suites. Just came to my mind what if we had non-blocking SecureRandom as a default in the startup scripts. And - if overwritten - the warning could give the feedback to the user.

        I think this link was cited in one of the issues. It argues for the usage of /dev/urandom:
        http://www.2uo.de/myths-about-urandom/

        Show
        mihaly.toth Mihaly Toth added a comment - I dont want to hijack the original intention of this issue. So let me know if this is not the point .. I think it is a valid point that entropy might run out also in production not just in test suites. Just came to my mind what if we had non-blocking SecureRandom as a default in the startup scripts. And - if overwritten - the warning could give the feedback to the user. I think this link was cited in one of the issues. It argues for the usage of /dev/urandom : http://www.2uo.de/myths-about-urandom/
        Hide
        esther.quansah Esther Quansah added a comment -

        Attached patch to include warning if available entropy is below 300

        Show
        esther.quansah Esther Quansah added a comment - Attached patch to include warning if available entropy is below 300
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 2ba54a36babd4cb6f2fb97e0f550d4980dbbced5 in lucene-solr's branch refs/heads/master from Ishan Chattopadhyaya
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=2ba54a3 ]

        SOLR-10352: bin/solr script now prints warning when available system entropy is lower than 300

        Show
        jira-bot ASF subversion and git services added a comment - Commit 2ba54a36babd4cb6f2fb97e0f550d4980dbbced5 in lucene-solr's branch refs/heads/master from Ishan Chattopadhyaya [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=2ba54a3 ] SOLR-10352 : bin/solr script now prints warning when available system entropy is lower than 300
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 6ecbe32dc2fdd2c3103e91cc9650395184f6499d in lucene-solr's branch refs/heads/branch_6x from Ishan Chattopadhyaya
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=6ecbe32 ]

        SOLR-10352: bin/solr script now prints warning when available system entropy is lower than 300

        Show
        jira-bot ASF subversion and git services added a comment - Commit 6ecbe32dc2fdd2c3103e91cc9650395184f6499d in lucene-solr's branch refs/heads/branch_6x from Ishan Chattopadhyaya [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=6ecbe32 ] SOLR-10352 : bin/solr script now prints warning when available system entropy is lower than 300
        Hide
        ichattopadhyaya Ishan Chattopadhyaya added a comment -
        Show
        ichattopadhyaya Ishan Chattopadhyaya added a comment - Thanks Esther Quansah !
        Hide
        ichattopadhyaya Ishan Chattopadhyaya added a comment -

        Apologies, I missed Mihaly's comment on this issue before closing it. Reopening this so that discussion can continue.

        Show
        ichattopadhyaya Ishan Chattopadhyaya added a comment - Apologies, I missed Mihaly's comment on this issue before closing it. Reopening this so that discussion can continue.
        Hide
        ichattopadhyaya Ishan Chattopadhyaya added a comment -

        Just came to my mind what if we had non-blocking SecureRandom as a default in the startup scripts

        My thought is that we should not change this by default, since /dev/random has been preferred by cryptographers and sysadmins for SSL. However, since the article argues that there are no downsides of using /dev/urandom, I think we can recommend that hte user use that when the entropy is low. This could be included in the warning message from the script. What do you think?

        Show
        ichattopadhyaya Ishan Chattopadhyaya added a comment - Just came to my mind what if we had non-blocking SecureRandom as a default in the startup scripts My thought is that we should not change this by default, since /dev/random has been preferred by cryptographers and sysadmins for SSL. However, since the article argues that there are no downsides of using /dev/urandom, I think we can recommend that hte user use that when the entropy is low. This could be included in the warning message from the script. What do you think?
        Hide
        hossman Hoss Man added a comment -

        Personally I think we really should not convolute the two issues of:

        • warning if entropy is low
        • changing the default source of entropy.

        those should really be 2 completely distinct discussions.

        one is an simple choice/discussion: is there any cost/overhead of giving the user a warning about entropy?

        The other is a more nuanced discussion about the risks/rewards of using diff sources of entropy and how that affects the confidence in our encryption based features: that deserves a lot more discussion in it's own jira.


        With that said: here are my thoughts on the current patch/commit made so far in this jira...

        I don't think it's useful as implemented.

        IIUC having this type of check solely on startup may be missleading to users – just because there is "low" entropy available when solr starts up doesn't mean there will be low of entropy for the (long) life of the solr server process. LIkewise if there is "high" entropy on startup that doesn't mean everything will be fine and there's nothing to worry about: the available entropy could drop over time and cause performance issues later.

        Rather then warning about this in bin/solr I feel like this type of information should be exposed by the solr metrics code, so people can easily monitor it over the life of the solr server process – either via a command line script we could provide, or via JMX, or via the admin UI ... we could even consider putting incorporating some specific "node health" metrics (entropy level, max open files, free disk, etc...) directly into the main screen of the Admin UI along with specific warnings/suggestions such as the text this issue added about SSL & UUIDField.

        Show
        hossman Hoss Man added a comment - Personally I think we really should not convolute the two issues of: warning if entropy is low changing the default source of entropy. those should really be 2 completely distinct discussions. one is an simple choice/discussion: is there any cost/overhead of giving the user a warning about entropy? The other is a more nuanced discussion about the risks/rewards of using diff sources of entropy and how that affects the confidence in our encryption based features: that deserves a lot more discussion in it's own jira. With that said: here are my thoughts on the current patch/commit made so far in this jira... I don't think it's useful as implemented. IIUC having this type of check solely on startup may be missleading to users – just because there is "low" entropy available when solr starts up doesn't mean there will be low of entropy for the (long) life of the solr server process. LIkewise if there is "high" entropy on startup that doesn't mean everything will be fine and there's nothing to worry about: the available entropy could drop over time and cause performance issues later. Rather then warning about this in bin/solr I feel like this type of information should be exposed by the solr metrics code, so people can easily monitor it over the life of the solr server process – either via a command line script we could provide, or via JMX, or via the admin UI ... we could even consider putting incorporating some specific "node health" metrics (entropy level, max open files, free disk, etc...) directly into the main screen of the Admin UI along with specific warnings/suggestions such as the text this issue added about SSL & UUIDField.
        Hide
        ichattopadhyaya Ishan Chattopadhyaya added a comment -

        Hoss Man, I think it is a good point that entropy could, in theory, go up and down over the course of the life of a Solr process. In practice, a host with high entropy (say baremetal) continues to remain in high entropy available state; and low entropy hosts (say VMs) continue to remain in a low entropy mode. So, although in theory a system could get in and out of the diminished available entropy state, in practice, afaik, a good system remains good and a bad one remains bad. Hence, a startup warning feels like a sensible thing to throw out there.

        Rather then warning about this in bin/solr I feel like this type of information should be exposed by the solr metrics code, so people can easily monitor it over the life of the solr server process

        I feel that a start up warning should definitely be thrown, since we already know that there will be a problem. Having metrics support and UI warning is a great idea. However, I think we should do both (startup warning and metrics/UI warning).

        Show
        ichattopadhyaya Ishan Chattopadhyaya added a comment - Hoss Man , I think it is a good point that entropy could, in theory, go up and down over the course of the life of a Solr process. In practice, a host with high entropy (say baremetal) continues to remain in high entropy available state; and low entropy hosts (say VMs) continue to remain in a low entropy mode. So, although in theory a system could get in and out of the diminished available entropy state, in practice, afaik, a good system remains good and a bad one remains bad. Hence, a startup warning feels like a sensible thing to throw out there. Rather then warning about this in bin/solr I feel like this type of information should be exposed by the solr metrics code, so people can easily monitor it over the life of the solr server process I feel that a start up warning should definitely be thrown, since we already know that there will be a problem. Having metrics support and UI warning is a great idea. However, I think we should do both (startup warning and metrics/UI warning).
        Hide
        manokovacs Mano Kovacs added a comment -

        Ishan Chattopadhyaya, Esther Quansah, may I ask why the final commit makes comparison with 30000? AFAIK, it never goes over 4096 [1].

        [1] "The read-only file entropy_avail gives the available entropy. Normally, this will be 4096 (bits), a full entropy pool." - https://linux.die.net/man/4/random

        Show
        manokovacs Mano Kovacs added a comment - Ishan Chattopadhyaya , Esther Quansah , may I ask why the final commit makes comparison with 30000? AFAIK, it never goes over 4096 [1] . [1] "The read-only file entropy_avail gives the available entropy. Normally, this will be 4096 (bits), a full entropy pool." - https://linux.die.net/man/4/random
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit efdb04d06c9d37b543ab0469c65f3474c34d455a in lucene-solr's branch refs/heads/master from Ishan Chattopadhyaya
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=efdb04d ]

        SOLR-10352: Fixing available entropy warning limit to 300

        Show
        jira-bot ASF subversion and git services added a comment - Commit efdb04d06c9d37b543ab0469c65f3474c34d455a in lucene-solr's branch refs/heads/master from Ishan Chattopadhyaya [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=efdb04d ] SOLR-10352 : Fixing available entropy warning limit to 300
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit e03d82eb45083a11ce97038d8054c3077068bed4 in lucene-solr's branch refs/heads/branch_6x from Ishan Chattopadhyaya
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=e03d82e ]

        SOLR-10352: Fixing available entropy warning limit to 300

        Show
        jira-bot ASF subversion and git services added a comment - Commit e03d82eb45083a11ce97038d8054c3077068bed4 in lucene-solr's branch refs/heads/branch_6x from Ishan Chattopadhyaya [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=e03d82e ] SOLR-10352 : Fixing available entropy warning limit to 300
        Hide
        ichattopadhyaya Ishan Chattopadhyaya added a comment -

        Thanks for the catch, Mano. I was testing Esther's patch with that high value to trigger that warning, but forgot revert to her patch before committing.

        Show
        ichattopadhyaya Ishan Chattopadhyaya added a comment - Thanks for the catch, Mano. I was testing Esther's patch with that high value to trigger that warning, but forgot revert to her patch before committing.

          People

          • Assignee:
            Unassigned
            Reporter:
            ichattopadhyaya Ishan Chattopadhyaya
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:

              Development