Lucene - Core
  1. Lucene - Core
  2. LUCENE-6795

Remove all accessClassInPackage permissions / remove uses AccessibleObject#setAccessible() to make ready for Java 9 Jigsaw

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 5.4, 6.0
    • Component/s: None
    • Labels:
    • Lucene Fields:
      New

      Description

      With jigsaw builds this stuff is not allowed, its no longer an option of security manager or not.

      So we should remove these permissions and fix the test leaks, crappy code, remove hacks, etc.

      If the hack is really somehow needed for some special reason (e.g. well known case like mmap), it needs to gracefully handle not being able to do this, the code and tests should still "work" if it cannot do the hack.

      Otherwise there will be problems for java 9.

      1. LUCENE-6795-forbids.patch
        29 kB
        Uwe Schindler
      2. LUCENE-6795-forbids.patch
        14 kB
        Uwe Schindler

        Issue Links

          Activity

          Hide
          ASF subversion and git services added a comment -

          Commit 1702139 from Robert Muir in branch 'dev/trunk'
          [ https://svn.apache.org/r1702139 ]

          LUCENE-6795: remove reflection hack completely, see http://mail.openjdk.java.net/pipermail/nio-dev/2015-September/003322.html

          Show
          ASF subversion and git services added a comment - Commit 1702139 from Robert Muir in branch 'dev/trunk' [ https://svn.apache.org/r1702139 ] LUCENE-6795 : remove reflection hack completely, see http://mail.openjdk.java.net/pipermail/nio-dev/2015-September/003322.html
          Hide
          ASF subversion and git services added a comment -

          Commit 1702141 from Robert Muir in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1702141 ]

          LUCENE-6795: remove reflection hack completely, see http://mail.openjdk.java.net/pipermail/nio-dev/2015-September/003322.html

          Show
          ASF subversion and git services added a comment - Commit 1702141 from Robert Muir in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1702141 ] LUCENE-6795 : remove reflection hack completely, see http://mail.openjdk.java.net/pipermail/nio-dev/2015-September/003322.html
          Hide
          ASF subversion and git services added a comment -

          Commit 1702144 from Robert Muir in branch 'dev/trunk'
          [ https://svn.apache.org/r1702144 ]

          LUCENE-6795: fix some test issues (mostly static leaks)

          Show
          ASF subversion and git services added a comment - Commit 1702144 from Robert Muir in branch 'dev/trunk' [ https://svn.apache.org/r1702144 ] LUCENE-6795 : fix some test issues (mostly static leaks)
          Hide
          ASF subversion and git services added a comment -

          Commit 1702148 from Robert Muir in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1702148 ]

          LUCENE-6795: fix some test issues (mostly static leaks)

          Show
          ASF subversion and git services added a comment - Commit 1702148 from Robert Muir in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1702148 ] LUCENE-6795 : fix some test issues (mostly static leaks)
          Hide
          ASF subversion and git services added a comment -

          Commit 1702149 from Robert Muir in branch 'dev/trunk'
          [ https://svn.apache.org/r1702149 ]

          LUCENE-6795: remove unnecessary reflection, everything in question is public

          Show
          ASF subversion and git services added a comment - Commit 1702149 from Robert Muir in branch 'dev/trunk' [ https://svn.apache.org/r1702149 ] LUCENE-6795 : remove unnecessary reflection, everything in question is public
          Hide
          ASF subversion and git services added a comment -

          Commit 1702151 from Robert Muir in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1702151 ]

          LUCENE-6795: remove unnecessary reflection, everything in question is public

          Show
          ASF subversion and git services added a comment - Commit 1702151 from Robert Muir in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1702151 ] LUCENE-6795 : remove unnecessary reflection, everything in question is public
          Hide
          ASF subversion and git services added a comment -

          Commit 1702152 from Robert Muir in branch 'dev/trunk'
          [ https://svn.apache.org/r1702152 ]

          LUCENE-6795: remove assert that is no longer allowed with jigsaw

          Show
          ASF subversion and git services added a comment - Commit 1702152 from Robert Muir in branch 'dev/trunk' [ https://svn.apache.org/r1702152 ] LUCENE-6795 : remove assert that is no longer allowed with jigsaw
          Hide
          ASF subversion and git services added a comment -

          Commit 1702153 from Robert Muir in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1702153 ]

          LUCENE-6795: remove assert that is no longer allowed with jigsaw

          Show
          ASF subversion and git services added a comment - Commit 1702153 from Robert Muir in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1702153 ] LUCENE-6795 : remove assert that is no longer allowed with jigsaw
          Hide
          ASF subversion and git services added a comment -

          Commit 1702154 from Robert Muir in branch 'dev/trunk'
          [ https://svn.apache.org/r1702154 ]

          LUCENE-6795: remove zero-byte file

          Show
          ASF subversion and git services added a comment - Commit 1702154 from Robert Muir in branch 'dev/trunk' [ https://svn.apache.org/r1702154 ] LUCENE-6795 : remove zero-byte file
          Hide
          ASF subversion and git services added a comment -

          Commit 1702155 from Robert Muir in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1702155 ]

          LUCENE-6795: remove zero-byte file

          Show
          ASF subversion and git services added a comment - Commit 1702155 from Robert Muir in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1702155 ] LUCENE-6795 : remove zero-byte file
          Hide
          Robert Muir added a comment -

          lucene tests are passing now under the jigsaw nightly.

          Show
          Robert Muir added a comment - lucene tests are passing now under the jigsaw nightly.
          Hide
          Uwe Schindler added a comment -

          Hi, I started to look into Solr. In fact after some fruitful discussions with the Jigsaw people I figured out what the "main" issue is.

          Also for Solr we should put AccessibleObject#setAccessible() onto the forbidden-apis list and only allow it for some very special cases (which are also supported by Oracle): The MMapDir cleaner. Any comments?

          Solr has some violations on this, because it tries to make hidden fields or methods available that are part of the Java runtime. With Java 9 Jigsaw (soon to be merged into Java 9 mainline), any call to setAccessible() on any class which is part of the Java runtime fails. PERIOD. No way to work around that. There are only some "exceptions" for some APIs (see JEP 260), but the general rule holds. If you try to make one of the internal members accessible, setAccessible throws a new RuntimeException. This lead to some discussions (e.g. the Groovy runtime engine is completely broken by that, see GROOVY-7587). Oracle is working on solutions for that, but the general rule holds: http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-September/004530.html

          I will now fix some issues in Solr. One of those is the SystemInfoHandler. This just uses the OperatingSystemMXBean in a "wrong way", because it tries to call hidden methods, which are only implemented by an interface. So it fails. You just need to cast to correct interface and then you can call the methods as usual. I will commit this fix in a minute.

          Show
          Uwe Schindler added a comment - Hi, I started to look into Solr. In fact after some fruitful discussions with the Jigsaw people I figured out what the "main" issue is. Also for Solr we should put AccessibleObject#setAccessible() onto the forbidden-apis list and only allow it for some very special cases (which are also supported by Oracle): The MMapDir cleaner. Any comments? Solr has some violations on this, because it tries to make hidden fields or methods available that are part of the Java runtime. With Java 9 Jigsaw (soon to be merged into Java 9 mainline), any call to setAccessible() on any class which is part of the Java runtime fails. PERIOD. No way to work around that. There are only some "exceptions" for some APIs (see JEP 260 ), but the general rule holds. If you try to make one of the internal members accessible, setAccessible throws a new RuntimeException. This lead to some discussions (e.g. the Groovy runtime engine is completely broken by that, see GROOVY-7587 ). Oracle is working on solutions for that, but the general rule holds: http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-September/004530.html I will now fix some issues in Solr. One of those is the SystemInfoHandler. This just uses the OperatingSystemMXBean in a "wrong way", because it tries to call hidden methods, which are only implemented by an interface. So it fails. You just need to cast to correct interface and then you can call the methods as usual. I will commit this fix in a minute.
          Hide
          ASF subversion and git services added a comment -

          Commit 1702613 from Uwe Schindler in branch 'dev/trunk'
          [ https://svn.apache.org/r1702613 ]

          LUCENE-6795: Fix SystemInfoHandler to work with Java 9 Jigsaw

          Show
          ASF subversion and git services added a comment - Commit 1702613 from Uwe Schindler in branch 'dev/trunk' [ https://svn.apache.org/r1702613 ] LUCENE-6795 : Fix SystemInfoHandler to work with Java 9 Jigsaw
          Hide
          ASF subversion and git services added a comment -

          Commit 1702614 from Uwe Schindler in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1702614 ]

          Merged revision(s) 1702613 from lucene/dev/trunk:
          LUCENE-6795: Fix SystemInfoHandler to work with Java 9 Jigsaw

          Show
          ASF subversion and git services added a comment - Commit 1702614 from Uwe Schindler in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1702614 ] Merged revision(s) 1702613 from lucene/dev/trunk: LUCENE-6795 : Fix SystemInfoHandler to work with Java 9 Jigsaw
          Hide
          ASF subversion and git services added a comment -

          Commit 1702628 from Uwe Schindler in branch 'dev/trunk'
          [ https://svn.apache.org/r1702628 ]

          LUCENE-6795: Improve SystemInfoHandler to get the bean directly instead of downcasting (thanks Peter Levart from jigsaw-dev), also more fixes with UOE thrown by RuntimeMXBean#getBootClassPath()

          Show
          ASF subversion and git services added a comment - Commit 1702628 from Uwe Schindler in branch 'dev/trunk' [ https://svn.apache.org/r1702628 ] LUCENE-6795 : Improve SystemInfoHandler to get the bean directly instead of downcasting (thanks Peter Levart from jigsaw-dev), also more fixes with UOE thrown by RuntimeMXBean#getBootClassPath()
          Hide
          ASF subversion and git services added a comment -

          Commit 1702629 from Uwe Schindler in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1702629 ]

          Merged revision(s) 1702628 from lucene/dev/trunk:
          LUCENE-6795: Improve SystemInfoHandler to get the bean directly instead of downcasting (thanks Peter Levart from jigsaw-dev), also more fixes with UOE thrown by RuntimeMXBean#getBootClassPath()

          Show
          ASF subversion and git services added a comment - Commit 1702629 from Uwe Schindler in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1702629 ] Merged revision(s) 1702628 from lucene/dev/trunk: LUCENE-6795 : Improve SystemInfoHandler to get the bean directly instead of downcasting (thanks Peter Levart from jigsaw-dev), also more fixes with UOE thrown by RuntimeMXBean#getBootClassPath()
          Hide
          ASF subversion and git services added a comment -

          Commit 1702656 from Uwe Schindler in branch 'dev/trunk'
          [ https://svn.apache.org/r1702656 ]

          LUCENE-6795: Make SystemInfoHandler more dynamic, add support for IBM J9 OperatingSystemMXBean

          Show
          ASF subversion and git services added a comment - Commit 1702656 from Uwe Schindler in branch 'dev/trunk' [ https://svn.apache.org/r1702656 ] LUCENE-6795 : Make SystemInfoHandler more dynamic, add support for IBM J9 OperatingSystemMXBean
          Hide
          ASF subversion and git services added a comment -

          Commit 1702657 from Uwe Schindler in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1702657 ]

          Merged revision(s) 1702656 from lucene/dev/trunk:
          LUCENE-6795: Make SystemInfoHandler more dynamic, add support for IBM J9 OperatingSystemMXBean

          Show
          ASF subversion and git services added a comment - Commit 1702657 from Uwe Schindler in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1702657 ] Merged revision(s) 1702656 from lucene/dev/trunk: LUCENE-6795 : Make SystemInfoHandler more dynamic, add support for IBM J9 OperatingSystemMXBean
          Hide
          ASF subversion and git services added a comment -

          Commit 1702669 from Uwe Schindler in branch 'dev/trunk'
          [ https://svn.apache.org/r1702669 ]

          LUCENE-6795: Improve SystemInfoHandler for IBM J9 OperatingSystemMXBean (some properties may throw UOE)

          Show
          ASF subversion and git services added a comment - Commit 1702669 from Uwe Schindler in branch 'dev/trunk' [ https://svn.apache.org/r1702669 ] LUCENE-6795 : Improve SystemInfoHandler for IBM J9 OperatingSystemMXBean (some properties may throw UOE)
          Hide
          ASF subversion and git services added a comment -

          Commit 1702670 from Uwe Schindler in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1702670 ]

          Merged revision(s) 1702669 from lucene/dev/trunk:
          LUCENE-6795: Improve SystemInfoHandler for IBM J9 OperatingSystemMXBean (some properties may throw UOE)

          Show
          ASF subversion and git services added a comment - Commit 1702670 from Uwe Schindler in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1702670 ] Merged revision(s) 1702669 from lucene/dev/trunk: LUCENE-6795 : Improve SystemInfoHandler for IBM J9 OperatingSystemMXBean (some properties may throw UOE)
          Hide
          Uwe Schindler added a comment -

          Patch to ban setAccessible. Only Lucene for now.

          Solr has some violations, but this is easy to fix. It starts almost successfully in single-core mode with Java 9 Jigsaw.

          Once I have done Solr I will commit this.

          There is the JakartaRegexCapabilities in Lucene's sandbox module. This is long outdated, we should remove in trunk and deprecate in 5.x. Any comments on this?

          Show
          Uwe Schindler added a comment - Patch to ban setAccessible. Only Lucene for now. Solr has some violations, but this is easy to fix. It starts almost successfully in single-core mode with Java 9 Jigsaw. Once I have done Solr I will commit this. There is the JakartaRegexCapabilities in Lucene's sandbox module. This is long outdated, we should remove in trunk and deprecate in 5.x. Any comments on this?
          Hide
          Robert Muir added a comment -

          great cleanup!

          +1 to deprecate/remove that slow regex query.

          Show
          Robert Muir added a comment - great cleanup! +1 to deprecate/remove that slow regex query.
          Hide
          Uwe Schindler added a comment -

          Final patch, also for Solr.

          Show
          Uwe Schindler added a comment - Final patch, also for Solr.
          Hide
          Uwe Schindler added a comment -

          Small fix with SolrDispatchFilter

          Show
          Uwe Schindler added a comment - Small fix with SolrDispatchFilter
          Hide
          ASF subversion and git services added a comment -

          Commit 1702690 from Uwe Schindler in branch 'dev/trunk'
          [ https://svn.apache.org/r1702690 ]

          LUCENE-6795: Forbid AccessibleObject#setAccessible and suppress legitimate uses / wrap with doPrivileged

          Show
          ASF subversion and git services added a comment - Commit 1702690 from Uwe Schindler in branch 'dev/trunk' [ https://svn.apache.org/r1702690 ] LUCENE-6795 : Forbid AccessibleObject#setAccessible and suppress legitimate uses / wrap with doPrivileged
          Hide
          ASF subversion and git services added a comment -

          Commit 1702691 from Uwe Schindler in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1702691 ]

          Merged revision(s) 1702690 from lucene/dev/trunk:
          LUCENE-6795: Forbid AccessibleObject#setAccessible and suppress legitimate uses / wrap with doPrivileged

          Show
          ASF subversion and git services added a comment - Commit 1702691 from Uwe Schindler in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1702691 ] Merged revision(s) 1702690 from lucene/dev/trunk: LUCENE-6795 : Forbid AccessibleObject#setAccessible and suppress legitimate uses / wrap with doPrivileged
          Hide
          Uwe Schindler added a comment -

          Committed. I will open new issue to remove RegexpQuery in Sandbox.

          Show
          Uwe Schindler added a comment - Committed. I will open new issue to remove RegexpQuery in Sandbox.
          Hide
          ASF subversion and git services added a comment -

          Commit 1702692 from Uwe Schindler in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1702692 ]

          LUCENE-6795: Improve RamUsageTester in 5.x (no lambdas available)

          Show
          ASF subversion and git services added a comment - Commit 1702692 from Uwe Schindler in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1702692 ] LUCENE-6795 : Improve RamUsageTester in 5.x (no lambdas available)
          Hide
          ASF subversion and git services added a comment -

          Commit 1702693 from Uwe Schindler in branch 'dev/trunk'
          [ https://svn.apache.org/r1702693 ]

          LUCENE-6795: Improve RamUsageTester in trunk

          Show
          ASF subversion and git services added a comment - Commit 1702693 from Uwe Schindler in branch 'dev/trunk' [ https://svn.apache.org/r1702693 ] LUCENE-6795 : Improve RamUsageTester in trunk
          Hide
          ASF subversion and git services added a comment -

          Commit 1702714 from Uwe Schindler in branch 'dev/trunk'
          [ https://svn.apache.org/r1702714 ]

          LUCENE-6795: Add CHANGES entry.

          Show
          ASF subversion and git services added a comment - Commit 1702714 from Uwe Schindler in branch 'dev/trunk' [ https://svn.apache.org/r1702714 ] LUCENE-6795 : Add CHANGES entry.
          Hide
          ASF subversion and git services added a comment -

          Commit 1702715 from Uwe Schindler in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1702715 ]

          Merged revision(s) 1702714 from lucene/dev/trunk:
          LUCENE-6795: Add CHANGES entry.

          Show
          ASF subversion and git services added a comment - Commit 1702715 from Uwe Schindler in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1702715 ] Merged revision(s) 1702714 from lucene/dev/trunk: LUCENE-6795 : Add CHANGES entry.
          Hide
          ASF subversion and git services added a comment -

          Commit 1702717 from Uwe Schindler in branch 'dev/trunk'
          [ https://svn.apache.org/r1702717 ]

          LUCENE-6795: Add CHANGES entry also to Solr (SystemInfoHandler improvements)

          Show
          ASF subversion and git services added a comment - Commit 1702717 from Uwe Schindler in branch 'dev/trunk' [ https://svn.apache.org/r1702717 ] LUCENE-6795 : Add CHANGES entry also to Solr (SystemInfoHandler improvements)
          Hide
          ASF subversion and git services added a comment -

          Commit 1702718 from Uwe Schindler in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1702718 ]

          Merged revision(s) 1702717 from lucene/dev/trunk:
          LUCENE-6795: Add CHANGES entry also to Solr (SystemInfoHandler improvements)

          Show
          ASF subversion and git services added a comment - Commit 1702718 from Uwe Schindler in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1702718 ] Merged revision(s) 1702717 from lucene/dev/trunk: LUCENE-6795 : Add CHANGES entry also to Solr (SystemInfoHandler improvements)
          Hide
          ASF subversion and git services added a comment -

          Commit 1702770 from Uwe Schindler in branch 'dev/trunk'
          [ https://svn.apache.org/r1702770 ]

          LUCENE-6795: Fix Solr tests that break RamUsageTester on Java 9 Jigsaw

          Show
          ASF subversion and git services added a comment - Commit 1702770 from Uwe Schindler in branch 'dev/trunk' [ https://svn.apache.org/r1702770 ] LUCENE-6795 : Fix Solr tests that break RamUsageTester on Java 9 Jigsaw
          Hide
          ASF subversion and git services added a comment -

          Commit 1702772 from Uwe Schindler in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1702772 ]

          Merged revision(s) 1702770 from lucene/dev/trunk:
          LUCENE-6795: Fix Solr tests that break RamUsageTester on Java 9 Jigsaw

          Show
          ASF subversion and git services added a comment - Commit 1702772 from Uwe Schindler in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1702772 ] Merged revision(s) 1702770 from lucene/dev/trunk: LUCENE-6795 : Fix Solr tests that break RamUsageTester on Java 9 Jigsaw
          Hide
          ASF subversion and git services added a comment -

          Commit 1702831 from Uwe Schindler in branch 'dev/trunk'
          [ https://svn.apache.org/r1702831 ]

          Revert the AbstractZkTestCase changes for now
          was: LUCENE-6795: Fix Solr tests that break RamUsageTester on Java 9 Jigsaw

          Show
          ASF subversion and git services added a comment - Commit 1702831 from Uwe Schindler in branch 'dev/trunk' [ https://svn.apache.org/r1702831 ] Revert the AbstractZkTestCase changes for now was: LUCENE-6795 : Fix Solr tests that break RamUsageTester on Java 9 Jigsaw
          Hide
          ASF subversion and git services added a comment -

          Commit 1702833 from Uwe Schindler in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1702833 ]

          Merged revision(s) 1702831 from lucene/dev/trunk:
          Revert the AbstractZkTestCase changes for now
          was: LUCENE-6795: Fix Solr tests that break RamUsageTester on Java 9 Jigsaw

          Show
          ASF subversion and git services added a comment - Commit 1702833 from Uwe Schindler in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1702833 ] Merged revision(s) 1702831 from lucene/dev/trunk: Revert the AbstractZkTestCase changes for now was: LUCENE-6795 : Fix Solr tests that break RamUsageTester on Java 9 Jigsaw

            People

            • Assignee:
              Uwe Schindler
              Reporter:
              Robert Muir
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development