Lucene.Net
  1. Lucene.Net
  2. LUCENENET-169

Changes to make Lucene.NET compatible with ASP.NET Medium Trust Level, in hosting environments (like GoDaddy...)

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Environment:

      ASP.NET

      Description

      Microsoft has a configuration file for shared hosting for what they call "Medium Trust". There are a couple places in FSDirectory.cs that violate the restrictions of Medium Trust, but I coded workarounds, shown below.

      #1)

      // Corey Trager, Oct 2008: Commented call to GetTempPath to workaround permission restrictions at shared host.
      // LOCK_DIR isn't used anyway.
      public static readonly System.String LOCK_DIR = null; // SupportClass.AppSettings.Get("Lucene.Net.lockDir", System.IO.Path.GetTempPath());

      #2)

      /// <summary>Returns an array of strings, one for each Lucene index file in the directory. </summary>
      public override System.String[] List()
      {

      /* Changes by Corey Trager, Oct 2008, to workaround permission restrictions at shared host */
      System.IO.DirectoryInfo dir = new System.IO.DirectoryInfo(directory.FullName);
      System.IO.FileInfo[] files = dir.GetFiles();
      string[] list = new string[files.Length];
      for (int i = 0; i < files.Length; i++)

      { list[i] = files[i].Name; }

      return list;
      /* end of changes */

      // System.String[] files = SupportClass.FileSupport.GetLuceneIndexFiles(directory.FullName, IndexFileNameFilter.GetFilter());
      // for (int i = 0; i < files.Length; i++)
      //

      { // System.IO.FileInfo fi = new System.IO.FileInfo(files[i]); // files[i] = fi.Name; // }

      // return files;
      }

      1. FSDirectory.patch
        0.7 kB
        Digy
      2. FSDirectory-mediumtrust.patch
        0.6 kB
        Martijn Boland
      3. FSDirectory.patch
        0.9 kB
        Digy

        Activity

        Hide
        Digy added a comment -

        Since LOCK_DIR is depreciated with 2.1 and not used anymore in 2.3.x, I plan to commit the "FSDirectory.patch" soon.

        DIGY

        Show
        Digy added a comment - Since LOCK_DIR is depreciated with 2.1 and not used anymore in 2.3.x, I plan to commit the "FSDirectory.patch" soon. DIGY
        Hide
        Digy added a comment -

        Orginal code

        public override System.String[] List()
        {
        System.String[] files = SupportClass.FileSupport.GetLuceneIndexFiles(directory.FullName, IndexFileNameFilter.GetFilter());
        for (int i = 0; i < files.Length; i++)
        {
        System.IO.FileInfo fi = new System.IO.FileInfo(files[i]);
        files[i] = fi.Name;
        }
        return files;
        }
        

        returns only the files belonging to Lucene.Net index
        but the proposed one

        public override System.String[] List()
        {
        
        /* Changes by Corey Trager, Oct 2008, to workaround permission restrictions at shared host */
        System.IO.DirectoryInfo dir = new System.IO.DirectoryInfo(directory.FullName);
        System.IO.FileInfo[] files = dir.GetFiles();
        string[] list = new string[files.Length];
        for (int i = 0; i < files.Length; i++)
        {
        list[i] = files[i].Name;
        }
        
        return list;
        }
        

        returns all the files in the index directory.

        A better solution would be to change the "SupportClass.FileSupport.GetLuceneIndexFiles" so that it supports Medium Trust level.

        DIGY

        Show
        Digy added a comment - Orginal code public override System . String [] List() { System . String [] files = SupportClass.FileSupport.GetLuceneIndexFiles(directory.FullName, IndexFileNameFilter.GetFilter()); for ( int i = 0; i < files.Length; i++) { System .IO.FileInfo fi = new System .IO.FileInfo(files[i]); files[i] = fi.Name; } return files; } returns only the files belonging to Lucene.Net index but the proposed one public override System . String [] List() { /* Changes by Corey Trager, Oct 2008, to workaround permission restrictions at shared host */ System .IO.DirectoryInfo dir = new System .IO.DirectoryInfo(directory.FullName); System .IO.FileInfo[] files = dir.GetFiles(); string[] list = new string[files.Length]; for ( int i = 0; i < files.Length; i++) { list[i] = files[i].Name; } return list; } returns all the files in the index directory. A better solution would be to change the "SupportClass.FileSupport.GetLuceneIndexFiles" so that it supports Medium Trust level. DIGY
        Hide
        Digy added a comment -

        After reading that doc; http://msdn.microsoft.com/en-us/library/ms998341.aspx

        • Is this really a problem of Lucene.Net?
        • Is it correct to use Lucene.Net in an ASP.Net app. or should there be a search service which can be accessed by IIS or directly from client apps?
        • Isn't it easier to control the lifetime of IndexReader and/or IndexWriter in an application(Console,Windows, Service) other than in IIS.

        DIGY

        Show
        Digy added a comment - After reading that doc; http://msdn.microsoft.com/en-us/library/ms998341.aspx Is this really a problem of Lucene.Net? Is it correct to use Lucene.Net in an ASP.Net app. or should there be a search service which can be accessed by IIS or directly from client apps? Isn't it easier to control the lifetime of IndexReader and/or IndexWriter in an application(Console,Windows, Service) other than in IIS. DIGY
        Hide
        Corey Trager added a comment -

        Regarding Medium Trust, Microsoft's out-of-the-box default Medium Trust config file is what it is. I read that some shared hosting companies tweak it, others don't. Using Lucene.NET in the real world just goes smoother if Lucene.NET doesn't conflict with the Microsoft defaults.

        Regarding:

        • Is it correct to use Lucene.Net in an ASP.Net app. or should there be a search service which can be accessed by IIS or directly from client apps?
        • Isn't it easier to control the lifetime of IndexReader and/or IndexWriter in an application(Console,Windows, Service) other than in IIS.

        I don't know of any shared hosting company that would let you run your own Windows service, so building Lucene.NET into the ASP.NET code itself is the only realistic practical option. And, for my app, BugTracker.NET, it's working great for me and the hundreds/thousands of people who are also running it. Nobody has reported any issues with the architecture.

        I've documented how I integrated Lucene.NET into my ASP.NET application here:
        http://ifdefined.com/blog/post/2009/02/Full-Text-Search-in-ASPNET-using-LuceneNET.aspx

        Show
        Corey Trager added a comment - Regarding Medium Trust, Microsoft's out-of-the-box default Medium Trust config file is what it is. I read that some shared hosting companies tweak it, others don't. Using Lucene.NET in the real world just goes smoother if Lucene.NET doesn't conflict with the Microsoft defaults. Regarding: Is it correct to use Lucene.Net in an ASP.Net app. or should there be a search service which can be accessed by IIS or directly from client apps? Isn't it easier to control the lifetime of IndexReader and/or IndexWriter in an application(Console,Windows, Service) other than in IIS. I don't know of any shared hosting company that would let you run your own Windows service, so building Lucene.NET into the ASP.NET code itself is the only realistic practical option. And, for my app, BugTracker.NET, it's working great for me and the hundreds/thousands of people who are also running it. Nobody has reported any issues with the architecture. I've documented how I integrated Lucene.NET into my ASP.NET application here: http://ifdefined.com/blog/post/2009/02/Full-Text-Search-in-ASPNET-using-LuceneNET.aspx
        Hide
        Digy added a comment -

        Alltough I still think that implementing search services in an Web Application is not a good idea, I can understand that you don't want to change your architecture.
        So can you send a patch that doesn't change the logic of GetLuceneIndexFiles and compatible with "Medium Trust Level"?

        (PS: I don't remember anybody who is blaming BugTracker.NET, it is more related with how to use the Lucene.Net)

        DIGY

        Show
        Digy added a comment - Alltough I still think that implementing search services in an Web Application is not a good idea, I can understand that you don't want to change your architecture. So can you send a patch that doesn't change the logic of GetLuceneIndexFiles and compatible with "Medium Trust Level"? (PS: I don't remember anybody who is blaming BugTracker.NET, it is more related with how to use the Lucene.Net) DIGY
        Hide
        Corey Trager added a comment -

        I'll see if I can change the code. I didn't even pay attention to the filter argument. Oops.

        ....Again, regarding the architecture, I'm not understanding. Before I integrated it into my ASP.NET, I didn't know if it would work well or not, but it has worked well, so, isn't the proof in the pudding? Do you have specific concerns? If you prefer to contact me directly, at ctrager@yahoo.com, that's ok, or if you prefer to drop the topic, that's ok too....

        Show
        Corey Trager added a comment - I'll see if I can change the code. I didn't even pay attention to the filter argument. Oops. ....Again, regarding the architecture, I'm not understanding. Before I integrated it into my ASP.NET, I didn't know if it would work well or not, but it has worked well, so, isn't the proof in the pudding? Do you have specific concerns? If you prefer to contact me directly, at ctrager@yahoo.com, that's ok, or if you prefer to drop the topic, that's ok too....
        Hide
        Digy added a comment -

        Although you can overcome all of them somehow;

        • controlling the the lifetime of IndexWriter/IndexReader in a naturally manner,
        • reopening the IndexReader only when needed using (for ex) FileSystemWatcher,
        • providing a separation between data & bussiness layer,
        • providing other apps an interface that may want to write its own user interface,
        • accessing a single search service from different web apps/from load balanced web servers
        • controlling the lifetime of searching/indexing code (without being effected by the restart of the IIS processes automatically when some memory limit is exceeded (for ex.) )
        • Ability to access some system resources that can be restricted by IIS
          etc.
          make me think a separete search service is a better idea.But at last, it is a design decision of you.
          (Think, A WebApp+Solr in Java world)

        DIGY

        Show
        Digy added a comment - Although you can overcome all of them somehow; controlling the the lifetime of IndexWriter/IndexReader in a naturally manner, reopening the IndexReader only when needed using (for ex) FileSystemWatcher, providing a separation between data & bussiness layer, providing other apps an interface that may want to write its own user interface, accessing a single search service from different web apps/from load balanced web servers controlling the lifetime of searching/indexing code (without being effected by the restart of the IIS processes automatically when some memory limit is exceeded (for ex.) ) Ability to access some system resources that can be restricted by IIS etc. make me think a separete search service is a better idea.But at last, it is a design decision of you. (Think, A WebApp+Solr in Java world) DIGY
        Hide
        Luis Fco. Ramirez Daza Glez. added a comment -

        Hi all

        Definitly there are a lot of applications that can benefit of using Lucene in MEDIUM TRUST.
        We already have an application that we want to use in shared hosting enviroments. Planning a new one. And I'm also thinking of adding Lucene to DotNetNuke, that also will require MEDIUM TRUST.

        COREY: Are you planning to work on the patch for the GetLuceneIndexFiles?

        Show
        Luis Fco. Ramirez Daza Glez. added a comment - Hi all Definitly there are a lot of applications that can benefit of using Lucene in MEDIUM TRUST. We already have an application that we want to use in shared hosting enviroments. Planning a new one. And I'm also thinking of adding Lucene to DotNetNuke, that also will require MEDIUM TRUST. COREY : Are you planning to work on the patch for the GetLuceneIndexFiles?
        Hide
        Corey Trager added a comment -

        @Luis - Here's what I did today. I fetched the latest code from svn, built, and tried to run. I got an error message "Required permissions cannot be acquired". The callstack does not tell me what the offending code or even permission is. I applied the changes already documented here (I only needed #2), but that didn't help.

        I then fetched the Lucene.NET cod with revision 703747 and I was able to run under Medium Trust on my machine without alterning the code at all. It was way back in October when I had the problems on GoDaddy and I'm not sure if my problems were ever with the default Medium Trust configuration like my machine. Maybe GoDaddy's customized their Medium Trust. configuration? I don't want to experiment with my site on GoDaddy because I don't want the disruption.

        It would take me a lot of trial and error to figure out what the changes are since svn revision 703747 that conflict with my own default Medium Trust, and then to experiment with workarounds..... I think I'm just going to have to stick wih 703747.

        We are the only ones who would like to see Lucene.NET compatible with Medium Trust:

        http://social.msdn.microsoft.com/Forums/en-US/windowsazure/thread/d0ec059b-f136-488b-ab74-c2f4c95c659c
        http://www.mojoportal.com/mediumtrust.aspx
        http://www.klopfenstein.net/lorenz.aspx/lucene-net-and-nhibernate-search-on-medium-trust
        http://dev.communityserver.com/forums/p/478315/544018.aspx

        Show
        Corey Trager added a comment - @Luis - Here's what I did today. I fetched the latest code from svn, built, and tried to run. I got an error message "Required permissions cannot be acquired". The callstack does not tell me what the offending code or even permission is. I applied the changes already documented here (I only needed #2), but that didn't help. I then fetched the Lucene.NET cod with revision 703747 and I was able to run under Medium Trust on my machine without alterning the code at all. It was way back in October when I had the problems on GoDaddy and I'm not sure if my problems were ever with the default Medium Trust configuration like my machine. Maybe GoDaddy's customized their Medium Trust. configuration? I don't want to experiment with my site on GoDaddy because I don't want the disruption. It would take me a lot of trial and error to figure out what the changes are since svn revision 703747 that conflict with my own default Medium Trust, and then to experiment with workarounds..... I think I'm just going to have to stick wih 703747. We are the only ones who would like to see Lucene.NET compatible with Medium Trust: http://social.msdn.microsoft.com/Forums/en-US/windowsazure/thread/d0ec059b-f136-488b-ab74-c2f4c95c659c http://www.mojoportal.com/mediumtrust.aspx http://www.klopfenstein.net/lorenz.aspx/lucene-net-and-nhibernate-search-on-medium-trust http://dev.communityserver.com/forums/p/478315/544018.aspx
        Hide
        Corey Trager added a comment -

        I found what was causing the "Required permissions cannot be acquired" with Medium Trust.

        You have to go to project settings, Build, and uncheck "Allow unsafe code".

        Pretty simple to say, but took me about three hours of trial and error to narrow down which svn revision the problem got introduced into, and then to narrow down the difference between that revision and the prior.

        Some links that I found after that fact:
        http://www.codeplex.com/AntiXSS/Thread/View.aspx?ThreadId=44517
        http://social.msdn.microsoft.com/Forums/en-US/clr/thread/77376b96-5512-4c38-8c07-a527e577ec3a

        Show
        Corey Trager added a comment - I found what was causing the "Required permissions cannot be acquired" with Medium Trust. You have to go to project settings, Build, and uncheck "Allow unsafe code". Pretty simple to say, but took me about three hours of trial and error to narrow down which svn revision the problem got introduced into, and then to narrow down the difference between that revision and the prior. Some links that I found after that fact: http://www.codeplex.com/AntiXSS/Thread/View.aspx?ThreadId=44517 http://social.msdn.microsoft.com/Forums/en-US/clr/thread/77376b96-5512-4c38-8c07-a527e577ec3a
        Hide
        Luis Fco. Ramirez Daza Glez. added a comment -

        Thank you Corey for your help and for share this with us.

        I'll do as you explained to make Lucene work in MEDIUM TRUST and post my results.

        Show
        Luis Fco. Ramirez Daza Glez. added a comment - Thank you Corey for your help and for share this with us. I'll do as you explained to make Lucene work in MEDIUM TRUST and post my results.
        Hide
        Martijn Boland added a comment -

        No need to change GetLuceneIndexFiles. The problem was creating a new FileInfo object with a relative path, which is not allowed under medium trust.

        Attached is a patch that fixes the issue. Tested it and it now works under medium trust.

        Show
        Martijn Boland added a comment - No need to change GetLuceneIndexFiles. The problem was creating a new FileInfo object with a relative path, which is not allowed under medium trust. Attached is a patch that fixes the issue. Tested it and it now works under medium trust.
        Hide
        Digy added a comment -

        The redundant for-loop in FSDirectory#List method is carried over from Lucene.Net 2.0 where "System.IO.Directory.GetFileSystemEntries" ( which incorrectly returned all files in index dir with full name) was used instead of "SupportClass.FileSupport.GetLuceneIndexFiles".

        All NUnite tests pass. If no objections, I will commit the last patch and close this issue.

        DIGY

        Show
        Digy added a comment - The redundant for-loop in FSDirectory#List method is carried over from Lucene.Net 2.0 where "System.IO.Directory.GetFileSystemEntries" ( which incorrectly returned all files in index dir with full name ) was used instead of "SupportClass.FileSupport.GetLuceneIndexFiles". All NUnite tests pass. If no objections, I will commit the last patch and close this issue. DIGY
        Hide
        Corey Trager added a comment -

        Digy - Please uncheck "Allow unsafe code".

        Show
        Corey Trager added a comment - Digy - Please uncheck "Allow unsafe code".
        Hide
        Digy added a comment -

        "Allow unsafe code" unchecked.
        Patch applied.

        DIGY

        Show
        Digy added a comment - "Allow unsafe code" unchecked. Patch applied. DIGY

          People

          • Assignee:
            Unassigned
            Reporter:
            Corey Trager
          • Votes:
            1 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development