Details

    • Type: Sub-task
    • Status: Closed
    • Priority: Major
    • Resolution: Implemented
    • Affects Version/s: Trunk
    • Fix Version/s: 16.11.01
    • Component/s: ALL COMPONENTS
    • Labels:
      None
    • Sprint:
      Bug Crush Event - 21/2/2015

      Description

      I recently (~4 weeks ago) started the "Performance over security, is that reasonable?" thread on dev ML. I think I did not explain me well then. I must say it's easy to drown down in details with this subject when you want to illustrate the reasons.

      So instead of only answering on the dev ML, I decided it will be good to create a Jira task with maybe related tasks, here it is.

      For now I consider it only an improvement, but since it's a security matter we can discuss backporting later.



      TL;DR

      Performance over security?

      So why was this thread opposing performance and security? First we need to understand that here performance stands for HTTP and security for HTTPS.

      Why is HTTP standing for performance?

      Actually is now not much performance difference between the 2 protocols, but you can't cache HTTPS requests and it sometimes (inter-continental requests) matters.

      And why the question about being reasonable or not?

      I think it's unreasonable to put performance over security. And nowadays you are not secure when you use HTTP mixed with HTTPS. Most of the time when you mix both is because you want to identity an user using a sessionId. So with HTTPS, after the user started with HTTP. As concisely explained Forrest in the above referenced thread

      If you're switching between HTTPS and HTTP based on some criteria, an attacker can leverage that to trick the user into all kind of things.

      It's also well and simply explained (with other things) in this article:

      The HTTP spec defines a “Secure” flag for cookies, which instructs the browser to only send that cookie value over SSL. If sites set that cookie like they’re supposed to, then yes, SSL is helping you out. Most sites don’t, however, and browsers will happily send the sensitive cookies over unencrypted HTTP. Our hypothetical skeezebag really just needs some way to trick you into opening a normal HTTP URL, maybe by e-mailing you a link to http://yourbank.com/a-picture-of-ponies-and-rainbows.gif so he can sniff the plain-text cookie off your unencrypted HTTP request, or by surreptitiously embedding a JavaScript file via some site’s XSS vulnerability.

      Of course if you site is only showing things but nobody has never to identify, then you are not at risk and HTTP only is perfect. But with ecommerce kind of site or such, it's rarely the case, most of the time users need to identify!



      So why are people still mixing HTTP and HTTPS on their site? In the 1st answer at [1] Thomas Pornin and others gave some interesting points and answers. At [2] Yves Lafon gave also a good summary even if a bit old now. I took some questions/answers from [3] also. So you might check those links by yourself, here is an abstract:

      1. "Some browsers may not support SSL" Only old Lynx versions, negligible
      2. "Connection initiation requires some extra network roundtrips" Negligible but for sites which serve mostly static contents, see "static content takes a hit" below.
      3. "the SSL initial key exchange adds to the latency" As completely explained here: "most TLS server use a RSA key and the client part of RSA is cheap (the server incurs most of the cost in RSA)". Still better to have not too short sessions as explained here
      4. "static content takes a hit" You should though store static content apart. OFBiz comes with ofbizContentUrl and content.properties for that. But you should still use HTTPS. The complete answer for the last question (just above this one) also applies here. Also this is quite interesting https://www.httpvshttps.com/ and proves HTTPS can be faster than HTTP
      5. "HTTPS servers must use one IP per server name" or "it doesn't work with virtual hosts" This issue has long been solved by Server Name Indication which is supported by all major browsers nowadays.
      6. Certificates are expensive For demos, etc. (ie not for real production sites where a certificate is mandatory anyway) but this no longer an issue with letsencrypt
      7. "Proxy servers cannot cache pages served with HTTPS" This is the more important point. Nowadays this is only a performance problem with inter-continental requests. Note that you can use HTTP for static content inside OFBiz
      Warning

      I must say the point 4 is disputable. If for instance you are serving a lot of static contents intercontinentally, you should measure and take any solution which fits with your case at the moment you measured it (means in few years it could change and be negligible)

      [1] https://security.stackexchange.com/questions/4369/why-is-https-not-the-default-protocol
      [2] http://arstechnica.com/business/2011/03/https-is-more-secure-so-why-isnt-the-web-using-it
      [3] https://stackoverflow.com/questions/2746047/why-not-use-https-for-everything

      As [2] concluded in 2011:

      In the Web of the future the main concern won't just be how fast a site loads, but how well it safeguards you and protects your data once it does load.
      

      And I you are really interested in every details you should read this other article from 2011. You might also notice that there are not much new articles on this subject. I still wonder why, I guess because most was already said and it's more to people (site developpers) now to take care

      1. OFBIZ-6849.patch
        26 kB
        Jacques Le Roux
      2. OFBIZ-6849.patch
        28 kB
        Jacques Le Roux
      3. OFBIZ-6849.patch
        3 kB
        Deepak Dixit

        Issue Links

          Activity

          Show
          deepak.dixit Deepak Dixit added a comment - +1 Jacques, https://googlewebmastercentral.blogspot.in/2015/12/indexing-https-pages-by-default.html
          Hide
          deepak.dixit Deepak Dixit added a comment -

          I added a patch for the same, but need to test it.

          Show
          deepak.dixit Deepak Dixit added a comment - I added a patch for the same, but need to test it.
          Hide
          jacques.le.roux Jacques Le Roux added a comment -

          Thanks Deepak, I will have a look.

          I must say it's still a WIP on my side. I have also some changes that I'm testing. I want 1st to better explain why and how I want to do it. For instance the issue description is still not complete...

          Show
          jacques.le.roux Jacques Le Roux added a comment - Thanks Deepak, I will have a look. I must say it's still a WIP on my side. I have also some changes that I'm testing. I want 1st to better explain why and how I want to do it. For instance the issue description is still not complete...
          Hide
          jacques.le.roux Jacques Le Roux added a comment -

          I completed the description today

          Show
          jacques.le.roux Jacques Le Roux added a comment - I completed the description today
          Show
          jacques.le.roux Jacques Le Roux added a comment - BTW this article is also interesting https://blog.mozilla.org/tanvi/2013/04/10/mixed-content-blocking-enabled-in-firefox-23/
          Hide
          jacques.le.roux Jacques Le Roux added a comment -

          Another interesting point: for 2 or 3 years now I'm using https://www.eff.org/https-everywhere in my preferred browsers (1= FF, 2= Chrome, I use both depending on circumstances). Initially there were very few cases where I had to downgrade sites to HTTP access. It's now a year or two I did not have to do that. Which means there are now very few sites which does not support HTTPS only.

          Show
          jacques.le.roux Jacques Le Roux added a comment - Another interesting point: for 2 or 3 years now I'm using https://www.eff.org/https-everywhere in my preferred browsers (1= FF, 2= Chrome, I use both depending on circumstances). Initially there were very few cases where I had to downgrade sites to HTTP access. It's now a year or two I did not have to do that. Which means there are now very few sites which does not support HTTPS only.
          Hide
          jacques.le.roux Jacques Le Roux added a comment -

          Deepak,

          As already "explained" in the related thread on dev ML, this is the patch I propose:

          Index: config/url.properties
          ===================================================================
          --- config/url.properties    (revision 1725003)
          +++ config/url.properties    (working copy)
          @@ -26,6 +26,7 @@
           force.https.host=
          
           # HTTP Port (Not Secure port)
          +no.http=Y
           port.http=8080
           force.http.host=
          
          Index: src/org/ofbiz/webapp/control/ConfigXMLReader.java
          ===================================================================
          --- src/org/ofbiz/webapp/control/ConfigXMLReader.java    (revision 1725003)
          +++ src/org/ofbiz/webapp/control/ConfigXMLReader.java    (working copy)
          @@ -41,6 +41,7 @@
           import org.ofbiz.base.util.FileUtil;
           import org.ofbiz.base.util.GeneralException;
           import org.ofbiz.base.util.UtilHttp;
          +import org.ofbiz.base.util.UtilProperties;
           import org.ofbiz.base.util.UtilValidate;
           import org.ofbiz.base.util.UtilXml;
           import org.ofbiz.base.util.cache.UtilCache;
          @@ -527,7 +528,7 @@
                   public boolean trackServerHit = true;
                   public String description;
                   public Event event;
          -        public boolean securityHttps = false;
          +        public boolean securityHttps = true;
                   public boolean securityAuth = false;
                   public boolean securityCert = false;
                   public boolean securityExternalView = true;
          @@ -544,7 +545,9 @@
                       // Check for security
                       Element securityElement = UtilXml.firstChildElement(requestMapElement, "security");
                       if (securityElement != null) {
          -                this.securityHttps = "true".equals(securityElement.getAttribute("https"));
          +                if (!UtilProperties.propertyValueEqualsIgnoreCase("url", "no.http", "Y")) {
          +                    this.securityHttps = "true".equals(securityElement.getAttribute("https"));
          +                }
                           this.securityAuth = "true".equals(securityElement.getAttribute("auth"));
                           this.securityCert = "true".equals(securityElement.getAttribute("cert"));
                           this.securityExternalView = !"false".equals(securityElement.getAttribute("external-view"));
          

          I have tested it locally and I so far found only few minor issues to fix.

          1. findSalesInvoicesByDueDate. I long ago naively introduced that myself with OFBIZ-4158
          2. NetworkError: 404 Not Found - https://localhost:8443/contentimages/contentForum.css. I have to check that...
          3. The W3 validator links no longer work => Jira creation WIP...
          4. ecomseo does not work at all. I have to check that...
          5. SOAP services don't pass. I have to check that...
          6. There are also few harcoded "<a href="http:" but none refer internally to OFBiz so we don't care.
          Show
          jacques.le.roux Jacques Le Roux added a comment - Deepak, As already "explained" in the related thread on dev ML, this is the patch I propose: Index: config/url.properties =================================================================== --- config/url.properties (revision 1725003) +++ config/url.properties (working copy) @@ -26,6 +26,7 @@ force.https.host= # HTTP Port (Not Secure port) +no.http=Y port.http=8080 force.http.host= Index: src/org/ofbiz/webapp/control/ConfigXMLReader.java =================================================================== --- src/org/ofbiz/webapp/control/ConfigXMLReader.java (revision 1725003) +++ src/org/ofbiz/webapp/control/ConfigXMLReader.java (working copy) @@ -41,6 +41,7 @@ import org.ofbiz.base.util.FileUtil; import org.ofbiz.base.util.GeneralException; import org.ofbiz.base.util.UtilHttp; + import org.ofbiz.base.util.UtilProperties; import org.ofbiz.base.util.UtilValidate; import org.ofbiz.base.util.UtilXml; import org.ofbiz.base.util.cache.UtilCache; @@ -527,7 +528,7 @@ public boolean trackServerHit = true ; public String description; public Event event; - public boolean securityHttps = false ; + public boolean securityHttps = true ; public boolean securityAuth = false ; public boolean securityCert = false ; public boolean securityExternalView = true ; @@ -544,7 +545,9 @@ // Check for security Element securityElement = UtilXml.firstChildElement(requestMapElement, "security" ); if (securityElement != null ) { - this .securityHttps = " true " .equals(securityElement.getAttribute( "https" )); + if (!UtilProperties.propertyValueEqualsIgnoreCase( "url" , "no.http" , "Y" )) { + this .securityHttps = " true " .equals(securityElement.getAttribute( "https" )); + } this .securityAuth = " true " .equals(securityElement.getAttribute( "auth" )); this .securityCert = " true " .equals(securityElement.getAttribute( "cert" )); this .securityExternalView = ! " false " .equals(securityElement.getAttribute( "external-view" )); I have tested it locally and I so far found only few minor issues to fix. findSalesInvoicesByDueDate. I long ago naively introduced that myself with OFBIZ-4158 NetworkError: 404 Not Found - https://localhost:8443/contentimages/contentForum.css . I have to check that... The W3 validator links no longer work => Jira creation WIP... ecomseo does not work at all. I have to check that... SOAP services don't pass. I have to check that... There are also few harcoded "<a href="http:" but none refer internally to OFBiz so we don't care.
          Hide
          jacques.le.roux Jacques Le Roux added a comment -

          The most important problem was ecomseo. I just retried (w/o changes) with most important browsers (last versions) trying both possibilities (though now only HTTPS should be used)

          1. https://localhost:8443/ecomseo
            all tried browsers work 1st time
          2. http://localhost:8080/ecomseo

          The difficulty here is that you don't need to login to access ecommerce or ecomseo (anonymoius access). Though there is any problems with http://localhost:8080/ecommerce/control/main. So this still needs to be clarifed but is not blocking

          Show
          jacques.le.roux Jacques Le Roux added a comment - The most important problem was ecomseo. I just retried (w/o changes) with most important browsers (last versions) trying both possibilities (though now only HTTPS should be used) https://localhost:8443/ecomseo all tried browsers work 1st time http://localhost:8080/ecomseo FF 404, works 2nd time (F5), redirects to https://localhost:8443/ecomseo Chrome 404, works 2nd time (F5), redirects to https://localhost:8443/ecomseo IE 11 works 404, stays. I guess it's related with HSTS, I will digg that Opera 404, works 2nd time (F5), redirects to https://localhost:8443/ecomseo The difficulty here is that you don't need to login to access ecommerce or ecomseo (anonymoius access). Though there is any problems with http://localhost:8080/ecommerce/control/main . So this still needs to be clarifed but is not blocking
          Hide
          jacques.le.roux Jacques Le Roux added a comment -

          Interesting tweet from Mozilla community : https://twitter.com/mozilla/status/708075463445450752
          The graph says it all

          Show
          jacques.le.roux Jacques Le Roux added a comment - Interesting tweet from Mozilla community : https://twitter.com/mozilla/status/708075463445450752 The graph says it all
          Hide
          gdraperi gregory draperi added a comment - - edited

          I share this point of view. For me it makes no sense to have a website in HTTPS but the landing page in HTTP since with this configuration you can not ensure that the link to the HTTPS has not been modified.

          Show
          gdraperi gregory draperi added a comment - - edited I share this point of view. For me it makes no sense to have a website in HTTPS but the landing page in HTTP since with this configuration you can not ensure that the link to the HTTPS has not been modified.
          Hide
          jacques.le.roux Jacques Le Roux added a comment -

          Thanks for your support Gregory, I have misc. patches pending, I'll review that a bit more next week and will propose what I have so far...

          Show
          jacques.le.roux Jacques Le Roux added a comment - Thanks for your support Gregory, I have misc. patches pending, I'll review that a bit more next week and will propose what I have so far...
          Hide
          jacques.le.roux Jacques Le Roux added a comment -

          Another interesting and quite documented case: https://twitter.com/matthew_d_green/status/716620369310916608

          Show
          jacques.le.roux Jacques Le Roux added a comment - Another interesting and quite documented case: https://twitter.com/matthew_d_green/status/716620369310916608
          Hide
          jacques.le.roux Jacques Le Roux added a comment -

          2. Was unrelated and resolved with OFBIZ-6956
          4. Not blocking (see comment just below)
          5. Simply for SOAP services you can't use HTTPS (must use HTTP and use SOAP security)
          6. Not a pb.

          So remains 1 and 3... 1 being the most annoying...

          Show
          jacques.le.roux Jacques Le Roux added a comment - 2. Was unrelated and resolved with OFBIZ-6956 4. Not blocking (see comment just below) 5. Simply for SOAP services you can't use HTTPS (must use HTTP and use SOAP security) 6. Not a pb. So remains 1 and 3... 1 being the most annoying...
          Hide
          jacques.le.roux Jacques Le Roux added a comment -

          OK 3 is also unrelated.
          With localhost the W3 message is

          Sorry, the IP address of localhost is not public. For security reasons, validating resources located at non-public IP addresses has been disabled in this service.

          On trunk demo:

          You have requested we check the referring page, but your browser did not send the HTTP "Referer" header field. This can be for several reasons, but most commonly it is because your browser does not know about this header, has been configured not to send one, transferred the referring document over a secure protocol such as https but is accessing the validator over ordinary non-secure http, or is behind a proxy or firewall that strips it out of the request before it reaches us. This is not an error in the referring page! Please use the form interface on the Validator Home Page to check the page by URL.

          Show
          jacques.le.roux Jacques Le Roux added a comment - OK 3 is also unrelated. With localhost the W3 message is Sorry, the IP address of localhost is not public. For security reasons, validating resources located at non-public IP addresses has been disabled in this service. On trunk demo: You have requested we check the referring page, but your browser did not send the HTTP "Referer" header field. This can be for several reasons, but most commonly it is because your browser does not know about this header, has been configured not to send one, transferred the referring document over a secure protocol such as https but is accessing the validator over ordinary non-secure http, or is behind a proxy or firewall that strips it out of the request before it reaches us. This is not an error in the referring page! Please use the form interface on the Validator Home Page to check the page by URL.
          Hide
          jacques.le.roux Jacques Le Roux added a comment - - edited

          I have created OFBIZ-7070 for the findSalesInvoicesByDueDate issue.

          I have checked several <security https="false" auth="true"/> entries in controller. It seems there are no other issues like the one exposed in OFBIZ-7070

          Show
          jacques.le.roux Jacques Le Roux added a comment - - edited I have created OFBIZ-7070 for the findSalesInvoicesByDueDate issue. I have checked several <security https="false" auth="true"/> entries in controller. It seems there are no other issues like the one exposed in OFBIZ-7070
          Hide
          jacques.le.roux Jacques Le Roux added a comment -

          Since now OFBIZ-7070 is done nothing is left. I'm attaching the pach with the changes I use locally for months without issues. I'll wait a few days and will commit it.

          Show
          jacques.le.roux Jacques Le Roux added a comment - Since now OFBIZ-7070 is done nothing is left. I'm attaching the pach with the changes I use locally for months without issues. I'll wait a few days and will commit it.
          Hide
          jacques.le.roux Jacques Le Roux added a comment -

          I just noticed some lines from the patch in OFBIZ-6931 ("Add XLS renderer") remained in webtools controller.xml, this last patch removes those

          Show
          jacques.le.roux Jacques Le Roux added a comment - I just noticed some lines from the patch in OFBIZ-6931 ("Add XLS renderer") remained in webtools controller.xml, this last patch removes those
          Hide
          jacques.le.roux Jacques Le Roux added a comment -

          BTW commons-safe-head.ftl is included but not used yet, it's a WIP...

          Show
          jacques.le.roux Jacques Le Roux added a comment - BTW commons-safe-head.ftl is included but not used yet, it's a WIP...
          Hide
          jacques.le.roux Jacques Le Roux added a comment -

          At revision: 1745525, I committed a version where the changes for XmlRpcTests were removed, they were not needed. All tests pass and things work correctly, including ecommerce where HTTP was most used.

          Show
          jacques.le.roux Jacques Le Roux added a comment - At revision: 1745525, I committed a version where the changes for XmlRpcTests were removed, they were not needed. All tests pass and things work correctly, including ecommerce where HTTP was most used.
          Hide
          jacques.le.roux Jacques Le Roux added a comment -

          Quoting Deepak on dev ML


          Hi Jacques,

          I think in fop.xconf file we have to keep http url, as FOP does not support
          https baseUrl for external graphics due to certificate issue (not remember
          exact reason).

           <!-- Base URL for resolving relative URLs -->
          -  <base>http://localhost:8080</base>
          +  <base>https://localhost:8443</base>
          

          If we set the https as <base> than images will not be rendered. Please
          refer
          http://demo-trunk-ofbiz.apache.org/ordermgr/control/order.pdf?orderId=DEMO10091

          and we get following error on console:

          
               [java] 2016-05-27 13:51:48,308 |http-nio-8443-exec-3 |FOUserAgent
                        |E| Image not found. URI: /images/ofbiz_logo.gif. (No context
          info available)
          
          

          If we set the http url than its working fine.


          Thanks Deepak, fixed revision: 1745732

          Show
          jacques.le.roux Jacques Le Roux added a comment - Quoting Deepak on dev ML Hi Jacques, I think in fop.xconf file we have to keep http url, as FOP does not support https baseUrl for external graphics due to certificate issue (not remember exact reason). <!-- Base URL for resolving relative URLs --> - <base>http: //localhost:8080</base> + <base>https: //localhost:8443</base> If we set the https as <base> than images will not be rendered. Please refer http://demo-trunk-ofbiz.apache.org/ordermgr/control/order.pdf?orderId=DEMO10091 and we get following error on console: [java] 2016-05-27 13:51:48,308 |http-nio-8443-exec-3 |FOUserAgent |E| Image not found. URI: /images/ofbiz_logo.gif. (No context info available) If we set the http url than its working fine. Thanks Deepak, fixed revision: 1745732
          Hide
          jacques.le.roux Jacques Le Roux added a comment -

          Mmm, it's embarrasing. I just realised that the main change (the one in ConfigXMLReader.java, exposed in a comment above) was not committed in r1745525. I guess it was because of tests I did for OFBIZ-4090, I certainly removed these changes then and forgot to put them back. So it's now committed at r1746159

          Show
          jacques.le.roux Jacques Le Roux added a comment - Mmm, it's embarrasing. I just realised that the main change (the one in ConfigXMLReader.java, exposed in a comment above) was not committed in r1745525. I guess it was because of tests I did for OFBIZ-4090 , I certainly removed these changes then and forgot to put them back. So it's now committed at r1746159
          Hide
          jacques.le.roux Jacques Le Roux added a comment -

          At revision: 1746254, I fixed and issue with xmlrpc tests. The tests did not pass because of xmlrpc. Simply adding it in http.request-map.list in url.properties and like for SOAPService request setting <security https="false"/> did it.

          Show
          jacques.le.roux Jacques Le Roux added a comment - At revision: 1746254, I fixed and issue with xmlrpc tests. The tests did not pass because of xmlrpc. Simply adding it in http.request-map.list in url.properties and like for SOAPService request setting <security https="false"/> did it.

            People

            • Assignee:
              jacques.le.roux Jacques Le Roux
              Reporter:
              jacques.le.roux Jacques Le Roux
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development

                  Agile