[FELIX-5759] StackOverflowError thrown during URL construction

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: framework-5.6.10
• Fix Version/s: framework-6.0.0
• Assignee: Karl Pauls
• Reporter: Ali Kamali
• Created: 06/Dec/17 18:27
• Updated: 09/Jul/18 10:31
• Resolved: 07/Dec/17 23:29

Description

I get the following callstack resulting in a stack overflow error when building a URL object:

[info]   at org.apache.felix.framework.URLHandlersStreamHandlerProxy.parseURL(URLHandlersStreamHandlerProxy.java:401)
[info]   at java.net.URL.<init>(URL.java:622)
[info]   at java.net.URL.<init>(URL.java:490)
[info]   at org.apache.felix.framework.URLHandlersStreamHandlerProxy.parseURL(URLHandlersStreamHandlerProxy.java:401)
[info]   at java.net.URL.<init>(URL.java:622)
[info]   at java.net.URL.<init>(URL.java:490)
[info]   at org.apache.felix.framework.URLHandlersStreamHandlerProxy.parseURL(URLHandlersStreamHandlerProxy.java:401)
[info]   at java.net.URL.<init>(URL.java:622)
[info]   at java.net.URL.<init>(URL.java:490)
[info]   at org.apache.felix.framework.URLHandlersStreamHandlerProxy.parseURL(URLHandlersStreamHandlerProxy.java:401)
[info]   at java.net.URL.<init>(URL.java:622)
[info]   at java.net.URL.<init>(URL.java:490)
[info]   at org.apache.felix.framework.URLHandlersStreamHandlerProxy.parseURL(URLHandlersStreamHandlerProxy.java:401)
[info]   at java.net.URL.<init>(URL.java:622)
[info]   at java.net.URL.<init>(URL.java:490)
[info]   at org.apache.felix.framework.URLHandlersStreamHandlerProxy.parseURL(URLHandlersStreamHandlerProxy.java:401)
[info]   at java.net.URL.<init>(URL.java:622)
[info]   at java.net.URL.<init>(URL.java:490)
[info]   at org.apache.felix.framework.URLHandlersStreamHandlerProxy.parseURL(URLHandlersStreamHandlerProxy.java:401)

Using org.apache.felix.framework 4.4.1 and Java 1.8.0_111 running on Ubuntu.

We started getting this exception after upgrading to Spark 2.2.0, after some investigation we realized Spark 2.2.0 registers its own URLStreamHandlerFactory:

package org.apache.spark.sql.internal

object SharedState extends Logging {
  try {
    URL.setURLStreamHandlerFactory(new FsUrlStreamHandlerFactory())
  } catch {
    case e: Error =>
      logWarning("URL.setURLStreamHandlerFactory failed to set FsUrlStreamHandlerFactory")
  }
...

Looks like the bug is related to line 128 in URLHandlers.java:

            URLStreamHandler handler = getBuiltInStreamHandler(protocol, factory);
            if (handler != null)
            {
                URL url = new URL(protocol, null, -1, "", handler);
                m_handlerToURL.put(handler, url);
            }

This code assumes there is a unique mapping from handlers to protocols, which doesn't seem to be a valid assumption, at least not with Spark. Spark URL handler factory is returning the same handler instance for both file and ftp protocols. When URLHandlers is initializing it first tries to register a handler for file and then for ftp, but because the factory returns the same handler we end up replacing the URL object we have for file with ftp in m_handlerToURL.

Later when a URL is being constructed it calls createURLStreamHandler from URLHandlers, at the end of this method:

        // If built-in content handler, then create a proxy handler.
        return addToStreamCache(protocol,
            new URLHandlersStreamHandlerProxy(protocol, m_secureAction,
                handler, (URL) m_handlerToURL.get(handler)));

Note that it's trying to use m_handlerToURL to look up the protocol for the handler, and in case of file instead of returning a URL with protocol set to file it returns a URL with protocol set to ftp, so URLHandlersStreamHandlerProxy gets constructed with m_builtInURL set to ftp.

Later in URL.java we have this code:

            if ((context != null) && ((newProtocol == null) ||
                            newProtocol.equalsIgnoreCase(context.protocol))) {
                // inherit the protocol handler from the context
                // if not specified to the constructor
                if (handler == null) {
                    handler = context.handler;
                }

This code only uses the handler if protocols match, but in this case protocols don't match because it's expected to be file but we receive ftp that comes from m_builtInURL, and the code falls back to asking the factory to create a new handler:

            if (handler == null &&
                (handler = getURLStreamHandler(protocol)) == null) {
                throw new MalformedURLException("unknown protocol: "+protocol);
            }

The factory returns a handler with protocol set to ftp again and we get stuck in a loop.

Looking at the newest Felix code looks like the assumption of having a unique handler per protocol is still there, so I believe this bug still exists in the newest Felix as well.

To reproduce this bug before starting a bundle you only need to register a factory that returns the same handler instance for file and ftp.