[DTACLOUD-553] 500: [SystemStackError] stack level too deep when curling DC for a url that needs credentials

• Type: Bug
• Status: Open
• Priority: Critical
• Resolution: Unresolved
• Component/s: Server

Environment

clean CentOS 6.2 64 bit, with yum -y update and the packages required for DC installed. DC from source (commit 1d9177495dd52fb01143c052e3fac945452837aa), running as not root, ruby
1.9.3p392 (2013-02-22 revision 39386) [x86_64-linux], installed using rbenv

• Assignee: Michal Fojtik
• Reporter: Dies Koper
• Created: 24/Apr/13 10:48
• Updated: 24/Apr/13 21:49

Description

When I list /cimi/cloudEntryPoint, it works fine.
When I list something that requires credentials (/cimi/systems?format=xml), I get an error:

<error status='500' url='/cimi/systems'>
<backend driver='fgcp' provider='default'></backend>
<code>500</code>
<message><![CDATA[stack level too deep]]></message>
<backtrace>

<![CDATA[/home/dc/deltacloud/server/lib/deltacloud/helpers/driver_helper.rb:57]]>
</backtrace>
<request>
<param name='format'><![CDATA["xml"]]></param>
<param name='splat'><![CDATA[[]]]></param>
<param name='captures'><![CDATA[[#<SystemStackError: stack level too
deep>]]]></param>
</request>
</error>

Adding a puts on the NameError gives:
uninitialized constant Deltacloud::Drivers::Fgcp

driver_source_name gives ../drivers/fgcp/fgcp_driver.rb

Line 57 has:
require_relative(driver_source_name) ? retry :
raise(LoadError.new(e.message))

Looks like whatever caused the NameError is causing an infinite loop due to the retry?
Relative patch from driver_helper.rb to fgcp_driver.rb looks fine, file is at that location on my machine.

Any idea what caused the error and should the infinite loop be guarded against or is it a rare case?

I posted this as a question to the ML, mentioning that I started DC as a daemon with:
~/deltacloud/server/bin/deltacloudd -i fgcp -f cimi -r 192.168.0.12 -d -c -u dc -g dc

With ~/.deltacloud/config containing credentials for fgcp:
fgcp:
user: user
password: mypwd

and at that time, when starting it without "-d -c -u dc -g dc" (i.e. passing in the credentials through curl), it worked fine.
But this morning I started it without "-d -c -u dc -g dc" again and got the same error. Restarting it a few times didn't make a difference.
Then suddenly, without restarting it again, it started working again.

So it seems 'require_relative' is not stable.

Marking it as critical as this could impact the reliability of DC in production environments.