Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
1.99.6
-
None
-
None
Description
On internal hackathon we we're hacking Sqoop 2 connector with singhashish and we went through few troubles that we should address.
For example any exception that is generated during connector initialization will cause Sqoop 2 to fail start. I think that being resilient and simply ignore malformed connectors is much better approach.
Here is exception that put down Sqoop 2 server if we incorrectly marked configuration objects:
2015-08-14 11:34:59,960 ERROR org.apache.sqoop.core.SqoopServer: Server startup failure org.apache.sqoop.common.SqoopException: MODEL_003:Object is not valid configuration object - Missing annotation ConfigurationClass on class com.cloudera.sqoop.Internal.conf.FromJobConfiguration at org.apache.sqoop.model.ConfigUtils.toConfigs(ConfigUtils.java:78) at org.apache.sqoop.model.ConfigUtils.toConfigs(ConfigUtils.java:67) at org.apache.sqoop.connector.ConnectorHandler.<init>(ConnectorHandler.java:95) at org.apache.sqoop.connector.ConnectorManager.initialize(ConnectorManager.java:177) at org.apache.sqoop.connector.ConnectorManager.initialize(ConnectorManager.java:155) at org.apache.sqoop.core.SqoopServer.initialize(SqoopServer.java:57) at org.apache.sqoop.server.ServerInitializer.contextInitialized(ServerInitializer.java:41) at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4210) at org.apache.catalina.core.StandardContext.start(StandardContext.java:4709) at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:802) at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:779) at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:583) at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:944) at org.apache.catalina.startup.HostConfig.deployWARs(HostConfig.java:779) at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:505) at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1322) at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:325) at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142) at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1068) at org.apache.catalina.core.StandardHost.start(StandardHost.java:822) at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1060) at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:463) at org.apache.catalina.core.StandardService.start(StandardService.java:525) at org.apache.catalina.core.StandardServer.start(StandardServer.java:759) at org.apache.catalina.startup.Catalina.start(Catalina.java:595) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289) at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414) 2015-08-14 11:34:59,963 ERROR org.apache.sqoop.server.ServerInitializer: Sqoop server failed to start
Or random exception when getting configuration classes:
2015-08-14 14:39:09,488 ERROR org.apache.sqoop.core.SqoopServer: Server startup failure
org.apache.sqoop.common.SqoopException: EXCEPTION_001:Unknown job type - TO
at com.cloudera.sqoop.Internal.Connector.getJobConfigurationClass(Connector.java:53)
at org.apache.sqoop.connector.ConnectorHandler.<init>(ConnectorHandler.java:100)
at org.apache.sqoop.connector.ConnectorManager.initialize(ConnectorManager.java:177)
at org.apache.sqoop.connector.ConnectorManager.initialize(ConnectorManager.java:155)
at org.apache.sqoop.core.SqoopServer.initialize(SqoopServer.java:57)
at org.apache.sqoop.server.ServerInitializer.contextInitialized(ServerInitializer.java:41)
at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4210)
at org.apache.catalina.core.StandardContext.start(StandardContext.java:4709)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:802)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:779)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:583)
at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:944)
at org.apache.catalina.startup.HostConfig.deployWARs(HostConfig.java:779)
at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:505)
at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1322)
at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:325)
at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1068)
at org.apache.catalina.core.StandardHost.start(StandardHost.java:822)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1060)
at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:463)
at org.apache.catalina.core.StandardService.start(StandardService.java:525)
at org.apache.catalina.core.StandardServer.start(StandardServer.java:759)
at org.apache.catalina.startup.Catalina.start(Catalina.java:595)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
2015-08-14 14:39:09,492 ERROR org.apache.sqoop.server.ServerInitializer: Sqoop server failed to start