[GOBBLIN-394] Using Zookeeper as state store leads to errors.

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.13.0
• Component/s: gobblin-helix
• Environment: OS: redhat ; gobblin: 0.13
• Assignee: Hung Tran
• Reporter: ROHIT KALHANS
• Created: 30/Jan/18 11:05
• Updated: 30/Jan/18 11:34

Description

When using zk for state store, I keep getting this error which terminates the job

o.a.g.r.AbstractJobLauncher [AbstractJobLauncher.java:468] - Failed to launch and run job job_kafkaStreaming_1517248081731: java.lang.NoClassDefFoundError: Could not initialize class org.apache.log4j.LogManager java.lang.NoClassDefFoundError: Could not initialize class org.apache.log4j.LogManager at org.apache.log4j.Logger.getLogger(Logger.java:117) at org.apache.helix.manager.zk.ZkCacheBaseDataAccessor.<clinit>(ZkCacheBaseDataAccessor.java:50) at org.apache.gobblin.metastore.ZkStateStore.<init>(ZkStateStore.java:89) at org.apache.gobblin.metastore.ZkStateStoreFactory.createStateStore(ZkStateStoreFactory.java:38) at org.apache.gobblin.runtime.StateStoreBasedWatermarkStorage.<init>(StateStoreBasedWatermarkStorage.java:101) at org.apache.gobblin.runtime.TaskContext.getWatermarkStorage(TaskContext.java:389) at org.apache.gobblin.runtime.Task.<init>(Task.java:234) at org.apache.gobblin.runtime.GobblinMultiTaskAttempt.createTaskRunnable(GobblinMultiTaskAttempt.java:363) at org.apache.gobblin.runtime.GobblinMultiTaskAttempt.runWorkUnits(GobblinMultiTaskAttempt.java:344) at org.apache.gobblin.runtime.GobblinMultiTaskAttempt.run(GobblinMultiTaskAttempt.java:134) at org.apache.gobblin.runtime.GobblinMultiTaskAttempt.runAndOptionallyCommitTaskAttempt(GobblinMultiTaskAttempt.java:369) at org.apache.gobblin.runtime.GobblinMultiTaskAttempt.runWorkUnits(GobblinMultiTaskAttempt.java:391) at org.apache.gobblin.runtime.local.LocalJobLauncher.runWorkUnitStream(LocalJobLauncher.java:142) at org.apache.gobblin.runtime.AbstractJobLauncher.launchJob(AbstractJobLauncher.java:443) at org.apache.gobblin.runtime.job_exec.JobLauncherExecutionDriver$DriverRunnable.call(JobLauncherExecutionDriver.java:159) at org.apache.gobblin.runtime.job_exec.JobLauncherExecutionDriver$DriverRunnable.call(JobLauncherExecutionDriver.java:147) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:748)

 

This looks like a misalignment between logger lib in  helix compared to other modules.