[GOBBLIN-477] Lib Jars in gobblin-mapreduce.sh are hardcoded

• Type: Bug
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 0.11.0, 0.12.0, 0.13.0
• Fix Version/s: 0.17.0
• Assignee: Unassigned
• Reporter: Benson Qiu
• Created: 26/Apr/18 06:48
• Updated: 14/Jun/23 04:13

Description

Some of the jars are hardcoded and don't match the actual jar versions in `gobblin-distribution*.tar.gz` that is generated when we build the source code. 

LIBJARS=(
$USER_JARS
$FWDIR_LIB/gobblin-api-$GOBBLIN_VERSION.jar
$FWDIR_LIB/gobblin-avro-json-$GOBBLIN_VERSION.jar
$FWDIR_LIB/gobblin-codecs-$GOBBLIN_VERSION.jar
$FWDIR_LIB/gobblin-core-$GOBBLIN_VERSION.jar
$FWDIR_LIB/gobblin-core-base-$GOBBLIN_VERSION.jar
$FWDIR_LIB/gobblin-crypto-$GOBBLIN_VERSION.jar
$FWDIR_LIB/gobblin-crypto-provider-$GOBBLIN_VERSION.jar
$FWDIR_LIB/gobblin-data-management-$GOBBLIN_VERSION.jar
$FWDIR_LIB/gobblin-metastore-$GOBBLIN_VERSION.jar
$FWDIR_LIB/gobblin-metrics-$GOBBLIN_VERSION.jar
$FWDIR_LIB/gobblin-metrics-base-$GOBBLIN_VERSION.jar
$FWDIR_LIB/gobblin-metadata-$GOBBLIN_VERSION.jar
$FWDIR_LIB/gobblin-utility-$GOBBLIN_VERSION.jar
$FWDIR_LIB/avro-1.8.1.jar
$FWDIR_LIB/avro-mapred-1.8.1.jar
$FWDIR_LIB/commons-lang3-3.4.jar
$FWDIR_LIB/config-1.2.1.jar
$FWDIR_LIB/data-11.0.0.jar
$FWDIR_LIB/gson-2.6.2.jar
$FWDIR_LIB/guava-15.0.jar
$FWDIR_LIB/guava-retrying-2.0.0.jar
$FWDIR_LIB/joda-time-2.9.3.jar
$FWDIR_LIB/javassist-3.18.2-GA.jar
$FWDIR_LIB/kafka_2.11-0.8.2.2.jar
$FWDIR_LIB/kafka-clients-0.8.2.2.jar
$FWDIR_LIB/metrics-core-2.2.0.jar
$FWDIR_LIB/metrics-core-3.1.0.jar
$FWDIR_LIB/metrics-graphite-3.1.0.jar
$FWDIR_LIB/scala-library-2.11.8.jar
$FWDIR_LIB/influxdb-java-2.1.jar
$FWDIR_LIB/okhttp-2.4.0.jar
$FWDIR_LIB/okio-1.4.0.jar
$FWDIR_LIB/retrofit-1.9.0.jar
$FWDIR_LIB/reflections-0.9.10.jar
)

Issue Links

• Is contained by: GOBBLIN-707 combine & standardize all gobblin scripts into one master script & restructure configs accordingly