Details
Description
This is not happening because Lineage is being exposed by MoveTask which checks for a LoadTableDesc. In case of CTAS the table is created after the MoveTask.
Proposed solution is to add a flag to CreateTableDesc to track a CTAS operation, and have the DDLTask expose the lineage if this flag is set.
Issue Links
- is depended upon by
-
ATLAS-247
Hive Column level lineage
-
- Resolved
-
- is related to
-
HIVE-14706
Lineage information not set properly
-
- Closed
-
Activity
- All
- Comments
- Work Log
- History
- Activity
- Transitions
The HookContext gets the Lineage from the SessionState. And AFAICT the Lineage in SessionState is only being set in the MoveTask.
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12789075/HIVE-13112.01.patch
ERROR: -1 due to no test(s) being added or modified.
ERROR: -1 due to 226 failed/errored test(s), 9781 tests executed
Failed tests:
TestSparkCliDriver-groupby3_map.q-sample2.q-auto_join14.q-and-12-more - did not produce a TEST-*.xml file TestSparkCliDriver-groupby_map_ppr_multi_distinct.q-table_access_keys_stats.q-groupby4_noskew.q-and-12-more - did not produce a TEST-*.xml file TestSparkCliDriver-join_rc.q-insert1.q-vectorized_rcfile_columnar.q-and-12-more - did not produce a TEST-*.xml file TestSparkCliDriver-ppd_join4.q-join9.q-ppd_join3.q-and-12-more - did not produce a TEST-*.xml file org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_allcolref_in_udf org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_rename_partition_authorization org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_table_update_status org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_table org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_6 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_create_temp_table org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_view_1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_view_disable_cbo_1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_autogen_colalias org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cbo_SortUnionTransposeRule org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cbo_rp_cross_product_check_2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cbo_rp_unionDistinct_2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_char_nested_types org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_colstats_all_nulls org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_create_default_prop org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cross_product_check_1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cross_product_check_2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ctas org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ctas_char org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ctas_colname org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ctas_date org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ctas_uses_database_location org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ctas_varchar org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cte_2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cte_4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_database org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_dbtxnmgr_ddl1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_decimal_6 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_decimal_join2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_decimal_serde org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_empty_join org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_explain_ddl org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_global_limit org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_duplicate_key org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input46 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_insert0 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join41 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join42 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join_filters_overlap org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_lateral_view_outer org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_llap_partitioned org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_llap_uncompressed org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_merge3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_multi_insert_lateral_view org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_multi_insert_union_src org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_multi_join_union org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_nestedvirtual org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_non_ascii_literal2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_nullformatCTAS org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_createas1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_llap org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_parquet_ctas org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_parquet_join org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_parquet_map_null org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_parquet_map_of_arrays_of_ints org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_parquet_map_of_maps org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_parquet_mixed_partition_formats2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_parquet_nested_complex org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_parquet_schema_evolution org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_parquet_write_correct_definition_levels org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_decode_name org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_special_char org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_query_result_fileformat org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_rcfile_createas1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_rcfile_default_format org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sample_islocalmode_hook org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_select_same_col org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_semijoin org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_semijoin3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_skewjoin_noskew org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin9 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_special_character_in_tabnames_1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats5 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_str_to_map org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_temp_table org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_temp_table_join1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf_unix_timestamp org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union24 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union27 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union31 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union32 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_unionDistinct_1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_unionDistinct_2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_fast_stats org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_top_level org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_updateAccessTime org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_updateBasicStats org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_varchar_nested_types org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_between_columns org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_between_in org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_char_mapjoin1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_decimal_10_0 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_decimal_3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_decimal_6 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_decimal_aggregate org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_decimal_expressions org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_decimal_math_funcs org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_grouping_sets org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_interval_mapjoin org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_join30 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_join_filters org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_join_nulls org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_leftsemi_mapjoin org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_multi_insert org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_nullsafe_join org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_outer_join1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_outer_join2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_outer_join3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_outer_join4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_outer_join5 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_outer_join6 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_partitioned_date_time org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_reduce_groupby_decimal org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_varchar_mapjoin1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_decimal_date org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_short_regress org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_windowing_navfn org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_windowing_streaming org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_2 org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_4 org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_dynamic_partition_pruning org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_hybridgrace_hashjoin_1 org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_llap_nullscan org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_llap_udf org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_llapdecider org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_dml org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_join_result_complex org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_union org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vectorized_dynamic_partition_pruning org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_parallel_orderby org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_vector_outer_join1 org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_vector_outer_join2 org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_vector_outer_join3 org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_vector_outer_join4 org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_vector_outer_join5 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_cross_product_check_1 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_cross_product_check_2 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_ctas org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_cte_2 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_cte_4 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_dynamic_partition_pruning org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_empty_join org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_explainuser_1 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_explainuser_3 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_hybridgrace_hashjoin_1 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_llap_nullscan org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_llapdecider org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_temp_table org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_dml org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_join_result_complex org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_union org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_unionDistinct_1 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_unionDistinct_2 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_union_fast_stats org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_between_columns org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_between_in org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_char_mapjoin1 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_decimal_10_0 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_decimal_3 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_decimal_6 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_decimal_aggregate org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_decimal_expressions org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_decimal_math_funcs org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_grouping_sets org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_interval_mapjoin org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_join30 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_join_filters org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_join_nulls org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_leftsemi_mapjoin org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_multi_insert org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_nullsafe_join org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_outer_join1 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_outer_join2 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_outer_join3 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_outer_join4 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_outer_join5 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_outer_join6 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_partitioned_date_time org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_reduce_groupby_decimal org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_varchar_mapjoin1 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorization_decimal_date org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorization_short_regress org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorized_dynamic_partition_pruning org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_parallel_orderby org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_skewjoin_onesideskew org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_part org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_view_1 org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_view_2 org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_view_3 org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_view_4 org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_view_disable_cbo_1 org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_view_disable_cbo_2 org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_view_disable_cbo_3 org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_view_disable_cbo_4 org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_updateBasicStats org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_cross_product_check_1 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_cross_product_check_2 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_ctas org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_join41 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_join_filters_overlap org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_multi_insert_lateral_view org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_multi_join_union org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_parquet_join org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_semijoin org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_skewjoin_noskew org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_stats5 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_temp_table_join1 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_union24 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_union27 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_union31 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_union32 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_union_top_level org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_vector_between_in org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_vector_decimal_aggregate org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_vectorization_decimal_date org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_vectorization_short_regress org.apache.hive.jdbc.TestSSL.testSSLVersion
Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/7158/testReport
Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/7158/console
Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-7158/
Messages:
Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 226 tests failed
This message is automatically generated.
ATTACHMENT ID: 12789075 - PreCommit-HIVE-TRUNK-Build
+1
Pushed to master with updated golden files.
I am using Hive version 2.1.0 and trying to get Column level lineage. However, when I try to access the Dependency value in LineageInfo, it is coming as null. Sample Code:
public static Map<ColumnName, List<HiveColumnLineageInfo>> buildLineageMap(LineageInfo lInfo) { for(Map.Entry<LineageInfo.DependencyKey, LineageInfo.Dependency> e : lInfo.entrySet()) { System.out.println("Col Lineage Key : " + e.getKey()); System.out.println("Col Lineage Value: " + e.getValue());
In the output, Key is set to destination column in a lineage but the Value which should contain origin column information is null. How can I verify whether the lineage information being set correctly in Hive Sessionstate.
Harish Butani Can you please verify the issue and help. I have created a JIRA for this issue at https://issues.apache.org/jira/browse/HIVE-14706
Have you tried the lineage hook: org.apache.hadoop.hive.ql.hooks.LineageLogger?