Uploaded image for project: 'Apache Trafodion'
  1. Apache Trafodion
  2. TRAFODION-3268

UDF|TMUDF returns various errors regarding dlopen cached_libs file

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 2.4
    • Component/s: sql-general
    • Labels:
      None

      Description

      In the latest version, we started to see various random errors returned from queries with UDF or TMUDF. The errors sometimes are 4316 error indicating failures to create the cached_libs directory, sometimes are 11245|11248 errors complaining about 'file too short' with the cached file. Whenever the errors occurred, the actual file appeared to exist in the cached_libs directory on every node. 

       It is a regression introduced  presumably because of the UDF|TMUDF cached_libs concept. This problem doesn't always happen. Sometimes the same sequence of statements failed previously would work fine the next time. It's also not clear if this only happens on a multi-node cluster, as the errors were seen on nap035, which happens to be a 4-node cluster.

      Here are a few examples of these failures seen from the SQL regression run:

      ===========================================================================
      SQL>select * from myview v, mytable t
      +>where (qa_udf_integer(v.a), qa_udf_integer(t.b)) >
      +>all (select qa_udf_integer(a), qa_udf_integer(b) from mytable2
      +>where qa_udf_integer(a) = qa_udf_integer(b));

          • ERROR[4316] An error was detected while extracting a udr routine library to local cache : Error 0 creating directory :/opt/trafodion/esgyndb/udr/DB__ROOT/cached_libs/MYTEST/ .

      ===========================================================================

      SQL>select * from
      UDF(MYTMFUNC(TABLE(select * from trafodion.zzz_single_stream_features_namespace.mytable),
      'TEST_CMPTIME_RUNTIME_OUT_TABLE_DEFINED_USING_PASSTHRU', '3')) t1,
      UDF(MYTMFUNC(TABLE(select * from trafodion.zzz_single_stream_features_namespace_myns.mytable),
      'TEST_CMPTIME_RUNTIME_OUT_TABLE_DEFINED_USING_PASSTHRU', '3')) t2,
      UDF(MYTMFUNC(TABLE(select * from trafodion.zzz_single_stream_features_namespace_myns.mytable_too),
      'TEST_CMPTIME_RUNTIME_OUT_TABLE_DEFINED_USING_PASSTHRU', '3')) t3
      where t1.a = t2.a and t2.a = t3.a order by 1;

          • ERROR[11245] An error occurred loading library 'qaTmudfTest_212414198362332378.so' from path '/opt/trafodion/esgyndb/udr/DB__ROOT/cached_libs/ZZZ_SINGLE_STREAM_FEATURES_NAMESPACE_MYNS'. [2019-01-14 00:50:22]
          • ERROR[11248] A call to dlopen returned errors 0 and 0. Details:
            /opt/trafodion/esgyndb/udr/DB__ROOT/cached_libs/ZZZ_SINGLE_STREAM_FEATURES_NAMESPACE_MYNS/qaTmudfTest_212414198362332378.so: file too short. [2019-01-14 00:50:22]

      ===========================================================================

      SQL>select * from myview v, mytable t
      where (qa_udf_integer(v.a), qa_udf_integer(t.b)) >
      all (select qa_udf_integer(a), qa_udf_integer(b) from mytable2
      where qa_udf_integer(a) = qa_udf_integer(b));

          • ERROR[11245] An error occurred loading library 'qaUdfTest_212414083371782103.so' from path '/opt/trafodion/esgyndb/udr/DB__ROOT/cached_libs/UDF_UDF_GENERAL'. [2019-01-12 15:20:07]
          • ERROR[11248] A call to dlopen returned errors 0 and 0. Details:
            /opt/trafodion/esgyndb/udr/DB__ROOT/cached_libs/UDF_UDF_GENERAL/qaUdfTest_212414083371782103.so: file too short. [2019-01-12 15:20:07]
          • ERROR[11245] An error occurred loading library 'qaUdfTest_212414083371782103.so' from path '/opt/trafodion/esgyndb/udr/DB__ROOT/cached_libs/UDF_UDF_GENERAL'. [2019-01-12 15:20:07]
          • ERROR[11248] A call to dlopen returned errors 0 and 0. Details:
            /opt/trafodion/esgyndb/udr/DB__ROOT/cached_libs/UDF_UDF_GENERAL/qaUdfTest_212414083371782103.so: file too short. [2019-01-12 15:20:07]
          • ERROR[11245] An error occurred loading library 'qaUdfTest_212414083371782103.so' from path '/opt/trafodion/esgyndb/udr/DB__ROOT/cached_libs/UDF_UDF_GENERAL'. [2019-01-12 15:20:07]
          • ERROR[11248] A call to dlopen returned errors 0 and 0. Details:
            /opt/trafodion/esgyndb/udr/DB__ROOT/cached_libs/UDF_UDF_GENERAL/qaUdfTest_212414083371782103.so: file too short. [2019-01-12 15:20:07]
          • ERROR[11245] An error occurred loading library 'qaUdfTest_212414083371782103.so' from path '/opt/trafodion/esgyndb/udr/DB__ROOT/cached_libs/UDF_UDF_GENERAL'. [2019-01-12 15:20:07]
          • ERROR[11248] A call to dlopen returned errors 0 and 0. Details:
            /opt/trafodion/esgyndb/udr/DB__ROOT/cached_libs/UDF_UDF_GENERAL/qaUdfTest_212414083371782103.so: file too short. [2019-01-12 15:20:07]
          • ERROR[11245] An error occurred loading library 'qaUdfTest_212414083371782103.so' from path '/opt/trafodion/esgyndb/udr/DB__ROOT/cached_libs/UDF_UDF_GENERAL'. [2019-01-12 15:20:07]
          • ERROR[11248] A call to dlopen returned errors 0 and 0. Details:
            /opt/trafodion/esgyndb/udr/DB__ROOT/cached_libs/UDF_UDF_GENERAL/qaUdfTest_212414083371782103.so: file too short. [2019-01-12 15:20:07]
          • ERROR[11245] An error occurred loading library 'qaUdfTest_212414083371782103.so' from path '/opt/trafodion/esgyndb/udr/DB__ROOT/cached_libs/UDF_UDF_GENERAL'. [2019-01-12 15:20:07]
          • ERROR[11248] A call to dlopen returned errors 0 and 0. Details:
            /opt/trafodion/esgyndb/udr/DB__ROOT/cached_libs/UDF_UDF_GENERAL/qaUdfTest_212414083371782103.so: file too short. [2019-01-12 15:20:07]

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                sandhya Sandhya Sundaresan
                Reporter:
                sandhya Sandhya Sundaresan
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 2h 20m
                  2h 20m