Derby
  1. Derby
  2. DERBY-4484

Add import/export support for UDTs.

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 10.6.1.0
    • Fix Version/s: 10.6.1.0
    • Component/s: SQL
    • Labels:
      None
    • Issue & fix info:
      Patch Available

      Description

      Import/export should work on UDTs.

        Issue Links

          Activity

          Hide
          Rick Hillegas added a comment -

          Attaching derby-4484-01-ac-basicExportImport.diff. This adds basic support for exporting and importing UDT columns. UDTs can be exported and imported alongside other columns to/from a file. Regression tests are running.

          This patch does NOT support dumping and retrieving UDT columns to/from a side file devoted to large objects--as can be done for LOBs and large binary columns. But that would be a useful next improvement.

          In exporting a UDT column, it is serialized first to a byte array. Then that array is turned into a hex string just like a VARBINARY column. Importing the UDT reverses this process.

          Touches the following files:

          ---------

          M java/engine/org/apache/derby/iapi/types/DataTypeDescriptor.java
          M java/engine/org/apache/derby/catalog/types/BaseTypeIdImpl.java
          M java/engine/org/apache/derby/catalog/types/UserDefinedTypeIdImpl.java
          M java/engine/org/apache/derby/impl/sql/compile/ValueNode.java

          Type system support for UDTs was improved to make it possible to create a UDT descriptor from the schema qualified name of the type. The Import VTI needs this. Along the way, the ValueNode was improved to bind a classname to these UDT descriptors.

          ---------

          M java/engine/org/apache/derby/impl/load/ExportAbstract.java
          M java/engine/org/apache/derby/impl/load/ColumnInfo.java
          M java/engine/org/apache/derby/impl/load/Import.java
          M java/engine/org/apache/derby/impl/load/ImportResultSetMetaData.java
          M java/engine/org/apache/derby/impl/load/ImportAbstract.java

          Changes to export and import to allow UDT columns in exported files. The import machinery is a little tricky because it relies on a VTI, whose metadata descriptions had to be improved.

          ---------

          M java/engine/org/apache/derby/impl/sql/compile/CreateTableNode.java

          The ImportExportTest uncovered some flaws in CREATE TABLE AS which prevented us from creating a table's shape from a query if one of the result columns was of UDT type. Fixed in this patch.

          ---------

          M java/testing/org/apache/derbyTesting/functionTests/tests/lang/UDTTest.java
          M java/testing/org/apache/derbyTesting/functionTests/tests/tools/ImportExportTest.java

          Regression tests for basic export/import of UDT columns.

          ---------

          Show
          Rick Hillegas added a comment - Attaching derby-4484-01-ac-basicExportImport.diff. This adds basic support for exporting and importing UDT columns. UDTs can be exported and imported alongside other columns to/from a file. Regression tests are running. This patch does NOT support dumping and retrieving UDT columns to/from a side file devoted to large objects--as can be done for LOBs and large binary columns. But that would be a useful next improvement. In exporting a UDT column, it is serialized first to a byte array. Then that array is turned into a hex string just like a VARBINARY column. Importing the UDT reverses this process. Touches the following files: --------- M java/engine/org/apache/derby/iapi/types/DataTypeDescriptor.java M java/engine/org/apache/derby/catalog/types/BaseTypeIdImpl.java M java/engine/org/apache/derby/catalog/types/UserDefinedTypeIdImpl.java M java/engine/org/apache/derby/impl/sql/compile/ValueNode.java Type system support for UDTs was improved to make it possible to create a UDT descriptor from the schema qualified name of the type. The Import VTI needs this. Along the way, the ValueNode was improved to bind a classname to these UDT descriptors. --------- M java/engine/org/apache/derby/impl/load/ExportAbstract.java M java/engine/org/apache/derby/impl/load/ColumnInfo.java M java/engine/org/apache/derby/impl/load/Import.java M java/engine/org/apache/derby/impl/load/ImportResultSetMetaData.java M java/engine/org/apache/derby/impl/load/ImportAbstract.java Changes to export and import to allow UDT columns in exported files. The import machinery is a little tricky because it relies on a VTI, whose metadata descriptions had to be improved. --------- M java/engine/org/apache/derby/impl/sql/compile/CreateTableNode.java The ImportExportTest uncovered some flaws in CREATE TABLE AS which prevented us from creating a table's shape from a query if one of the result columns was of UDT type. Fixed in this patch. --------- M java/testing/org/apache/derbyTesting/functionTests/tests/lang/UDTTest.java M java/testing/org/apache/derbyTesting/functionTests/tests/tools/ImportExportTest.java Regression tests for basic export/import of UDT columns. ---------
          Hide
          Rick Hillegas added a comment -

          Tests passed cleanly for me on derby-4484-01-ac-basicExportImport.diff.

          Show
          Rick Hillegas added a comment - Tests passed cleanly for me on derby-4484-01-ac-basicExportImport.diff.
          Hide
          Rick Hillegas added a comment -

          Committed derby-4484-01-ac-basicExportImport.diff at subversion revision 905289.

          Show
          Rick Hillegas added a comment - Committed derby-4484-01-ac-basicExportImport.diff at subversion revision 905289.
          Hide
          Knut Anders Hatlen added a comment -

          In UserDefinedTypeIdImpl, there's an error handler that looks like this:

          + } catch (Exception e)

          { throw new IllegalArgumentException( e.getMessage() ); }

          Here, the stack trace of the original error is lost, and it's also not obvious to me that any error happening in that method must be because of an illegal argument. I'd suggest that this method is declared to throw StandardException instead (I think that's the only checked exception in the method) and the try/catch is removed. Alternatively, use iapi.error.PassThroughException instead of IllegalArgumentException so that the original exception and stack trace can be unwrapped by higher-level error handlers.

          Similarly, I think this error handler in ColumnInfo.getUDTClassName()

          + catch (Exception e)

          { throw new SQLException( e.getMessage() ); }

          should throw LoadError.unexpectedError(e) instead (for preservation of stack trace and SQLState).

          Show
          Knut Anders Hatlen added a comment - In UserDefinedTypeIdImpl, there's an error handler that looks like this: + } catch (Exception e) { throw new IllegalArgumentException( e.getMessage() ); } Here, the stack trace of the original error is lost, and it's also not obvious to me that any error happening in that method must be because of an illegal argument. I'd suggest that this method is declared to throw StandardException instead (I think that's the only checked exception in the method) and the try/catch is removed. Alternatively, use iapi.error.PassThroughException instead of IllegalArgumentException so that the original exception and stack trace can be unwrapped by higher-level error handlers. Similarly, I think this error handler in ColumnInfo.getUDTClassName() + catch (Exception e) { throw new SQLException( e.getMessage() ); } should throw LoadError.unexpectedError(e) instead (for preservation of stack trace and SQLState).
          Hide
          Rick Hillegas added a comment -

          Thanks, Knut. Those are good suggestions. Attaching derby-4484-02-aa-exceptionHandling.diff, which makes those changes. Touches the following files:

          M java/engine/org/apache/derby/impl/load/ColumnInfo.java
          M java/engine/org/apache/derby/impl/sql/compile/JavaValueNode.java
          M java/engine/org/apache/derby/impl/sql/compile/sqlgrammar.jj
          M java/engine/org/apache/derby/impl/sql/catalog/SYSCOLUMNSRowFactory.java
          M java/engine/org/apache/derby/impl/sql/catalog/SYSALIASESRowFactory.java
          M java/engine/org/apache/derby/impl/sql/catalog/SYSCOLPERMSRowFactory.java
          M java/engine/org/apache/derby/impl/sql/catalog/SYSSTATISTICSRowFactory.java
          M java/engine/org/apache/derby/impl/sql/catalog/SYSSEQUENCESRowFactory.java
          M java/engine/org/apache/derby/impl/sql/catalog/SYSROUTINEPERMSRowFactory.java
          M java/engine/org/apache/derby/impl/sql/catalog/SYSVIEWSRowFactory.java
          M java/engine/org/apache/derby/impl/sql/catalog/SYSTABLEPERMSRowFactory.java
          M java/engine/org/apache/derby/impl/sql/catalog/SYSCHECKSRowFactory.java
          M java/engine/org/apache/derby/impl/sql/catalog/SYSFILESRowFactory.java
          M java/engine/org/apache/derby/impl/sql/catalog/SYSTABLESRowFactory.java
          M java/engine/org/apache/derby/impl/sql/catalog/SYSKEYSRowFactory.java
          M java/engine/org/apache/derby/impl/sql/catalog/SYSDEPENDSRowFactory.java
          M java/engine/org/apache/derby/impl/sql/catalog/SYSSCHEMASRowFactory.java
          M java/engine/org/apache/derby/impl/sql/catalog/SYSROLESRowFactory.java
          M java/engine/org/apache/derby/impl/sql/catalog/SYSFOREIGNKEYSRowFactory.java
          M java/engine/org/apache/derby/impl/sql/catalog/SYSTRIGGERSRowFactory.java
          M java/engine/org/apache/derby/impl/sql/catalog/SYSPERMSRowFactory.java
          M java/engine/org/apache/derby/impl/sql/catalog/SYSSTATEMENTSRowFactory.java
          M java/engine/org/apache/derby/impl/sql/catalog/SYSCONGLOMERATESRowFactory.java
          M java/engine/org/apache/derby/impl/sql/catalog/SystemColumnImpl.java
          M java/engine/org/apache/derby/iapi/sql/dictionary/CatalogRowFactory.java
          M java/engine/org/apache/derby/iapi/types/TypeId.java
          M java/engine/org/apache/derby/iapi/types/DataTypeDescriptor.java
          M java/engine/org/apache/derby/iapi/types/JSQLType.java
          M java/engine/org/apache/derby/catalog/types/UserDefinedTypeIdImpl.java

          Show
          Rick Hillegas added a comment - Thanks, Knut. Those are good suggestions. Attaching derby-4484-02-aa-exceptionHandling.diff, which makes those changes. Touches the following files: M java/engine/org/apache/derby/impl/load/ColumnInfo.java M java/engine/org/apache/derby/impl/sql/compile/JavaValueNode.java M java/engine/org/apache/derby/impl/sql/compile/sqlgrammar.jj M java/engine/org/apache/derby/impl/sql/catalog/SYSCOLUMNSRowFactory.java M java/engine/org/apache/derby/impl/sql/catalog/SYSALIASESRowFactory.java M java/engine/org/apache/derby/impl/sql/catalog/SYSCOLPERMSRowFactory.java M java/engine/org/apache/derby/impl/sql/catalog/SYSSTATISTICSRowFactory.java M java/engine/org/apache/derby/impl/sql/catalog/SYSSEQUENCESRowFactory.java M java/engine/org/apache/derby/impl/sql/catalog/SYSROUTINEPERMSRowFactory.java M java/engine/org/apache/derby/impl/sql/catalog/SYSVIEWSRowFactory.java M java/engine/org/apache/derby/impl/sql/catalog/SYSTABLEPERMSRowFactory.java M java/engine/org/apache/derby/impl/sql/catalog/SYSCHECKSRowFactory.java M java/engine/org/apache/derby/impl/sql/catalog/SYSFILESRowFactory.java M java/engine/org/apache/derby/impl/sql/catalog/SYSTABLESRowFactory.java M java/engine/org/apache/derby/impl/sql/catalog/SYSKEYSRowFactory.java M java/engine/org/apache/derby/impl/sql/catalog/SYSDEPENDSRowFactory.java M java/engine/org/apache/derby/impl/sql/catalog/SYSSCHEMASRowFactory.java M java/engine/org/apache/derby/impl/sql/catalog/SYSROLESRowFactory.java M java/engine/org/apache/derby/impl/sql/catalog/SYSFOREIGNKEYSRowFactory.java M java/engine/org/apache/derby/impl/sql/catalog/SYSTRIGGERSRowFactory.java M java/engine/org/apache/derby/impl/sql/catalog/SYSPERMSRowFactory.java M java/engine/org/apache/derby/impl/sql/catalog/SYSSTATEMENTSRowFactory.java M java/engine/org/apache/derby/impl/sql/catalog/SYSCONGLOMERATESRowFactory.java M java/engine/org/apache/derby/impl/sql/catalog/SystemColumnImpl.java M java/engine/org/apache/derby/iapi/sql/dictionary/CatalogRowFactory.java M java/engine/org/apache/derby/iapi/types/TypeId.java M java/engine/org/apache/derby/iapi/types/DataTypeDescriptor.java M java/engine/org/apache/derby/iapi/types/JSQLType.java M java/engine/org/apache/derby/catalog/types/UserDefinedTypeIdImpl.java
          Hide
          Knut Anders Hatlen added a comment -

          Thanks for making these changes, Rick. I didn't realize that adding a throws declaration to UserDefinedTypeIdImpl's constructor would affect so many files... But the changes look good to me. +1 to commit if the regression tests still pass.

          Show
          Knut Anders Hatlen added a comment - Thanks for making these changes, Rick. I didn't realize that adding a throws declaration to UserDefinedTypeIdImpl's constructor would affect so many files... But the changes look good to me. +1 to commit if the regression tests still pass.
          Hide
          Rick Hillegas added a comment -

          Thanks, Knut. The tests ran cleanly for me. Committed derby-4484-02-aa-exceptionHandling.diff at subversion revision 905472.

          Show
          Rick Hillegas added a comment - Thanks, Knut. The tests ran cleanly for me. Committed derby-4484-02-aa-exceptionHandling.diff at subversion revision 905472.

            People

            • Assignee:
              Rick Hillegas
              Reporter:
              Rick Hillegas
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development