Derby
  1. Derby
  2. DERBY-2839

initial problems with the 10.3.1.0 beta candidate

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 10.3.1.4
    • Fix Version/s: 10.3.1.4, 10.4.1.3
    • Component/s: Build tools
    • Labels:
      None
    • Urgency:
      Blocker

      Description

      1) Problem unzipping derby_doc_plugin_1.1.0.zip and derby_ui_plugin_1.1.0.zip:

      rick doc-plugin (1.4) > unzip derby_doc_plugin_1.1.0.zip
      Archive: derby_doc_plugin_1.1.0.zip
      End-of-central-directory signature not found. Either this file is not
      a zipfile, or it constitutes one disk of a multi-part archive. In the
      latter case the central directory and zipfile comment will be found on
      the last disk(s) of this archive.
      unzip: cannot find zipfile directory in one of derby_doc_plugin_1.1.0.zip or
      derby_doc_plugin_1.1.0.zip.zip, and cannot find derby_doc_plugin_1.1.0.zip.ZIP, period.

      2) Old problem: Filenames in the src-tar distribution are truncated. What
      is the use of this distribution?

      3) Broken links in index.html:

      • Could not find link to pdf for Getting Started. The file is called
        getstartderby.pdf but the link references getstart.pdf
      • Could not find link to either of the html files for Working with Derby. The actual directory
        is workingwithderbytemp but should be workingwithderby
      • Could not find the pdf for Working with Derby either. The
        workingwithderby subdirectory doesn't exist under the pdf branch.
      • Could not find pdf for Admin Guide. The file is called
        derbyadmin.pdf but the link specifies adminguide.pdf
      1. DERBY-2839_3-src.diff
        7 kB
        Myrna van Lunteren
      2. DERBY-2839_3-doc.diff
        14 kB
        Myrna van Lunteren

        Issue Links

          Activity

          Rick Hillegas created issue -
          Daniel John Debrunner made changes -
          Field Original Value New Value
          Link This issue relates to DERBY-2619 [ DERBY-2619 ]
          Daniel John Debrunner made changes -
          Link This issue relates to DERBY-2840 [ DERBY-2840 ]
          Myrna van Lunteren made changes -
          Assignee Myrna van Lunteren [ myrna ]
          Hide
          Myrna van Lunteren added a comment -

          Re the last item...(broken links in index.html)...So...The working with derby manual has been removed - it's been merged with the getting start manual.

          However, we still have the working with derby demos, which now have been orphaned...Most of their function was to illustrate steps in the workingwithderby manual...
          Can these demo files be removed also?
          Or just not included / built?

          Show
          Myrna van Lunteren added a comment - Re the last item...(broken links in index.html)...So...The working with derby manual has been removed - it's been merged with the getting start manual. However, we still have the working with derby demos, which now have been orphaned...Most of their function was to illustrate steps in the workingwithderby manual... Can these demo files be removed also? Or just not included / built?
          Hide
          Rick Hillegas added a comment -

          Hm. I think that the demo code will continue to be copied into the source distribution, along with the rest of the source tree. That sounds good to me. I don't see the demo code mentioned in the Getting Started guide. Where, for instance, is toursdb mentioned? Here's another puzzle: the Getting Started guide mentions a sample program called gsEmbedded.java and a utility class called gsUtils.java. Where do these live?

          Show
          Rick Hillegas added a comment - Hm. I think that the demo code will continue to be copied into the source distribution, along with the rest of the source tree. That sounds good to me. I don't see the demo code mentioned in the Getting Started guide. Where, for instance, is toursdb mentioned? Here's another puzzle: the Getting Started guide mentions a sample program called gsEmbedded.java and a utility class called gsUtils.java. Where do these live?
          Hide
          Myrna van Lunteren added a comment -

          ah, I see what happened, they were renamed in the doc, but not in the code.
          I'll fix that up...

          Show
          Myrna van Lunteren added a comment - ah, I see what happened, they were renamed in the doc, but not in the code. I'll fix that up...
          Hide
          Myrna van Lunteren added a comment -

          Re 2)Old problem: Filenames in the src-tar distribution are truncated. What
          is the use of this distribution:

          If this is an old problem, was there ever a solution? I couldn't find any info about it.

          Show
          Myrna van Lunteren added a comment - Re 2)Old problem: Filenames in the src-tar distribution are truncated. What is the use of this distribution: If this is an old problem, was there ever a solution? I couldn't find any info about it.
          Hide
          Rick Hillegas added a comment -

          Re (2): I'm not aware of an existing JIRA or discussion of this. It has simply been true all along as far as I can see. I have a hard time understanding how someone would use this distribution which has truncated file names. Does Apache require us to provide a src tarball in addition to a src zip? If so, then I think the src tarball ought to be usable.

          Show
          Rick Hillegas added a comment - Re (2): I'm not aware of an existing JIRA or discussion of this. It has simply been true all along as far as I can see. I have a hard time understanding how someone would use this distribution which has truncated file names. Does Apache require us to provide a src tarball in addition to a src zip? If so, then I think the src tarball ought to be usable.
          Hide
          Myrna van Lunteren added a comment -

          Attaching patches for item 3
          1. for doc:

          • fixing up the getting started guide to rever to the actual file names in the demo/workingwithderby directory
            2 for src:
          • fixing up references to pdf manuals, and removing links to no longer existing workingwithderby manual.
          Show
          Myrna van Lunteren added a comment - Attaching patches for item 3 1. for doc: fixing up the getting started guide to rever to the actual file names in the demo/workingwithderby directory 2 for src: fixing up references to pdf manuals, and removing links to no longer existing workingwithderby manual.
          Myrna van Lunteren made changes -
          Attachment DERBY-2839_3-src.diff [ 12360231 ]
          Attachment DERBY-2839_3-doc.diff [ 12360232 ]
          Hide
          Rick Hillegas added a comment -

          Thanks, Myrna. These patches look good just with a visual inspection. However, I haven't tried to build a release with these changes to verify that the end-product looks good.

          Show
          Rick Hillegas added a comment - Thanks, Myrna. These patches look good just with a visual inspection. However, I haven't tried to build a release with these changes to verify that the end-product looks good.
          Hide
          Myrna van Lunteren added a comment -

          I committed revisions 549267, 549279 and 549288 to code trunk, backported to 10.3 with 549280 and 549289; and revision 549263 to docs/trunk, backported to 10.3 with 549281.
          I think this resolves item 3)
          I did an ant release and all links work and I spotted no more references that were off after my latest iteration.

          Show
          Myrna van Lunteren added a comment - I committed revisions 549267, 549279 and 549288 to code trunk, backported to 10.3 with 549280 and 549289; and revision 549263 to docs/trunk, backported to 10.3 with 549281. I think this resolves item 3) I did an ant release and all links work and I spotted no more references that were off after my latest iteration.
          Hide
          Myrna van Lunteren added a comment -

          I don't understand item 2...
          I put db-derby-10.3.1.0-src.tar.gz file on a machine, did gzip -d, then tar -xvf and I see no truncated files. Maybe I'm not looking at the right things?
          Which file is getting truncated for you?
          Maybe there's an issue with older versions of gnu tar?

          Show
          Myrna van Lunteren added a comment - I don't understand item 2... I put db-derby-10.3.1.0-src.tar.gz file on a machine, did gzip -d, then tar -xvf and I see no truncated files. Maybe I'm not looking at the right things? Which file is getting truncated for you? Maybe there's an issue with older versions of gnu tar?
          Hide
          Knut Anders Hatlen added a comment -

          Re 2)
          GNU tar uses a (non-standard) format to store file names longer than 100 characters. For instance, using Solaris tar to extract the src-tar distribution, I get lots of messages like this

          tar: ././@LongLink: typeflag 'L' not recognized, converting to regular file
          tar: ././@LongLink: typeflag 'L' not recognized, converting to regular file
          tar: ././@LongLink: typeflag 'L' not recognized, converting to regular file
          tar: ././@LongLink: typeflag 'L' not recognized, converting to regular file
          ....

          and long names get truncated, for instance
          db-derby-10.3.1.0-src/java/client/org/apache/derby/client/am/UpdateSensitiveBlobLocatorInputStream.java -> db-derby-10.3.1.0-src/java/client/org/apache/derby/client/am/UpdateSensitiveBlobLocatorInputStream.j

          Extracting the archive with GNU tar does not truncate the names.

          If I recreate the archive with GNU tar and tells it to use a standard format with "--format=ustar", Solaris tar has no problem with the long names. According to the GNU tar manual, --format=ustar will produce a POSIX.1-1988 compatible archive, which supports file names up to 256 characters. As far as I can see, 256 characters is sufficient for the source distribution.

          Probably not a big problem since GNU tar is available on most platforms, but generally I think using a standards-compliant format is better.

          Show
          Knut Anders Hatlen added a comment - Re 2) GNU tar uses a (non-standard) format to store file names longer than 100 characters. For instance, using Solaris tar to extract the src-tar distribution, I get lots of messages like this tar: ././@LongLink: typeflag 'L' not recognized, converting to regular file tar: ././@LongLink: typeflag 'L' not recognized, converting to regular file tar: ././@LongLink: typeflag 'L' not recognized, converting to regular file tar: ././@LongLink: typeflag 'L' not recognized, converting to regular file .... and long names get truncated, for instance db-derby-10.3.1.0-src/java/client/org/apache/derby/client/am/UpdateSensitiveBlobLocatorInputStream.java -> db-derby-10.3.1.0-src/java/client/org/apache/derby/client/am/UpdateSensitiveBlobLocatorInputStream.j Extracting the archive with GNU tar does not truncate the names. If I recreate the archive with GNU tar and tells it to use a standard format with "--format=ustar", Solaris tar has no problem with the long names. According to the GNU tar manual, --format=ustar will produce a POSIX.1-1988 compatible archive, which supports file names up to 256 characters. As far as I can see, 256 characters is sufficient for the source distribution. Probably not a big problem since GNU tar is available on most platforms, but generally I think using a standards-compliant format is better.
          Hide
          Knut Anders Hatlen added a comment -

          See also description of the longfile attribute in ant. http://ant.apache.org/manual/CoreTasks/tar.html
          tools/release/build.xml specifies longfile="gnu" for src.tgz.

          Show
          Knut Anders Hatlen added a comment - See also description of the longfile attribute in ant. http://ant.apache.org/manual/CoreTasks/tar.html tools/release/build.xml specifies longfile="gnu" for src.tgz.
          Hide
          Myrna van Lunteren added a comment -

          As gnu is really the only viable option for src distribution with ant - 'fail' would never create a tar file; 'truncate' would result in files getting truncated with all tar implementations, and 'omit' would result in source files missing - I don't see how we can get around this.

          I thought of doing a manual step of untar-ring with a non gnu-tar, then tarring back up with a non-tar utility, or tar with --format=ustar, and using that, but then the md5 checksum will fail...Is that acceptable?

          Or can we state one needs gnu-tar?

          Show
          Myrna van Lunteren added a comment - As gnu is really the only viable option for src distribution with ant - 'fail' would never create a tar file; 'truncate' would result in files getting truncated with all tar implementations, and 'omit' would result in source files missing - I don't see how we can get around this. I thought of doing a manual step of untar-ring with a non gnu-tar, then tarring back up with a non-tar utility, or tar with --format=ustar, and using that, but then the md5 checksum will fail...Is that acceptable? Or can we state one needs gnu-tar?
          Hide
          Rick Hillegas added a comment -

          I think that we should state that you need gnu-tar to unspool these tarballs. That clarification would improve the out-of-box experience.

          Show
          Rick Hillegas added a comment - I think that we should state that you need gnu-tar to unspool these tarballs. That clarification would improve the out-of-box experience.
          Hide
          Knut Anders Hatlen added a comment -

          I agree, we should state that GNU tar (or compatible) is required.

          > I thought of doing a manual step of untar-ring with a non gnu-tar, then tarring back up with a non-tar utility, or tar with --format=ustar, and using that, but then the md5 checksum will fail...Is that acceptable?

          No, that's not acceptable. The md5 checksums would have to be recreated too, otherwise they are useless. But I'm not sure we want too many manual steps in the release process, so stating that GNU tar is required is probably sufficient. Most platforms come with unzip or GNU tar installed by default, so it shouldn't be a big problem as long as the requirements are clearly stated.

          There was a similar discussion on the Ivy list a couple of months ago.
          <URL:http://mail-archives.apache.org/mod_mbox/incubator-ivy-dev/200704.mbox/%3C20070402183548.GA25940@petunia.outback.escape.de%3E>

          I think they ended up with a disclaimer on the download page. One of the messages in the thread said:

          > Ant and many other projects include a disclaimer about requiring GNU
          > tar on the download page. I'd suggest to do the same for Ivy as well.

          A good idea for Derby too, I think.

          Show
          Knut Anders Hatlen added a comment - I agree, we should state that GNU tar (or compatible) is required. > I thought of doing a manual step of untar-ring with a non gnu-tar, then tarring back up with a non-tar utility, or tar with --format=ustar, and using that, but then the md5 checksum will fail...Is that acceptable? No, that's not acceptable. The md5 checksums would have to be recreated too, otherwise they are useless. But I'm not sure we want too many manual steps in the release process, so stating that GNU tar is required is probably sufficient. Most platforms come with unzip or GNU tar installed by default, so it shouldn't be a big problem as long as the requirements are clearly stated. There was a similar discussion on the Ivy list a couple of months ago. <URL: http://mail-archives.apache.org/mod_mbox/incubator-ivy-dev/200704.mbox/%3C20070402183548.GA25940@petunia.outback.escape.de%3E > I think they ended up with a disclaimer on the download page. One of the messages in the thread said: > Ant and many other projects include a disclaimer about requiring GNU > tar on the download page. I'd suggest to do the same for Ivy as well. A good idea for Derby too, I think.
          Hide
          Myrna van Lunteren added a comment -

          Thx for all the input! So, when I post the final release, I'll make sure to add a comment next to/above the src.tar.gz, indicating this requires gnu tar.

          Now, for the first item...
          I don't see a problem unzipping the plugin zip files...Although I noticed Rick apparently saved the file as a zip.zip, instead of .zip...Not sure if that matters.

          However, I noticed that there was a difference between how previous releases produced the zip file. Finally I figured out in a 'new' eclipse installation there was an option re Jar files checked on the 'Options' tab in the project export (once unchecked, it stays unchecked, even with new plugin projects). I'm not sure this is only with eclipse 3.1, but I'll make a note in the plugins/Readme.txt assuming it is eclipse 3.2, and that it should be unchecked.

          Also with previous derby releases, the plugins were combined, instead of 2 separate zipfiles as I made them.
          But, I thought it was a little strange to call the combined zip the same name as the single one, as was done with previous releases, so, I propose to give it a different name: derby_eclipse_plugins_1.1.0.zip.

          I created a new 103 co, svn reverted to revision 548006 (the original 10.3.1.0beta revision), created the core ui, used the core ui created (rather than some old version as it says in the plugins/Readme.txt - although at this point it doesn't matter as the core ui has never changed), build the combo zip file and uploaded the new file to http://people.apache.org/~myrnavl/10.3.1.0.beta/derby_eclipse_plugins_1.1.0.zip.

          Please have a look and see if that one unzips correctly...

          Show
          Myrna van Lunteren added a comment - Thx for all the input! So, when I post the final release, I'll make sure to add a comment next to/above the src.tar.gz, indicating this requires gnu tar. Now, for the first item... I don't see a problem unzipping the plugin zip files...Although I noticed Rick apparently saved the file as a zip.zip, instead of .zip...Not sure if that matters. However, I noticed that there was a difference between how previous releases produced the zip file. Finally I figured out in a 'new' eclipse installation there was an option re Jar files checked on the 'Options' tab in the project export (once unchecked, it stays unchecked, even with new plugin projects). I'm not sure this is only with eclipse 3.1, but I'll make a note in the plugins/Readme.txt assuming it is eclipse 3.2, and that it should be unchecked. Also with previous derby releases, the plugins were combined, instead of 2 separate zipfiles as I made them. But, I thought it was a little strange to call the combined zip the same name as the single one, as was done with previous releases, so, I propose to give it a different name: derby_eclipse_plugins_1.1.0.zip. I created a new 103 co, svn reverted to revision 548006 (the original 10.3.1.0beta revision), created the core ui, used the core ui created (rather than some old version as it says in the plugins/Readme.txt - although at this point it doesn't matter as the core ui has never changed), build the combo zip file and uploaded the new file to http://people.apache.org/~myrnavl/10.3.1.0.beta/derby_eclipse_plugins_1.1.0.zip . Please have a look and see if that one unzips correctly...
          Hide
          Rick Hillegas added a comment -

          Thanks, Myrna. This one unzips fine for me.

          Show
          Rick Hillegas added a comment - Thanks, Myrna. This one unzips fine for me.
          Hide
          Myrna van Lunteren added a comment -

          Marking resolved, because of item 3) fixes, with the note that I'll have to add a comment re requiring gnu tar for the src.tar.gz when I put out the full release.

          Show
          Myrna van Lunteren added a comment - Marking resolved, because of item 3) fixes, with the note that I'll have to add a comment re requiring gnu tar for the src.tar.gz when I put out the full release.
          Myrna van Lunteren made changes -
          Fix Version/s 10.3.1.0 [ 12312541 ]
          Status Open [ 1 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Fix Version/s 10.4.0.0 [ 12312540 ]
          Fix Version/s 10.3.1.1 [ 12312542 ]
          Hide
          Dyre Tjeldvoll added a comment -

          Blocker issues that have been 'Resolved' for a long time (6 months+)

          Show
          Dyre Tjeldvoll added a comment - Blocker issues that have been 'Resolved' for a long time (6 months+)
          Dyre Tjeldvoll made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Kathey Marsden made changes -
          Component/s Build tools [ 11405 ]
          Gavin made changes -
          Workflow jira [ 12406539 ] Default workflow, editable Closed status [ 12797394 ]

            People

            • Assignee:
              Myrna van Lunteren
              Reporter:
              Rick Hillegas
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development