FTPClient listFiles returns incorrect timestamp on freshly uploaded file but corrects itself after about 15 minutes

• Key: NET-140
• Type: Bug
• Status: Closed
• Resolution: Fixed
• Priority: Major
• Assignee: Unassigned
• Reporter: Bill Giel
• Affects Version/s: 1.4
• Fix Version/s: 2.0

Description

This is an odd one:

We upload GPS data each hour to a public site using FTPClient. Every 24 hours we check for files older than 60 days using listFiles and getting the timestamps do decide if we want to delete older files.

When we list the files, the most recently uploaded files have a time stamp exactly one year too old. After about 15 minutes, it seems to correct itself and eventually displays the correct timestamp.

During this time while FTPFile.getTimestamp is giving the incorrect timestamp, browsing the folder with a web browser, a commercial FTP client, or actually checking the file info in a shell shows the correct timestamp (i.e. does not seem to be a problem on the remote site)

commons-net-1.4.1 (as well as commons-net-20060901) exhibits this behavior.

commons-net-1.3.0 works properly

I did a little investigating, and it seems to happen with every file written to the remote directory each hour, and the incorrect timestamp will be returned using listFiles for about 15 minutes... and then it corrects itself.

Comments

[Rory Winston]

Bill

Would you be able to provide some raw listings of files that exhibit this behaviour? I think this is related to a previous issue regarding date handling and old/new date formats.

[Bill Giel]

[[ Old comment, sent by email on Sun, 03 Sep 2006 01:29:31 -0400 ]]

I may not have made it clear, the only machine that corrects itself is
the Sphera Linux machine. I have a feeling that there is a process
running there that causes this behavior. The timestamps do not correct
themselves on the RedHat9 machine.

Bill

–
William Giel, LS
======================
Rocco V. D'Andrea, Inc.
PO Box 549 / Six Neil Lane
Riverside, CT 06460
203-637-1779 (Voice)
203-637-1770 (Fax)
[e-mail]
www.rvdi.com

[Bill Giel]

[[ Old comment, sent by email on Sun, 03 Sep 2006 01:22:07 -0400 ]]

Thanks for the quick reply, Rory.

See the attached, which contains a small test source and some results
for a few different sites. A couple of sites worked well, but a couple
do not. Unfortunately for me, my public site is one that does not work.
(But commons-net-1.3.0 works for all, so I'll be using that for now.)

Let me know if you need more info, I'll try my best to get what you need.

Regards,

Bill

–
William Giel, LS
======================
Rocco V. D'Andrea, Inc.
PO Box 549 / Six Neil Lane
Riverside, CT 06460
203-637-1779 (Voice)
203-637-1770 (Fax)
[e-mail]
www.rvdi.com

[demime 1.01d removed an attachment of type application/x-zip-compressed which had a name of timestamp-issue.zip]

[Bill Giel]

[[ Old comment, sent by email on Sun, 03 Sep 2006 16:11:57 -0400 ]]

Some more info... Sphera VDS Linux mentioned previously, ftpd is version
wu-2.6.2(1).

That did not have a problem on the Sun machine, but it does have a
problem on the Sphera Linux machine.

–
William Giel, LS
======================
Rocco V. D'Andrea, Inc.
PO Box 549 / Six Neil Lane
Riverside, CT 06460
203-637-1779 (Voice)
203-637-1770 (Fax)
[e-mail]
www.rvdi.com

[Julien Aymé]

Isn't this issue related to the issue <a href="https://issues.apache.org/jira/browse/NET-159">NET-159 </a> ?

[Rory Winston]

The only reason I can see that the timestamp will be out of sync for 15 minutes and then ok is if the FTP server sends a different timestamp for files that are older than 15 minutes. This will result in the above behaviour.

I have implemented a workaround for this problem - by calling FTPClient::setDateRollbackPermitted(false), the rollback by 1year should not occur for these situations.

Any issues, feel free to reopen.

[James Fu]

Hi

I still see this timestamp-display-incorrect issue even when using commons-net-2.0.jar.
Any newly uploaded file in year 2009 will be displayed as year 2008, and on the next day, it will display as year 2009 correctly.

I've read articles about issue NET-140 and related NET-159, it seems that this issue has already been fixed? I've tried the following code segments and it seems to have been resolved, just wonder if this is the official way to handle this issue.

FTPClientConfig conf = new FTPClientConfig(FTPClientConfig.SYST_UNIX);
conf.setLenientFutureDates(true);
this.ftp.configure(conf);

[Rory Winston]

Hi James

Could you provide some examples of the files in question (i.e. would you be able to provide the raw output from an FTP listing)?

[Sebb]

IIRC, Commons NET has to use the local system time to determine what to do with file listings that have no year information.

So it would also be useful to know whether the client and ftp server times are in synch. or not - and which is ahead - and also the timezones on the two systems.

[James Fu]

Hi Rory and Sebb, thanks for the reply!

To Rory:
Here's the raw output from ftp server corresponding to the file that I just uploaded:

rw-rr- 1 182390 eng2 91184 Apr 2 10:06 0C5F8D76F819BF060A791469835B1BF0.commons-net-ftp-2.0.jar

If not using following code segments, I got display year 2008; if used, go correct year 2009.

FTPClientConfig conf = new FTPClientConfig(FTPClientConfig.SYST_UNIX);
conf.setLenientFutureDates(true);
this.ftp.configure(conf);

To Sebb:
The timezone of client and ftp server are both GMT. I've both tried to set the time on client side to be ahead of server time and behind server time, but seems the same. All need to setLenientFutureDates(true) to make it display correctly.