[SQOOP-3225] Mainframe module FTP listing parser should cater for larger datasets on disk

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 1.5.0
• Assignee: Chris Teoh
• Reporter: Chris Teoh
• Created: 24/Aug/17 12:35
• Updated: 24/Oct/17 14:54
• Resolved: 24/Oct/17 14:54

Description

The parser can handle smaller datasets but when a dataset reaches a certain size, the FTP listing changes such that the current FTP parser doesn't read the listing correctly. Current workaround is to move the datasets to tape. Sample is below of some tape based datasets followed by a large dataset followed by a small dataset:-

{{ftp> ls
227 Entering Passive Mode (Xx,XX,XXX,XXX,XX,XXX)
125 List started OK
Volume Unit Referred Ext Used Recfm Lrecl BlkSz Dsorg Dsname
Migrated G1622V00
Migrated G1623V00
Migrated G1624V00
Migrated G1625V00
Migrated G1626V00
Migrated G1627V00
Migrated G1628V00
Migrated G1629V00
Migrated G1630V00
Migrated G1631V00
Migrated G1632V00
Migrated G1633V00
Migrated G1634V00
Migrated G1635V00
Migrated G1636V00
Migrated G1637V00
Migrated G1638V00
Migrated G1639V00
Migrated G1640V00
xxxx39 3390 2017/08/02 633540 FB 847 27951 PS G1641V00
xxxx09 3390 2017/08/02 1 15 FB 132 27984 PS OUTPUT}}

Issue Links

• links to: https://reviews.apache.org/r/62653/