[SPARK-27405] Restrict the range of generated random timestamps

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.4.0
• Fix Version/s: 3.0.0
• Component/s: Tests
• Assignee: Max Gekk
• Reporter: Max Gekk
• Created: 07/Apr/19 22:03
• Updated: 11/Apr/19 11:34
• Resolved: 11/Apr/19 11:34

Description

The timestampLiteralGen of LiteralGenerator can produce instances of java.sql.Timestamp that cause Long arithmetic overflow on conversion milliseconds to microseconds. The former conversion is performed because Catalyst's Timestamp type stores microseconds since epoch internally. The ticket aims to restrict the range of generated random timestamps to [Long.MaxValue / 1000, Long.MinValue/ 1000].

Issue Links

• links to: GitHub Pull Request #24316