[FLUME-3173] Upgrade joda-time

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.7.0
• Fix Version/s: 1.8.0
• Component/s: Sinks+Sources
• Assignee: Miklos Csanady
• Reporter: Azat Nizametdinov
• Created: 11/Sep/17 08:19
• Updated: 12/Sep/17 14:33
• Resolved: 12/Sep/17 13:26

Description

Flume 1.7 depends on joda-time version 2.1 which uses outdated tz database.
For example following code

new org.joda.time.DateTime(
    org.joda.time.DateTimeZone.forID("Europe/Moscow")
).toString()

returns time with offset +04:00, but Moscow timezone is UTC+3 since 2014.

Furthermore this version of joda-time does not allow to specify custom tz databse folder in contrast to newer versions.

It affects RegexExtractorInterceptorMillisSerializer. Test to reproduce the bug:

public void testMoscowTimezone() throws Exception {
    TimeZone.setDefault(TimeZone.getTimeZone("Europe/Moscow"));

    String pattern = "yyyy-MM-dd HH:mm:ss";
    SimpleDateFormat format = new SimpleDateFormat(pattern);
    String dateStr = "2017-09-10 10:00:00";
    Date expectedDate = format.parse(dateStr);

    RegexExtractorInterceptorMillisSerializer sut = new RegexExtractorInterceptorMillisSerializer();
    Context context = new Context();
    context.put("pattern", pattern);
    sut.configure(context);

    assertEquals(String.valueOf(expectedDate.getTime()), sut.serialize(dateStr));
}

Issue Links

• is required by: FLUME-3174 HdfsSink AWS S3A authentication does not work on JDK 8
• relates to: FLUME-3177 Replace joda-time with Java 8 Date and Time API
• links to: Pull request