[#HADOOP-2461] Configuration should trim property names and accept decimal, hexadecimal, and octal numbers

Hadoop Common

Configuration should trim property names and accept decimal, hexadecimal, and octal numbers

Created: 18/Dec/07 11:08 PM Updated: 22/Aug/08 07:50 PM

Component/s:

conf

Affects Version/s:

None

Fix Version/s:

0.18.0

Time Tracking:

Not Specified

File Attachments:

File Name	Date Attached	Attached By	Size
2461_20071218.patch	2007-12-18 11:19 PM	Tsz Wo (Nicholas), SZE	5 kB
2461_20080208.patch	2008-02-08 06:28 PM	Tsz Wo (Nicholas), SZE	1 kB
2461_20080425.patch	2008-04-25 06:11 PM	Tsz Wo (Nicholas), SZE	3 kB

Hadoop Flags:	Reviewed
Resolution Date:	29/Apr/08 10:42 PM

Description

« Hide

I suggest two improvements in reading configuration:

Suppose we have the following property in a conf file.

<property>
<name>
testing.property</name>
<value>something</value>
</property>

Try to get it by

Configuration conf = new Configuration();
String value = conf.get("testing.property"); //value == null here

We will get null since there is an eol in

<name>
testing.property</name>

I suggest to trim all property names.

I also suggest configuration to accept decimal, hexadecimal, and octal numbers (e.g. 011 is 9, 0xA is 10)
It can be easily done by replacing Integer.parseInt(...) with Integer.decode(...) in Configuration.getInt(...), similarly, in Configuration.getLong(...).

All

Comments

Work Log

Change History

Subversion Commits

Sort Order:

[ Permlink | « Hide ]

Tsz Wo (Nicholas), SZE added a comment - 18/Dec/07 11:19 PM

If the suggestions are good, here is a patch.

[ Permlink | « Hide ]

Owen O'Malley added a comment - 07/Feb/08 10:01 PM

I think removing white space from the start and end of the values is a good thing. I'm pretty skeptical of using hexadecimal and octal numbers. I think it will confuse users a lot if 010 == 8.

[ Permlink | « Hide ]

Konstantin Shvachko added a comment - 07/Feb/08 10:14 PM - edited

I am positive on both trimming and decoding.
I don't think anybody will be setting say replication to 011.
But having an opportunity to set dfs.block.size to 0x4000000 rather than to 67108864 is an advantage imo.

[ Permlink | « Hide ]

Tsz Wo (Nicholas), SZE added a comment - 07/Feb/08 11:16 PM

> I think it will confuse users a lot if 010 == 8.

I agree that it might be a problem and such problem is hard to be detected.

How about we make a special syntax for octal? Maybe something like: octal10 = 8

[ Permlink | « Hide ]

Robert Chansler added a comment - 08/Feb/08 12:09 AM

No special invented formats!

I'm persuaded that it is convenient and less error prone to set a file mode to 0777 or to set a length of 0x10000 bytes.

How large is the community of the confused if we choose a representation that is in the intersection of C and Java?

[ Permlink | « Hide ]

Tsz Wo (Nicholas), SZE added a comment - 08/Feb/08 06:28 PM

2461_20080208.patch: syn with current trunk

[ Permlink | « Hide ]

Amareshwari Sriramadasu added a comment - 25/Apr/08 05:55 AM

I think Nicholas's patch here suffices the requirements.
It does trimming of configuration names. And accepts decimal, hexadecimal, and octal numbers, the representation followed in Java.
Do we need to do anything more here?

If not, Nicholas could you please submit patch in sync with the trunk? And in the latest patch the testcase was missing, could you include testcase also in the patch?

[ Permlink | « Hide ]

Tsz Wo (Nicholas), SZE added a comment - 25/Apr/08 06:11 PM

It seems that "accept decimal, hexadecimal, and octal numbers" needs more discussion since it introduces a user-level incompatible change and the octal notation may confuse some users.

2461_20080425.patch: added a test.