Issue 3687 - Make CSV import/export filter settings persistent (e.g. the delimiter character). Very helpful for batch tasks.
Summary: Make CSV import/export filter settings persistent (e.g. the delimiter charact...
Status: CLOSED FIXED
Alias: None
Product: Calc
Classification: Application
Component: open-import (show other issues)
Version: 642
Hardware: All All
: P3 Trivial with 26 votes (vote)
Target Milestone: ---
Assignee: oc
QA Contact: issues@sc
URL: http://wiki.services.openoffice.org/w...
Keywords: ms_interoperability, oooqa, rfe_eval_ok, third_party_support, usability
: 3766 14675 21331 32651 40552 77215 (view as issue list)
Depends on:
Blocks: 97765 97416
  Show dependency tree
 
Reported: 2002-03-27 09:50 UTC by chris319
Modified: 2017-05-20 10:20 UTC (History)
19 users (show)

See Also:
Issue Type: PATCH
Latest Confirmation in: ---
Developer Difficulty: ---


Attachments
Proposed patch (5.79 KB, patch)
2005-08-01 11:35 UTC, muthusuba
no flags Details | Diff
Updated with er's requirements. (7.91 KB, patch)
2005-09-07 18:51 UTC, muthusuba
no flags Details | Diff
updated to preserve the fixed width(s) also. (11.20 KB, patch)
2005-09-19 18:32 UTC, muthusuba
no flags Details | Diff
Data loss example (15.03 KB, image/png)
2008-04-10 12:48 UTC, rail_ooo
no flags Details
revised patch to disable loading and saving of dialog settings (14.44 KB, patch)
2008-04-29 01:28 UTC, kyoshida
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this issue.
Description chris319 2002-03-27 09:50:49 UTC
Under "Save As -> Save As Type -> Text CSV -> Export of text files -> Text 
delimiter" please provide a null text delimiter (no character) in addition to " 
and '
Comment 1 oc 2002-03-27 11:53:30 UTC
Hi Falko,
I think this is something for you.
Comment 2 niklas.nebel 2002-03-27 17:47:24 UTC
You can already manually enter any character instead of selecting one,
or leave it empty in which case no text delimiter is used.
Comment 3 chris319 2002-03-28 22:16:57 UTC
If the user can edit the delimiter character, the drop-box with " 
and ' is unnecessary.
Comment 4 bobthehamster 2003-08-27 23:16:32 UTC
It would be really wonderful if the delimiter you select could persist.
I hate having to delete the delimiter every single time (but indeed I
am glad that I _can_ have a null delimiter)
Comment 5 falko.tesch 2003-10-01 14:42:48 UTC
Make the setting that are used with the CSV import/Export filter
persistent.
Example: I need to import a 100 .CSV files with some special
delimeters. Now every time I open one of them I have to re-set the
import settings of the .CSV filter.
Comment 6 falko.tesch 2003-10-01 14:43:02 UTC
started
Comment 7 falko.tesch 2003-10-01 14:43:33 UTC
*** Issue 3766 has been marked as a duplicate of this issue. ***
Comment 8 frank 2003-10-22 10:18:41 UTC
*** Issue 21331 has been marked as a duplicate of this issue. ***
Comment 9 frank 2004-08-06 09:39:35 UTC
*** Issue 32651 has been marked as a duplicate of this issue. ***
Comment 10 erwin.tenhumberg 2004-09-30 14:54:00 UTC
enhanced summary, set keywords and reassigned issue according to RFE process
Comment 11 frank 2005-01-13 10:06:43 UTC
*** Issue 40552 has been marked as a duplicate of this issue. ***
Comment 12 frank 2005-01-13 10:06:52 UTC
*** Issue 40552 has been marked as a duplicate of this issue. ***
Comment 13 abeelias 2005-07-27 04:01:22 UTC
Another year 2002 enhancement that would probably take no more than 8 hours to
address.  I urge you all to vote on an even greater injustice bug 4929.  This is
a BUG that has had no attention in 3 years.
Comment 14 abeelias 2005-07-27 04:08:59 UTC
bug 4925 oops
Comment 15 muthusuba 2005-08-01 11:35:29 UTC
Created attachment 28401 [details]
Proposed patch
Comment 16 frank 2005-08-01 11:55:23 UTC
Hi Daniel,

could you have a look at this patch please ?

Frank
Comment 17 lohmaier 2005-08-03 20:42:19 UTC
type is now "PATCH". Please reconsider the target.
Comment 18 ooo 2005-08-29 16:51:28 UTC
IMHO not only the "separated by" delimited case should be handled, but also the
"fixed width" case.
Comment 19 muthusuba 2005-09-07 18:51:52 UTC
Created attachment 29379 [details]
Updated with er's requirements.
Comment 20 ooo 2005-09-09 11:13:08 UTC
Hi Muthu,

Right direction, but what I meant was to also include the column positions of
the fixed width setting.

  Eike
Comment 21 ooo 2005-09-09 11:23:25 UTC
Hi Falko,

I added you to Cc of this issue because the patch introduces a new behavior,
where dialog settings are stored in the configuration, and are not only
remembered throughout a running OOo session. Does the User Experience team have
any input on this topic?

In case we wanted to add such behavior also to other dialogs I suggest to move
the Office.Calc/CSVImport configuration group to an own subgroup of dialogs,
e.g. Office.Calc/Dialogs/CSVImport

Thanks
  Eike
Comment 22 muthusuba 2005-09-19 18:32:51 UTC
Created attachment 29682 [details]
updated to preserve the fixed width(s) also.
Comment 23 stx123 2006-03-29 16:01:24 UTC
I guess Frank could be more helpful nowadays. To support the reconsideration of
the target I set it to "not determined".
Comment 24 kendy 2006-08-02 09:17:06 UTC
Ping? ;-)  What about the new version of the patch, please?
Comment 25 ooo 2006-08-02 11:27:54 UTC
Kendy, please be careful when fiddling around with targets! Changing it was
probably ok in this case (2.x), though having set it to 'not determined' earlier
was done on pupose. Thanks.

As there was no input from Falko back in those days and he isn't with us
anymore, I'm Cc'ing Matthias ('mmp') of the User Experience team now.

Matthias, please read
------ Additional comments from er Fri Sep 9 03:23:25 -0700 2005 -------
whether UE has some input on that.

Thanks
  Eike
Comment 26 utomo99 2006-11-19 09:25:26 UTC
The patch already exist since 2005, and until now not yet Integrated. 
and it has ms_interoperability, rfe_eval_ok, usability. 
I hope it is considered for OOo 2.2 
Thanks

Comment 27 daniel.rentz 2006-11-21 11:00:27 UTC
DR->MMP: Please comment on Eike's questions from 
Fri Sep 9 02:23:25 -0800 2005 
Comment 28 kpalagin 2007-04-23 14:46:23 UTC
Dear developers,
any news on this enhancement?
Comment 29 ooo 2007-04-25 11:27:35 UTC
The developers don't know. This issue waits for input from the User Experience team.
Comment 30 frank 2007-07-06 13:21:02 UTC
*** Issue 77215 has been marked as a duplicate of this issue. ***
Comment 31 dlm 2007-07-06 14:53:34 UTC
*** Issue 77215 has been marked as a duplicate of this issue. ***

ok, looking at the long history of this bug (3687) and its many friends I'm 
kind of despairing of ever seeing a fix for my issue (which is quite a 
different matter from choice of delimiter).

anyway, the minimal fix for the lossy csv export after an import that I 
described in #77215 would be to offer the same export menu when selecting "save 
as" on an opened csv file that you receive when selecting "save as -> csv" for 
a regular calc spreadsheet. This would give the ability to uncheck "save as 
displayed". Ideally csv import than export should be as close to idempotent as 
possible.
Comment 32 dlm 2007-07-06 14:58:39 UTC
seemingly I want the feature that user Abeelias considered a bug in #4925. I 
think my use case (importing csv files, editing, then exporting without loss of 
precision) is valid, and it is exactly how excel works on imported csv files.
Comment 33 krabbel 2007-08-23 13:20:02 UTC
I was sent to this issue for my question:

is there a way to change the default separator char for csv-export dialog?

why not user-depend-config-values for import & export like this:

-list-of-delimiter-chars (example: "')
-list-of-separator-chars (example: ,:;{TAB}{SPACE})
-default-delimiter (e.g. first of list)
-default-separator (e.g. first of list)
-use-separator with the values: import-depended, always-default
-use-delimiter-chars: import-depended, always-default

and a default like Excel does?

Sincerely yours
Comment 34 rene 2007-09-14 14:29:10 UTC
Note Issue 75076 and the patch (mypatch.diff) there. Needed to fix some import
problem.
Comment 35 rene 2007-09-14 14:37:29 UTC
(beware: mypatch.diff is a reversed one....)

Adding eknagy to CC
Comment 36 ooo 2007-10-11 21:03:01 UTC
@mmp: Matthias, any input on my #desc22 comment of Fri Sep 9 10:23:25 +0000 2005
directed to Falko? Repeating here:

[...] the patch introduces a new behavior,
where dialog settings are stored in the configuration, and are not only
remembered throughout a running OOo session. Does the User Experience team have
any input on this topic?

In case we wanted to add such behavior also to other dialogs I suggest to move
the Office.Calc/CSVImport configuration group to an own subgroup of dialogs,
e.g. Office.Calc/Dialogs/CSVImport

Thanks
  Eike
Comment 37 Martin Hollmichel 2008-01-22 07:49:45 UTC
move target from 2.x to 3.0
Comment 38 Martin Hollmichel 2008-01-22 07:53:52 UTC
reassign issue to fl. Frank, can you please take action on it, since this is
voted patch ?!
Comment 39 kpalagin 2008-03-11 16:07:07 UTC
With feature freeze delayed untill 15th of March we have a chance of 
integrating this patch for 3.0.
Comment 40 ooo 2008-03-11 16:17:01 UTC
@kpalagin: No, even if feature freeze was delayed (check back with the releases@
mailing list, the delay was redrawn) there would be no chance to get this in. It
would need yet another CWS that had to be QA'ed. There wasn't even a decision
whether we want this patch as is, we're still awaiting UX input.
Comment 41 rail_ooo 2008-04-10 12:47:06 UTC
muthusuba:
I suggest to change the behavior of this patch this way:

change 

if( nCharSet >= 0 )
  aLbCharSet.SelectEntryPos( nCharSet );

to

if( !bPreselectUnicode && nCharSet >= 0 )
  aLbCharSet.SelectEntryPos( nCharSet );

or to

if( !bClipboard && nCharSet >= 0 )
  aLbCharSet.SelectEntryPos( nCharSet );


Charset field is disabled in "Text Import" and "Text to Column" modes
(bClipboard and bPreselectUnicode in both case). Using the character set
previously selected in "CSV Import" mode may cause data loss (see attached
screen shot) and it's of course an UX issue.
Comment 42 rail_ooo 2008-04-10 12:48:51 UTC
Created attachment 52723 [details]
Data loss example
Comment 43 muthusuba 2008-04-11 09:05:55 UTC
rail,
  I am sorry, but i didn't clearly understand:
if( nCharSet >= 0 )
  aLbCharSet.SelectEntryPos( nCharSet );
changes the 'Character set' - which can be manually changed if required. But in
the screen shot I see the field to be disabled (how?!).

thanks!
muthusuba
Comment 44 rail_ooo 2008-04-11 10:04:50 UTC
muthusuba,

it's disabled by ScImportAsciiDlg::SetTextToColumnsMode() in the same file, when
you call this dialog via Data - Text to Column (introduced in 2.4).
Comment 45 muthusuba 2008-04-11 13:18:56 UTC
rail, 
 hmm...that's interesting...any idea on the reason for doing that? That would
solve our problem, i guess.

muthusuba
Comment 46 rail_ooo 2008-04-11 14:39:21 UTC
The reason is simple, clipboard is in unicode always, so there is no need to
choose the character encoding.

bClipboard is true if we use text from clipboard, bPreselectUnicode is true if
unicode characters were "sniffed".
Comment 47 muthusuba 2008-04-12 09:08:20 UTC
we'll have to do that for 'storing' also then - thanx rail!
Comment 48 kyoshida 2008-04-29 01:28:01 UTC
Created attachment 53246 [details]
revised patch to disable loading and saving of dialog settings
Comment 49 kyoshida 2008-04-29 01:34:04 UTC
The revised patch disables loading and saving of the dialog settings when the
dialog is launched for the text-to-columns functionality.  We needed to do this
to address several bug reports where the character set was incorrectly set to
'Unicode' when importing a csv file.

Here is the downstream bug report:
https://bugzilla.novell.com/show_bug.cgi?id=376473
Comment 50 frank.loehmann 2008-05-22 09:35:55 UTC
This issue is important and listed on the quarterly review for Calc:
http://wiki.services.openoffice.org/wiki/2008_Q2_Review_of_Spreadsheet_Project
Comment 51 frank.loehmann 2008-06-10 14:36:02 UTC
FL: New target.
Comment 52 kpalagin 2008-11-12 11:31:44 UTC
Any news with integration of this patch?
Comment 53 kpalagin 2009-01-16 10:44:08 UTC
I guess for whatever reason we missed 3.1 with this enhancement.
Is it possible that we do not miss 3.2?
Comment 54 matthias.mueller-prove 2009-01-16 11:42:56 UTC
remove mmp
Comment 55 farvardin 2009-01-20 10:53:34 UTC
I'd really like to see this integrated in the next release of OOo.
We seldom use simple comma in CSV, but rather semicolon for exemple :)
Comment 56 Martin Hollmichel 2009-01-27 19:45:21 UTC
what kind of input is needed for this issue from User Experience ? This issue
seems to be translation relevant, there move target to 3.2 :-(
Comment 57 ooo 2009-01-27 21:31:13 UTC
See my comments in #desc22 from er Fri Sep 9 10:23:25 +0000 2005

"[...] the patch introduces a new behavior,
where dialog settings are stored in the configuration, and are not only
remembered throughout a running OOo session. Does the User Experience team have
any input on this topic?

In case we wanted to add such behavior also to other dialogs I suggest to move
the Office.Calc/CSVImport configuration group to an own subgroup of dialogs,
e.g. Office.Calc/Dialogs/CSVImport"


Actually that was a question whether we may need some "store settings / forget
settings / forget temporarily" or whatever buttons, and if storing/retrieving
dialog settings would be expanded to a more application wide behavior in future,
to not end up with some "island solution". If UX doesn't have any input on this
we may as well proceed without. However, I'd still suggest to use the
configuration path Office.Calc/Dialogs/CSVImport nevertheless, just in case
we'll add other dialogs later.
Comment 58 frank.loehmann 2009-01-28 10:22:30 UTC
I think it is common sense that storing dialog settings does make sense.
Normally OOo stores settings for runtime of the program only, but in this case
is makes sense to store them persistently, because a csv import is often a
periodically recurring task. Most times the imported csv files will have the
same format, so restoring the latest defaults makes sense. Even if the csv
format changes the restored settings will not generate more work for the user,
but saves time if the format keeps unchanged.

Resetting or explizit storing/loading of settings sems to be overkill due to the
limited settings of this dialog. 
Comment 59 ooo 2009-01-28 11:29:35 UTC
*** Issue 14675 has been marked as a duplicate of this issue. ***
Comment 60 ooo 2009-01-28 11:33:25 UTC
@kohei: please go ahead in one of your next CWSs for OOo3.2, thanks. Note my
comment about the configuration path Office.Calc/Dialogs/CSVImport above.
Comment 61 muthusuba 2009-01-28 12:01:46 UTC
Do let me know if I can do anything for this.
Comment 62 kpalagin 2009-05-15 20:32:18 UTC
Kohei,
do you have some expected date when you integrate the patch so that we do not 
miss 3.2?

Regards,
KP.
Comment 63 kyoshida 2009-05-15 20:43:10 UTC
not at the moment.  When is the feature freeze for 3.2?
Comment 64 kpalagin 2009-05-15 21:05:14 UTC
http://wiki.services.openoffice.org/wiki/OOoRelease32
2009-09-21  Branch off date (UI and feature freeze)  
It will come much faster than we expect.
Comment 65 kyoshida 2009-08-07 03:30:27 UTC
Spec is now available here:
http://wiki.services.openoffice.org/wiki/Calc/Features/Numbers_import_for_plain_text_files

The development is on-going in koheicsvimport cws.
Comment 66 kyoshida 2009-08-07 03:52:42 UTC
fixed in koheicsvimport cws.
Comment 67 kyoshida 2009-08-07 04:00:15 UTC
re-assign to oc for qa.
Comment 68 oc 2009-11-05 12:54:36 UTC
adjusting target according to cws target
Comment 69 oc 2010-01-28 13:36:19 UTC
verified in internal build cws_koheicsvimport
Comment 70 jrw32982 2010-05-14 12:03:55 UTC
(Hopefully, this is the right place to enter these comments)

I think this issue needs to be solved (from the user's perspective) as follows:

1) There needs to be a way to persistently configure the default settings.  This
could be by making Calc's Text Import dialog's settings persistent or by having
a Calc Import tab in the Options dialog.

2) There needs to be a way to choose to use the (saved, persistent) default
settings without the Text Import dialog being displayed at all, when opening a
new file.

What I want to do is to start Calc from the command line (or via a File Manager
by selecting a csv file) and have the specified file loaded into Calc without
any Text Import dialog at all, using the default settings which I have
previously chosen.
Comment 71 kyoshida 2010-05-14 12:44:13 UTC
@jrw32982: You need to open up a new issue for that.  This issue has already
been resolved, verified, and is done for.
Comment 72 benni-b-aus-b 2011-05-10 21:22:24 UTC
Hi

I'm a German user of open office.
The default setting for exporting csv files is ";" as field delimitor and "," as decimal delimitor.

Unfortunately, the bugfix did not seem to work for my version.
As long as i use "save as" and "edit filter settings", everything works fine, and he saves it in the format i used latest.
Yet when i simply use the "save" button, he always saves in the german format (";" and ","), even when i changed the format before to "," and ".".

Language settings do not change this behaviour.

Best regards
Thio
Comment 73 jago25_98 2013-11-27 00:36:39 UTC
 Is this the same thing? 

Version: 4.1.3.2
Build ID: 4.1.3.2-4.fc19 (old in fedora?)

I was unable to categorise this both into components open-import and save-export. 

To duplicate:

Try reordering this series of words:
AUDCAD,AUDCHF,AUDNZD,AUDUSD,CADJPY,CADCHF,CHFJPY,EURUSD,EURGBP,EURAUD,EURCHF,EURJPY,EURNZD,EURCAD,AUDJPY,GBPUSD,GBPCHF,GBPJPY,GBPAUD,GBPNZD,GBPCAD,

To do this I:
- saved as a .csv from a txt editor, 
- imported with , delimit
- right click copy/paste special transpose
- try to save... 
- find I can't export in the same format file as I imported


 I think the best behaviour would be:
- detecting the delimiting, 
- remembering this delimit
- offer the option to export in the same format as what it was imported with, ideally with a preview