[Octave-patch-tracker] [patch #8140] Speed up importdata() ASCII CSV processing using dlmread() as core

[Dan Sebald]

Follow-up Comment #9, patch #8140 (project octave):

I rolled back to a few days ago to compile octave and download the latest
importdata.m.  Looks nice.  I'll have to look through some of the regexp, etc.
mods to see the improvements at some other time.

The CR/LF case works, at least on Unix.  I suspect you are worried that
creating the test file:


fputs (fid, "3.1t-7.2t0rn0.012t6.5t128");


will create a file on Windows systems in which the linebreaks are CR-CR-LF and
the test will fail.  However, we might be able to change the way in which the
data file is generated by using explicit output data 0x0D (CR) and 0x0A (LF).


fprintf (fid, "3.1t-7.2t0%c%c0.012t6.5t128", 0x0D, 0x0A);



fprintf (fid, "3.1t-7.2t0%c0.012t6.5t128", 0x0A);


So we know we can generate files that are sure to have CR, or LF, or CR-LF. 
The question is whether these can all be handled properly on all platforms. 
If dlmread() can do so, then I see no problems with having these tests in
place.  If in the future dlmread() does not support these, then the tests can
be removed.  Both of the above pass the tests.  This one:


fprintf (fid, "3.1t-7.2t0%c0.012t6.5t128", 0x0D);


fails.  (We knew that.)

I looked at dlmread.  It is using "getline (*input, line);" to get a line of
the file.  I think getline is a C or C++ library function.  We could program a
fix for the last case easy enough by searching for the first CR or LF and
check whether the line delimiter is CR, LF or CRLF.  Then we can use a
delimiter as part of the getline() routine:

http://www.cplusplus.com/reference/string/string/getline/

I think if we support CR, LF and CRLF we are preserving compatibility whatever
the case.

    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/patch/?8140>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/