Re: use of xlsread in Octave

[Philip Nienhuis]

Sergei Steshenko wrote:
> --- On Sun, 5/12/13, PhilipNienhuis<address@hidden>  wrote:
>
> > From: PhilipNienhuis<address@hidden>
> > Subject: Re: use of xlsread in Octave
> > To: address@hidden
> > Date: Sunday, May 12, 2013, 11:34 PM
> > PhilipNienhuis wrote
> > > Hi Jean Rene,
> > > <snip>
> > > JR Cliche wrote:
> > > > Hello Philip,
> > > >
> > > > I am using Octave 3.7.2+, and I am having problem
> > with the function
> > > > xlsread.m.
> > > >
> > > > I have the io package version 1.2.1. I ensure to do
> > pkg load all at the
> > > > prompt.
> > > >
> > > > When I use xlsread, I get:
> > > > "java_invoke is obsolete and will be removed from a
> > future version of
> > > > Octave, please use javaMethod instead"
> > >
> > > I suppose you use Michael Goffioul's 3.7.2+ MSVC
> > binary.
> > > 1. What you see is just a warning, you can ignore that.
> > You only see it
> > > once per Octave session.
> > >
> > > 2. A more profound issue is that the 3.7.2+ binary is
> > really a
> > > developers snapshot, ordinary users do have to expect
> > to run into
> > > problems and indeed - it seems you have.
> > >
> > > 3. As to developers snapshots, I'm a bit reluctant to
> > provide much
> > > support for their general use. Those snapshots are only
> > made for testing.
> > > But OK I'll just give some hints below. Maybe others
> > will help you
> > > further.
> > >
> > > Java is built-in in that 3.7.2+ version. So that
> > traditional stumbling
> > > block has been conquered - you don't need to install
> > the java package.
> > > A next thing then is to hunt for the Java class libs
> > required for
> > > spreadsheet I/O, and to add them to the javaclasspath.
> > > In the ./doc subdir of the io package you'll find
> > adequate info on this.
> > > Just put all class libs together in some subdir (I
> > usually use
> > > ./lib/java) and invoke chk_spreadsheet_support.m to add
> > them to the
> > > javaclasspath.
> >
> > It just occurred to me that the above hints are probably all
> > futile.
> > The Java package has been moved into core Octave and is
> > already in the dev
> > version, incl. 3.7.2+ you use. Along that course that Java
> > subsystem has
> > changed quite a bit, so much that the Java stuff in the io
> > package doesn't
> > work anymore with 3.7.2+.
> > Futhermore, since the 3.7.2+ windows binary was made,
> > several bugs have been
> > found in that new Java subsystem that affect the io package.
> > Only once these
> > have been solved I can proceed making a new io package.
> >
> > I do have a io package version patched to present 3.7.3+
> > state (being io pkg
> > maintainer of course I have) but as the Octave dev version
> > is still unstable
> > I'm not going to disseminate it - support would be an undue
> > time-consuming
> > affair for me.
> >
> > So your only options for xlsread are:
> >
> > 1. Just use the stable Windows version (there's a very
> > usable 3.6.4 MinGW
> > binary available). Why didn't you use that in the first
> > place?
> >
> > 2. Wait until a new stable version incl. experimental GUI is
> > available (this
> > summer). Usually OF packages (like the io package) lag a bit
> > behind so you
> > might have to wait even a little bit longer.
> >
> > 3. Use the windows package (COM/ActiveX) for xlsread. It has
> > to be patched
> > however.
> > In the thread "xlsread in Octave 3.6.4" you'll find hints
> > how to patch it so
> > that it works with Octave 3.7.2+. In fact that's the setup
> > that worked for
> > me.
> >
> > Philip
> >
> >
> >
> >
> > --
> > View this message in context: 
> > http://octave.1599824.n4.nabble.com/Re-use-of-xlsread-in-Octave-tp4652816p4652840.html
> > Sent from the Octave - General mailing list archive at
> > Nabble.com.
> > _______________________________________________
> > Help-octave mailing list
> > address@hidden
> > https://mailman.cae.wisc.edu/listinfo/help-octave
>
> The amount of complications is simply staggering.

Heh - shit happens with development versions. One can say that's what 
they are meant for.
The OP probably was a bit naive in that respect.

> Inclusion of Java into Octave makes it unnecessarily inflated. I think it's a 
> horrible architectural decision.

Yes, I understand your reasoning.
But I also see the other side(s). Once Java works properly things get 
much easier for the Java users than with the OF Java package.
To expect things works OOTB after inclusion is a bit naive. It'll be 
some time before it works satisfactorily for everyone.
And - you can disable it when building Octave. So there's choice. And 
choice is what OSS is meant for.

> Regarding 'xlsread'.
>
> There stand alone Strawberry Perl: http://strawberryperl.com/ . And there is 
> even a portable one, e.g.: 
> http://strawberryperl.com/download/5.16.3.1/strawberry-perl-5.16.3.1-32bit-portable.zip
>  , 
> http://strawberryperl.com/download/5.16.3.1/strawberry-perl-5.16.3.1-64bit-portable.zip
>  .
>
> It even comes with 'gcc'.
>
> There is a whole bunch of Excel related modules: 
> http://search.cpan.org/search?query=excel&mode=all ->  
> http://search.cpan.org/~jmcnamara/Spreadsheet-ParseExcel-0.59/lib/Spreadsheet/ParseExcel.pm
>  .
>
> If the modules work, translating format from Perl data structures into Octave 
> struct is trivial.
>
> And Octave has 'system' function, i.e. this whole Perl stuff is neatly kept 
> separate from Octave.
>
> And on UNIX(-like) boxes Perl is present by default. And Perl can trivially be 
> built from source on UNIX(-like) boxes and can trivially be built portable - 
> just in case default Perl is not new enough. I am routinely building portable 
> Perl on my Linux box.

Thanks for the overview.
Besides Perl there exist many more software environments offering 
reading from Excel files. Python, Scilab, R, you name it. One can safely 
conclude that there is great demand for that.

Anyway, have you tried your suggestions? Do you have:

- Any benchmarks? Are your suggestions faster?

- How flexible & robust are your suggestions?
  * What happens with merged cells?
  * What happens to formulas in cells (do we get the evaluated formula 
results, the formulas as strings, or cached values? do we have any 
choice at all?)
  * Do they read .xls (both BIFF5 and BIFF8), .xlsx, .dbf, .xlsm, .csv, 
.wk1, etc etc?
  * What happens with erroneous results in cells?
  * What happens in case of user error? (like requesting more data than 
available in the file, or requesting reading from non-existent sheets)

- How maintainable is it? Are upstream maintainers sufficiently responsive?

- And of course, can you use Perl as a replacement for xlswrite?
I see little use to be able to only read spreadsheets. There's 
definitely demand for writing spreadsheets as well.
From my own experience I can tell you that implementing flexible and 
robust writing to spreadsheet files gets you into the next levels of 
complexity, not to mention trade-offs between flexibility and speed.

It would be nice to compare your benchmark results with the present io 
package support for spreadsheet I/O.
I'm not quite against an overhaul of xlsread and friends, and/or adding 
another interface, yet I think alternatives should have at least 
comparable flexibility, robustness, speed, and maintainability. Or at 
least if one these assets is lacking it should be compensated for by the 
others (choice, remember, that's why I offer a number of interfaces 
rather than just one).

And last but not least, alternatives should be "Matlab-compatible". That 
in itself makes for a lot of complications.

Philip