Re: incremental read of gzipped matrix

[Markus Mützel]

Am 08. Dezember 2019 um 22:30 Uhr schrieb "Andreas Weber":
> Am 08.12.19 um 19:25 schrieb "Markus Mützel":
> > Octave has the gzifstream (in zfstream.h) which might be useful for your 
> > purpose.
> 
> I've used this before and only got an improvement by factor 2.
> The factor 2 is because load currently parses the inputfile twice: the
> first run is to detect number of columns and rows, then the Matrix is
> allocated then parsed the second time.

Do you know why gzifstream is so much slower compared to your code? Given that 
you measured a 10 fold performance increase, there might be room for a factor 
of 5 (taking the double parsing into account).

> In the meanwhile I've enabled incremental reads in load_gz which is very
> important for me if I periodically poll an increasing .gz file.
> 
> -- Andy
>