Aw: Re: memory leak in regexprep ? (windows/3.7.2+/VS)

[Christoph Ellenberger]

> --- On Wed, 2013/3/27, Tatsuro MATSUOKA wrote:
> > --- On Wed, 2013/3/27, Tatsuro MATSUOKA wrote:
> > > --- On Tue, 2013/3/26, Christoph Ellenberger  wrote:
> > > > I am experiencing some strange behavior and wanted to check if it is  a 
> > > > bug or me....
> > > > It happens when I try to read in and manipulating some ascii data 
> > > > files. It only happens with the VS version of octave. (tested with 
> > > > several gcc versions on windows and didn't showed up).
> > > > Ok here is some example code to reproduce:
> > > > First generating an ascii file:
> > > > s="0123456789";
> > > > for i=1:21, s=cstrcat(s,s);,endfor
> > > > save TestData.dat s
> > > >  
> > > > Then readin and manipulate:
> > > > fid = fopen("TestData.dat","r");
> > > > s = fscanf(fid,'%c');
> > > > fclose(fid);
> > > > s = regexprep(s,'0','2');
> > > >  
> > > > Clear all and repeat:
> > > > clear all
> > > > 
> > > > fid = fopen("TestData.dat","r");
> > > > s = fscanf(fid,'%c');
> > > > fclose(fid);
> > > > s = regexprep(s,'0','2');
> > > >  
> > > > Depending on datasize and how many replacement were performed it either 
> > > > fails on the new readin or regexprep with a: "error: out of memory or 
> > > > dimension too large for Octave's index type"
> > > > Any help is appreciated
> > > > Christoph
> > > 
> > > I have tested on octave-3.6.2 VS, 3.6.2 MinGW, 3.7.2+ VS and 3.6.2 cygwin
> > > (win 7 64 bit Home premium, 4GB memory)
> > > 
> > > 
> > > 1. octave-3.6.2 VS and 3.6.2 MinGW*****************************
> > > >> fid = fopen("TestData.dat","r");
> > > >> s = fscanf(fid,'%c');
> > > warning: range error for conversion to character value
> > > >> fclose(fid);
> > > >> s = regexprep(s,'0','2');
> > > >> clear all
> > > >> fid = fopen("TestData.dat","r");
> > > >> s = fscanf(fid,'%c');
> > > error: memory exhausted or requested size too large for range of Octave's 
> > > index type -- trying to return to prompt
> > > ********************************************************************
> > > 
> > > 
> > > 2. octave-3.7.2+ VS*****************************
> > > >> fid = fopen("TestData.dat","r");
> > > >> s = fscanf(fid,'%c');
> > > warning: range error for conversion to character value
> > > >> fclose(fid);
> > > >> s = regexprep(s,'0','2');
> > > >> clear all
> > > >> fid = fopen("TestData.dat","r");
> > > >> s = fscanf(fid,'%c');
> > > error: out of memory or dimension too large for Octave's index type
> > > ********************************************************************
> > > 
> > > 3.Octave-3.6.2 Cygwin**********************************************
> > > >> fid = fopen("TestData.dat","r");
> > > >> s = fscanf(fid,'%c');
> > > >> fclose(fid);
> > > >> s = regexprep(s,'0','2');
> > > >> clear all
> > > >> fid = fopen("TestData.dat","r");
> > > >> s = fscanf(fid,'%c');
> > > >> fclose(fid);
> > > >> s = regexprep(s,'0','2');
> > > ********************************************************************
> > > 
> > > For me, both VS and MinGW version caused problem. Cygwin version seemed 
> > > to work correctly.
> > > 
> > > Regards
> > > 
> > > Tatsuro
> > 
> > 
> > I have also tested on 
> > octave-3.2.4 and 3.4.3 MinGW version.
> > 
> > 4. octave-3.2.4 and 3.4.3 MinGW *****************************
> > >> fid = fopen("TestData.dat","r");
> > >> s = fscanf(fid,'%c');
> > warning: range error for conversion to character value
> > >> fclose(fid);
> > >> s = regexprep(s,'0','2');
> > >> clear all
> > >> fid = fopen("TestData.dat","r");
> > >> s = fscanf(fid,'%c');
> > warning: range error for conversion to character value
> > >> fclose(fid);
> > >> s = regexprep(s,'0','2');
> > **************************************************
> > 
> > The problem seemed to be happened on octave-3.6 or later on native windows.
> > 
> > Regards
> > 
> > Tatsuro 
> 
> Sorry for successive post.
> 
> I have tested the following  :
> 
> %*********************************
> s="0123456789";
> for i=1:21, s=cstrcat(s,s);,endfor
> save TestData.dat s
> load TestData.dat
> s = regexprep(s,'0','2');
> clear all
> load TestData.dat
> s = regexprep(s,'0','2');
> %*********************************
> 
> The above worked correctly on octave-3.2.4 (MinGW), 3.4.3 (MinGW), 3.6.2 
> (MinGW,VS, Cygwin) and 3.7.2+ (VS).
> 
> I think that the problem does not rely on regexprep but on fscanf for large 
> data.
> 
> Regards
> 
> Tatsuro
> 
Thank you very much for the fast confirmation of the same problem. Actually it 
must be a strange combination of both I tested:

octave-3.7.2+ VS*****************************
fid = fopen("TestData.dat","r");
s = fscanf(fid,'%c');
warning: range error for conversion to character value
fclose(fid);
clear all
fid = fopen("TestData.dat","r");
s = fscanf(fid,'%c');
warning: range error for conversion to character value
fclose(fid);
clear all
fid = fopen("TestData.dat","r");
s = fscanf(fid,'%c');
warning: range error for conversion to character value
fclose(fid);
clear all
********************************************************************

So it can't be fscanf only. On the other hand I tested as well (because your 
last example the second regexprep does nothing, as all 0 were already exchanged)

octave-3.7.2+ VS*****************************
fid = fopen("TestData.dat","r");
s = fscanf(fid,'%c');
warning: range error for conversion to character value
fclose(fid);
s = regexprep(s,'0','2');
s = regexprep(s,'1','2');
s = regexprep(s,'2','3');
error: out of memory or dimension too large for Octave's index type
********************************************************************

So it clearly also depends on how many replacements are performed within the 
regexprep. (same fails as well in 3.6.0_gcc4.6.2 and 3.2.4_gcc4.4.0)
As something like this always works:

octave-3.7.2+ VS*****************************
fid = fopen("TestData.dat","r");
s = fscanf(fid,'%c');
warning: range error for conversion to character value
fclose(fid);
s = ctsrcat(s,"aaaaaaaaa", s);
s = regexprep(s,'a','2');
clear all
fid = fopen("TestData.dat","r");
s = fscanf(fid,'%c');
warning: range error for conversion to character value
fclose(fid);
clear all
********************************************************************

Kind regards
Christoph