Re: performance problem with unique on sparse arrays

[David Bateman]

Jaroslav Hajek wrote:
> It would be nice to let the Sparse indexing benefit from similar
> improvements as the Array indexing, and also clean up the interface in
> a similar manner. Since sparse indexing is only 1D or 2D, reductions
> are not needed here, but the innermost loop specializations would
> likely make the code both cleaner and more efficient.
> I'd like to do some work here after 3.4, but I would appreciate
> assistance, because the coding will probably be more difficult (except
> for the lack of n-d case, which will make it easier).
>   
I understand the sparse indexing pretty well and would be happy to help 
out. The existing index, maybe_delete_elements and assign methods of the 
Sparse<T> class were written by taking the Array<T> methods from version 
2.1.63 (at least I think it was about then) and then just converting 
them ad-hoc to Sparse versions that did exactly the same thing.  I 
suppose the same process needs to be done with a recent version of your 
Array<T> indexing changes and will probably be easier as these functions 
are now much simpler. A nice thing to add if we do that  is to make

sm = spalloc (r, c, n);
for i = 1 : r
 for j = 1 : c
    y = foo (i, j);
    if (y)
      sm(i, j) = y;
    endif
 endfor
endfor

not reallocate sm on every call to the Sparse<T>::assign method as it 
currently does. Fixing the indexing such that something like

sm = sprandn (1e6,1e6,1e-6); sm(sm<1)= 0;

does cause a memory overflow with sparse logical indexing would be 
really nice as well.

Cheers
David