Re: problem with OCTAVE WINDOWS out of memory

[Markus Bergholz]

On Tue, Jan 26, 2016 at 3:24 PM, Marco Mazzucco <address@hidden> wrote:

I sent the email before finishing it.. (skip the part in italian, please..)

Anyway, I cannot understand which is the problem..

I am able to load the same .mat files on FreeMAT without problems, 

and even perform an SVD on the matrix A...

Do you know what is wrong?

My operating system is Windows Professional 64bit.

Thank you,

Kind regards,

Marco Mazzucco

As a workaround, you can use Redis with octave.


octave:1> r = redis();
octave:2> r.loadcreateArrays('../mex/createArrays.lua');
octave:3> r.rand('A', 120000, 330)
ans =                    1
octave:4> whos
Variables in the current scope:

   Attr Name        Size                     Bytes  Class
   ==== ====        ====                     =====  =====
        ans         1x1                          1  logical
        r           1x1                          0  redis

Total is 2 elements using 1 byte

octave:5> r.size('A')
ans =

                   120000                   330

theoretically the only memory limit is your available memory (or memory server ... or server cluster with redis cluster (distributed arrays is not supported/implemented!)).

but it will consume more memory in redis as it would consume in octave... and calculations will be slower too. but that's the price for handling very huge datasets (theoretically).

AND (again!) it's high experimentel. Next to rand(), ones(), zeros(), size(), and numel() nothing more is implemented.

So you have to implement more calculation by your self. And it depends on your calculations if you can do it with octave itself inside the redis() class or if you have to use Lua inside of Redis.

https://github.com/markuman/go-redis

Examples for octave are inst/redis.m from line 369
Examples for lua are in the mex folder (createArray.lua and gaussian.lua)

this kind of lua-redis-octave "extension" was just a side-product while playing with redis, lua and the redis client for octave itself.

 

2016-01-26 15:20 GMT+01:00 Marco Mazzucco <address@hidden>:

Good afternoon,

I am trying to adapt a program I wrote on MATLAB and run it on OCTAVE.

The program has input data saved in .mat files, some of them containing quite big matrices (120000x330, real).

I am working with a workstation HP Z820 with 32 GB of RAM.

When I try to load the files on OCTAVE, an error occurs:

"error: out of memory or dimension too large for Octave's index type".

Also when I generate a random real matrix of those dimensions, 

        A = randn (120000, 330);

        >> whos A
        Variables in the current scope:

        Attr Name        Size                     Bytes  Class
        ==== ====        ====                     =====  =====
        A      120000x330                316800000  double

        Total is 39600000 elements using 316800000 bytes

then I cannot do not even the transpose of the matrix A:

B=A';

error: out of memory or dimension too large for Octave's index type

Sto cercando di adattare un programma che ho scritto su MATLAB 

e farlo girare su OCTAVE. Il programma ha dei dati in ingresso salvati su file .mat, 

ed alcuni di essi sono matrici abbastanza grandi (120000x330).

Sto attualmente lavorando con una workstation HP Z820 con 32 GB di RAM.

Quando provo a caricare i file su OCTAVE mi da un errore 

"error: out of memory or dimension too large for Octave's index type"

e anche quando genero una matrice random di quelle dimensioni,

poi non riesco neanche a fare la trasposta, mi da sempre l'errore di sopra.

Penso che il problema fondamentale sia che la versione scaricabile dal sito

di OCTAVE è 32 bit. Ho provato a caricare gli stessi file su FreeMAT e li carica senza problemi,

nonostante mi sembri più limitato come programma, però lavora a 64bit. Fa anche una SVD 

sulla matrice..

Come posso risolvere questo problema in OCTAVE?

E' possibile far girare OCTAVE a 64 bit?

Come posso fare?

Grazie.

_______________________________________________
Help-octave mailing list
address@hidden
https://lists.gnu.org/mailman/listinfo/help-octave

--

icq: 167498924
XMPP|Jabber: address@hidden