Re: Octave Compiler

[JD Cole]

Hi back Jens,

> thanks for your mail. Thanks for the oct-compiler frame. This really
> gave me a clue, about the architecture of octave.
> I finally got it running and now I am finding out, what in detail you
> were doing.

Well, the truth is, I hadn't done so much on that code, for the most 
part I too was trying to get a good feel for how to attack that probem.

But first I'll explain some more about my project:

JD Cole wrote:

> > There are two quite interesting alternatives to converting to C++ 
> > that I had been kicking around. One was creating a Just-In-Time 
> > compiler for Octave. The other is actually writing a front end for 
> > gcc to compile directly to binary from octave code. While the former 
> > could potentially provide a good performance increase for the 
> > standard Octave user, the latter may not be relevant, it just sounds 
> > like a cool project to me. 
>
>
> The latter is much more than just a cool project - this way octave can
> make its way out of the laboratory. If you can generate C++ from Octave
> code you can take it and plug it seamless into your software project.

I agree with you here, and it was definitely one of the reasons  octave 
-> C++ would be cool. As an intermediate step, it may not be "too" 
difficult to convert directly to Octave-based stand alone functions, 
however, since we'd be linking with the Octave libraries, the generated 
code would still fall under GPL since the whole of Octave source is GPL. 
(As opposed to LGPL. If you're not clear on this I'd be happy to explain 
more.) A similar problem would arise when you tried to convert a script 
file which makes use of other scripts which also fall under 
GPL...especially so if those were .oct files which could also be 
converted using the compiler.

Here's a big question: If I write a script file and put it under GPL and 
you convert it to pure C++ code, does that C-code fall under GPL. My 
guess is yes, since you are in effect duplicating the algorithm which I 
developed. Well, maybe....it deserves a little more thought.

   That said, I think that the response from the user community would 
be huge just to allow anyone to compile their script file into a .oct 
file and benefit from the speedup. I had a convesation with someone and 
they said a simple "no optimization" C++ implementation of some of their 
.oct files gave something like a 6x speedup. This actually sounds like a 
good place to start...create a .m ---> .cc ---> .oct "compiler". It 
would give the user community a pretty immediate benefit and would give 
the compiler authors a good stepping stone to understanding how to 
convert to independent C++.

   [My interest] While I would love to reap the benefits of converting 
Octave code to C++, I also have another aim, as I told you I've put some 
time into adding parallel support to Octave. My grand view is to allow 
Octave users to make use of large parallel systems to do their 
computations. However, large simulations take a lot of time and money 
(if it's a lease type situation), plus it isn't always possible to 
install a large application such as Octave for single run. So I'm 
thinking it may be possible to pre-compile the code which will run on 
remote nodes into a form independent of octave and then execute it 
during an Octave session. (This idea is still a little gray....in the 
short term I'm just working on getting a bunch of machines running 
Octave to cooperatively run arbitrary code.)

> Today developers don't use Octave because they have to reingineer their
> prototype sooner or later in C++. So they take C++ for the prototype as
> well.

Yes, I know that one well, I found myself procrastinating in converting 
my working Octave code to C/C++ code for an application a few days ago. 
It just hurts.

> Do you know Ptolemy? Ptolemy is a simulation environment. Its kind alike
> Simulink - but much more powerful. Its based on C++ classes connected by
> Tcl Functions. But C++ has limitation in numerics - its just to far away
> from the problems of engineers. So we want to use Octave as the language
> for describing the blocks. Just like Simulink is able to use Matlab.

Ya, I have come across this project before, it's a moutain they've done. 
Definitely a good direction to go in, however, it may again be worth 
investigating various license compatibilies....'cause the purists will 
get you later on. I'm basically saying that  UC Regents copyrights and 
GPL may not mix...but I'm no lawyer. (Historically UC Regents, i.e. the 
governing body of all the Univesities of California has had a pretty 
tight fist on anything coming out of the university. This copyright 
seems to be a little more lenient, though.) Sorry, I've been reading so 
much about court cases lately all this stuff is coming out my ears!

> > There is plenty of literature on the JIT compiler in regards to Matlab.
> Do you have some links?

Ya, the best one I recall is here:

http://www.cs.uiuc.edu/Dienst/UI/2.0/Describe/ncstrl.uiuc_cs/UIUCDCS-R-2001-2213

and if you're into that check out GNU Lightning:

http://www.gnu.org/software/lightning/lightning.html

it's a somewhat cross-platform library for generating and executing code 
at run-time.

when I was looking at this stuff, someone had also suggested that rather 
than create a new runtime compiler, code could be converted to an 
existing one like Java....something everybody has access too and is 
rather mature. (I'm not sure if this is appropriate or not, but 
definitely worth some investigation.

> There is a problem, I am hanging in understanding your code and getting
> started with coding my self. I don`t have to much experience in coding
> with gnu tools. (Never did big projects)

> Do you know how I can debug .oct files? How I can trace into them with a
> debugger? 

Ya. OK, I'm going to assume you have used gdb at least once, if you 
haven't feel free to speak up and I give you more. Basically the problem 
with .oct files is that they are dynamically linked, the implication is 
that symbols and code will not be available via the debugger until 
Octave has loaded that specific file. My way around this is to do the 
following, let's assume we're debugging the .oct file oregano.cc (the 
example included with Octave).

1. Start the debugger "gdb octave".
2. at the prompt  start octave using the "run" command
3. Once at the Octave command prompt you need to "coax" Octave into 
loading the command, but if we're trying to debug something it might not 
be desirable to actually execute the code, so I just type "help 
oregano". This forces Octave to load the file into memory.
4. Now that oregano.oct has been loaded, do a "control-c". This should 
drop you back to the debugger prompt where you should have access to the 
symbols and code in your .oct file. At this point you would presumably 
set breakpoint(s) within your code and continue down the debugging path.

NOTE: A couple of caveauts to debugging Octave code. 1) Octave source 
makes have use  of inlined functions, so often when you arrive at a 
breakpoints which lies within your code. When stepping through code I 
have written, it hasn't been uncommon that I see none of my original 
code. 2) Compiler optimizations will further obfuscate your code. For 
example using "-g -O2" is sometimes confusing to step through.

> What development environment do you use? I'm using Emacs (due
> to my former small projects) - but  I'm still not convinced, its the
> best decision.

Well, I wouldn't call emacs a development environment, but I do use it 
for all my coding. I like the fact that it works on almost every 
platform I use and everything is accessible from a combination of keys 
(dont let that scare you off). Plus it's the right price. There are more 
"complete" development environments, but I just use a combination of 
emacs and gdb in one of three forms (command line gdb, emacs gdb or DDD 
which is a really nice graphical interface to gdb.) If you're used to a 
VC++ development environment going to emacs may have a shock factor 
since there are so many pictures.

If you would like to work to together on your idea, I'd be more than 
happy to...perhaps we could turn something out that really gets people 
going.

JD