Re: [Tinycc-devel] Using TCC as a backbone for our compiler

[Jerome St-Louis]

Hi Basile, thank you very much for giving such insights and asking all
these good questions.
If it does happen that this becomes off topic on this tinycc list,
please let me know, hopefully with a suggestion on where to move the
discussion.

Yes our compiler currently generates C code as an intermediate
language, then pass it on to GCC to produce object code. The main
reason why we haven't tried plugging in TCC yet is the use of GCC
extensions in that generated C code, as previously mentioned. One
example of such extensions is the use of compound block expressions,
which comes in very handy.

Compiling the C code very quickly is definitely a desired feature.
Slow resulting execution speed is indeed a drawback, but in the usage
we'll make of TCC, it might be a sound tradeoff. For one, the SDK is
mainly made off a runtime library where most of the CPU intensive code
should reside, which we will keep compiling with GCC. The code
compiled with TCC would be for the more dynamic aspect (JIT
compilation).

As for our goals, they are many. This particular interest in TCC is to
allow deploying web based Ecere solutions. The idea is to have a JIT
and caching system and developing a client which would be able to run
any such application at native speed. We already have a prototype
which externally calls GCC, but it requires GCC as well as the whole
SDK to be installed on the client machine. The next step will be to
have a standalone (self-contained) client, which could eventually also
take the form of a browser plugin.

As for the language, I wouldn't say it is a dialect of C. It is called
eC (for Ecere C), and yes its home is at http://www.ecere.com along
with the rest of the SDK. It is an object oriented language which
supports the entire C syntax as it is, with many new additions and
features. The compiler does parse and represent the entire syntax
tree. The compiler is also pretty advanced and does a lot of type
checking etc. The eC language adds a lot of dynamism to C, and a lot
of very nice features. All of which are implemented, used and work
very well. As I mentioned already, it uses GCC for the object code
generation part. The compiler basically first parses the eC source
files and store it in an AST, then in multiple passes walks through it
and converts eC specific features into C features. Then it spits out
the resulting C AST at the end as C code.

I would have tried using TCC already if it wasn't for the use of GCC
specific extensions, which is one of my questions to this mailing list
if there is any plan to support those in GCC, particularly the ones
I'm making use of, and how hard it would be to implement (unless they
are already supported). The one I'm most worried about is the compound
expression, but then there is also the issues of including system
header files which often do make use of extensions on some systems,
particularly Linux.

I've also looked at TCC as a library before, as someone developed a C
scripting system using TCC with an Ecere GUI, some very nice work I've
been able to take a look at. I saw it as the next natural step to make
it support eC as well as C. Using it as a library is a key point here,
because we're trying to build a stand alone embedded client. Tiny is
key in this endeavor of bringing Ecere to the web.

We do typically invoke GCC with -O2. However as I mentioned, it is
likely that the performance of the higher level code, specifically the
scripting code ran on the client side, which will in turn invoke a
natively precompiled library for all performance critical tasks. The
idea is to have native performance applications over the web, and
we're up against JavaScript, which shouldn't be so difficult to beat
with C code. Again the key point is that things that absolutely must
run fast can be precompiled into native libraries (with GCC).

We're not interested in all those high level languages you mentioned,
I'd say the project is at the other end of the spectrum. We care about
optimal performance and minimalistic size.

So as you're putting it libjit, llvm, GNU lighting and TCC are all
potential candidates. I still feel strongly for TCC in this particular
project.

There is a very important code base, the user base is relatively
small, the most active users being part of the development team. I
don't see how this project should require changes resulting in
incompatibilities for them, as the entire C generation process it is
an internal implementation detail, but if it does it is not likely to
be a problem, we've dealt with small incremental modifications in the
past and surely will continue to do so for a while.

As for a GCC front end for Ecere, that is also something we've been
looking into, which might be the future of the actual compiler instead
of externally calling GCC. We actually had a student which was
interested in doing his Google Summer of Code project on that, but
unfortunately we didn't get accepted as a mentor organization and he
had to go with another org. I'm a little bit wary of the license
restrictions however, I'm not very familiar with the GCC front ends
and licenses. Ecere is BSD licensed. I'm not very familiar with GIMPLE
either, but it does sound very promising. Still, for this particular
scripting system, it might very well be that GCC is too big and too
slow to compile.

I'm familiar with the things I came across and had to figure out in
the process of designing and implementing the eC compiler, though I'll
admit it has been developed in a somewhat brute force approach and
would benefit immensely from a redesign by an expert in the field. I
still consider the current compiler a proof of concept for the eC
language, definitely a successful one without the shadow of a doubt
however. I have personally put many years of sweat into this compiler,
and I do agree with you that it is very hard work.

The eC language is meant as an all purpose programming language, to
the same extent as C++, C#, Java or Python are.
The Ecere SDK is now made up of the compiler tools, as well as the
runtime library which contains a cross platform GUI toolkit (a lighter
alternative to Qt, GTK, etc.), and a basic 2D/3D graphics engine. The
eC language was designed in order to provide an easier to use and more
well structured API for the toolkit that could have been done with any
other language. I've heard of Vala, and you could say Vala is to GTK
what eC is to Ecere. However I believe eC predated Vala, and of course
this is with a lot of inherent bias, but I wouldn't switch eC for Vala
anytime ;)

A little summary from
http://www.ecere.com/wiki/index.php?title=Ecere_Programming_Philosophy
:

Main Goals

    * Elegance
    * Simplicity
    * Power
    * Efficiency


Quick Mission Statemnent

    * eC is meant to be C with an elegance factor
    * eC is meant to have well featured object-oriented constructs
    * eC is meant to be ideal for implementing a complete cross-platform SDK
    * eC is meant to have simple and powerful modularity with elegant APIs
    * eC is meant to allow simple implementation of fast and powerful
cross-platform applications
    * eC is meant to allow elegant implementation of flexible
cross-environment GUIs

Thank you very much for your time and interest again, and I apologize
if this is getting off-topic

Best regards,

Jerome


On Sat, Jun 13, 2009 at 9:21 AM, Basile
STARYNKEVITCH<address@hidden> wrote:
>
>> On Sat, Jun 13, 2009 at 5:37 AM, Basile
>> STARYNKEVITCH<address@hidden> wrote:
>>
>>>
>>> Jerome St-Louis wrote:
>>>
>>>>
>>>> Hi all...
>>>>
>>>> We are contemplating using TCC as the backbone engine to generate
>>>> object code for an eC language Just-In-Time compiler system.
>>>>
>>>
>>> Did you consider using something more appropriate for that purpose, like
>>> LLVM or perhaps LibJit?
>
> Jerome St-Louis wrote:
>>
>> Hi, thanks for the reply.
>>
>> Of course we're considering all options, and I'll admit we're not very
>> familiar with any of them yet.
>>
>> I was particularly fond of TCC however, and maybe the thing that
>> naturally brought us to TCC is the fact that we currently output
>> intermediate C code and uses GCC, and TCC is a lighter and smaller
>> alternative...
>>
>>
>
> I am not an expert on TCC. I am a GCC contributor (but I won't claim to be a
> GCC expert yet. FWIW, my main GCC work is the MELT branch, which also
> generates C code -inside a GCC plugin; so I clearly understand what
> generating C code means in practice.).
>
> Of course, if your compiler already generate C code, switching to TCC should
> be easy. However, the generated code performance is not good (w.r.t. GCC).
> The main strength of TCC is that it compiles C code very quickly, to a slow
> executable.
>
> I am not sure to understand your goals. And I do not understand what you are
> currently doing: is your language a C dialect, very similar to C (enough so
> that representing all the abstract syntax tree is not done inside your
> compiler) or not. You really should explain alot more what is already done.
> Is your language http://www.ecere.com/ ?
>
> First, you should try (temporarily, as an experiment) to replace gcc in your
> program by tcc. That should be really trivial (just fork a tcc process, not
> a gcc one; basically change the "gcc" string inside your compiler into
> "tcc"). That should give you a feeling of what tcc would give you (or not).
> In particular, you'll measure that tcc generate really awful machine code -
> but the generation time is very very fast! You could also temporarily
> experiment replacing "gcc" with "lvm-gcc" or even "nwcc" ; all this are very
> easy experiments!
>
> If that suits your expectations, then you may take some time to use tcc as a
> library. But you probably won't win a lot. That brings you the libtcc.h API
> & library; *mostly the following function.
> /* compile a string containing a C source. Return non zero if
>  error. */
>  LIBTCCAPI int tcc_compile_string(TCCState *s, const char *buf);
>
>
>
> There is also another point you should care about: generating good machine
> code from any kind of source language is a hard task (because there is an
> increasing impedance mismatch between C, and even more higher level
> languages like Ocaml, and current processors). You should define what you
> really care about. Is the execution time of the generated code an important
> point to you? In practice, in your current use of gcc, do you invoke it
> usually with -O1 or -O2? If you do, that means that you care about the
> performance of the generated code. Tcc generates machine code roughly  in
> the same quality as  gcc -O0!
>
> You might also consider generating CIL or JVM bytecode, or C# or Java source
> code. Or even Ocaml.
>
> You might also consider using other metaprogramming languages, in particular
> Common Lisp, Clojure, ...
>
> Again, I don't really understand your goals, your needs, the things you
> really care about.
>
> In addition of libjit & LLVM, you might also consider GNU lightning (which
> share with tcc the facet of generating quickly code which does not perform
> well).
>
> Also, do you have an important code base? Do you have a lot of users for
> Ecere? Will they complain of incompatible changes?
>
> And of course, you could make a front-end from Ecere for GCC (ie for its
> GIMPLE internal representation). This is not as hard as you believe: you
> just have to adapt your parser to build GIMPLE internal representation. If
> you did that, you'll get all the code generation power of GCC.
>
> What part of compiler science are you familiar with (compilation is no more
> a parsing problem: the hard work happens on internal intermediate
> representations, and it is really hard work!!).
>
> http://llvm.org/
> http://nwcc.sourceforge.net/
>
> And please explain in one paragraph what is the interest of Ecere. The
> http://www.ecere.com/action.html page suggest that most of it is a graphical
> library, not really a new programming language (would it make sense to code
> in Ecere any non-graphical application, eg a formal theorem demonstrator?).
> See also Vala http://live.gnome.org/Vala/
>
> Regards.
>
> PS. It could happen that the discussion becomes off topic on a tinycc list.
>
> --
> Basile STARYNKEVITCH         http://starynkevitch.net/Basile/
> email: basile<at>starynkevitch<dot>net mobile: +33 6 8501 2359
> 8, rue de la Faiencerie, 92340 Bourg La Reine, France
> *** opinions {are only mines, sont seulement les miennes} ***
>
>
>
> _______________________________________________
> Tinycc-devel mailing list
> address@hidden
> http://lists.nongnu.org/mailman/listinfo/tinycc-devel
>