[Help-zile] [ANN] Zile Syntax Highlighter

[Gary V. Vaughan]

For those of you who are not following development on the Zile Github mirror
(if you'd like to do that, please login to your own githob account and then
go to http://github.com/gvvaughan/zile and press the "Star" button near the
top left of the page)...

I finally ported forward the syntax highlighter proof-of-concept I wrote in
the old Zi fork {ZILE without the L(isp) E(ngine)} into the Zz editor tree
of current master, and you can install it right now, as long as you have a
working Lua 5.2 with LuaRocks in your PATH, using:

  luarocks install \
  https://raw.githubusercontent.com/gvvaughan/zile/release/zile-git-1.rockspec

You might need to install your distro's liboniguruma-dev packages first on
Linux; also on Mac OSX, unfortunately some of the luarocks dependencies need
a bit of manual help to install at the moment (lrex-gnu in particularly is
quite fiddly as it relies on glibc's GNU regex APIs -- but I'm happy to help
anyone who gets stuck by answering questions on the list).

Once luarocks has successfully installed the latest Zile master, you'll have
a `zemacs` command, which is a souped up version of the C Editor formerly
known as `zile` rebuilt over the new zile libraries in Lua, and a new `zz`
command with a colorized UI including syntax highlighting for Lua sources.

Obligatory Screenshot (dropbox link to save bandwidth):

https://www.dropbox.com/s/bcy2n8zpp2aoj6v/Screenshot%202014-05-09%2021.01.28.png

Things to note:

  1. Multi-line comments are highlighted correctly; further-more, the
     syntax format supports backreferences in end expressions, so [===[
     ]===] are matched correctly regardless of number of = characters.
  2. Embedded escapes (\n) in strings are highlighted properly according
     to whether the string is quote or [[...]] delimited;
  3. Percent-placeholders are highlighted, including the white-on-red
     invalid warning for `%0` -- not a legal string.format escape sequence.
  4. There's a color theme bundle format for changing the color selections
     by syntax type separate from the highlighter which merely recognizes
     types... if you don't like my color scheme, write your own -- adding
     support for 256 color terminals for theming independently of your
     terminal's color scheme would be very straight forward, should you
     feel so inclined.
  5. The green Zz modeline, and the terminal chrome both know that this
     is a Lua source file!
  6. I've been too lazy to tweak the TextMate Oniguruma expressions to
     make function 'end' keywords blue, but it would be straight-forward
     enough and not require any Lua coding, just tweaks to the lua-bundle.lua
     syntax file.

Two major issues remain:

  1. The current highlighter is a proof-of-concept, and consequently very
     slow (especially on files with more than 200 lines or so).  The algorithms
     are fully featured and correct though, just the implementation needs to
     be optimized *a lot*.  I think some of the slowness can also be attributed
     to Zile's curses redrawing being suboptimal, but 90% of it is definitely
     in the new highlighting code.

  2. Highlighting for more kinds of files.  The use of oniguruma is to
     leverage the hundreds of TextMate language bundles that are now freely
     available.  I didn't reimplement an Apple Property List parser, so the
     tmLang files need mechanically translating to Lua read syntax and adding
     to the Zile source repository.

  2.5. In addition to leveraging the vast library of existing tmLang files,
       basing the Zz syntax highlighter on TextMate also enables nested
       syntaxes (not yet implemented!).  That is, the format supports proper
       highlighting and editing of an HTML file with embedded CSS, JS and
       PHP for example!  Or more important to me, shell syntax in Make files;
       m4 and shell syntax in configure.ac files...

I'll be tackling (1) slowly over the coming months, and will probably convert
a handful of tmLang files for languages I use regularly in due course.  It
will likely be quite some time before I get to (2.5).

A really easy way for anyone else who would like to contribute to Zile would
be to convert some tmLang files for *your* favorite languages, which will also
help me shake out any remaining bugs in the highlighter in advance of ploughing
into the optimizations.

And of course, there's still a ton of work ahead in converting the legacy tests
into Specl format; adding BDD tests to Zz; finishing the Zlisp->Lua compiler and
not to mention properly separating the reusable Zile library components from the
editor implementations that build on it - maybe to include yours?.  Any and all
contributions (or feedback on the list) are very welcome, are in fact very
easy now that Zile has a github mirror and continuous integration; and your
contributions will speed along the process of taking over the world, or 
something
like that ;-)

Enjoy!
-- 
Gary V. Vaughan (gary AT gnu DOT org)

signature.asc
Description: Message signed with OpenPGP using GPGMail