Book: Lex & Yacc

I picked up a copy of Lex & Yacc off TradeMe the yesterday, and I’m quite excited.

I’m hoping it will help me get my head around parser writing, for my still to-be revealed side project…

For a while now I have been working on a side project to add editor support for Erlang in Visual Studio.

I have a Google Code project for it, and it currently has syntax colour highlighting, but I was getting stuck with writing othe parser.

This is where the project has stalled over the last few months as I have been reading lots.

Originally I was basing my work on Lua Langage Pack for Visual Sudio. This package is a C# based plug-in that uses the CSTools project by Malcolm Crowe.

With the help of Code Project documents, MSDN and Blogs I slowly got a plug-in that has colour highlighting working (if you run it in the debug Visual Studio).

I was also reviewing how the IronPython project does it’s parser, but that used seemed to use python to do some parsing.. so

I then was trying to get Antlr to work, via AntlrWorks, but was banging my head on the debuggers limited support.

The next problem was how do I actually define the grammar for Erlang.

Erlang ships with a  grammar defined in yecc, it’s version-thing of yacc.  So I started translating this to Antrl but was getting left-right recursion errors, even though the problem was not the standard definition of left-right recursion problem.  Yecc is recursive decent as Erlang does recursion so well, but this was not playing well with Antlr. I then discovered I was looking at a subset of the Erlang gramma, and the full yecc gramma was huge, so hand translations was not an option.

So I then found the yecc grammar for yecc, and thought that I could hand roll a C# recursive decent parser for yecc, which would allow the auto-writing of a proper grammar for CSTools. But I wasn’t so keen on the .dll dependency of that tool chain.

I started reading the dragon book (Compilers: Principles, Techniques, and Tools) around this time, so was starting to get my head a little better positioned.

I then stumbled on to the Irony project which is a Visual Studio language development framework.  Eek they have most of what I was trying to workout how to-do, mostly worked out. But they have their own lex and yacc like tools.  This project also refers to the Visual Studio’s lex and yacc tools called MPLex and MPPG (distributed in the Visual Studio SDK)

I have just found that the newer version of the Lua project for Visual Studio 2008 uses the Irony project, and is hosted on CodePlex.

So I was getting keen again to work out how to use lex and yacc correctly, thus why when I saw this book for $3, I bid and anxiously waited to se if I’d win.  Of the 19 counted page view on the auction, 17 were mine.

So I’m half way through the first chapter and have just realised I don’t need to write a hand parser of yecc, I just need to write a lex parser that translates yecc to yacc, and rebuild that with MPPG to get a C# Erlang parser that is not hand rolled. Which means if the Erlang language changes, I can just re-run the whole process on the new gramma, and still be compliant.

Comments:

Stu 2009-06-28 13:59:14

aah the lexx + yacc book.. I have this, there is a new version coming out soon that should be a good upgrade. I have the pragprog antlr book, its interesting but antlr seems massively complex :(

right now I’m going over writing compilers + interpreters by mak, retargetable compiler design, and holub’s old compiler design book. I have others I dont like as much (dragon bleach, modern compiler design, some other old dusty tomes).

Edited by Simeon to add links.


Simeon 2009-06-29 12:57:28

Is your interest in compilers based on your game, or some other tangent?

Also by retargetable compiler design, there seems to be three books that could match which one do you mean.


Stu 2009-06-30 09:05:37

Retargetable is the LCC book ( http://www.amazon.com/dp/0805316701 )…

my interest in compilers goes waaaay back, ever since I ran into the original smallc implementations. that and VM’s since I tried cracking zmachine way back in the 80s and that magnetic scrolls wrote an entire 68k cpu emulator to port sinclair ql games to every other machine around…


Stu 2009-07-02 04:53:05

I should add I have the really cool dr dobbs small-c resource cd wich has all the small c stuff, the classic book from jimi hendrix etc articles from all the ddj magazines.

quite cool.