Perl Hand Coded Lexical Analyzer (Tokenizer)

[LeafSy]

I just want to ask how will i develop a Perl Lexical analyzer that will mimic language like C, for example the input will be:

main(){
printf("Hello World");
}

How will i broken down the input into tokens like main = reserve words, printf = function, ( = character symbol etc so that the output will be

reserveword_openparenthesis_closeparenthesis_opencurlybraces
function_openparenthesis_doublequote_stringConstants_doubleqoute_closeparenthesis_semicolon
closecurlybraces

I know the basic of perl like using regular expressions, I just dont know how will i break does lines of codes into tokens.

 

[MillerH]

No need to reinvent the wheel using REGEX. This is already solved using PPI:

http://search.cpan.org/perldoc?PPI

- Miller