下面的例子展示了如何使用lex.py對輸入進行標記 ~~~ # ------------------------------------------------------------ # calclex.py # # tokenizer for a simple expression evaluator for # numbers and +,-,*,/ # ------------------------------------------------------------ import ply.lex as lex # List of token names. This is always required tokens = ( 'NUMBER', 'PLUS', 'MINUS', 'TIMES', 'DIVIDE', 'LPAREN', 'RPAREN', ) # Regular expression rules for simple tokens t_PLUS = r'\+' t_MINUS = r'-' t_TIMES = r'\*' t_DIVIDE = r'/' t_LPAREN = r'\(' t_RPAREN = r'\)' # A regular expression rule with some action code def t_NUMBER(t): r'\d+' t.value = int(t.value) return t # Define a rule so we can track line numbers def t_newline(t): r'\n+' t.lexer.lineno += len(t.value) # A string containing ignored characters (spaces and tabs) t_ignore = ' \t' # Error handling rule def t_error(t): print "Illegal character '%s'" % t.value[0] t.lexer.skip(1) # Build the lexer lexer = lex.lex() ~~~ 為了使lexer工作，你需要給定一個輸入，并傳遞給`input()`方法。然后，重復調用`token()`方法來獲取標記序列，下面的代碼展示了這種用法： ~~~ # Test it out data = ''' 3 + 4 * 10 + -20 *2 ''' # Give the lexer some input lexer.input(data) # Tokenize while True: tok = lexer.token() if not tok: break # No more input print tok ~~~ 程序執行，將給出如下輸出： ~~~ $ python example.py LexToken(NUMBER,3,2,1) LexToken(PLUS,'+',2,3) LexToken(NUMBER,4,2,5) LexToken(TIMES,'*',2,7) LexToken(NUMBER,10,2,10) LexToken(PLUS,'+',3,14) LexToken(MINUS,'-',3,16) LexToken(NUMBER,20,3,18) LexToken(TIMES,'*',3,20) LexToken(NUMBER,2,3,21) ~~~ Lexers也同時支持迭代，你可以把上面的循環寫成這樣： ~~~ for tok in lexer: print tok ~~~ 由lexer.token()方法返回的標記是LexToken類型的實例，擁有`tok.type`,`tok.value`,`tok.lineno`和`tok.lexpos`屬性，下面的代碼展示了如何訪問這些屬性： ~~~ # Tokenize while True: tok = lexer.token() if not tok: break # No more input print tok.type, tok.value, tok.line, tok.lexpos ~~~ `tok.type`和`tok.value`屬性表示標記本身的類型和值。`tok.line`和`tok.lexpos`屬性包含了標記的位置信息，`tok.lexpos`表示標記相對于輸入串起始位置的偏移。