A Crystal wrapper around liblex.
Go to file
2018-07-25 19:30:26 -07:00
spec Initial commit, plus liblex submodule. 2018-07-22 16:14:38 -07:00
src Create a simple wrapper around liblex and libds. 2018-07-22 16:15:15 -07:00
.editorconfig Initial commit, plus liblex submodule. 2018-07-22 16:14:38 -07:00
.gitignore Initial commit, plus liblex submodule. 2018-07-22 16:14:38 -07:00
.gitmodules Remove the use of submodules. 2018-07-22 16:59:49 -07:00
.travis.yml Initial commit, plus liblex submodule. 2018-07-22 16:14:38 -07:00
compile.sh Remove the use of submodules. 2018-07-22 16:59:49 -07:00
LICENSE Initial commit, plus liblex submodule. 2018-07-22 16:14:38 -07:00
README.md Add proper README. 2018-07-25 19:30:26 -07:00
shard.yml Put postinstall script back. 2018-07-22 17:02:07 -07:00

lex

Use the liblex library to tokenize text.

Installation

Add this to your application's shard.yml:

dependencies:
  lex:
    git: https://dev.danilafe.com/Chip-8-Wizardry/lex.git

Usage

require "lex"

# Create a lexer
lexer = Lex::Lexer.new

# Add tokens using their regular expression and value.
# Tokens with larger value take higher priority.
lexer.add_pattern(".", 0) # Matches any one character.
lexer.add_pattern("ab+", 1) # Matches ab, abb, abbb...
lexer.add_pattern("(ab)+", 2) # Matches ab, abab, ababab...
lexer.add_pattern("ab*", 3) # Matches a, ab, abb...
lexer.add_pattern("[a-d]", 4) # matches a, b, c, d
lexer.add_pattern("[^a-d]", 5) # matches all chars but a, b, c, d

# Lex some text
tokens = lexer.lex "ab abb"

An array of tuples of type Tuple(String, Int32) is returned.

Contributing

  1. Fork it (https://dev.danilafe.com/Chip-8-Wizardry/lex)
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Add some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create a new Pull Request

Contributors

  • DanilaFe Danila Fedorin - creator, maintainer