lex/README.md

# lex

Use the liblex library to tokenize text.

## Installation

Add this to your application's `shard.yml`:

```yaml
dependencies:
  lex:
    git: https://dev.danilafe.com/Chip-8-Wizardry/lex.git
```

## Usage

```crystal
require "lex"

# Create a lexer
lexer = Lex::Lexer.new

# Add tokens using their regular expression and value.
# Tokens with larger value take higher priority.
lexer.add_pattern(".", 0) # Matches any one character.
lexer.add_pattern("ab+", 1) # Matches ab, abb, abbb...
lexer.add_pattern("(ab)+", 2) # Matches ab, abab, ababab...
lexer.add_pattern("ab*", 3) # Matches a, ab, abb...
lexer.add_pattern("[a-d]", 4) # matches a, b, c, d
lexer.add_pattern("[^a-d]", 5) # matches all chars but a, b, c, d

# Lex some text
tokens = lexer.lex "ab abb"
```

An array of tuples of type `Tuple(String, Int32)` is returned.

## Contributing

1. Fork it (<https://dev.danilafe.com/Chip-8-Wizardry/lex>)
2. Create your feature branch (`git checkout -b my-new-feature`)
3. Commit your changes (`git commit -am 'Add some feature'`)
4. Push to the branch (`git push origin my-new-feature`)
5. Create a new Pull Request

## Contributors

- [DanilaFe](https://dev.danilafe.com/DanilaFe) Danila Fedorin - creator, maintainer