49 lines
1.2 KiB
Markdown
49 lines
1.2 KiB
Markdown
# lex
|
|
|
|
Use the liblex library to tokenize text.
|
|
|
|
## Installation
|
|
|
|
Add this to your application's `shard.yml`:
|
|
|
|
```yaml
|
|
dependencies:
|
|
lex:
|
|
git: https://dev.danilafe.com/Chip-8-Wizardry/lex.git
|
|
```
|
|
|
|
## Usage
|
|
|
|
```crystal
|
|
require "lex"
|
|
|
|
# Create a lexer
|
|
lexer = Lex::Lexer.new
|
|
|
|
# Add tokens using their regular expression and value.
|
|
# Tokens with larger value take higher priority.
|
|
lexer.add_pattern(".", 0) # Matches any one character.
|
|
lexer.add_pattern("ab+", 1) # Matches ab, abb, abbb...
|
|
lexer.add_pattern("(ab)+", 2) # Matches ab, abab, ababab...
|
|
lexer.add_pattern("ab*", 3) # Matches a, ab, abb...
|
|
lexer.add_pattern("[a-d]", 4) # matches a, b, c, d
|
|
lexer.add_pattern("[^a-d]", 5) # matches all chars but a, b, c, d
|
|
|
|
# Lex some text
|
|
tokens = lexer.lex "ab abb"
|
|
```
|
|
|
|
An array of tuples of type `Tuple(String, Int32)` is returned.
|
|
|
|
## Contributing
|
|
|
|
1. Fork it (<https://dev.danilafe.com/Chip-8-Wizardry/lex>)
|
|
2. Create your feature branch (`git checkout -b my-new-feature`)
|
|
3. Commit your changes (`git commit -am 'Add some feature'`)
|
|
4. Push to the branch (`git push origin my-new-feature`)
|
|
5. Create a new Pull Request
|
|
|
|
## Contributors
|
|
|
|
- [DanilaFe](https://dev.danilafe.com/DanilaFe) Danila Fedorin - creator, maintainer
|