# liblex

A library for converting input text into tokens defined by regular expressions.

## Rationale

liblex is a part of an attempt to write / create a compiler entirely from scratch.
This part of the compiler would be used to convert input text into tokens
to be evaluated by the parser.

## Usage

First of all, an evulation configuration has to be created. This configuration
is used to store the various regular expressions to be used during
lexing. The below code is an example of initializing and configuring an
evalutation configuration.

```C
/* Declares the configuration */
eval_config config;
/* Initializes the configuration for use */
eval_config_init(&config);
/* Registers regular expressions to be used. The IDs, given as the third
   parameter are also used for priority - the higher the ID, the higher the
   priority. */
eval_config_add(&config, "[ \n]+", 0);
eval_config_add(&config, "[a-zA-Z_][a-zA-Z_0-9]*", 1);
eval_config_add(&config, "if", 2);
eval_config_add(&config, "else", 3);
eval_config_add(&config, "[0-9]+", 4);
eval_config_add(&config, "{|}", 5);
```

It should be noted that this example is incomplete. `eval_config_add` returns
a `liblex_result`, which represents the result of the operation. `LIBLEX_SUCCESS`
means that no errors occured. `LIBLEX_MALLOC`, on the other hand, means that
the function failed to allocate the necessary memory, and `LIBLEX_INVALID`
means that the regular expression provided was not correctly formatted.

After the eval configuration has been configured, tokenizing a string is
done by creating a linked list and populating it with the resulting tokens
(called matches). 

```
/* Declares the linked list. */
ll match_ll;
/* Initializes the linked list. */
ll_init(&match_ll);

/* The first parameter is the input string, the second is the index at which
   to begin parsing. */
eval_all(string, 0, &config, &match_ll);
```

Once done, some things need to be cleaned up. The `eval_foreach_match_free`
function can be passed to `ll_foreach` containing the matches to release them:
```
ll_foreach(&match_ll, NULL, compare_always, eval_foreach_match_free);
ll_clear(&match_ll);
```
And the configuration can be freed using:
```
eval_config_free(&config);
```