From 6add9ae89835fd71a44bdc3bc756fce23e112ac0 Mon Sep 17 00:00:00 2001 From: Danila Date: Wed, 15 Feb 2017 03:28:52 +0000 Subject: [PATCH] Create README.md --- README.md | 64 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 64 insertions(+) create mode 100644 README.md diff --git a/README.md b/README.md new file mode 100644 index 0000000..95d3246 --- /dev/null +++ b/README.md @@ -0,0 +1,64 @@ +# liblex + +A library for converting input text into tokens defined by regular expressions. + +## Rationale + +liblex is a part of an attempt to write / create a compiler entirely from scratch. +This part of the compiler would be used to convert input text into tokens +to be evaluated by the parser. + +## Usage + +First of all, an evulation configuration has to be created. This configuration +is used to store the various regular expressions to be used during +lexing. The below code is an example of initializing and configuring an +evalutation configuration. + +```C +/* Declares the configuration */ +eval_config config; +/* Initializes the configuration for use */ +eval_config_init(&config); +/* Registers regular expressions to be used. The IDs, given as the third + parameter are also used for priority - the higher the ID, the higher the + priority. */ +eval_config_add(&config, "[ \n]+", 0); +eval_config_add(&config, "[a-zA-Z_][a-zA-Z_0-9]*", 1); +eval_config_add(&config, "if", 2); +eval_config_add(&config, "else", 3); +eval_config_add(&config, "[0-9]+", 4); +eval_config_add(&config, "{|}", 5); +``` + +It should be noted that this example is incomplete. `eval_config_add` returns +a `liblex_result`, which represents the result of the operation. `LIBLEX_SUCCESS` +means that no errors occured. `LIBLEX_MALLOC`, on the other hand, means that +the function failed to allocate the necessary memory, and `LIBLEX_INVALID` +means that the regular expression provided was not correctly formatted. + +After the eval configuration has been configured, tokenizing a string is +done by creating a linked list and populating it with the resulting tokens +(called matches). + +``` +/* Declares the linked list. */ +ll match_ll; +/* Initializes the linked list. */ +ll_init(&match_ll); + +/* The first parameter is the input string, the second is the index at which + to begin parsing. */ +eval_all(string, 0, &config, &match_ll); +``` + +Once done, some things need to be cleaned up. The `eval_foreach_match_free` +function can be passed to `ll_foreach` containing the matches to release them: +``` +ll_foreach(&match_ll, NULL, compare_always, eval_foreach_match_free); +ll_clear(&match_ll); +``` +And the configuration can be freed using: +``` +eval_config_free(&config); +``` \ No newline at end of file