Compare commits

..

2 Commits

6 changed files with 91 additions and 36 deletions

View File

@ -302,7 +302,7 @@ void ast_let::translate(global_scope& scope) {
mangled_env->bind(def.first, env->lookup(def.first), visibility::global);
mangled_env->set_mangled_name(def.first, global_definition.name);
ast_ptr global_app(new ast_lid(global_definition.name));
ast_ptr global_app(new ast_lid(original_name));
global_app->env = mangled_env;
for(auto& param : global_definition.params) {
if(!(captured--)) break;

View File

@ -139,7 +139,7 @@ void definition_group::typecheck(type_mgr& mgr, type_env_ptr& env) {
def_defn->typecheck(mgr);
}
for(auto& def_defnn_name : group->members) {
this->env->generalize(def_defnn_name, mgr);
this->env->generalize(def_defnn_name, *group, mgr);
}
}
}

View File

@ -205,7 +205,13 @@ void type_mgr::find_free(const type_ptr& t, std::set<std::string>& into) const {
void type_mgr::find_free(const type_scheme_ptr& t, std::set<std::string>& into) const {
std::set<std::string> monotype_free;
find_free(t->monotype, monotype_free);
type_mgr limited_mgr;
for(auto& binding : types) {
auto existing_position = std::find(t->forall.begin(), t->forall.end(), binding.first);
if(existing_position != t->forall.end()) continue;
limited_mgr.types[binding.first] = binding.second;
}
limited_mgr.find_free(t->monotype, monotype_free);
for(auto& not_free : t->forall) {
monotype_free.erase(not_free);
}

View File

@ -8,11 +8,11 @@ void type_env::find_free(const type_mgr& mgr, std::set<std::string>& into) const
}
}
void type_env::find_free_except(const type_mgr& mgr, const std::string& avoid,
void type_env::find_free_except(const type_mgr& mgr, const group& avoid,
std::set<std::string>& into) const {
if(parent != nullptr) parent->find_free(mgr, into);
for(auto& binding : names) {
if(binding.first == avoid) continue;
if(avoid.members.find(binding.first) != avoid.members.end()) continue;
mgr.find_free(binding.second.type, into);
}
}
@ -65,7 +65,7 @@ void type_env::bind_type(const std::string& type_name, type_ptr t) {
type_names[type_name] = t;
}
void type_env::generalize(const std::string& name, type_mgr& mgr) {
void type_env::generalize(const std::string& name, const group& grp, type_mgr& mgr) {
auto names_it = names.find(name);
if(names_it == names.end()) throw 0;
if(names_it->second.type->forall.size() > 0) throw 0;
@ -73,7 +73,7 @@ void type_env::generalize(const std::string& name, type_mgr& mgr) {
std::set<std::string> free_in_type;
std::set<std::string> free_in_env;
mgr.find_free(names_it->second.type->monotype, free_in_type);
find_free_except(mgr, name, free_in_env);
find_free_except(mgr, grp, free_in_env);
for(auto& free : free_in_type) {
if(free_in_env.find(free) != free_in_env.end()) continue;
names_it->second.type->forall.push_back(free);

View File

@ -2,6 +2,7 @@
#include <map>
#include <string>
#include <set>
#include "graph.hpp"
#include "type.hpp"
struct type_env;
@ -29,7 +30,7 @@ struct type_env {
type_env() : type_env(nullptr) {}
void find_free(const type_mgr& mgr, std::set<std::string>& into) const;
void find_free_except(const type_mgr& mgr, const std::string& avoid,
void find_free_except(const type_mgr& mgr, const group& avoid,
std::set<std::string>& into) const;
type_scheme_ptr lookup(const std::string& name) const;
bool is_global(const std::string& name) const;
@ -41,7 +42,7 @@ struct type_env {
void bind(const std::string& name, type_scheme_ptr t,
visibility v = visibility::local);
void bind_type(const std::string& type_name, type_ptr t);
void generalize(const std::string& name, type_mgr& mgr);
void generalize(const std::string& name, const group& grp, type_mgr& mgr);
};

View File

@ -6,9 +6,9 @@ description: "In this post, we extend our language with let/in expressions and l
draft: true
---
Now that our language's type system is more fleshed out and pleasant to use, it's time to shift our focus to the ergonomics of the language itself. I've been mentioning `let/in` expressions and __lambda expressions__ for a while now. The former will let us create names for expressions that are limited to a certain scope (without having to create global variable bindings), while the latter will allow us to create functions without giving them any name at all.
Now that our language's type system is more fleshed out and pleasant to use, it's time to shift our focus to the ergonomics of the language itself. I've been mentioning `let/in` and __lambda__ expressions for a while now. The former will let us create names for expressions that are limited to a certain scope (without having to create global variable bindings), while the latter will allow us to create functions without giving them any name at all.
Let's take a look at `let/in` expressions first, to make sure we're all on the same page about what it is we're trying to implement. Let's start with some rather basic examples, and then move on to more complex examples. The most basic use of a `let/in` expression is, in Haskell:
Let's take a look at `let/in` expressions first, to make sure we're all on the same page about what it is we're trying to implement. Let's start with some rather basic examples, and then move on to more complex ones. A very basic use of a `let/in` expression is, in Haskell:
```Haskell
let x = 5 in x + x
@ -93,7 +93,7 @@ addSingle6 x = 6 + x
-- ... and so on ...
```
But now, we end up creating several functions with almost identical bodies, with the exception of the free variables themselves. Wouldn't it be better to perform the well-known strategy of reducing code duplication by factoring out parameters, and leaving only instance of the repeated code? We would end up with:
But now, we end up creating several functions with almost identical bodies, with the exception of the free variables themselves. Wouldn't it be better to perform the well-known strategy of reducing code duplication by factoring out parameters, and leaving only one instance of the repeated code? We would end up with:
```Haskell {linenos=table}
addToAll n xs = map (addSingle n) xs
@ -145,11 +145,48 @@ to `let/in`, and that's what we'll be using in our language.
This technique of replacing captured variables with arguments, and pulling closures into the global scope to aid compilation, is called [Lambda Lifting](https://en.wikipedia.org/wiki/Lambda_lifting). Its name is no coincidence - lambda functions need to undergo the same kind of transformation as our nested definitions (unlike nested definitions, though, lambda functions need to be named). This is why they are included in this post together with `let/in`!
What are lambda functions, by the way? A lambda function is just a function
expression that doesn't have a name. For example, if we had Haskell code like
this:
```Haskell
double x = x + x
doubleList xs = map double xs
```
We could rewrite it using a lambda function as follows:
```Haskell
doubleList xs = map (\x -> x + x) xs
```
As you can see, a lambda is an expression in the form `\x -> y` where `x` can
be any variable and `y` can be any expression (including another lambda).
This represents a function that, when applied to a value `x`, will perform
the computation given by `y`. Lambdas are useful when creating single-use
functions that we don't want to make globally available.
Lifting lambda functions will effectively rewrite our program in the
opposite direction to the one shown, replacing the lambda with a reference
to a global declaration which will hold the function's body. Just like
with `let/in`, we will represent captured variables using arguments
and partial appliciation. For instance, when starting with:
```Haskell
addToAll n xs = map (\x -> n + x) xs
```
We would output the following:
```Haskell
addToAll n xs = map (lambda n) xs
lambda n x = n + x
```
### Implementation
Now that we understand what we have to do, it's time to jump straight into
doing it. First, we need to refactor our current code so allow for the changes
we're going to make; then, we can implement `let/in` expressions; finally,
we'll tackle lambda functions.
doing it. First, we need to refactor our current code to allow for the changes
we're going to make; then, we will use the new tools we defined to implement `let/in` expressions and lambda functions.
#### Infrastructure Changes
When finding captured variables, the notion of _free variables_ once again
@ -168,8 +205,8 @@ since it's not defined locally.
The algorithm that we used for computing free variables was rather biased.
Previously, we only cared about the difference between a local variable
(defined somewhere in a function's body, or referring to one of the function's
parameters) and a global variable (referring to a function name). This shows in
our code for `find_free`. Consider, for example, this segment:
parameters) and a global variable (referring to a global function).
This shows in our code for `find_free`. Consider, for example, this snippet:
{{< codelines "C++" "compiler/11/ast.cpp" 33 36 >}}
@ -449,17 +486,17 @@ we're trying to operate on is global or not? I propose a flag in our
this, we update the implementation of `type_env` to map variables to
values of a struct `variable_data`:
{{< codelines "C++" "compiler/12/type_env.hpp" 13 22 >}}
{{< codelines "C++" "compiler/12/type_env.hpp" 14 23 >}}
The `visibility` enum is defined as follows:
{{< codelines "C++" "compiler/12/type_env.hpp" 10 10 >}}
{{< codelines "C++" "compiler/12/type_env.hpp" 11 11 >}}
As you can see from the above snippet, we also added a `mangled_name` field
to the new `variable_data` struct. We will be using this field shortly. We
also add a few methods to our `type_env`, and end up with the following:
{{< codelines "C++" "compiler/12/type_env.hpp" 31 44 >}}
{{< codelines "C++" "compiler/12/type_env.hpp" 32 45 >}}
We will come back to `find_free` and `find_free_except`, as well as
`set_mangled_name` and `get_mangled_name`. For now, we just adjust `bind` to
@ -536,7 +573,7 @@ And the latter:
{{< codelines "C++" "compiler/12/type_env.cpp" 39 45 >}}
We don't allow the `set_mangled_name` to affect variables that are declared
We don't allow `set_mangled_name` to affect variables that are declared
above the receiving `type_env`, and use the empty string as a 'none' value.
Now, when lifting data type constructors, we'll be able to use
`set_mangled_name` to make sure constructor calls are made correctly. We
@ -630,7 +667,7 @@ void ast::translate(global_scope& scope);
The `scope` parameter and its `add_function` and `add_constructor` methods will
be used to add definitions to the global scope. Each AST node will also
uses this method to implement the second step. Currently, only
use this method to implement the second step. Currently, only
`ast_let` and `ast_lambda` will need to modify themselves - all other
nodes will simply recursively call this method on their children. Let's jump
straight into implementing this method for `ast_let`:
@ -639,7 +676,7 @@ straight into implementing this method for `ast_let`:
Since data type definitions don't really depend on anything else, we process
them first. This amounts to simply calling the `definition_data::into_globals`
methd, which itself simply calls `global_scope::add_constructor`:
method, which itself simply calls `global_scope::add_constructor`:
{{< codelines "C++" "compiler/12/definition.cpp" 86 92 >}}
@ -659,7 +696,7 @@ First, this method collects all the non-global free variables in
its body, which will need to be passed to the global definition
as arguments. It then combines this list with the arguments
the user explicitly added to it, recursively translates
its body, creates a new global definition using `add_function`.
its body, and creates a new global definition using `add_function`.
We return to `ast_let::translate` at line 299. Here,
we determine how many variables ended up being captured, by
@ -675,7 +712,7 @@ of the function, but this seems inelegant, especially since we
alreaady keep track of mangling information in `type_env`. Instead,
we create a new, local environment, in which we place an updated
binding for the function, marking it global, and setting
its mangled name to one generated by `global_sope`. This work is done
its mangled name to the one generated by `global_sope`. This work is done
on lines 301-303. We create a reference to the global function
using the new environment on lines 305 and 306, and apply it to
all the implict arguments on lines 307-313. Finally, we
@ -730,7 +767,7 @@ closer to the top of the G-machine stack. Thus, when we
iterate the definitions again, this time to compile their
bodies, we have to do so starting with the highest offset,
and working our way down to __Update__-ing the top of the stack.
One the definitions have been compiled, we proceed to compiling
Once the definitions have been compiled, we proceed to compiling
the `in` part of the expression as normal, using our updated
environment. Finally, we use __Slide__ to get rid of the definition
graphs, cleaning up the stack.
@ -738,16 +775,16 @@ graphs, cleaning up the stack.
Compiling the `ast_lambda` is far more straightforward. We just
compile the resulting partial application as we normally would have:
{{< codelines "C++" "compiler/12/ast.cpp" 393 395 >}}
{{< codelines "C++" "compiler/12/ast.cpp" 394 396 >}}
One more thing. Let's adopt the convention of storing __mangled__
names into the environment. This way, rather than looking up
names into the compilation environment. This way, rather than looking up
mangled names only for global functions, which would be a 'gotcha'
for anyone working on the compiler, we will always use the mangled
names during compilation. To make this change, we make sure that
`ast_case` also uses `mangled_name`:
{{< codelines "C++" "compiler/12/ast.cpp" 228 228 >}}
{{< codelines "C++" "compiler/12/ast.cpp" 242 242 >}}
We also update the logic for `ast_lid::compile` to use the mangled
name information:
@ -775,9 +812,12 @@ void type_env::find_free_except(const type_mgr& mgr, const std::string& avoid,
Why `find_free_except`? When generalizing a variable whose type was already
stored in the environment, all the type variables we could generalize would
not be 'free'. If they only occur in the type we're generalizing, though,
we shouldn't let that stop us! Thus, when finding free type variables, we will
avoid looking at the particular variable whose type is being generalized. The
implementations of the two methods are straightforward:
we shouldn't let that stop us! More generally, if we see type variables that
are only found in the same mutually recursive group as the binding we're
generalizing, we are free to generalize them too. Thus, we pass in
a reference to a `group`, and check if a variable is a member of that group
before searching it for free type variables. The implementations of the two
methods are straightforward:
{{< codelines "C++" "compiler/12/type_env.cpp" 4 18 >}}
@ -787,7 +827,15 @@ that have the same name as the variable we're generalizing, but aren't found
in the same scope. As far as we're concerned, they're different variables!
The two methods use another `find_free` method which we add to `type_mgr`:
{{< codelines "C++" "compiler/12/type.cpp" 206 213 >}}
{{< codelines "C++" "compiler/12/type.cpp" 206 219 >}}
This one is a bit of a hack. Typically, while running `find_free`, a
`type_mgr` will resolve any type variables. However, variables from the
`forall` quantifier of a type scheme should not be resolved, since they
are explicitly generic. To prevent the type manager from erroneously resolving
such type variables, we create a new type manager that does not have
these variables bound to anything, and thus marks them as free. We then
filter these variables out of the final list of free variables.
Finally, `generalize` makes sure not to use variables that it finds free:
@ -859,7 +907,7 @@ in our language, perhaps to create an infinite list of ones:
We want `sumTwo` to take the first two elements from the list,
and return their sum. For an infinite list of ones, we expect
this sum to equal to 2, and so it does:
this sum to be equal to 2, and it is:
```
Result: 2
@ -873,9 +921,9 @@ dependency tracking works as expected:
{{< codeblock "text" "compiler/12/examples/letin.txt" >}}
Here, we have a function `mergeUntil` which, given two lists
and a predicate, combines the two lists until as long as
and a predicate, combines the two lists as long as
the predicate returns `True`. It does so using a convoluted
pair of two mutually recursive functions, one of which
pair of mutually recursive functions, one of which
unpacks the left list, and the other the right. Each of the
functions calls the global function `if`. We also use two
definitions inside of `main` to create the two lists we're