ChapelCon/ChapelCon2025-Slides
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Finalize slides given timing constraints

Browse Source 
Signed-off-by: Danila Fedorin <daniel.fedorin@hpe.com>
This commit is contained in:
Danila Fedorin 2025-10-10 13:05:15 -07:00
parent 86f1ba686a
commit 6ccf99656c
4 changed files with 780 additions and 716 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

BIN
alloy/alloy-blog.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 1.9 KiB

814
alloy/slides.md
View File

@ -29,25 +29,15 @@ Daniel Fedorin, HPE
--- ---
# Terms
* What are **formal methods**?
* Techniques rooted in computer science and mathematics to specify and verify systems
* What part of the **Chapel compiler**?
* The 'Dyno' compiler front-end, particularly its use/import resolution phase.
* This piece is used by the production Chapel compiler.
---
# The Story # The Story
I have a story in three parts. I have a story in three parts.
1. I found it very hard to think through a section of compiler code. 1. I found it very hard to think through a section of compiler code.
- Specifically, code that performed lookups in `use`s/`import`s - Specifically, code that performed scope lookups for variables
2. I used the [Alloy Analyzer](https://alloytools.org/) to model the assumptions and behavior of the code. 2. I used the [Alloy Analyzer](https://alloytools.org/) to model the assumptions and behavior of the code.
- I had little background in Alloy, but some background in (formal) logic - I had little background in Alloy, but some background in formal logic
3. This led me to discover a bug in the compiler that I then fixed. 3. This led me to discover a bug in the compiler that I then fixed.
- Re-creating the bug required some gymnastics that were unlikely to occur in practice. - Re-creating the bug required some gymnastics that were unlikely to occur in practice.
@ -66,11 +56,388 @@ Imagine you see the following snippet of Chapel code:
foo(); foo();
``` ```
Where do you look for `foo`? The answer is quite complicated, and depends Where do you look for `foo`? The answer is quite complicated, and depends strongly on the context of the call.
strongly on the context of the call.
Moreover, the order of where to look matters: method calls are preferred over global functions, "nearer" functions are preferred over "farther" ones.
---
# The Humble `foo` (example 1)
```chapel
module M1 {
record R {}
proc R.foo() { writeln("R.foo"); }
proc foo() { writeln("M1.foo"); }
proc R.someMethod() {
foo(); // which 'foo'?
}
}
```
* Both `R.foo` and `M1.foo` would be valid candidates.
* We give priority to methods over global functions. So, the compiler would:
* Search `R` and its scope (`M1`) for methods named `foo`.
* If that fails, search `M1` for any symbols `foo`.
* We've had to look at `M1` twice! (once for methods, once for non-methods)
---
# The Humble `foo` (example 2)
```chapel
module M1 {
record R {}
proc foo() { writeln("R.foo"); }
}
module M2 {
use M1;
proc R.someMethod() {
foo(); // which 'foo'?
}
}
```
Here, we search the scope of `R` and `M1`, but **only for public symbols**.
---
# How Chapel's Compiler Handles This
We want to:
- Respect the priority order
- Including preferring methods over non-methods
- As a result, we search the scopes multiple times
- Avoid any extra work
- This includes redundant re-searches
- Example redundant search: looked up "all symbols", then later "all public symbols"
---
# Encoding Search Configuration
```C++
enum { PUBLIC = 1, NOT_PUBLIC = 2, METHOD_FIELD = 4, NOT_METHOD_FIELD = 8, /* ... */ };
```
1. For each scope, save the flags we've already searched with
2. When searching a scope again, exclude the flags we've already searched with
This was handled by two bitfields: `filter` and `excludeFilter`.
---
# Populating `filter` and `excludeFilter`
```C++
if (skipPrivateVisibilities) { // depends on context of 'foo()'
filter |= IdAndFlags::PUBLIC;
}
else if (!includeMethods && receiverScopes.empty()) {
filter |= IdAndFlags::NOT_METHOD;
}
```
```C++
excludeFilter = previousFilter;
```
```C++
// scary!
previousFilter = filter & previousFilter;
```
Code notes `previousFilter` is an approximation.
---
# Possible Problems
* `previousFilter` is an approximation.
* However, no case we knew of hit this combination of searches, or any like it.
* All of our language tests passed.
* Code seemed to work.
* If only there was a way I could get a computer to check whether such a combination
could occur...
---
<!-- _class: lead -->
# Formal Methods
---
# Model Checking
Model checking involves formally describing the behavior of a system, then having a solver check whether other desired properties hold.
- Alloy is an example of a model checker.
- TLA is another famous example.
---
# A Primer on Logic (example)
Model checkers like Alloy are rooted in temporal logic, which builds on first-order logic.
Example statement: "Bob has a son who likes all compilers".
$$
\exists x. (\text{Son}(x, \text{Bob}) \land \forall y. (\text{Compiler}(y) \Rightarrow \text{Likes}(x, y)))
$$
In Alloy:
```alloy
some x { Son[x, Bob] and all y { Compiler[y] implies Likes[x, y] } }
```
---
# A Primer on Temporal Logic
Temporal logic provides additional operators to reason about how properties change over time.
- $\Box p$ (in Alloy: `always p`): A statement that is always true.
- Example: $\Box(\text{like charges repel})$
- $\Diamond p$ (in Alloy: `eventually p`) : A statement that will be true at some point in the future.
- Example: $\Diamond(\text{the sun is in the sky})$
In Alloy specifically, we can mention the next state of a variable using `'`.
```alloy
// pseudocode:
// the next future value of previousFilter will be the intersection of filter
// and the current value
previousFilter' = filter & previousFilter;
```
---
# Modeling Possible Searches
Alloy isn't an imperative language. We can't mutate variables like we do in C++. Instead, we model how each statement changes the state, by relating the "current" state to the "next" state.
<div class="side-by-side">
<div>
```C++
filter |= IdAndFlags::PUBLIC;
```
</div>
<div>
```alloy
addBitfieldFlag[filterNow, filterNext, Public]
```
</div>
</div>
This might remind you of [Hoare Logic](https://en.wikipedia.org/wiki/Hoare_logic), where statements like:
$$
\{ P \} \; s \; \{ Q \}
$$
Read as:
> If $P$ is true before executing $s$, then $Q$ will be true after executing $s$.
$$
\{ \text{filter} = \text{filterNow} \} \; \texttt{filter |= PUBLIC} \; \{ \text{filter} = \text{filterNext} \}
$$
---
# Modeling Possible Searches
To combine several statements, we make it so that the "next" state of one statement is the "current" state of the next statement.
<div class="side-by-side">
<div>
```C++
curFilter |= IdAndFlags::PUBLIC;
curFilter |= IdAndFlags::METHOD_FIELD;
```
</div>
<div>
```alloy
addBitfieldFlag[filterNow, filterNext1, Public]
addBitfieldFlag[filterNext1, filterNext2, Method]
```
</div>
</div>
This is reminiscent of sequencing Hoare triples:
$$
\{ P \} \; s_1 \; \{ Q \} \; s_2 \; \{ R \}
$$
---
# Modeling Possible Searches
Finally, if C++ code has conditionals, we need to allow for the possibility of either branch being taken. We do this by using "or" on descriptions of the next state.
<div class="side-by-side">
<div>
```C++
if (skipPrivateVisibilities) {
curFilter |= IdAndFlags::PUBLIC;
}
if (!includeMethods && receiverScopes.empty()) {
curFilter |= IdAndFlags::NOT_METHOD;
}
```
</div>
<div>
```alloy
addBitfieldFlag[initialState.curFilter, bitfieldMiddle, Public] or
bitfieldEqual[initialState.curFilter, bitfieldMiddle]
// if it's not a receiver, filter to non-methods (could be overridden)
addBitfieldFlagNeg[bitfieldMiddle, filterState.curFilter, Method] or
bitfieldEqual[bitfieldMiddle, filterState.curFilter]
```
</div>
</div>
Putting this into a predicate, `possibleState`, we encode what searches the compiler can undertake.
**Takeaway**: We encoded the logic that configures possible searches in Alloy. This instructs the analyzer about possible cases to consider.
---
# Are there any bugs?
Model checkers ensure that all properties we want to hold, do hold. To find a counter example, we ask it to prove the negation of what we want.
```C
wontFindNeeded: run {
all searchState: SearchState {
eventually some props: Props, fs: FilterState, fsBroken: FilterState {
// Some search (fs) will cause a transition / modification of the search state...
configureState[fs]
updateOrSet[searchState.previousFilter, fs]
// Such that a later, valid search... (fsBroken)
configureState[fsBroken]
// Will allow for a set of properties...
// ... that are left out of the original search...
not bitfieldMatchesProperties[searchState.previousFilter, props]
// ... and out of the current search
not (bitfieldMatchesProperties[fs.include, props] and not bitfieldMatchesProperties[searchState.previousFilter, props])
// But would be matched by the broken search...
bitfieldMatchesProperties[fsBroken.include, props]
// ... to not be matched by a search with the new state:
not (bitfieldMatchesProperties[fsBroken.include, props] and not bitfieldOrNotSetMatchesProperties[searchState.previousFilter', props])
}
}
}
```
---
# Uh-Oh!
![](./instancefound.png)
---
# The Bug
![bg left width:600px](./bug.png)
We need some gymnastics to figure out what variables make this model possible.
Alloy has a nice visualizer, but it has a lot of information.
In the interest of time, I found:
* If the compiler searches a scope first for `PUBLIC` symbols, ...
* ...then for `METHOD_OR_FIELD`, ...
* ...then for any symbols, they will miss things!
---
<style scoped>
section {
font-size: 20px;
}
</style>
# The Reproducer
To trigger this sequence of searches, we needed a lot more gymnastics.
<div class="side-by-side">
<div>
```chapel
module TopLevel {
module XContainerUser {
public use TopLevel.XContainer;
}
module XContainer {
private var x: int;
record R {}
module MethodHaver {
use TopLevel.XContainerUser;
use TopLevel.XContainer;
proc R.foo() {
var y = x;
}
}
}
}
```
</div>
<div>
* the scope of `R` is searched with for methods
* The scope of `R`’s parent (`XContainer`) is searched for methods
* The scope of `XContainerUser` is searched for public symbols (via the `use`)
* The scope of `XContainer` is searched with public symbols (via the `public use`)
* The scope of `XContainer` searched for with no filters via the second use; but the stored filter is bad, so the lookup returns early, not finding `x`.
</div>
</div>
---
<!-- _class: lead -->
# **Thank you!**
![](./alloy-blog.png)
Read more
---
<!-- _class: lead -->
# Extra Slides
---
# Terms
* What are **formal methods**?
* Techniques rooted in computer science and mathematics to specify and verify systems
* What part of the **Chapel compiler**?
* The 'Dyno' compiler front-end, particularly its scope lookup phase
* This piece is used by the production Chapel compiler.
Moreover, the order of where to look matters: method calls are preferred
over global functions, "nearer" functions are preferred over "farther" ones.
--- ---
@ -155,96 +522,6 @@ Here, we search the scope of `R` and `M1`, but **only for public symbols**.
--- ---
# How Chapel's Compiler Handles This
We want to:
- Respect the priority order
- Including preferring methods over non-methods
- As a result, we search the scopes multiple times
- Avoid any extra work
- This includes redundant re-searches
- Example redundant search: looked up "all symbols", then later "all public symbols"
```C++
enum { PUBLIC = 1, NOT_PUBLIC = 2, METHOD_FIELD = 4, NOT_METHOD_FIELD = 8, /* ... */ };
```
1. For each scope, save the flags we've already searched with
2. When searching a scope again, exclude the flags we've already searched with
This was handled by two bitfields: `filter` and `excludeFilter`.
---
# Populating `filter` and `excludeFilter`
```C++
if (skipPrivateVisibilities) { // depends on context of 'foo()'
filter |= IdAndFlags::PUBLIC;
}
if (onlyMethodsFields) {
filter |= IdAndFlags::METHOD_FIELD;
} else if (!includeMethods && receiverScopes.empty()) {
filter |= IdAndFlags::NOT_METHOD;
}
```
```C++
excludeFilter = previousFilter;
```
```C++
// scary!
previousFilter = filter & previousFilter;
```
Code notes `previousFilter` is an approximation.
---
# Possible Problems
* `previousFilter` is an approximation.
* However, no case we knew of hit this combination of searches, or any like it.
* All of our language tests passed.
* Code seemed to work.
* If only there was a way I could get a computer to check whether such a combination
could occur...
---
<!-- _class: lead -->
# Formal Methods
---
# Types of Formal Methods
- Model checking involves formally describing the behavior of a system, then having a solver check whether other desired properties hold.
- Alloy is an example of a model checker.
- TLA is another famous example.
- Theorem proving is a heavier weight approach that involves building a formal proof of correctness.
- Coq and Isabelle are examples of theorem provers.
---
<style scoped>
li:nth-child(2) { color: lightgrey; }
</style>
# Types of Formal Methods
- Model checking involves formally describing the behavior of a system, then having a solver check whether other desired properties hold.
- Alloy is an example of a model checker.
- TLA is another famous example.
- Theorem proving is a heavier weight approach that involves building a formal proof of correctness.
- Coq and Isabelle are examples of theorem provers.
**Reason**: I was in the middle of developing compiler code. I wanted to sketch
the assumptions I was making and see if they held up.
---
# A Primer on Logic # A Primer on Logic
Model checkers like Alloy are rooted in temporal logic, which builds on Model checkers like Alloy are rooted in temporal logic, which builds on
@ -263,40 +540,6 @@ first-order logic. This includes:
--- ---
# A Primer on Logic (example)
Example statement: "Bob has a son who likes all compilers".
$$
\exists x. (\text{Son}(x, \text{Bob}) \land \forall y. (\text{Compiler}(y) \Rightarrow \text{Likes}(x, y)))
$$
In Alloy:
```alloy
some x { Son[x, Bob] and all y { Compiler[y] implies Likes[x, y] } }
```
---
# A Primer on Temporal Logic
Temporal logic provides additional operators to reason about how properties change over time.
- $\Box p$ (in Alloy: `always p`): A statement that is always true.
- $\Diamond p$ (in Alloy: `eventually p`) : A statement that will be true at some point in the future.
In Alloy specifically, we can mention the next state of a variable using `'`.
```alloy
// pseudocode:
// the next future value of previousFilter will be the intersection of filter
// and the current value
previousFilter' = filter & previousFilter;
```
---
# A Primer on Temporal Logic (example) # A Primer on Temporal Logic (example)
Some examples: Some examples:
@ -387,131 +630,6 @@ pred bitfieldEqual[b1: Bitfield, b2: Bitfield] {
--- ---
# Modeling Possible Searches
Alloy isn't an imperative language. We can't mutate variables like we do in C++. Instead, we model how each statement changes the state, by relating the "current" state to the "next" state.
<div class="side-by-side">
<div>
```C++
filter |= IdAndFlags::PUBLIC;
```
</div>
<div>
```alloy
addBitfieldFlag[filterNow, filterNext, Public]
```
</div>
</div>
This might remind you of [Hoare Logic](https://en.wikipedia.org/wiki/Hoare_logic), where statements like:
$$
\{ P \} \; s \; \{ Q \}
$$
Read as:
> If $P$ is true before executing $s$, then $Q$ will be true after executing $s$.
$$
\{ \text{filter} = \text{filterNow} \} \; \texttt{filter |= PUBLIC} \; \{ \text{filter} = \text{filterNext} \}
$$
---
# Modeling Possible Searches
To combine several statements, we make it so that the "next" state of one statement is the "current" state of the next statement.
<div class="side-by-side">
<div>
```C++
curFilter |= IdAndFlags::PUBLIC;
curFilter |= IdAndFlags::METHOD_FIELD;
```
</div>
<div>
```alloy
addBitfieldFlag[filterNow, filterNext1, Public]
addBitfieldFlag[filterNext1, filterNext2, Method]
```
</div>
</div>
This is reminiscent of sequencing Hoare triples:
$$
\{ P \} \; s_1 \; \{ Q \} \; s_2 \; \{ R \}
$$
---
# Modeling Possible Searches
Finally, if C++ code has conditionals, we need to allow for the possibility of either branch being taken. We do this by using "or" on descriptions of the next state.
<div class="side-by-side">
<div>
```C++
if (skipPrivateVisibilities) {
curFilter |= IdAndFlags::PUBLIC;
}
if (onlyMethodsFields) {
curFilter |= IdAndFlags::METHOD_FIELD;
} else if (!includeMethods && receiverScopes.empty()) {
curFilter |= IdAndFlags::NOT_METHOD;
}
```
</div>
<div>
```alloy
addBitfieldFlag[initialState.curFilter, bitfieldMiddle, Public] or
bitfieldEqual[initialState.curFilter, bitfieldMiddle]
// If it's a method receiver, add method or field restriction
addBitfieldFlag[bitfieldMiddle, filterState.curFilter, MethodOrField] or
// if it's not a receiver, filter to non-methods (could be overridden)
addBitfieldFlagNeg[bitfieldMiddle, filterState.curFilter, Method] or
// Maybe methods are not being curFilterd but it's not a receiver, so no change.
bitfieldEqual[bitfieldMiddle, filterState.curFilter]
```
</div>
</div>
Putting this into a predicate, `possibleState`, we encode what searches the compiler can undertake.
**Takeaway**: We encoded the logic that configures possible searches in Alloy. This instructs the analyzer about possible cases to consider.
---
# Modeling `previousFilter` # Modeling `previousFilter`
So far, all we've done is encoded what queries the compiler might make about a scope. So far, all we've done is encoded what queries the compiler might make about a scope.
@ -531,6 +649,55 @@ one sig SearchState {
Above, `+` is used for union. `previousFilter` can either be a `Bitfield` or `NotSet`. Above, `+` is used for union. `previousFilter` can either be a `Bitfield` or `NotSet`.
---
# Types of Formal Methods
- Model checking involves formally describing the behavior of a system, then having a solver check whether other desired properties hold.
- Alloy is an example of a model checker.
- TLA is another famous example.
- Theorem proving is a heavier weight approach that involves building a formal proof of correctness.
- Coq and Isabelle are examples of theorem provers.
---
<style scoped>
li:nth-child(2) { color: lightgrey; }
</style>
# Types of Formal Methods
- Model checking involves formally describing the behavior of a system, then having a solver check whether other desired properties hold.
- Alloy is an example of a model checker.
- TLA is another famous example.
- Theorem proving is a heavier weight approach that involves building a formal proof of correctness.
- Coq and Isabelle are examples of theorem provers.
**Reason**: I was in the middle of developing compiler code. I wanted to sketch
the assumptions I was making and see if they held up.
---
# Putting it Together
We now have a model of what our C++ program is doing: it computes some set of filter flags, then runs a search, excluding the previous flags. It then updates the previous flags with the current search. We can encode this as follows:
```
fact step {
always {
// Model that a new doLookupInScope could've occurred, with any combination of flags.
all searchState: SearchState {
some fs: FilterState {
// This is a possible combination of lookup flags
possibleState[fs]
// If a search has been performed before, take the intersection; otherwise,
// just insert the current filter flags.
updateOrSet[searchState.previousFilter, fs]
}
}
}
}
```
--- ---
# Modeling `previousFilter` # Modeling `previousFilter`
@ -566,120 +733,3 @@ Otherwise, we set `previousFilter` to the intersection of `filter` and `previous
</div> </div>
</div> </div>
---
# Putting it Together
We now have a model of what our C++ program is doing: it computes some set of filter flags, then runs a search, excluding the previous flags. It then updates the previous flags with the current search. We can encode
this as follows:
```
fact step {
always {
// Model that a new doLookupInScope could've occurred, with any combination of flags.
all searchState: SearchState {
some fs: FilterState {
// This is a possible combination of lookup flags
possibleState[fs]
// If a search has been performed before, take the intersection; otherwise,
// just insert the current filter flags.
updateOrSet[searchState.previousFilter, fs]
}
}
}
}
```
---
# Are there any bugs?
Model checkers ensure that all properties we want to hold, do hold. To find a counter example, we ask it to prove the negation of what we want.
```C
wontFindNeeded: run {
all searchState: SearchState {
eventually some props: Props, fs: FilterState, fsBroken: FilterState {
// Some search (fs) will cause a transition / modification of the search state...
configureState[fs]
updateOrSet[searchState.previousFilter, fs]
// Such that a later, valid search... (fsBroken)
configureState[fsBroken]
// Will allow for a set of properties...
// ... that are left out of the original search...
not bitfieldMatchesProperties[searchState.previousFilter, props]
// ... and out of the current search
not (bitfieldMatchesProperties[fs.include, props] and not bitfieldMatchesProperties[searchState.previousFilter, props])
// But would be matched by the broken search...
bitfieldMatchesProperties[fsBroken.include, props]
// ... to not be matched by a search with the new state:
not (bitfieldMatchesProperties[fsBroken.include, props] and not bitfieldOrNotSetMatchesProperties[searchState.previousFilter', props])
}
}
}
```
---
# Uh-Oh!
![](./instancefound.png)
---
# The Bug
![bg left width:600px](./bug.png)
We need some gymnastics to figure out what variables make this model possible.
Alloy has a nice visualizer, but it has a lot of information.
In the interest of time, I found:
* If the compiler searches a scope first for `PUBLIC` symbols, ...
* ...then for `METHOD_OR_FIELD`, ...
* ...then for any symbols, they will miss things!
---
<style scoped>
section {
font-size: 20px;
}
</style>
# The Reproducer
To trigger this sequence of searches, we needed a lot more gymnastics.
<div class="side-by-side">
<div>
```chapel
module TopLevel {
module XContainerUser {
public use TopLevel.XContainer;
}
module XContainer {
private var x: int;
record R {}
module MethodHaver {
use TopLevel.XContainerUser;
use TopLevel.XContainer;
proc R.foo() {
var y = x;
}
}
}
}
```
</div>
<div>
* the scope of `R` is searched with for methods
* The scope of `R`’s parent (`XContainer`) is searched for methods
* The scope of `XContainerUser` is searched for public symbols (via the `use`)
* The scope of `XContainer` is searched with public symbols (via the `public use`)
* The scope of `XContainer` searched for with no filters via the second use; but the stored filter is bad, so the lookup returns early, not finding `x`.
</div>
</div>

BIN
type-level/linear-multistep.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 2.0 KiB

682
type-level/slides.md
View File

@ -235,6 +235,354 @@ There is no runtime overhead!
<!-- _class: lead --> <!-- _class: lead -->
# Linear Multi-Step Method Approximator # Linear Multi-Step Method Approximator
![](./linear-multistep.png)
---
<!-- _class: lead -->
# Type-Safe `printf`
---
# The `printf` Function
The `printf` function accepts a format string, followed by a variable number of arguments that should match:
```C
// totally fine:
printf("Hello, %s! Your ChapelCon submission is #%d\n", "Daniel", 18);
// not good:
printf("Hello, %s! Your ChapelCon submission is #%d\n", 18, "Daniel");
```
Can we define a `printf` function in Chapel that is type-safe?
---
# Yet Another Type-Level List
- The general idea for type-safe `printf`: take the format string, and extract a list of the expected argument types.
- To make for nicer error messages, include a human-readable description of each type in the list.
- I've found it more convenient to re-define lists for various problems when needed, rather than having a single canonical list definition.
```chapel
record _nil {
proc type length param do return 0;
}
record _cons {
type expectedType; // type of the argument to printf
param name: string; // human-readable name of the type
type rest;
proc type length param do return 1 + rest.length();
}
```
---
# Extracting Types from Format Strings
```Chapel
proc specifiers(param s: string, param i: int = 0) type {
if i >= s.size then return _nil;
if s[i] == "%" {
if i + 1 >= s.size then
compilerError("Invalid format string: unterminted %");
select s[i + 1] {
when "%" do return specifiers(s, i + 2);
when "s" do return _cons(string, "a string", specifiers(s, i + 2));
when "i" do return _cons(int, "a signed integer", specifiers(s, i + 2));
when "u" do return _cons(uint, "an unsigned integer", specifiers(s, i + 2));
when "n" do return _cons(numeric, "a numeric value", specifiers(s, i + 2));
otherwise do compilerError("Invalid format string: unknown format type");
}
} else {
return specifiers(s, i + 1);
}
}
```
---
# Extracting Types from Format Strings
Let's give it a quick try:
```Chapel
writeln(specifiers("Hello, %s! Your ChapelCon submission is #%i\n") : string);
```
The above prints:
```Chapel
_cons(string,"a string",_cons(int(64),"a signed integer",_nil))
```
---
# Validating Argument Types
* The Chapel standard library has a nice `isSubtype` function that we can use to check if an argument matches the expected type.
* Suppose the `.length` of our type specifiers matches the number of arguments to `printf`
* Chapel doesn't currently support empty tuples, so if the lengths match, we know that `specifiers` is non-empty.
* Then, we can validate the types as follows:
```Chapel
proc validate(type specifiers: _cons(?t, ?s, ?rest), type argTup, param idx) {
if !isSubtype(argTup[idx], t) then
compilerError("Argument " + (idx + 1) : string + " should be " + s + " but got " + argTup[idx]:string, idx+2);
if idx + 1 < argTup.size then
validate(rest, argTup, idx + 1);
}
```
* The `idx+2` argument to `compilerError` avoids printing the recursive `validate` calls in the error message.
---
# The `fprintln` overloads
* I named it `fprintln` for "formatted print line".
* To support the empty-specifier case (Chapel varargs don't allow zero arguments):
```Chapel
proc fprintln(param format: string) where specifiers(format).length == 0 {
writeln(format);
}
```
* If we do have type specifiers, to ensure our earlier assumption of `size` matching:
```Chapel
proc fprintln(param format: string, args...)
where specifiers(format).length != args.size {
compilerError("'fprintln' with this format string expects " +
specifiers(format).length : string +
" argument(s) but got " + args.size : string);
}
```
---
# The `fprintln` overloads
* All that's left is the main `fprintln` implementation:
```Chapel
proc fprintln(param format: string, args...) {
validate(specifiers(format), args.type, 0);
writef(format + "\n", (...args));
}
```
---
# Using `fprintln`
```Chapel
fprintln("Hello, world!"); // fine, prints "Hello, world!"
fprintln("The answer is %i", 42); // fine, prints "The answer is 42"
// compiler error: Argument 3 should be a string but got int(64)
fprintln("The answer is %i %i %s", 1, 2, 3);
```
More work could be done to support more format specifiers, escapes, etc., but the basic idea is there.
---
<!-- _class: lead -->
# Beyond Lists
---
# Beyond Lists
* I made grand claims earlier
- "Write functional-ish program at the type level!"
* So far, we've just used lists and some recursion.
* Is that all there is?
---
# Algebraic Data Types
* The kinds of data types that Haskell supports are called *algebraic data types*.
* At a fundamental level, they can be built up from two operations: _Cartesian product_ and _disjoint union_.
* There are other concepts to build recursive data types, but we won't need them in Chapel.
- To prove to you I know what I'm talking about, some jargon:
_initial algebras_, _the fixedpoint functor_, _catamorphisms_...
- Check out _Bananas, Lenses, Envelopes and Barbed Wire_ by Meijer et al. for more.
* __Claim__: Chapel supports disjoint union and Cartesian product, so we can build any data type that Haskell can.
---
<style scoped>
li:nth-child(3) { color: lightgrey; }
</style>
# Algebraic Data Types
- The kinds of data types that Haskell supports are called *algebraic data types*.
- At a fundamental level, they can be built up from two operations: _Cartesian product_ and _disjoint union_.
- There are other concepts to build recursive data types, but we won't need them in Chapel.
- To prove to you I know what I'm talking about, some jargon:
_initial algebras_, _the fixedpoint functor_, _catamorphisms_...
- Check out _Bananas, Lenses, Envelopes and Barbed Wire_ by Meijer et al. for more.
- __Claim__: Haskell supports disjoint union and Cartesian product, so we can build any data type that Haskell can.
---
# A General Recipe
To translate a Haskell data type definition to Chapel:
* For each constructor, define a `record` with that constructor's name
* The fields of that record are `type` fields for each argument of the constructor
- If the argument is a value (like `Int`), you can make it a `param` field instead
* A visual example, again:
<div class="side-by-side">
<div>
```Chapel
record C1 { type arg1; /* ... */ type argi; }
// ...
record Cn { type arg1; /* ... */ type argj; }
```
</div>
<div>
```Haskell
data T = C1 arg1 ... argi
| ...
| Cn arg1 ... argj
```
</div>
</div>
---
# Inserting and Looking Up in a BST
<div class="side-by-side">
<div>
```Chapel
proc insert(type t: Empty, param x: int) type do return Node(x, Empty, Empty);
proc insert(type t: Node(?v, ?left, ?right), param x: int) type do
select true {
when x < v do return Node(v, insert(left, x), right);
otherwise do return Node(v, left, insert(right, x));
}
type test = insert(insert(insert(Empty, 2), 1), 3);
proc lookup(type t: Empty, param x: int) param do return false;
proc lookup(type t: Node(?v, ?left, ?right), param x: int) param do
select true {
when x == v do return true;
when x < v do return lookup(left, x);
otherwise do return lookup(right, x);
}
```
</div>
<div>
```Haskell
insert :: Int -> BSTree -> BSTree
insert x Empty = Node x Empty Empty
insert x (Node v left right)
| x < v = Node v (insert x left) right
| otherwise = Node v left (insert x right)
test = insert 3 (insert 1 (insert 2 Empty))
lookup :: Int -> BSTree -> Bool
lookup x Empty = False
lookup x (Node v left right)
| x == v = True
| x < v = lookup x left
| otherwise = lookup x right
```
</div>
</div>
It really works!
```Chapel
writeln(test : string);
// prints Node(2,Node(1,Empty,Empty),Node(3,Empty,Empty))
writeln(lookup(test, 1));
// prints true for this one, but false for '4'
```
---
# A Key-Value Map
```Chapel
record Empty {}
record Node { param key: int; param value; type left; type right; }
proc insert(type t: Empty, param k: int, param v) type do return Node(k, v, Empty, Empty);
proc insert(type t: Node(?k, ?v, ?left, ?right), param nk: int, param nv) type do
select true {
when nk < k do return Node(k, v, insert(left, nk, nv), right);
otherwise do return Node(k, v, left, insert(right, nk, nv));
}
proc lookup(type t: Empty, param k: int) param do return "not found";
proc lookup(type t: Node(?k, ?v, ?left, ?right), param x: int) param do
select true {
when x == k do return v;
when x < k do return lookup(left, x);
otherwise do return lookup(right, x);
}
type test = insert(insert(insert(Empty, 2, "two"), 1, "one"), 3, "three");
writeln(lookup(test, 1)); // prints "one"
writeln(lookup(test, 3)); // prints "three"
writeln(lookup(test, 4)); // prints "not found"
```
---
# Conclusion
* Chapel's type-level programming is surprisingly powerful.
* We can write compile-time programs that are very similar to Haskell programs.
* This allows us to write highly parameterized code without paying runtime overhead.
* This also allows us to devise powerful compile-time checks and constraints on our code.
* This approach allows for general-purpose programming, which can be applied to `your use-case`
---
<!-- _class: lead -->
# Extra Slides
---
<!-- _class: lead -->
# Linear Multi-Step Method Approximator
--- ---
# Euler's Method # Euler's Method
@ -433,210 +781,6 @@ We can now construct solvers for any explicit Adams-Bashforth method, without wr
--- ---
<!-- _class: lead -->
# Type-Safe `printf`
---
# The `printf` Function
The `printf` function accepts a format string, followed by a variable number of arguments that should match:
```C
// totally fine:
printf("Hello, %s! Your ChapelCon submission is #%d\n", "Daniel", 18);
// not good:
printf("Hello, %s! Your ChapelCon submission is #%d\n", 18, "Daniel");
```
Can we define a `printf` function in Chapel that is type-safe?
---
# Yet Another Type-Level List
- The general idea for type-safe `printf`: take the format string, and extract a list of the expected argument types.
- To make for nicer error messages, include a human-readable description of each type in the list.
- I've found it more convenient to re-define lists for various problems when needed, rather than having a single canonical list definition.
```chapel
record _nil {
proc type length param do return 0;
}
record _cons {
type expectedType; // type of the argument to printf
param name: string; // human-readable name of the type
type rest;
proc type length param do return 1 + rest.length();
}
```
---
# Extracting Types from Format Strings
```Chapel
proc specifiers(param s: string, param i: int = 0) type {
if i >= s.size then return _nil;
if s[i] == "%" {
if i + 1 >= s.size then
compilerError("Invalid format string: unterminted %");
select s[i + 1] {
when "%" do return specifiers(s, i + 2);
when "s" do return _cons(string, "a string", specifiers(s, i + 2));
when "i" do return _cons(int, "a signed integer", specifiers(s, i + 2));
when "u" do return _cons(uint, "an unsigned integer", specifiers(s, i + 2));
when "n" do return _cons(numeric, "a numeric value", specifiers(s, i + 2));
otherwise do compilerError("Invalid format string: unknown format type");
}
} else {
return specifiers(s, i + 1);
}
}
```
---
# Extracting Types from Format Strings
Let's give it a quick try:
```Chapel
writeln(specifiers("Hello, %s! Your ChapelCon submission is #%i\n") : string);
```
The above prints:
```Chapel
_cons(string,"a string",_cons(int(64),"a signed integer",_nil))
```
---
# Validating Argument Types
* The Chapel standard library has a nice `isSubtype` function that we can use to check if an argument matches the expected type.
* Suppose the `.length` of our type specifiers matches the number of arguments to `printf`
* Chapel doesn't currently support empty tuples, so if the lengths match, we know that `specifiers` is non-empty.
* Then, we can validate the types as follows:
```Chapel
proc validate(type specifiers: _cons(?t, ?s, ?rest), type argTup, param idx) {
if !isSubtype(argTup[idx], t) then
compilerError("Argument " + (idx + 1) : string + " should be " + s + " but got " + argTup[idx]:string, idx+2);
if idx + 1 < argTup.size then
validate(rest, argTup, idx + 1);
}
```
* The `idx+2` argument to `compilerError` avoids printing the recursive `validate` calls in the error message.
---
# The `fprintln` overloads
* I named it `fprintln` for "formatted print line".
* To support the empty-specifier case (Chapel varargs don't allow zero arguments):
```Chapel
proc fprintln(param format: string) where specifiers(format).length == 0 {
writeln(format);
}
```
* If we do have type specifiers, to ensure our earlier assumption of `size` matching:
```Chapel
proc fprintln(param format: string, args...)
where specifiers(format).length != args.size {
compilerError("'fprintln' with this format string expects " +
specifiers(format).length : string +
" argument(s) but got " + args.size : string);
}
```
---
# The `fprintln` overloads
* All that's left is the main `fprintln` implementation:
```Chapel
proc fprintln(param format: string, args...) {
validate(specifiers(format), args.type, 0);
writef(format + "\n", (...args));
}
```
---
# Using `fprintln`
```Chapel
fprintln("Hello, world!"); // fine, prints "Hello, world!"
fprintln("The answer is %i", 42); // fine, prints "The answer is 42"
// compiler error: Argument 3 should be a string but got int(64)
fprintln("The answer is %i %i %s", 1, 2, 3);
```
More work could be done to support more format specifiers, escapes, etc., but the basic idea is there.
---
<!-- _class: lead -->
# Beyond Lists
---
# Beyond Lists
* I made grand claims earlier
- "Write functional-ish program at the type level!"
* So far, we've just used lists and some recursion.
* Is that all there is?
---
# Algebraic Data Types
* The kinds of data types that Haskell supports are called *algebraic data types*.
* At a fundamental level, they can be built up from two operations: _Cartesian product_ and _disjoint union_.
* There are other concepts to build recursive data types, but we won't need them in Chapel.
- To prove to you I know what I'm talking about, some jargon:
_initial algebras_, _the fixedpoint functor_, _catamorphisms_...
- Check out _Bananas, Lenses, Envelopes and Barbed Wire_ by Meijer et al. for more.
* __This matters because, if Chapel has these operations, we can build any data type that Haskell can.__
---
<style scoped>
li:nth-child(3) { color: lightgrey; }
</style>
# Algebraic Data Types
- The kinds of data types that Haskell supports are called *algebraic data types*.
- At a fundamental level, they can be built up from two operations: _Cartesian product_ and _disjoint union_.
- There are other concepts to build recursive data types, but we won't need them in Chapel.
- To prove to you I know what I'm talking about, some jargon:
_initial algebras_, _the fixedpoint functor_, _catamorphisms_...
- Check out _Bananas, Lenses, Envelopes and Barbed Wire_ by Meijer et al. for more.
- __This matters because, if Chapel has these operations, we can build any data type that Haskell can.__
---
# Cartesian Product # Cartesian Product
For any two types, the _Cartesian product_ of these two types defines all pairs of values from these types. For any two types, the _Cartesian product_ of these two types defines all pairs of values from these types.
- This is like a two-element tuple _at the value level_ in Chapel. - This is like a two-element tuple _at the value level_ in Chapel.
@ -853,133 +997,3 @@ balancedOneTwoThree' = InR (2 `MkPair` (InR (1 `MkPair` (InL MkUnit `MkPair` InL
--- ---
# A General Recipe
To translate a Haskell data type definition to Chapel:
* For each constructor, define a `record` with that constructor's name
* The fields of that record are `type` fields for each argument of the constructor
- If the argument is a value (like `Int`), you can make it a `param` field instead
* A visual example, again:
<div class="side-by-side">
<div>
```Chapel
record C1 { type arg1; /* ... */ type argi; }
// ...
record Cn { type arg1; /* ... */ type argj; }
```
</div>
<div>
```Haskell
data T = C1 arg1 ... argi
| ...
| Cn arg1 ... argj
```
</div>
</div>
---
# Inserting and Looking Up in a BST
<div class="side-by-side">
<div>
```Chapel
proc insert(type t: Empty, param x: int) type do return Node(x, Empty, Empty);
proc insert(type t: Node(?v, ?left, ?right), param x: int) type do
select true {
when x < v do return Node(v, insert(left, x), right);
otherwise do return Node(v, left, insert(right, x));
}
type test = insert(insert(insert(Empty, 2), 1), 3);
proc lookup(type t: Empty, param x: int) param do return false;
proc lookup(type t: Node(?v, ?left, ?right), param x: int) param do
select true {
when x == v do return true;
when x < v do return lookup(left, x);
otherwise do return lookup(right, x);
}
```
</div>
<div>
```Haskell
insert :: Int -> BSTree -> BSTree
insert x Empty = Node x Empty Empty
insert x (Node v left right)
| x < v = Node v (insert x left) right
| otherwise = Node v left (insert x right)
test = insert 3 (insert 1 (insert 2 Empty))
lookup :: Int -> BSTree -> Bool
lookup x Empty = False
lookup x (Node v left right)
| x == v = True
| x < v = lookup x left
| otherwise = lookup x right
```
</div>
</div>
It really works!
```Chapel
writeln(test : string);
// prints Node(2,Node(1,Empty,Empty),Node(3,Empty,Empty))
writeln(lookup(test, 1));
// prints true for this one, but false for '4'
```
---
# A Key-Value Map
```Chapel
record Empty {}
record Node { param key: int; param value; type left; type right; }
proc insert(type t: Empty, param k: int, param v) type do return Node(k, v, Empty, Empty);
proc insert(type t: Node(?k, ?v, ?left, ?right), param nk: int, param nv) type do
select true {
when nk < k do return Node(k, v, insert(left, nk, nv), right);
otherwise do return Node(k, v, left, insert(right, nk, nv));
}
proc lookup(type t: Empty, param k: int) param do return "not found";
proc lookup(type t: Node(?k, ?v, ?left, ?right), param x: int) param do
select true {
when x == k do return v;
when x < k do return lookup(left, x);
otherwise do return lookup(right, x);
}
type test = insert(insert(insert(Empty, 2, "two"), 1, "one"), 3, "three");
writeln(lookup(test, 1)); // prints "one"
writeln(lookup(test, 3)); // prints "three"
writeln(lookup(test, 4)); // prints "not found"
```
---
# Conclusion
* Chapel's type-level programming is surprisingly powerful.
* We can write compile-time programs that are very similar to Haskell programs.
* This allows us to write highly parameterized code without paying runtime overhead.
* This also allows us to devise powerful compile-time checks and constraints on our code.
* This approach allows for general-purpose programming, which can be applied to `your use-case`