diff --git a/alloy/alloy-blog.png b/alloy/alloy-blog.png new file mode 100644 index 0000000..8fc64f7 Binary files /dev/null and b/alloy/alloy-blog.png differ diff --git a/alloy/slides.md b/alloy/slides.md index 5a7ce64..b54da45 100644 --- a/alloy/slides.md +++ b/alloy/slides.md @@ -29,25 +29,15 @@ Daniel Fedorin, HPE --- -# Terms - -* What are **formal methods**? - * Techniques rooted in computer science and mathematics to specify and verify systems -* What part of the **Chapel compiler**? - * The 'Dyno' compiler front-end, particularly its use/import resolution phase. - * This piece is used by the production Chapel compiler. - ---- - # The Story I have a story in three parts. 1. I found it very hard to think through a section of compiler code. - - Specifically, code that performed lookups in `use`s/`import`s + - Specifically, code that performed scope lookups for variables 2. I used the [Alloy Analyzer](https://alloytools.org/) to model the assumptions and behavior of the code. - - I had little background in Alloy, but some background in (formal) logic + - I had little background in Alloy, but some background in formal logic 3. This led me to discover a bug in the compiler that I then fixed. - Re-creating the bug required some gymnastics that were unlikely to occur in practice. @@ -66,11 +56,388 @@ Imagine you see the following snippet of Chapel code: foo(); ``` -Where do you look for `foo`? The answer is quite complicated, and depends -strongly on the context of the call. +Where do you look for `foo`? The answer is quite complicated, and depends strongly on the context of the call. + +Moreover, the order of where to look matters: method calls are preferred over global functions, "nearer" functions are preferred over "farther" ones. + +--- + +# The Humble `foo` (example 1) + +```chapel +module M1 { + record R {} + + proc R.foo() { writeln("R.foo"); } + proc foo() { writeln("M1.foo"); } + + proc R.someMethod() { + foo(); // which 'foo'? + } +} +``` + +* Both `R.foo` and `M1.foo` would be valid candidates. +* We give priority to methods over global functions. So, the compiler would: + * Search `R` and its scope (`M1`) for methods named `foo`. + * If that fails, search `M1` for any symbols `foo`. +* We've had to look at `M1` twice! (once for methods, once for non-methods) + +--- + +# The Humble `foo` (example 2) + +```chapel +module M1 { + record R {} + + proc foo() { writeln("R.foo"); } +} +module M2 { + use M1; + + proc R.someMethod() { + foo(); // which 'foo'? + } +} +``` + +Here, we search the scope of `R` and `M1`, but **only for public symbols**. + +--- + + +# How Chapel's Compiler Handles This + +We want to: +- Respect the priority order + - Including preferring methods over non-methods + - As a result, we search the scopes multiple times +- Avoid any extra work + - This includes redundant re-searches + - Example redundant search: looked up "all symbols", then later "all public symbols" + +--- + +# Encoding Search Configuration + +```C++ +enum { PUBLIC = 1, NOT_PUBLIC = 2, METHOD_FIELD = 4, NOT_METHOD_FIELD = 8, /* ... */ }; +``` + +1. For each scope, save the flags we've already searched with +2. When searching a scope again, exclude the flags we've already searched with + +This was handled by two bitfields: `filter` and `excludeFilter`. + +--- + +# Populating `filter` and `excludeFilter` + +```C++ +if (skipPrivateVisibilities) { // depends on context of 'foo()' + filter |= IdAndFlags::PUBLIC; +} + +else if (!includeMethods && receiverScopes.empty()) { + filter |= IdAndFlags::NOT_METHOD; +} +``` + +```C++ +excludeFilter = previousFilter; +``` + +```C++ +// scary! +previousFilter = filter & previousFilter; +``` + +Code notes `previousFilter` is an approximation. + +--- +# Possible Problems + +* `previousFilter` is an approximation. +* However, no case we knew of hit this combination of searches, or any like it. + * All of our language tests passed. + * Code seemed to work. +* If only there was a way I could get a computer to check whether such a combination + could occur... + +--- + + +# Formal Methods + +--- + +# Model Checking + +Model checking involves formally describing the behavior of a system, then having a solver check whether other desired properties hold. + - Alloy is an example of a model checker. + - TLA is another famous example. + +--- + +# A Primer on Logic (example) + +Model checkers like Alloy are rooted in temporal logic, which builds on first-order logic. + +Example statement: "Bob has a son who likes all compilers". + +$$ +\exists x. (\text{Son}(x, \text{Bob}) \land \forall y. (\text{Compiler}(y) \Rightarrow \text{Likes}(x, y))) +$$ + +In Alloy: + +```alloy +some x { Son[x, Bob] and all y { Compiler[y] implies Likes[x, y] } } +``` + +--- + +# A Primer on Temporal Logic + +Temporal logic provides additional operators to reason about how properties change over time. + +- $\Box p$ (in Alloy: `always p`): A statement that is always true. + - Example: $\Box(\text{like charges repel})$ +- $\Diamond p$ (in Alloy: `eventually p`) : A statement that will be true at some point in the future. + - Example: $\Diamond(\text{the sun is in the sky})$ + +In Alloy specifically, we can mention the next state of a variable using `'`. + +```alloy +// pseudocode: +// the next future value of previousFilter will be the intersection of filter +// and the current value +previousFilter' = filter & previousFilter; +``` + +--- + +# Modeling Possible Searches + +Alloy isn't an imperative language. We can't mutate variables like we do in C++. Instead, we model how each statement changes the state, by relating the "current" state to the "next" state. + +
+
+ +```C++ + filter |= IdAndFlags::PUBLIC; +``` + +
+
+ +```alloy + addBitfieldFlag[filterNow, filterNext, Public] +``` + +
+
+ +This might remind you of [Hoare Logic](https://en.wikipedia.org/wiki/Hoare_logic), where statements like: + +$$ + \{ P \} \; s \; \{ Q \} +$$ + +Read as: + +> If $P$ is true before executing $s$, then $Q$ will be true after executing $s$. + +$$ + \{ \text{filter} = \text{filterNow} \} \; \texttt{filter |= PUBLIC} \; \{ \text{filter} = \text{filterNext} \} +$$ + +--- + +# Modeling Possible Searches + +To combine several statements, we make it so that the "next" state of one statement is the "current" state of the next statement. + +
+
+ +```C++ + curFilter |= IdAndFlags::PUBLIC; + curFilter |= IdAndFlags::METHOD_FIELD; +``` + +
+
+ +```alloy + addBitfieldFlag[filterNow, filterNext1, Public] + addBitfieldFlag[filterNext1, filterNext2, Method] +``` + +
+
+ +This is reminiscent of sequencing Hoare triples: + +$$ + \{ P \} \; s_1 \; \{ Q \} \; s_2 \; \{ R \} +$$ + +--- + +# Modeling Possible Searches + +Finally, if C++ code has conditionals, we need to allow for the possibility of either branch being taken. We do this by using "or" on descriptions of the next state. + +
+
+ +```C++ + if (skipPrivateVisibilities) { + curFilter |= IdAndFlags::PUBLIC; + } + + if (!includeMethods && receiverScopes.empty()) { + curFilter |= IdAndFlags::NOT_METHOD; + } +``` +
+
+ +```alloy + addBitfieldFlag[initialState.curFilter, bitfieldMiddle, Public] or + bitfieldEqual[initialState.curFilter, bitfieldMiddle] + + + // if it's not a receiver, filter to non-methods (could be overridden) + addBitfieldFlagNeg[bitfieldMiddle, filterState.curFilter, Method] or + bitfieldEqual[bitfieldMiddle, filterState.curFilter] +``` +
+
+ +Putting this into a predicate, `possibleState`, we encode what searches the compiler can undertake. + +**Takeaway**: We encoded the logic that configures possible searches in Alloy. This instructs the analyzer about possible cases to consider. + +--- + +# Are there any bugs? + +Model checkers ensure that all properties we want to hold, do hold. To find a counter example, we ask it to prove the negation of what we want. + +```C +wontFindNeeded: run { + all searchState: SearchState { + eventually some props: Props, fs: FilterState, fsBroken: FilterState { + // Some search (fs) will cause a transition / modification of the search state... + configureState[fs] + updateOrSet[searchState.previousFilter, fs] + // Such that a later, valid search... (fsBroken) + configureState[fsBroken] + + // Will allow for a set of properties... + // ... that are left out of the original search... + not bitfieldMatchesProperties[searchState.previousFilter, props] + // ... and out of the current search + not (bitfieldMatchesProperties[fs.include, props] and not bitfieldMatchesProperties[searchState.previousFilter, props]) + // But would be matched by the broken search... + bitfieldMatchesProperties[fsBroken.include, props] + // ... to not be matched by a search with the new state: + not (bitfieldMatchesProperties[fsBroken.include, props] and not bitfieldOrNotSetMatchesProperties[searchState.previousFilter', props]) + } + } +} +``` + +--- +# Uh-Oh! + +![](./instancefound.png) + +--- +# The Bug + +![bg left width:600px](./bug.png) + +We need some gymnastics to figure out what variables make this model possible. + +Alloy has a nice visualizer, but it has a lot of information. + +In the interest of time, I found: + +* If the compiler searches a scope first for `PUBLIC` symbols, ... +* ...then for `METHOD_OR_FIELD`, ... +* ...then for any symbols, they will miss things! + +--- + + +# The Reproducer +To trigger this sequence of searches, we needed a lot more gymnastics. + +
+
+ +```chapel +module TopLevel { + module XContainerUser { + public use TopLevel.XContainer; + } + module XContainer { + private var x: int; + record R {} + module MethodHaver { + use TopLevel.XContainerUser; + use TopLevel.XContainer; + proc R.foo() { + var y = x; + } + } + } +} +``` +
+
+ +* the scope of `R` is searched with for methods +* The scope of `R`’s parent (`XContainer`) is searched for methods +* The scope of `XContainerUser` is searched for public symbols (via the `use`) +* The scope of `XContainer` is searched with public symbols (via the `public use`) +* The scope of `XContainer` searched for with no filters via the second use; but the stored filter is bad, so the lookup returns early, not finding `x`. + +
+
+ +--- + + +# **Thank you!** + +![](./alloy-blog.png) + +Read more + +--- + + +# Extra Slides + +--- + +# Terms + +* What are **formal methods**? + * Techniques rooted in computer science and mathematics to specify and verify systems +* What part of the **Chapel compiler**? + * The 'Dyno' compiler front-end, particularly its scope lookup phase + * This piece is used by the production Chapel compiler. -Moreover, the order of where to look matters: method calls are preferred -over global functions, "nearer" functions are preferred over "farther" ones. --- @@ -155,96 +522,6 @@ Here, we search the scope of `R` and `M1`, but **only for public symbols**. --- - -# How Chapel's Compiler Handles This - -We want to: -- Respect the priority order - - Including preferring methods over non-methods - - As a result, we search the scopes multiple times -- Avoid any extra work - - This includes redundant re-searches - - Example redundant search: looked up "all symbols", then later "all public symbols" - -```C++ -enum { PUBLIC = 1, NOT_PUBLIC = 2, METHOD_FIELD = 4, NOT_METHOD_FIELD = 8, /* ... */ }; -``` - -1. For each scope, save the flags we've already searched with -2. When searching a scope again, exclude the flags we've already searched with - -This was handled by two bitfields: `filter` and `excludeFilter`. - ---- - -# Populating `filter` and `excludeFilter` - -```C++ -if (skipPrivateVisibilities) { // depends on context of 'foo()' - filter |= IdAndFlags::PUBLIC; -} - -if (onlyMethodsFields) { - filter |= IdAndFlags::METHOD_FIELD; - -} else if (!includeMethods && receiverScopes.empty()) { - filter |= IdAndFlags::NOT_METHOD; -} -``` - -```C++ -excludeFilter = previousFilter; -``` - -```C++ -// scary! -previousFilter = filter & previousFilter; -``` - -Code notes `previousFilter` is an approximation. - ---- -# Possible Problems - -* `previousFilter` is an approximation. -* However, no case we knew of hit this combination of searches, or any like it. - * All of our language tests passed. - * Code seemed to work. -* If only there was a way I could get a computer to check whether such a combination - could occur... - ---- - - -# Formal Methods - ---- -# Types of Formal Methods - -- Model checking involves formally describing the behavior of a system, then having a solver check whether other desired properties hold. - - Alloy is an example of a model checker. - - TLA is another famous example. -- Theorem proving is a heavier weight approach that involves building a formal proof of correctness. - - Coq and Isabelle are examples of theorem provers. ---- - -# Types of Formal Methods - -- Model checking involves formally describing the behavior of a system, then having a solver check whether other desired properties hold. - - Alloy is an example of a model checker. - - TLA is another famous example. -- Theorem proving is a heavier weight approach that involves building a formal proof of correctness. - - Coq and Isabelle are examples of theorem provers. - - - -**Reason**: I was in the middle of developing compiler code. I wanted to sketch -the assumptions I was making and see if they held up. - ---- - # A Primer on Logic Model checkers like Alloy are rooted in temporal logic, which builds on @@ -263,40 +540,6 @@ first-order logic. This includes: --- -# A Primer on Logic (example) - -Example statement: "Bob has a son who likes all compilers". - -$$ -\exists x. (\text{Son}(x, \text{Bob}) \land \forall y. (\text{Compiler}(y) \Rightarrow \text{Likes}(x, y))) -$$ - -In Alloy: - -```alloy -some x { Son[x, Bob] and all y { Compiler[y] implies Likes[x, y] } } -``` - ---- - -# A Primer on Temporal Logic - -Temporal logic provides additional operators to reason about how properties change over time. - -- $\Box p$ (in Alloy: `always p`): A statement that is always true. -- $\Diamond p$ (in Alloy: `eventually p`) : A statement that will be true at some point in the future. - -In Alloy specifically, we can mention the next state of a variable using `'`. - -```alloy -// pseudocode: -// the next future value of previousFilter will be the intersection of filter -// and the current value -previousFilter' = filter & previousFilter; -``` - ---- - # A Primer on Temporal Logic (example) Some examples: @@ -387,131 +630,6 @@ pred bitfieldEqual[b1: Bitfield, b2: Bitfield] { --- -# Modeling Possible Searches - -Alloy isn't an imperative language. We can't mutate variables like we do in C++. Instead, we model how each statement changes the state, by relating the "current" state to the "next" state. - -
-
- -```C++ - filter |= IdAndFlags::PUBLIC; -``` - -
-
- -```alloy - addBitfieldFlag[filterNow, filterNext, Public] -``` - -
-
- -This might remind you of [Hoare Logic](https://en.wikipedia.org/wiki/Hoare_logic), where statements like: - -$$ - \{ P \} \; s \; \{ Q \} -$$ - -Read as: - -> If $P$ is true before executing $s$, then $Q$ will be true after executing $s$. - -$$ - \{ \text{filter} = \text{filterNow} \} \; \texttt{filter |= PUBLIC} \; \{ \text{filter} = \text{filterNext} \} -$$ - ---- - -# Modeling Possible Searches - -To combine several statements, we make it so that the "next" state of one statement is the "current" state of the next statement. - -
-
- -```C++ - curFilter |= IdAndFlags::PUBLIC; - curFilter |= IdAndFlags::METHOD_FIELD; -``` - -
-
- -```alloy - addBitfieldFlag[filterNow, filterNext1, Public] - addBitfieldFlag[filterNext1, filterNext2, Method] -``` - -
-
- -This is reminiscent of sequencing Hoare triples: - -$$ - \{ P \} \; s_1 \; \{ Q \} \; s_2 \; \{ R \} -$$ - ---- - -# Modeling Possible Searches - -Finally, if C++ code has conditionals, we need to allow for the possibility of either branch being taken. We do this by using "or" on descriptions of the next state. - -
-
- -```C++ - if (skipPrivateVisibilities) { - curFilter |= IdAndFlags::PUBLIC; - } - - - - - - - if (onlyMethodsFields) { - curFilter |= IdAndFlags::METHOD_FIELD; - } else if (!includeMethods && receiverScopes.empty()) { - curFilter |= IdAndFlags::NOT_METHOD; - } - - - -``` -
-
- -```alloy - addBitfieldFlag[initialState.curFilter, bitfieldMiddle, Public] or - bitfieldEqual[initialState.curFilter, bitfieldMiddle] - - - - - - - - // If it's a method receiver, add method or field restriction - addBitfieldFlag[bitfieldMiddle, filterState.curFilter, MethodOrField] or - - // if it's not a receiver, filter to non-methods (could be overridden) - addBitfieldFlagNeg[bitfieldMiddle, filterState.curFilter, Method] or - - // Maybe methods are not being curFilterd but it's not a receiver, so no change. - bitfieldEqual[bitfieldMiddle, filterState.curFilter] -``` -
-
- -Putting this into a predicate, `possibleState`, we encode what searches the compiler can undertake. - -**Takeaway**: We encoded the logic that configures possible searches in Alloy. This instructs the analyzer about possible cases to consider. - ---- - # Modeling `previousFilter` So far, all we've done is encoded what queries the compiler might make about a scope. @@ -531,6 +649,55 @@ one sig SearchState { Above, `+` is used for union. `previousFilter` can either be a `Bitfield` or `NotSet`. +--- +# Types of Formal Methods + +- Model checking involves formally describing the behavior of a system, then having a solver check whether other desired properties hold. + - Alloy is an example of a model checker. + - TLA is another famous example. +- Theorem proving is a heavier weight approach that involves building a formal proof of correctness. + - Coq and Isabelle are examples of theorem provers. +--- + +# Types of Formal Methods + +- Model checking involves formally describing the behavior of a system, then having a solver check whether other desired properties hold. + - Alloy is an example of a model checker. + - TLA is another famous example. +- Theorem proving is a heavier weight approach that involves building a formal proof of correctness. + - Coq and Isabelle are examples of theorem provers. + + + +**Reason**: I was in the middle of developing compiler code. I wanted to sketch +the assumptions I was making and see if they held up. + +--- + +# Putting it Together + +We now have a model of what our C++ program is doing: it computes some set of filter flags, then runs a search, excluding the previous flags. It then updates the previous flags with the current search. We can encode this as follows: + +``` +fact step { + always { + // Model that a new doLookupInScope could've occurred, with any combination of flags. + all searchState: SearchState { + some fs: FilterState { + // This is a possible combination of lookup flags + possibleState[fs] + + // If a search has been performed before, take the intersection; otherwise, + // just insert the current filter flags. + updateOrSet[searchState.previousFilter, fs] + } + } + } +} +``` + --- # Modeling `previousFilter` @@ -566,120 +733,3 @@ Otherwise, we set `previousFilter` to the intersection of `filter` and `previous - ---- - -# Putting it Together - -We now have a model of what our C++ program is doing: it computes some set of filter flags, then runs a search, excluding the previous flags. It then updates the previous flags with the current search. We can encode -this as follows: - -``` -fact step { - always { - // Model that a new doLookupInScope could've occurred, with any combination of flags. - all searchState: SearchState { - some fs: FilterState { - // This is a possible combination of lookup flags - possibleState[fs] - - // If a search has been performed before, take the intersection; otherwise, - // just insert the current filter flags. - updateOrSet[searchState.previousFilter, fs] - } - } - } -} -``` - ---- -# Are there any bugs? - -Model checkers ensure that all properties we want to hold, do hold. To find a counter example, we ask it to prove the negation of what we want. - -```C -wontFindNeeded: run { - all searchState: SearchState { - eventually some props: Props, fs: FilterState, fsBroken: FilterState { - // Some search (fs) will cause a transition / modification of the search state... - configureState[fs] - updateOrSet[searchState.previousFilter, fs] - // Such that a later, valid search... (fsBroken) - configureState[fsBroken] - - // Will allow for a set of properties... - // ... that are left out of the original search... - not bitfieldMatchesProperties[searchState.previousFilter, props] - // ... and out of the current search - not (bitfieldMatchesProperties[fs.include, props] and not bitfieldMatchesProperties[searchState.previousFilter, props]) - // But would be matched by the broken search... - bitfieldMatchesProperties[fsBroken.include, props] - // ... to not be matched by a search with the new state: - not (bitfieldMatchesProperties[fsBroken.include, props] and not bitfieldOrNotSetMatchesProperties[searchState.previousFilter', props]) - } - } -} -``` - ---- -# Uh-Oh! - -![](./instancefound.png) - ---- -# The Bug - -![bg left width:600px](./bug.png) - -We need some gymnastics to figure out what variables make this model possible. - -Alloy has a nice visualizer, but it has a lot of information. - -In the interest of time, I found: - -* If the compiler searches a scope first for `PUBLIC` symbols, ... -* ...then for `METHOD_OR_FIELD`, ... -* ...then for any symbols, they will miss things! - ---- - - -# The Reproducer -To trigger this sequence of searches, we needed a lot more gymnastics. - -
-
- -```chapel -module TopLevel { - module XContainerUser { - public use TopLevel.XContainer; - } - module XContainer { - private var x: int; - record R {} - module MethodHaver { - use TopLevel.XContainerUser; - use TopLevel.XContainer; - proc R.foo() { - var y = x; - } - } - } -} -``` -
-
- -* the scope of `R` is searched with for methods -* The scope of `R`’s parent (`XContainer`) is searched for methods -* The scope of `XContainerUser` is searched for public symbols (via the `use`) -* The scope of `XContainer` is searched with public symbols (via the `public use`) -* The scope of `XContainer` searched for with no filters via the second use; but the stored filter is bad, so the lookup returns early, not finding `x`. - -
-
diff --git a/type-level/linear-multistep.png b/type-level/linear-multistep.png new file mode 100644 index 0000000..b88f763 Binary files /dev/null and b/type-level/linear-multistep.png differ diff --git a/type-level/slides.md b/type-level/slides.md index 5ed2ba4..71d8835 100644 --- a/type-level/slides.md +++ b/type-level/slides.md @@ -235,6 +235,354 @@ There is no runtime overhead! # Linear Multi-Step Method Approximator +![](./linear-multistep.png) + +--- + + +# Type-Safe `printf` + +--- + +# The `printf` Function + +The `printf` function accepts a format string, followed by a variable number of arguments that should match: + +```C +// totally fine: +printf("Hello, %s! Your ChapelCon submission is #%d\n", "Daniel", 18); + +// not good: +printf("Hello, %s! Your ChapelCon submission is #%d\n", 18, "Daniel"); +``` + +Can we define a `printf` function in Chapel that is type-safe? + +--- + +# Yet Another Type-Level List + +- The general idea for type-safe `printf`: take the format string, and extract a list of the expected argument types. + +- To make for nicer error messages, include a human-readable description of each type in the list. + +- I've found it more convenient to re-define lists for various problems when needed, rather than having a single canonical list definition. + +```chapel +record _nil { + proc type length param do return 0; +} +record _cons { + type expectedType; // type of the argument to printf + param name: string; // human-readable name of the type + type rest; + + proc type length param do return 1 + rest.length(); +} +``` + +--- + +# Extracting Types from Format Strings + +```Chapel +proc specifiers(param s: string, param i: int = 0) type { + if i >= s.size then return _nil; + + if s[i] == "%" { + if i + 1 >= s.size then + compilerError("Invalid format string: unterminted %"); + + select s[i + 1] { + when "%" do return specifiers(s, i + 2); + when "s" do return _cons(string, "a string", specifiers(s, i + 2)); + when "i" do return _cons(int, "a signed integer", specifiers(s, i + 2)); + when "u" do return _cons(uint, "an unsigned integer", specifiers(s, i + 2)); + when "n" do return _cons(numeric, "a numeric value", specifiers(s, i + 2)); + otherwise do compilerError("Invalid format string: unknown format type"); + } + } else { + return specifiers(s, i + 1); + } +} +``` + +--- + +# Extracting Types from Format Strings + +Let's give it a quick try: + +```Chapel +writeln(specifiers("Hello, %s! Your ChapelCon submission is #%i\n") : string); +``` + +The above prints: + +```Chapel +_cons(string,"a string",_cons(int(64),"a signed integer",_nil)) +``` + +--- + +# Validating Argument Types + +* The Chapel standard library has a nice `isSubtype` function that we can use to check if an argument matches the expected type. + +* Suppose the `.length` of our type specifiers matches the number of arguments to `printf` + +* Chapel doesn't currently support empty tuples, so if the lengths match, we know that `specifiers` is non-empty. + +* Then, we can validate the types as follows: + ```Chapel + proc validate(type specifiers: _cons(?t, ?s, ?rest), type argTup, param idx) { + if !isSubtype(argTup[idx], t) then + compilerError("Argument " + (idx + 1) : string + " should be " + s + " but got " + argTup[idx]:string, idx+2); + + if idx + 1 < argTup.size then + validate(rest, argTup, idx + 1); + } + ``` + +* The `idx+2` argument to `compilerError` avoids printing the recursive `validate` calls in the error message. + +--- + +# The `fprintln` overloads + +* I named it `fprintln` for "formatted print line". + +* To support the empty-specifier case (Chapel varargs don't allow zero arguments): + + ```Chapel + proc fprintln(param format: string) where specifiers(format).length == 0 { + writeln(format); + } + ``` +* If we do have type specifiers, to ensure our earlier assumption of `size` matching: + ```Chapel + proc fprintln(param format: string, args...) + where specifiers(format).length != args.size { + compilerError("'fprintln' with this format string expects " + + specifiers(format).length : string + + " argument(s) but got " + args.size : string); + } + ``` + +--- + +# The `fprintln` overloads + +* All that's left is the main `fprintln` implementation: + + ```Chapel + proc fprintln(param format: string, args...) { + validate(specifiers(format), args.type, 0); + + writef(format + "\n", (...args)); + } + ``` + +--- + +# Using `fprintln` + +```Chapel +fprintln("Hello, world!"); // fine, prints "Hello, world!" +fprintln("The answer is %i", 42); // fine, prints "The answer is 42" + +// compiler error: Argument 3 should be a string but got int(64) +fprintln("The answer is %i %i %s", 1, 2, 3); +``` + +More work could be done to support more format specifiers, escapes, etc., but the basic idea is there. + +--- + + + +# Beyond Lists + +--- + +# Beyond Lists + +* I made grand claims earlier + - "Write functional-ish program at the type level!" +* So far, we've just used lists and some recursion. +* Is that all there is? + +--- + +# Algebraic Data Types + +* The kinds of data types that Haskell supports are called *algebraic data types*. +* At a fundamental level, they can be built up from two operations: _Cartesian product_ and _disjoint union_. +* There are other concepts to build recursive data types, but we won't need them in Chapel. + - To prove to you I know what I'm talking about, some jargon: + _initial algebras_, _the fixedpoint functor_, _catamorphisms_... + - Check out _Bananas, Lenses, Envelopes and Barbed Wire_ by Meijer et al. for more. +* __Claim__: Chapel supports disjoint union and Cartesian product, so we can build any data type that Haskell can. + +--- + + + +# Algebraic Data Types + +- The kinds of data types that Haskell supports are called *algebraic data types*. +- At a fundamental level, they can be built up from two operations: _Cartesian product_ and _disjoint union_. +- There are other concepts to build recursive data types, but we won't need them in Chapel. + - To prove to you I know what I'm talking about, some jargon: + _initial algebras_, _the fixedpoint functor_, _catamorphisms_... + - Check out _Bananas, Lenses, Envelopes and Barbed Wire_ by Meijer et al. for more. +- __Claim__: Haskell supports disjoint union and Cartesian product, so we can build any data type that Haskell can. + +--- + +# A General Recipe + +To translate a Haskell data type definition to Chapel: + +* For each constructor, define a `record` with that constructor's name +* The fields of that record are `type` fields for each argument of the constructor + - If the argument is a value (like `Int`), you can make it a `param` field instead +* A visual example, again: + +
+
+ + ```Chapel + record C1 { type arg1; /* ... */ type argi; } + // ... + record Cn { type arg1; /* ... */ type argj; } + ``` +
+
+ + ```Haskell + data T = C1 arg1 ... argi + | ... + | Cn arg1 ... argj + ``` +
+
+ +--- + +# Inserting and Looking Up in a BST + +
+
+ +```Chapel + +proc insert(type t: Empty, param x: int) type do return Node(x, Empty, Empty); +proc insert(type t: Node(?v, ?left, ?right), param x: int) type do + select true { + when x < v do return Node(v, insert(left, x), right); + otherwise do return Node(v, left, insert(right, x)); + } + +type test = insert(insert(insert(Empty, 2), 1), 3); + + + +proc lookup(type t: Empty, param x: int) param do return false; +proc lookup(type t: Node(?v, ?left, ?right), param x: int) param do + select true { + when x == v do return true; + when x < v do return lookup(left, x); + otherwise do return lookup(right, x); + } +``` +
+
+ +```Haskell +insert :: Int -> BSTree -> BSTree +insert x Empty = Node x Empty Empty +insert x (Node v left right) + + | x < v = Node v (insert x left) right + | otherwise = Node v left (insert x right) + + +test = insert 3 (insert 1 (insert 2 Empty)) + + +lookup :: Int -> BSTree -> Bool +lookup x Empty = False +lookup x (Node v left right) + + | x == v = True + | x < v = lookup x left + | otherwise = lookup x right + +``` +
+
+ +It really works! + +```Chapel +writeln(test : string); +// prints Node(2,Node(1,Empty,Empty),Node(3,Empty,Empty)) + +writeln(lookup(test, 1)); +// prints true for this one, but false for '4' +``` + +--- + +# A Key-Value Map +```Chapel +record Empty {} +record Node { param key: int; param value; type left; type right; } + +proc insert(type t: Empty, param k: int, param v) type do return Node(k, v, Empty, Empty); +proc insert(type t: Node(?k, ?v, ?left, ?right), param nk: int, param nv) type do + select true { + when nk < k do return Node(k, v, insert(left, nk, nv), right); + otherwise do return Node(k, v, left, insert(right, nk, nv)); + } + +proc lookup(type t: Empty, param k: int) param do return "not found"; +proc lookup(type t: Node(?k, ?v, ?left, ?right), param x: int) param do + select true { + when x == k do return v; + when x < k do return lookup(left, x); + otherwise do return lookup(right, x); + } + +type test = insert(insert(insert(Empty, 2, "two"), 1, "one"), 3, "three"); +writeln(lookup(test, 1)); // prints "one" +writeln(lookup(test, 3)); // prints "three" +writeln(lookup(test, 4)); // prints "not found" +``` + +--- + +# Conclusion + +* Chapel's type-level programming is surprisingly powerful. +* We can write compile-time programs that are very similar to Haskell programs. +* This allows us to write highly parameterized code without paying runtime overhead. +* This also allows us to devise powerful compile-time checks and constraints on our code. +* This approach allows for general-purpose programming, which can be applied to `your use-case` + +--- + + +# Extra Slides + +--- + + +# Linear Multi-Step Method Approximator + --- # Euler's Method @@ -433,210 +781,6 @@ We can now construct solvers for any explicit Adams-Bashforth method, without wr --- - - -# Type-Safe `printf` - ---- - -# The `printf` Function - -The `printf` function accepts a format string, followed by a variable number of arguments that should match: - -```C -// totally fine: -printf("Hello, %s! Your ChapelCon submission is #%d\n", "Daniel", 18); - -// not good: -printf("Hello, %s! Your ChapelCon submission is #%d\n", 18, "Daniel"); -``` - -Can we define a `printf` function in Chapel that is type-safe? - ---- - -# Yet Another Type-Level List - -- The general idea for type-safe `printf`: take the format string, and extract a list of the expected argument types. - -- To make for nicer error messages, include a human-readable description of each type in the list. - -- I've found it more convenient to re-define lists for various problems when needed, rather than having a single canonical list definition. - -```chapel -record _nil { - proc type length param do return 0; -} -record _cons { - type expectedType; // type of the argument to printf - param name: string; // human-readable name of the type - type rest; - - proc type length param do return 1 + rest.length(); -} -``` - ---- - -# Extracting Types from Format Strings - -```Chapel -proc specifiers(param s: string, param i: int = 0) type { - if i >= s.size then return _nil; - - if s[i] == "%" { - if i + 1 >= s.size then - compilerError("Invalid format string: unterminted %"); - - select s[i + 1] { - when "%" do return specifiers(s, i + 2); - when "s" do return _cons(string, "a string", specifiers(s, i + 2)); - when "i" do return _cons(int, "a signed integer", specifiers(s, i + 2)); - when "u" do return _cons(uint, "an unsigned integer", specifiers(s, i + 2)); - when "n" do return _cons(numeric, "a numeric value", specifiers(s, i + 2)); - otherwise do compilerError("Invalid format string: unknown format type"); - } - } else { - return specifiers(s, i + 1); - } -} -``` - ---- - -# Extracting Types from Format Strings - -Let's give it a quick try: - -```Chapel -writeln(specifiers("Hello, %s! Your ChapelCon submission is #%i\n") : string); -``` - -The above prints: - -```Chapel -_cons(string,"a string",_cons(int(64),"a signed integer",_nil)) -``` - ---- - -# Validating Argument Types - -* The Chapel standard library has a nice `isSubtype` function that we can use to check if an argument matches the expected type. - -* Suppose the `.length` of our type specifiers matches the number of arguments to `printf` - -* Chapel doesn't currently support empty tuples, so if the lengths match, we know that `specifiers` is non-empty. - -* Then, we can validate the types as follows: - ```Chapel - proc validate(type specifiers: _cons(?t, ?s, ?rest), type argTup, param idx) { - if !isSubtype(argTup[idx], t) then - compilerError("Argument " + (idx + 1) : string + " should be " + s + " but got " + argTup[idx]:string, idx+2); - - if idx + 1 < argTup.size then - validate(rest, argTup, idx + 1); - } - ``` - -* The `idx+2` argument to `compilerError` avoids printing the recursive `validate` calls in the error message. - ---- - -# The `fprintln` overloads - -* I named it `fprintln` for "formatted print line". - -* To support the empty-specifier case (Chapel varargs don't allow zero arguments): - - ```Chapel - proc fprintln(param format: string) where specifiers(format).length == 0 { - writeln(format); - } - ``` -* If we do have type specifiers, to ensure our earlier assumption of `size` matching: - ```Chapel - proc fprintln(param format: string, args...) - where specifiers(format).length != args.size { - compilerError("'fprintln' with this format string expects " + - specifiers(format).length : string + - " argument(s) but got " + args.size : string); - } - ``` - ---- - -# The `fprintln` overloads - -* All that's left is the main `fprintln` implementation: - - ```Chapel - proc fprintln(param format: string, args...) { - validate(specifiers(format), args.type, 0); - - writef(format + "\n", (...args)); - } - ``` - ---- - -# Using `fprintln` - -```Chapel -fprintln("Hello, world!"); // fine, prints "Hello, world!" -fprintln("The answer is %i", 42); // fine, prints "The answer is 42" - -// compiler error: Argument 3 should be a string but got int(64) -fprintln("The answer is %i %i %s", 1, 2, 3); -``` - -More work could be done to support more format specifiers, escapes, etc., but the basic idea is there. - ---- - - - -# Beyond Lists - ---- - -# Beyond Lists - -* I made grand claims earlier - - "Write functional-ish program at the type level!" -* So far, we've just used lists and some recursion. -* Is that all there is? - ---- - -# Algebraic Data Types - -* The kinds of data types that Haskell supports are called *algebraic data types*. -* At a fundamental level, they can be built up from two operations: _Cartesian product_ and _disjoint union_. -* There are other concepts to build recursive data types, but we won't need them in Chapel. - - To prove to you I know what I'm talking about, some jargon: - _initial algebras_, _the fixedpoint functor_, _catamorphisms_... - - Check out _Bananas, Lenses, Envelopes and Barbed Wire_ by Meijer et al. for more. -* __This matters because, if Chapel has these operations, we can build any data type that Haskell can.__ - ---- - - - -# Algebraic Data Types - -- The kinds of data types that Haskell supports are called *algebraic data types*. -- At a fundamental level, they can be built up from two operations: _Cartesian product_ and _disjoint union_. -- There are other concepts to build recursive data types, but we won't need them in Chapel. - - To prove to you I know what I'm talking about, some jargon: - _initial algebras_, _the fixedpoint functor_, _catamorphisms_... - - Check out _Bananas, Lenses, Envelopes and Barbed Wire_ by Meijer et al. for more. -- __This matters because, if Chapel has these operations, we can build any data type that Haskell can.__ - ---- - # Cartesian Product For any two types, the _Cartesian product_ of these two types defines all pairs of values from these types. - This is like a two-element tuple _at the value level_ in Chapel. @@ -853,133 +997,3 @@ balancedOneTwoThree' = InR (2 `MkPair` (InR (1 `MkPair` (InL MkUnit `MkPair` InL --- -# A General Recipe - -To translate a Haskell data type definition to Chapel: - -* For each constructor, define a `record` with that constructor's name -* The fields of that record are `type` fields for each argument of the constructor - - If the argument is a value (like `Int`), you can make it a `param` field instead -* A visual example, again: - -
-
- - ```Chapel - record C1 { type arg1; /* ... */ type argi; } - // ... - record Cn { type arg1; /* ... */ type argj; } - ``` -
-
- - ```Haskell - data T = C1 arg1 ... argi - | ... - | Cn arg1 ... argj - ``` -
-
- ---- - -# Inserting and Looking Up in a BST - -
-
- -```Chapel - -proc insert(type t: Empty, param x: int) type do return Node(x, Empty, Empty); -proc insert(type t: Node(?v, ?left, ?right), param x: int) type do - select true { - when x < v do return Node(v, insert(left, x), right); - otherwise do return Node(v, left, insert(right, x)); - } - -type test = insert(insert(insert(Empty, 2), 1), 3); - - - -proc lookup(type t: Empty, param x: int) param do return false; -proc lookup(type t: Node(?v, ?left, ?right), param x: int) param do - select true { - when x == v do return true; - when x < v do return lookup(left, x); - otherwise do return lookup(right, x); - } -``` -
-
- -```Haskell -insert :: Int -> BSTree -> BSTree -insert x Empty = Node x Empty Empty -insert x (Node v left right) - - | x < v = Node v (insert x left) right - | otherwise = Node v left (insert x right) - - -test = insert 3 (insert 1 (insert 2 Empty)) - - -lookup :: Int -> BSTree -> Bool -lookup x Empty = False -lookup x (Node v left right) - - | x == v = True - | x < v = lookup x left - | otherwise = lookup x right - -``` -
-
- -It really works! - -```Chapel -writeln(test : string); -// prints Node(2,Node(1,Empty,Empty),Node(3,Empty,Empty)) - -writeln(lookup(test, 1)); -// prints true for this one, but false for '4' -``` - ---- - -# A Key-Value Map -```Chapel -record Empty {} -record Node { param key: int; param value; type left; type right; } - -proc insert(type t: Empty, param k: int, param v) type do return Node(k, v, Empty, Empty); -proc insert(type t: Node(?k, ?v, ?left, ?right), param nk: int, param nv) type do - select true { - when nk < k do return Node(k, v, insert(left, nk, nv), right); - otherwise do return Node(k, v, left, insert(right, nk, nv)); - } - -proc lookup(type t: Empty, param k: int) param do return "not found"; -proc lookup(type t: Node(?k, ?v, ?left, ?right), param x: int) param do - select true { - when x == k do return v; - when x < k do return lookup(left, x); - otherwise do return lookup(right, x); - } - -type test = insert(insert(insert(Empty, 2, "two"), 1, "one"), 3, "three"); -writeln(lookup(test, 1)); // prints "one" -writeln(lookup(test, 3)); // prints "three" -writeln(lookup(test, 4)); // prints "not found" -``` - ---- - -# Conclusion - -* Chapel's type-level programming is surprisingly powerful. -* We can write compile-time programs that are very similar to Haskell programs. -* This allows us to write highly parameterized code without paying runtime overhead. -* This also allows us to devise powerful compile-time checks and constraints on our code. -* This approach allows for general-purpose programming, which can be applied to `your use-case`