fix(report): improve layout

add pagebreaks to improve awkward layouts and relieve some pressure on layout algorithm

report/appendices/function_subtyping.typ

@@ -37,6 +37,8 @@ For a parameter specification $S_2$ to be a subtype of another $S_1$, the latter
 
 After mapping parameters of $S_1$ and $S_2$, types must be checked such that if a parameter $p_i: T in S_1$ is mapped to a parameter $q_j: U in S_2$, $U <: T$.
 
+#pagebreak(weak: true)
+
 = Detailed rules for *ParamSpec* subtyping
 
 #gc.info(title: [Notation])[

report/chapters/03_theory.typ

@@ -79,6 +79,8 @@ Finally, the rules expressed in @sec:theory-syntax are written in the form of th
   [Evaluating $P$ produces the new context $Gamma'$]
 ))
 
+#pagebreak(weak: true)
+
 == Syntax rules <sec:theory-syntax>
 
 In this section, we define syntax rules for both Python and our type definition language Midas. Obviously, Python already has syntax rules defining what is or is not valid code. What is defined in @sec:theory-syntax-python are a subset of constructs which our type checker will be able to process.
@@ -141,6 +143,8 @@ First and foremost, we can define elementary typing rules for all literal values
 
 Other constructs, although not constants, directly map to builtin types. These include literal lists, tuples and dictionaries. Each of these has a corresponding `list[T]`, `tuple` and `dict[K, V]` type. We will not define formal rules in this section regarding these elements, but a more in depth explanation will be given in the implementation of the type checker, in @sec:python-check-literals.
 
+#pagebreak(weak: true)
+
 === Expressions <sec:typing-expressions>
 
 Apart from literals, developers also need some building blocks to express their programs. These expressions include variables, function calls, operations, etc. and need their own typing rules. The rules listed in @tab:typing-expressions cover most expressions we will handle, as defined in @sec:theory-syntax-python. Some syntaxes are omitted to keep this chapter short and because they don't necessarily bring new theoretical concepts.
@@ -162,6 +166,8 @@ Apart from literals, developers also need some building blocks to express their
 
 #smallcaps[T-Op] is used for all binary operations. In Python, these are implemented in _dunder-methods_, such as `__add__` for the `+` operator. A similar rule could be defined for unary operations but is omitted here for brevity.
 
+#pagebreak(weak: true)
+
 === Statements <sec:typing-statements>
 
 Finally, expressions can be used in statements, which are special constructs that can have side-effects. We listed in @sec:theory-syntax-python the particular statements that Midas should support, and @tab:typing-statements provides rules to type check their components and effects.

report/chapters/04_implementation.typ

@@ -23,7 +23,9 @@ In @sec:impl-overview, we will first look at the whole system from a top-down pe
 #include-offset(path("04_implementation/01_overview.typ"))
 #include-offset(path("04_implementation/02_types.typ"))
 #include-offset(path("04_implementation/03_registry.typ"))
+#pagebreak(weak: true)
 #include-offset(path("04_implementation/04_midas_language.typ"))
+#pagebreak(weak: true)
 #include-offset(path("04_implementation/05_python_checking.typ"))
 #include-offset(path("04_implementation/06_generation.typ"))
 

report/chapters/04_implementation/03_registry.typ

@@ -57,7 +57,6 @@ The `define_*` and `lookup_*`/`get_*` methods are quite straightforward, inserti
                   + " only methods can be overloaded"
               )
               return
-
           combined: Type
           match current.type:
               case OverloadedFunction(overloads=overloads):
@@ -65,7 +64,6 @@ The `define_*` and `lookup_*`/`get_*` methods are quite straightforward, inserti
               case _:
                   combined = OverloadedFunction(overloads=[current.type, member_type])
           members[member_name] = Member(kind=current.kind, type=combined)
-
       else:
           members[member_name] = Member(kind=kind, type=member_type)
   ```,
@@ -141,7 +139,6 @@ Because we defined our internal type representations with dataclasses, we can ea
       match type:
           case DerivedType(name=name, type=base):
               return DerivedType(name=name, type=self.apply_generic(base, args))
-
           case GenericType(name=name, params=type_vars, body=body):
               n_args: int = len(args)
               n_type_vars: int = len(type_vars)
@@ -167,12 +164,9 @@ Because we defined our internal type representations with dataclasses, we can ea
                   args=args,
                   body=substitute_typevars(body, substitutions),
               )
-
           case BaseType(name="tuple"):
               return TupleType(items=tuple(args))
-
-          case _:
-              raise ValueError(f"{type} is not a generic type")
+          case _: raise ValueError(f"{type} is not a generic type")
   ```,
   caption: [Implementation of `TypesRegistry.apply_generic`]
 ) <fig:apply_generic>
@@ -189,6 +183,8 @@ The first case handling `DerivedType`, @fig:apply_generic:3 to @fig:apply_generi
   caption: [Generic subtype syntactic sugar]
 ) <fig:generic-subtype>
 
+#pagebreak(weak: true)
+
 == `is_subtype` <sec:is_subtype>
 
 This method is one of the central parts of our type system. Its role is simple: judge, according to the rules we defined in @sec:typing-subtyping (and some we skipped), whether a given type (`type1`) should be considered a subtype of another (`type2`).
@@ -259,6 +255,8 @@ This verification is implemented as in @fig:is_subtype-applied-type.
 
 The second case simply handles other situations where `type1` is an `AppliedType`, recursively checking its body against `type2` similarly to derived types.
 
+#pagebreak(weak: true)
+
 Only three rules remain to check for dataframes, columns and most importantly functions.
 Columns are simply regarded as invariant generic types. They are not implement using a regular `GenericType`/`AppliedType` because it makes many mechanisms much simpler to implement and reason about, especially regarding attributes and methods, which is worth making them a special construct.
 
@@ -280,7 +278,6 @@ Finally, as shown in @fig:is_subtype-df-cols-funcs, functions are checked separa
 == `is_func_subtype` <sec:is_func_subtype>
 
 This section describes the implementation of the function subtyping verification algorithm. Please refer to @app:function-subtyping for more information on the underlying theory and formal rules.
-
 The complete implementation of `is_func_subtype` is given in @fig:is_func_subtype.
 
 The first thing `is_func_subtype` must check is whether the return types of the given functions are subtypes of one another (see #smallcaps[S-Func] in @tab:typing-subtyping-function). A simple early return can be added right at the beginning of the method, as shown in @fig:is_func_subtype-returns.
@@ -294,6 +291,8 @@ The first thing `is_func_subtype` must check is whether the return types of the
   caption: [`is_func_subtype`: check return types]
 ) <fig:is_func_subtype-returns>
 
+#pagebreak(weak: true)
+
 As we will need to get parameters of each function by kind, name and position, we will first extract the different lists and dictionaries in short variables as outlined in @fig:is_func_subtype-extract-params. These correspond to $(P_1, M_1, K_1)$ and $(P_2, M_2, K_2)$ in the theoretical description.
 
 #codly(
@@ -309,7 +308,7 @@ We first check that `func2`'s positional- and keyword-only parameters are approp
 @fig:is_func_subtype-pos-kw shows how this is implemented with simple loops.
 
 #codly(
-  range: (24, 52),
+  range: (24, 49),
   smart-skip: true
 )
 #figure(
@@ -320,7 +319,7 @@ We first check that `func2`'s positional- and keyword-only parameters are approp
 Verifying proper coverage of mixed arguments is slightly more complicated but a similar method can be used to implement the theoretical algorithm, such as in @fig:is_func_subtype-mixed.
 
 #codly(
-  range: (54, 80),
+  range: (51, 77),
   smart-skip: true
 )
 #figure(
@@ -331,7 +330,7 @@ Verifying proper coverage of mixed arguments is slightly more complicated but a
 Finally in @fig:is_func_subtype-extra-subtypes, we check that `func1` does not introduce new required parameters and that matching parameters respect contravariance.
 
 #codly(
-  range: (82, 98),
+  range: (79, 95),
   smart-skip: true
 )
 #figure(

report/chapters/04_implementation/04_midas_language.typ

@@ -11,7 +11,9 @@ For maximum flexibility and control over the whole process, it has been chosen t
 
 Processing a Midas definitions file is done in 3 steps:
 + lexing, i.e. turning raw text bytes into tokens
+
 + parsing, i.e. assembling tokens into an #acr("AST") according to syntax rules defined in @sec:theory-syntax-midas (and @app:midas-syntax)
+
 + typing, i.e. processing each statement, registering types and predicates in the registry
 
 Each of these steps map to a dedicated class, respectively `MidasLexer`, `MidasParser` and `MidasTyper`.
@@ -152,6 +154,8 @@ Before we can register a new type or an alias, we must be able to convert an #ac
 
 As an example, visiting a `m.NamedType` node simply looks up the name in the registry to retrieve the definition for that type. If it cannot be found, an error is reported to the user and `UnknownType` is returned. This is the standard behavior we will implement whenever the type checker cannot make a definite judgement about something. As you may notice in @fig:midas-visit_named_type, the implementation checks `self._current_name` to detect cyclic references. In @fig:midas-visit_named_type:4, `self.get_type` is used instead of directly calling the registry's method to handle type variables, as explained in @sec:midas-type-stmt.
 
+#pagebreak(weak: true)
+
 #figure(
   ```python
   def visit_named_type(self, type: m.NamedType) -> Type:

report/chapters/04_implementation/05_python_checking.typ

@@ -114,6 +114,7 @@ The input given to `PythonTyper` is basically a sequence of statements (`p.Stmt`
 
 One of the main mechanics manipulating the environment is variable declaration and assignment.
 During parsing, a statement such as ```python foo: int = 3```, represented in Python as a single `ast.AnnAssign` node, is split into two #acr("AST") nodes. The first is a type assignment (`p.TypeAssign`), which declares a new variable with the given type. The second is a variable assignment (`p.AssignStmt`). Implementing `visit_type_assign` is straightforward: we first resolve the type annotation expression and then define a new variable in the current environment, as demonstrated in @fig:python-visit_type_assign.
+This method basically materializes #smallcaps[T-Annot] from @tab:typing-statements.
 
 #figure(
   ```python
@@ -124,8 +125,6 @@ During parsing, a statement such as ```python foo: int = 3```, represented in Py
   caption: [Python Typer: implementation of `visit_type_assign`]
 ) <fig:python-visit_type_assign>
 
-This method basically materializes #smallcaps[T-Annot] from @tab:typing-statements.
-
 Handling the variable assignment part is a bit more involved, because Python (and our type checker) allows both assigning to multiple targets simultaneously (@fig:python-visit_assign_stmt:3) and assigning to attributes and subscripts (not only variables). This is the reason we use a `match` statement in @fig:python-visit_assign_stmt. This function embodies #smallcaps[T-Assign].
 
 #codly(
@@ -204,9 +203,7 @@ To easily implement the first effect, we will take advantage of exceptions and r
           except ReturnException:
               returned = True
               if i < len(block) - 1:
-                  self.reporter.warning(
-                      block[i + 1].location, "Unreachable statement"
-                  )
+                  self.reporter.warning(block[i + 1].location, "Unreachable statement")
               break
       self.env = previous_env
       return returned
@@ -468,11 +465,9 @@ Now that we have a method to choose an overload given some arguments, we can add
 === Generic function
 
 Generic functions are useful for defining some kind of template behavior while allowing different types to be used. However, when it comes to a call to such a function, things get a bit more complicated. Indeed, type parameters must be mapped to concrete types and unified depending on the actual call-site arguments before getting the return type. Taking the example of a simple generic doubling function ```py def double(value: T) -> T```, the return type depends on the type of the argument. Furthermore, in a more complex case like ```py def add(v1: T, v2: T) -> T```, the type variable `T` must be mapped to the same type for both `v1` and `v2`.
-
 This process is covered more in depth in chapter 22 of #acr("TaPL")@tapl.
 
 For this purpose, we can implement a dedicated `Unifier` class whose role is to find appropriate substitutions for type variables in a generic call. Its source code is available in the repository in #code-ref(<unifier>, "midas/checker/unifier.py").
-
 Using this class, we can complete `get_result` with the last case handling calls to a `GenericType`, as shown in @fig:dispatcher-match-generic.
 
 #codly(
@@ -484,6 +479,8 @@ Using this class, we can complete `get_result` with the last case handling calls
   caption: [Call Dispatcher: call to `GenericType`]
 ) <fig:dispatcher-match-generic>
 
+#pagebreak(weak: true)
+
 == Casts and Static Evaluation <sec:python-check-casts>
 
 From a type checking point of view, cast expressions are trivial to implement by applying #smallcaps[T-Cast]. `visit_cast_expr` simply returns the type passed to `cast`. There is however a fundamental premise making casts sounds, that is, the expression must conform to the given type. Most of the time, users will use cast expressions when the expression is not properly typeable at compile-time, but will always be valid at runtime. To maintain type safety, we will thus generate a runtime assertion checking that premise (see @sec:gen-assertions). However, these check might be quite computationally expensive, so we will provide users with an unsafe alternative that does not produce any runtime verification, `unsafe_cast`.
@@ -525,6 +522,8 @@ One particularity of Midas is how it allows curried application of predicates. G
   caption: [Example of curried predicate application]
 ) <fig:example-curried-application>
 
+#pagebreak(weak: true)
+
 In practice, we keep track of a scope dictionary containing all defined variables (parameters). When the first call is evaluated, we insert `min = 0.0` and `max = 1.0` in the scope and build a `PartialPredicate` object. This object contains the inner predicate function signature (`fn(float) -> bool`), the predicate's body (`min <= v & v <= max`) and the scope dictionary. The second call receives this `PartialPredicate` as the callee, inserts `_ = <the literal value>` in the scope and evaluates the predicate's body. This whole process is detailed in @fig:static-predicate-eval.
 
 #figure(
@@ -568,6 +567,8 @@ There is still one big part of the type checker that we have not covered: datafr
 Midas stands in between by providing some static type-checking for dataframe schemas and operations as well as generating runtime checks when a value is cast to such a type.
 We will explore in this section how we can not only provide type-checking for frame columns, for either accessing or assigning to them, but also handle some method calls with best effort inference of the resulting type.
 
+#pagebreak(weak: true)
+
 === Schema Manipulation <sec:python-df-manager>
 
 When `PythonTyper` encounters a `p.SubscriptExpr` where the object is a `DataFrameType`, either in a getter context or in an assignment, we defer the resolution to a dedicated `FrameManager` class. This manager's responsibility is to check that referenced columns exist in the schema (and return their types). When assigning to a dataframe, it also build a modified version of the schema to include new columns.

report/chapters/05_results_discussion.typ

@@ -12,8 +12,7 @@ In @chap:theory and @chap:impl, we have theorized and implemented a complete typ
 - structural subtyping for functions
 - static evaluation of `cast` expressions on literal values
 - runtime assertion generation for `cast` expressions
-- dataframe schema definition and manipulation
-- arithmetic and aggregation methods on dataframes and columns
+- schema definition and manipulation, arithmetic and aggregation methods on dataframes and columns
 
 This is more than enough to make Midas usable in a wide range of contexts, including data science. The following sections demonstrate how Midas can be used to provide powerful type checking, both statically and at runtime.
 
@@ -26,7 +25,7 @@ As an example, we will consider a sample weather-data transformation pipeline as
 Using the Midas language, we are able to define domain-specific types, as demonstrated in @fig:pipeline-base-types.
 
 #codly(
-  range: (1, 15),
+  range: (1, 14),
   smart-skip: true
 )
 #figure(
@@ -34,10 +33,12 @@ Using the Midas language, we are able to define domain-specific types, as demons
   caption: [Example Pipeline: domain specific types]
 ) <fig:pipeline-base-types>
 
+#pagebreak(weak: true)
+
 Additionally, we can define operations to preserve some of these semantics, as shown in @fig:pipeline-operations.
 
 #codly(
-  range: (17, 29),
+  range: (16, 28),
   smart-skip: true
 )
 #figure(
@@ -48,7 +49,7 @@ Additionally, we can define operations to preserve some of these semantics, as s
 Finally, we can concisely define frame schemas that will become useful when manipulating dataframes in Python, as shown in @fig:pipeline-schemas.
 
 #codly(
-  range: (31, 62),
+  range: (30, 61),
   smart-skip: true
 )
 #figure(
@@ -56,10 +57,11 @@ Finally, we can concisely define frame schemas that will become useful when mani
   caption: [Example Pipeline: dataframe schemas]
 ) <fig:pipeline-schemas>
 
+#pagebreak(weak: true)
+
 === Type checking in action
 
 Now what is the time for Midas to shine. The first step in a transformation pipeline is load some data. We will used `pandas` to read a dataframe from a #acr("CSV") file. Now, the compiler or type checker has no idea what this #acr("CSV") file might look like. Even if we specify a schema, there is no guarantee that the runtime file will conform to it. At most, the type checker can say that the result of `read_csv` is a dataframe. We thus introduce a `cast` expression to actually tell the type checker what the dataframe contains. Furthermore, a runtime assertion will check that the value returned by `read_csv` does indeed match our expectations.
-
 Our load function thus looks like @fig:pipeline-load.
 
 #codly(
@@ -74,7 +76,7 @@ Our load function thus looks like @fig:pipeline-load.
 In a second step, we might want to transform some values to more appropriate types, such as parsing timestamps from strings. This is also the place where we cast the dataframe to a schema with our domain-specific types, which will ensure that values conform to the defined constraints, as shown in @fig:pipeline-convert.
 
 #codly(
-  range: (15, 24),
+  range: (15, 23),
   smart-skip: true
 )
 #figure(
@@ -86,7 +88,7 @@ This does highlight two current weaknesses of Midas. The first is the need to co
 The second issue, which is more of a possible optimization, is the fact that `cast` will re-check the whole dataframe at runtime, even though some checks are irrelevant given the parameter's type. This is made even more noticeable in @fig:pipeline-aggregation which will check base types again (e.g. `float`).
 
 #codly(
-  range: (27, 38),
+  range: (26, 37),
   smart-skip: true
 )
 #figure(
@@ -94,10 +96,12 @@ The second issue, which is more of a possible optimization, is the fact that `ca
   caption: [Example Pipeline: arithmetic operations on columns]
 ) <fig:pipeline-heat-index>
 
+#pagebreak(weak: true)
+
 As implemented in @sec:python-df-methods, Midas will type check many operations on dataframes and columns, including `groupby` and aggregation methods. The result the computation shown in @fig:pipeline-aggregation is already typed as a dataframe of `float` columns by the type checker. We only add a `cast` to bring back our domain specific types, while re-checking value constraints. This latter point reveals one great feature missing from Midas: constraint unification (this will be discussed in @chap:conclusion).
 
 #codly(
-  range: (41, 54),
+  range: (40, 53),
   smart-skip: true
 )
 #figure(
@@ -110,7 +114,7 @@ Finally, there are some cases where Midas is not capable of properly type-checki
 Moreover, users may want to cast an expression to a type but cannot afford the cost of checking it at runtime or feel it is too redundant with a previous known typing judgment. Alternatively, they may want to use a value with an unknown type which _behaves_ as another for all practical purposes (e.g. `np.float32`). In that case, they can use an escape hatch with `unsafe_cast` which blindly accepts that the given expression is of the specified type, as used in @fig:pipeline-plot.
 
 #codly(
-  range: (57, 65),
+  range: (56, 64),
   smart-skip: true
 )
 #figure(
@@ -118,6 +122,8 @@ Moreover, users may want to cast an expression to a type but cannot afford the c
   caption: [Example Pipeline: unknown types and `unsafe_cast`]
 ) <fig:pipeline-plot>
 
+#pagebreak(weak: true)
+
 == Type errors
 
 In the previous section, we focused on dataframe operations, casts and runtime type errors, but Midas also catches static type errors. One classical but sneaky kind of error is mixing incompatible units. While using millimeters instead of centimeters when 3D-printing a pen holder might be comical, mixing monetary currencies while handling enterprise assets will probably get you fired. As demonstrated in @fig:caught-errors, Midas can help you avoid this kind of errors that can happen when some values share the same base representation (`float`).

report/code/example_pipeline/custom_types.midas

@@ -9,7 +9,6 @@ type Temperature = Celsius where in_range(-30.0, 100.0)(_)
 type Pressure = Hectopascal where in_range(800.0, 1100.0)(_)
 type Humidity = float where is_percentage(_)
 type HeatIndex = float
-
 type StationID = str where len(_) == 3 & _.isupper()
 
 type Mean[T <: float] = float

report/code/example_pipeline/pipeline.py

@@ -18,7 +18,6 @@ def convert_data(raw_df: RawData) -> Data:
         Column[object],
         pd.to_datetime(new_df["timestamp"]),
     )
-
     # Check types and constraints at runtime, catches out-of-range values and
     # invalid types / malformed data
     return cast(Data, new_df)

report/code/is_func_subtype.py

@@ -22,10 +22,8 @@ def is_func_subtype(self, func1: Function, func2: Function) -> bool:
     }
 
     matches: list[Match] = []
-
     for param2 in pos2:
         param1: Function.Parameter
-
         if param2.pos < len(pos1):
             param1 = pos1[param2.pos]
         elif param2.pos in mixed_by_pos:
@@ -39,7 +37,6 @@ def is_func_subtype(self, func1: Function, func2: Function) -> bool:
 
     for name, param2 in kw2.items():
         param1: Function.Parameter
-
         if name in kw1:
             param1 = kw1[name]
         elif name in mixed_by_name: