refactor(report): split types and registry sections
This commit is contained in:
@@ -21,10 +21,11 @@ In @sec:impl-overview, we will first look at the whole system from a top-down pe
|
||||
#pagebreak()
|
||||
|
||||
#include-offset(path("04_implementation/01_overview.typ"))
|
||||
#include-offset(path("04_implementation/02_types_registry.typ"))
|
||||
#include-offset(path("04_implementation/03_midas_language.typ"))
|
||||
#include-offset(path("04_implementation/04_python_checking.typ"))
|
||||
#include-offset(path("04_implementation/05_generation.typ"))
|
||||
#include-offset(path("04_implementation/02_types.typ"))
|
||||
#include-offset(path("04_implementation/03_registry.typ"))
|
||||
#include-offset(path("04_implementation/04_midas_language.typ"))
|
||||
#include-offset(path("04_implementation/05_python_checking.typ"))
|
||||
#include-offset(path("04_implementation/06_generation.typ"))
|
||||
|
||||
/*
|
||||
Subjects:
|
||||
|
||||
151
report/chapters/04_implementation/02_types.typ
Normal file
151
report/chapters/04_implementation/02_types.typ
Normal file
@@ -0,0 +1,151 @@
|
||||
#import "../../requirements.typ": isc-hei-bthesis
|
||||
#import isc-hei-bthesis: todo
|
||||
#import "../../utils.typ": fn-link, code-ref
|
||||
#import "@preview/acrostiche:0.7.0": acr
|
||||
|
||||
= Internal Type Representations <sec:impl-types>
|
||||
|
||||
Before we can parse type definitions or type check Python code, we need to define structures to represent our types.
|
||||
|
||||
In our type system, we will indeed have different kinds of types with different properties. All types described in this section are grouped under a union type named `Type` for easier references in annotations. Additionally, we will later need to define some basic functions to manipulate types, for example to substitute parameters in a generic type. These functions will be introduced where needed in later sections. The source code for internal type representations and these basic functions is available in the repository in #code-ref(<types>, "midas/checker/types.py").
|
||||
|
||||
== Base Types
|
||||
|
||||
The simplest types are those at the root of the type lattice with no or few properties. These include top-types like `Any`, `UnknownType` and `UnitType`/`None`, as described in @sec:theory-special-types, which can be represented as simple empty classes like in @fig:types-base-types. Builtin types also have a dedicated `BaseType` kind which holds the type's name.
|
||||
For the special `tuple` type, we can define a more specific class which can hold the each item's type. This will allow nice type checking of subscript expressions for example.
|
||||
|
||||
#figure(
|
||||
```python
|
||||
class TopType: ...
|
||||
class UnknownType: ...
|
||||
class UnitType: ...
|
||||
class BaseType:
|
||||
name: str
|
||||
class TupleType:
|
||||
items: tuple[Type, ...]
|
||||
```,
|
||||
caption: [Type kinds: base types]
|
||||
) <fig:types-base-types>
|
||||
|
||||
== Derived Types
|
||||
|
||||
When defining subtypes in Midas, we will need to store that link in the registry. There are multiple approaches to subtyping, which can either be nominal or structural. In the case of Midas, we will generally stick to nominal subtyping, i.e. subtypes are explicitly defined, with some exceptions like constraint types and function types, as described in @sec:typing-subtyping.
|
||||
|
||||
Instead of storing the subtype relationship in a separate registry, we will embed it in the type itself by using a `DerivedType` class containing the type's name and super-type. This allows easy pattern matching without additional queries to the registry, and keeps the type hierarchy clear and explicit.
|
||||
A similar approach is used for constraint types, by embedding the base type and the constraint expression in the same `ConstraintType` object, as shown in @fig:types-derived-types.
|
||||
|
||||
#figure(
|
||||
```python
|
||||
class DerivedType:
|
||||
name: str
|
||||
type: Type
|
||||
class ConstraintType:
|
||||
type: Type
|
||||
constraint: m.Expr
|
||||
```,
|
||||
caption: [Type kinds: derived types]
|
||||
) <fig:types-derived-types>
|
||||
|
||||
== Generic Types
|
||||
|
||||
To define generic types, we first need type variables, which are kinds of placeholder types. These type variables have a name, an optional bounding type and a variance. The latter can be represented as a simple enum. As explained in @sec:typing-subtyping, variance is inferred rather than explicitly annotated by the user. This inference process is described in @sec:impl-midas #todo[replace ref].
|
||||
Once type variables are defined, generic types are simply a kind of derived type with a list of parameters.
|
||||
We also need to define an application counterpart to represent a generic type that has been instantiated with some concrete type arguments. Here, the choice of having this `AppliedType` kind instead of simply using the generic's body with its parameters substituted has been made for multiple reasons. Most importantly, it allows diagnostics to be more specific and informative and eases some comparisons between different applied types.
|
||||
|
||||
These types are implemented by the classes in @fig:types-generics.
|
||||
|
||||
#figure(
|
||||
```python
|
||||
class Variance(StrEnum):
|
||||
INVARIANT = "INVARIANT"
|
||||
COVARIANT = "COVARIANT"
|
||||
CONTRAVARIANT = "CONTRAVARIANT"
|
||||
class TypeVar:
|
||||
name: str
|
||||
bound: Optional[Type]
|
||||
variance: Variance = Variance.INVARIANT
|
||||
class GenericType:
|
||||
name: str
|
||||
params: list[TypeVar]
|
||||
body: Type
|
||||
class AppliedType:
|
||||
name: str
|
||||
args: list[Type]
|
||||
body: Type
|
||||
```,
|
||||
caption: [Type kinds: generic types]
|
||||
) <fig:types-generics>
|
||||
|
||||
== Function Types
|
||||
|
||||
As noted in the theory chapter (@chap:theory), functions in Python are complex and highly flexible.
|
||||
A function type thus have a parameter specification and a return type. That parameter specification contains three kinds of parameters: positional-only (`pos`), keyword-only (`kw`) and mixed (`mixed`).
|
||||
@fig:types-functions shows the implemented structures to handle function types and their parameters.
|
||||
For simplicity, each parameter will store both its position and name, as well as its type and a flag indicating whether it is required or not (e.g. parameters with default values are optional). You may notice an additional `unsupported` flag in @fig:types-functions:10, which is added for some dataframe-related methods. More information shall be given in @sec:impl-python #todo[replace ref].
|
||||
|
||||
Finally, we will also define a structure to hold multiple signature of an overloaded function because it cannot be represented as a single `Function`. You may also notice that `OverloadedFunction.overloads` does not store a list of `Function` but simply a list of `Type`, in @fig:types-functions:16. This allows the use of generic functions, which are `Function` types wrapped inside a `GenericType`.
|
||||
|
||||
#figure(
|
||||
```python
|
||||
class Function:
|
||||
params: ParamSpec
|
||||
returns: Type
|
||||
|
||||
class Parameter:
|
||||
pos: int
|
||||
name: str
|
||||
type: Type
|
||||
required: bool
|
||||
unsupported: bool = False
|
||||
class ParamSpec:
|
||||
pos: list[Function.Parameter] = field(default_factory=list)
|
||||
mixed: list[Function.Parameter] = field(default_factory=list)
|
||||
kw: list[Function.Parameter] = field(default_factory=list)
|
||||
class OverloadedFunction:
|
||||
overloads: list[Type]
|
||||
```,
|
||||
caption: [Type kinds: function types]
|
||||
) <fig:types-functions>
|
||||
|
||||
== Dataframe types
|
||||
|
||||
Because Midas has been designed with data scientists in mind, it will also try and type check some common dataframe operations. To do so, we want the user to be able to define dataframe schemas and use their columns. We thus define `ColumnType` and `DataFrameType` in @fig:types-dataframes, which can be thought of as more abstract representations of Pandas' `Series` and `DataFrame`. Additionally, we define special objects to handle group-by entities which are intermediary values used when computing aggregations. These group-by types are necessary to distinguish them from the base dataframe or column because they provide different methods.
|
||||
|
||||
#figure(
|
||||
```python
|
||||
class ColumnType:
|
||||
type: Type
|
||||
class DataFrameType:
|
||||
columns: list[Column]
|
||||
|
||||
class Column:
|
||||
index: int
|
||||
name: Optional[str]
|
||||
type: ColumnType
|
||||
class FrameGroupBy:
|
||||
frame: DataFrameType
|
||||
class ColumnGroupBy:
|
||||
column: ColumnType
|
||||
```,
|
||||
caption: [Type kinds: dataframe types]
|
||||
) <fig:types-dataframes>
|
||||
|
||||
== Predicates
|
||||
|
||||
Although not types, predicates will also be represented by similar dataclasses in the types registry, as defined in @fig:types-predicate.
|
||||
|
||||
#figure(
|
||||
```python
|
||||
class Predicate:
|
||||
type: Type
|
||||
body: m.Expr
|
||||
alias: bool
|
||||
```,
|
||||
caption: [Predicate Dataclass]
|
||||
) <fig:types-predicate>
|
||||
|
||||
This class holds three properties of predicates:
|
||||
- `type`: the function signature of the predicate.\
|
||||
For example, ```midas predicate less_than(max: float)(v: float) = v < max``` has the signature ```midas fn(max: float) -> fn(v: float) -> bool```
|
||||
- `body`: the raw predicate expression. This will be used to generate equivalent Python code and perform static checks against literal values
|
||||
- `alias`: a flag indicating whether a predicate is a simple alias of another, used for example with partially applied predicates (see @sec:impl-midas #todo[replace ref])
|
||||
@@ -1,159 +1,8 @@
|
||||
#import "../../requirements.typ": isc-hei-bthesis
|
||||
#import isc-hei-bthesis: todo
|
||||
#import "../../utils.typ": fn-link, code-ref
|
||||
#import "@preview/acrostiche:0.7.0": acr
|
||||
#import "../../utils.typ": code-ref
|
||||
#import "@preview/codly:1.3.0": codly
|
||||
#import "../../appendices/is_subtype.typ": is-subtype-code, is-func-subtype-code
|
||||
|
||||
= Internal Type Representations <sec:impl-types>
|
||||
|
||||
Before we can parse type definitions or type check Python code, we need to define structures to represent our types.
|
||||
|
||||
In our type system, we will indeed have different kinds of types with different properties. All types described in this section are grouped under a union type named `Type` for easier references in annotations. Additionally, we will later need to define some basic functions to manipulate types, for example to substitute parameters in a generic type. These functions will be introduced where needed in later sections. The source code for internal type representations and these basic functions is available in the repository in #code-ref(<types>, "midas/checker/types.py").
|
||||
|
||||
== Base Types
|
||||
|
||||
The simplest types are those at the root of the type lattice with no or few properties. These include top-types like `Any`, `UnknownType` and `UnitType`/`None`, as described in @sec:theory-special-types, which can be represented as simple empty classes like in @fig:types-base-types. Builtin types also have a dedicated `BaseType` kind which holds the type's name.
|
||||
For the special `tuple` type, we can define a more specific class which can hold the each item's type. This will allow nice type checking of subscript expressions for example.
|
||||
|
||||
#figure(
|
||||
```python
|
||||
class TopType: ...
|
||||
class UnknownType: ...
|
||||
class UnitType: ...
|
||||
class BaseType:
|
||||
name: str
|
||||
class TupleType:
|
||||
items: tuple[Type, ...]
|
||||
```,
|
||||
caption: [Type kinds: base types]
|
||||
) <fig:types-base-types>
|
||||
|
||||
== Derived Types
|
||||
|
||||
When defining subtypes in Midas, we will need to store that link in the registry. There are multiple approaches to subtyping, which can either be nominal or structural. In the case of Midas, we will generally stick to nominal subtyping, i.e. subtypes are explicitly defined, with some exceptions like constraint types and function types, as described in @sec:typing-subtyping.
|
||||
|
||||
Instead of storing the subtype relationship in a separate registry, we will embed it in the type itself by using a `DerivedType` class containing the type's name and super-type. This allows easy pattern matching without additional queries to the registry, and keeps the type hierarchy clear and explicit.
|
||||
A similar approach is used for constraint types, by embedding the base type and the constraint expression in the same `ConstraintType` object, as shown in @fig:types-derived-types.
|
||||
|
||||
#figure(
|
||||
```python
|
||||
class DerivedType:
|
||||
name: str
|
||||
type: Type
|
||||
class ConstraintType:
|
||||
type: Type
|
||||
constraint: m.Expr
|
||||
```,
|
||||
caption: [Type kinds: derived types]
|
||||
) <fig:types-derived-types>
|
||||
|
||||
== Generic Types
|
||||
|
||||
To define generic types, we first need type variables, which are kinds of placeholder types. These type variables have a name, an optional bounding type and a variance. The latter can be represented as a simple enum. As explained in @sec:typing-subtyping, variance is inferred rather than explicitly annotated by the user. This inference process is described in @sec:impl-midas #todo[replace ref].
|
||||
Once type variables are defined, generic types are simply a kind of derived type with a list of parameters.
|
||||
We also need to define an application counterpart to represent a generic type that has been instantiated with some concrete type arguments. Here, the choice of having this `AppliedType` kind instead of simply using the generic's body with its parameters substituted has been made for multiple reasons. Most importantly, it allows diagnostics to be more specific and informative and eases some comparisons between different applied types.
|
||||
|
||||
These types are implemented by the classes in @fig:types-generics.
|
||||
|
||||
#figure(
|
||||
```python
|
||||
class Variance(StrEnum):
|
||||
INVARIANT = "INVARIANT"
|
||||
COVARIANT = "COVARIANT"
|
||||
CONTRAVARIANT = "CONTRAVARIANT"
|
||||
class TypeVar:
|
||||
name: str
|
||||
bound: Optional[Type]
|
||||
variance: Variance = Variance.INVARIANT
|
||||
class GenericType:
|
||||
name: str
|
||||
params: list[TypeVar]
|
||||
body: Type
|
||||
class AppliedType:
|
||||
name: str
|
||||
args: list[Type]
|
||||
body: Type
|
||||
```,
|
||||
caption: [Type kinds: generic types]
|
||||
) <fig:types-generics>
|
||||
|
||||
== Function Types
|
||||
|
||||
As noted in the theory chapter (@chap:theory), functions in Python are complex and highly flexible.
|
||||
A function type thus have a parameter specification and a return type. That parameter specification contains three kinds of parameters: positional-only (`pos`), keyword-only (`kw`) and mixed (`mixed`).
|
||||
@fig:types-functions shows the implemented structures to handle function types and their parameters.
|
||||
For simplicity, each parameter will store both its position and name, as well as its type and a flag indicating whether it is required or not (e.g. parameters with default values are optional). You may notice an additional `unsupported` flag in @fig:types-functions:10, which is added for some dataframe-related methods. More information shall be given in @sec:impl-python #todo[replace ref].
|
||||
|
||||
Finally, we will also define a structure to hold multiple signature of an overloaded function because it cannot be represented as a single `Function`. You may also notice that `OverloadedFunction.overloads` does not store a list of `Function` but simply a list of `Type`, in @fig:types-functions:16. This allows the use of generic functions, which are `Function` types wrapped inside a `GenericType`.
|
||||
|
||||
#figure(
|
||||
```python
|
||||
class Function:
|
||||
params: ParamSpec
|
||||
returns: Type
|
||||
|
||||
class Parameter:
|
||||
pos: int
|
||||
name: str
|
||||
type: Type
|
||||
required: bool
|
||||
unsupported: bool = False
|
||||
class ParamSpec:
|
||||
pos: list[Function.Parameter] = field(default_factory=list)
|
||||
mixed: list[Function.Parameter] = field(default_factory=list)
|
||||
kw: list[Function.Parameter] = field(default_factory=list)
|
||||
class OverloadedFunction:
|
||||
overloads: list[Type]
|
||||
```,
|
||||
caption: [Type kinds: function types]
|
||||
) <fig:types-functions>
|
||||
|
||||
== Dataframe types
|
||||
|
||||
Because Midas has been designed with data scientists in mind, it will also try and type check some common dataframe operations. To do so, we want the user to be able to define dataframe schemas and use their columns. We thus define `ColumnType` and `DataFrameType` in @fig:types-dataframes, which can be thought of as more abstract representations of Pandas' `Series` and `DataFrame`. Additionally, we define special objects to handle group-by entities which are intermediary values used when computing aggregations. These group-by types are necessary to distinguish them from the base dataframe or column because they provide different methods.
|
||||
|
||||
#figure(
|
||||
```python
|
||||
class ColumnType:
|
||||
type: Type
|
||||
class DataFrameType:
|
||||
columns: list[Column]
|
||||
|
||||
class Column:
|
||||
index: int
|
||||
name: Optional[str]
|
||||
type: ColumnType
|
||||
class FrameGroupBy:
|
||||
frame: DataFrameType
|
||||
class ColumnGroupBy:
|
||||
column: ColumnType
|
||||
```,
|
||||
caption: [Type kinds: dataframe types]
|
||||
) <fig:types-dataframes>
|
||||
|
||||
== Predicates
|
||||
|
||||
Although not types, predicates will also be represented by similar dataclasses in the types registry, as defined in @fig:types-predicate.
|
||||
|
||||
#figure(
|
||||
```python
|
||||
class Predicate:
|
||||
type: Type
|
||||
body: m.Expr
|
||||
alias: bool
|
||||
```,
|
||||
caption: [Predicate Dataclass]
|
||||
) <fig:types-predicate>
|
||||
|
||||
This class holds three properties of predicates:
|
||||
- `type`: the function signature of the predicate.\
|
||||
For example, ```midas predicate less_than(max: float)(v: float) = v < max``` has the signature ```midas fn(max: float) -> fn(v: float) -> bool```
|
||||
- `body`: the raw predicate expression. This will be used to generate equivalent Python code and perform static checks against literal values
|
||||
- `alias`: a flag indicating whether a predicate is a simple alias of another, used for example with partially applied predicates (see @sec:impl-midas #todo[replace ref])
|
||||
|
||||
|
||||
= Types Registry <sec:registry>
|
||||
= Types Registry <sec:impl-registry>
|
||||
|
||||
With structures to represent our types, we now need to implement the registry which hold all builtin and user-defined types, as well as named predicates. As a reminder of requirements given in @sec:impl-overview, the registry must also provide a method `is_subtype` to check whether according to our subtyping rules one type is subtype of another, a method to register a type with a name, a method to lookup a type given a name, and some methods to manipulate types, such as instantiating a generic type given some arguments. We will also registration and lookup methods for predicates, as well as for type members (defined in `extend` statements).
|
||||
|
||||
Reference in New Issue
Block a user