A Short Note on Collecting Dependently Typed Values
aa r X i v : . [ c s . P L ] A ug A Short Note on Collecting Dependently Typed Values ∗ JAN DE MUIJNCK-HUGHES,
University of GlasgowWithin dependently typed languages, such as Idris, types can depend on values. This dependency, however,can limit the collection of items in standard containers: all elements must have the same type, and as suchtheir types must contain the same values. We present two dependently typed data structures for collectingdependent types:
DList and
PList . Use of these new data structures allow for the creation of single succinctinductive
ADTs whose constructions were previously verbose and split across many data structures.Additional Key Words and Phrases: Dependent Types
Dependently typed languages such as Idris [2] and Agda [8], provide programmers with a richand expressive type-system that facilitates greater precision when reasoning about our softwareprograms. However, this expressiveness comes with a cost when collecting values within a con-tainer. Take, for example, ‘cons’-style lists that provides an inductive Algebraic Data Type (ADT)for collecting values: data List a = Nil | (::) a (List a)
The type
List is indexed by the type of the list’s elements, and the constructors
Nil and (::) allow lists to be constructed by appending elements to an initially empty list. For example: hoi : List Stringhoi = "H" :: "o" :: "i" :: Nil
With the list hoi , each element has the type
String : This is fine, and by design, as the type
List is indexed by a single value. Suppose, however, we are to work with a dependently typed valueand collect several different values together using
List . For instance, suppose we are modelling aTODO list in which each TODO item has a type parameterised by the item’s TODO state. data Status = TODO | STARTED | DONEdata Item : Status -> Type whereMkItem : (state : Status)-> (title : String)-> Item state
A question arises concerning how we are to use
List to contain a list of TODO items that mayhave differing TODO states. items : List ?myTypeIsitems = MkItem STARTED "Write Paper":: MkItem TODO "Write Introduction":: Nil
The
List data type is not able to collect elements from the same indexed families. Each elementmust have the same type, and in a dependently typed language this also means the types mustalso depend on the same value. Lists of type
List can be made for
TODO items,
STARTED items, and
DONE items, but not for a mixture of such items.A natural solution would be to represent items as a list of dependent pairs: ∗ A short paper illustrating minor semi-unpublished work.Author’s address: Jan de Muijnck-Hughes, University of Glasgow, School of Computing Science, [email protected] 01:07. Page 1 of 1–9.
Jan de Muijnck-Hughes items : List ( ty : Status ** Item ty )items = (STARTED ** MkItem STARTED "Write Paper"):: (TODO ** MkItem TODO "Write Introduction"):: Nil
With this construction the type-level values are also represented at the value level for each ele-ment. While we can now have a list of dependently typed values, working with the resulting datastructure is cumbersome. We have to take into account type-level only information at the valuelevel when operating on individual elements within the list.We can do better!
This paper presents two dependently typed list data structures that allows for values from thesame dependent type to be collected in the same container.(1) Section 3 presents
DList a dependently typed list that collects, at the type level, a singlevalue from a dependent type.(2) Section 4 introduces
PList an extended definition of
DList where the collected values mustalso satisfy a provided predicated.Both data structures have been made available as part of the idris-containers library [6]. Sec-tion 2 further motivates the need for these data structures by examining the specification of anADT for JavaScript Object Notation (
Json ) documents. While such ADTs are naturally inductive,we further motivate the paper by examining a version of
Json in which the root element must bea key-value store. This provides a minimum motivating example suitable for examination in thepaper. Section 5 discuses the limitations of the data structures presented in this paper; discussessimilar structures; and other larger examples in which these structures prove useful.
Json is a well known serialisation format.
Json documents can contain elements that are either objects or values . An object is either a key value store (associative array) mapping String valuesto other
Json elements, or an array or elements.
Json values are either:
String , Double , Bool , or
Null . The natural shape of a
Json document makes it ideally suited for modelling as an inductiveADT. For example: data JSONDoc =JStr String | JNum Double | JBool Bool| JNull | JArray JSONDoc| JMap (List (String, JSONDoc))| JDoc JSONDoc
This is fine. Suppose, however, that the root element in a
Json document must be a key value store.With this restriction our once reasonable data structure becomes problematic in its use. First, it isnot trivial to declare a function that, through its type signature, is guaranteed to accept or return acomplete
Json document. For example, take the following type signatures for reading and writing
Json documents. writeDoc : String -> JSONDoc -> IO ()readDoc : String -> IO (Either JSONDoc Error)
The second argument passed in to writeDoc is only guaranteed to be of type
JSONDoc and notnecessarily a value constructed using
JDoc . Likewise, when using readDoc , we are not guranteed
Short Note on Collecting Dependently Typed Values 3 to return a complete
Json document. To provide such guarantees, one has take a defensive pro-gramming stance and ensure that the functions works with full documents only or fail gracefully.Secondly, how can the internal structure of a document be specified such that only valid doc-uments are created. For example, the constructors
JArray , JMap , and
JDoc can take in any validvalue or document. This violates the requirement that the root of a
Json document is an associativearray.A natural way to address these concerns is to introduce more data types to model specific sub-sections of a
Json document. For example: data JVal = JStr String | JNum Float | JBool Bool | JNulldata JObj = JDict (List String, JObj) | JArray (List JObj) | JValue JValdata JRoot = JMap (List (String, JObj))data JSONDoc = JDoc JRoot
However, with this approach the natural inductive structure of the original attempt has been lost.Further, the
Json document is no longer a single data type, it is now made up of four distinctones. It is now no longer possible to write simple recursive functions that traverse or query
Json documents. Multiple functions and instances must now be created to work with each different datatype used to model the document.Following from the
Well-Typed Interpreter [1], dependent types can capture the shape of indi-vidual sections within a
Json document directly within the document’s type. We begin by definingthe following enumerated type
JTy . data JTy = DOC | OBJECT | VALUE Values of
JTy allow us to distinguish between a complete
Json document itself, and the objects andvalues contained therein. By indexing
JSONDoc with
JTy the allowed structure of a
Json documentcan be capture more accurately. data JSONDoc : JTy -> Type whereJStr : String -> JSONDoc VALUEJNum : Float -> JSONDoc VALUEJBool : Bool -> JSONDoc VALUEJNull : JSONDoc VALUEJArray : List (JSONDoc VALUE)-> JSONDoc OBJECTJMap : List (String, (JSONDoc VALUE))-> JSONDoc OBJECTJDoc : JSONDoc OBJECT -> JSONDoc DOC
With
JSONDoc , we are now able to specify functions that operate on documents and not values.For example: writeDoc : JSONDoc DOC -> IO ()readDoc : String -> IO (JSONDoc DOC)
However, the internal structure of a document is not well-formed. We need to be able to specifythat: (a) the constructor
JDoc takes a map as its input; and (b) that both
JArray and
JMap haveelements that are either values or objects.. A naïve attempt to address these issues would be tointroduce a fourth constructor to
JTy , MAP to represent associative arrays, and introduce versionsof
JMap and
JMap that collects objects. This doubles the number of duplicate data constructors, andAs with our introductory example, A standard list construct is not sufficient; all contents of thelist must have the same type and in a dependently typed language, the same values. We need to
Jan de Muijnck-Hughes data DList : (aTy : Type)-> (elemTy : aTy -> Type)-> (as : List aTy)-> Type whereNil : DList aTy elemTy Nil(::) : (elem : elemTy x)-> (rest : DList aTy elemTy xs)-> DList aTy elemTy (x::xs)
Fig. 1. A Cons-Style ADT for collecting Dependently Typed Values. head : (xs : DList aTy eTy (a::as))-> {auto ok : NonEmpty xs}-> {auto ok' : NonEmpty (a::as)}-> eTy ahead (y::rest) {ok = IsNonEmpty} {ok' = IsNonEmpty} = y (a) Head tail : (xs : DList aTy eTy (a::as))-> {auto ok : NonEmpty xs}-> DList aTy eTy astail (x::rest) ok = IsNonEmpty = rest (b) Tail take : (n : Nat)-> DList aTy eTy as-> DList aTy eTy (take n as)take Z rest = Niltake (S k) Nil = Niltake (S k) (e::rest) = e :: take k rest (c) Take drop : (n : Nat)-> DList aTy eTy as-> DList aTy eTy as (drop n as)drop Z rest = restdrop (S k) Nil = Nildrop (S k) (e::rest) = drop k rest (d) DropFig. 2. Example functions operating on
DList instances. be able to construct a list that contains elements from the same dependent type but whose typelevel values differ.
DLIST
CONTAINER
Christiansen [3] presented the
UList a dependently typed ADT for encoding lists of valuesencoded using a
Universe Pattern . UList is a generalised cons-style ADT that allows for a value contained within the type of a dependent type to be collected at the type-level. All elements withinthe list come from the same family of indexed types and that the index within the type of theelement can differ. With
UList , the family of indexed types is constrained to a singular instance.Although,
UList is useful for encoding constraints on types, the pattern can be used more gen-erally and be used for collecting elements of a dependent type regardless using a cons-style ADT.This was observed in de Muijnck-Hughes [4, Chapter 9] in which the author developed (independ-ently)
DList that was designed for collecting type-level information.Figure 1 presents the definition for
DList . In this definition: aTy is the type of the value con-tained within the list element type; elemTy is the type of the elements within the list; and xs isthe List containing the collected values.
DList data structure only collects a single value fromthe type. Dependent types that are parameterised using multiple elements must ensure that allrequired values are collected. Structurally,
UList and
DList are the same . Using DList a singlelibrary of operations operating on generic instances can now be specified. For example Figure 2presents several common functions on lists as replicated for
DList . Notice how the actions per-formed at the value level are mirrored at the type-level. For the remainder of the paper, we will use
DList to distinguish from the original use of
UList
Short Note on Collecting Dependently Typed Values 5
DList allows us to collect dependently typed values. Returning to the introductory example ofTODO lists, we can use
DList to collect the individual TODO items. items : DList Status Item [STARTED, TODO]items = MkItem STARTED "Write Paper":: MkItem TODO "Write Introduction":: Nil
Notice, how the structure of items mirrors that of a standard list. Values are appended to the list,and at the type level the values indexing each element are collected.Using
DList a more accurate description of the internal shape of our running
JSONDoc examplecan be attempted. First we extend
JTy with constructors to differentiate between associative arraysand arrays using
ARRAY and
MAP . data JTy = DOC | ARRAY | MAP | VALUE Secondly, we change the definition of
JArray and
JMap to use
DList . For
JMap , we also introducean anonymous function to ensure that the correct value is collected at the type-level. data JSONDoc : JTy -> Type whereJStr : String -> JSONDoc VALUEJNum : Float -> JSONDoc VALUEJBool : Bool -> JSONDoc VALUEJNull : JSONDoc VALUEJArray : DList JTy JSONDoc ts -> JSONDoc ARRAYJMap : DList JTy(\ty => (String, JSONDoc ty)) ts-> JSONDoc MAPJDoc : JSONDoc MAP -> JSONDoc DOC
Unfortunately, use of
DList here has resulted in too permissive a collection of
JSONDoc elementsand as such a
Json object can not contain any
Json value or object. To address this permissivenesswe need to be able to constrain the types in a
DList to have the following values of type
JTy : MAP , ARRAY , or
VALUE .One solution would be to introduce a predicate (
JPred ) that models a constraint on instancesof
JTy that are allowed
Json values. For example: data JPred : JTy -> Type whereJMap : JPred MAPJArr : JPred ARRAYJVal : JPred VALUE
Such a predicate can be enforced using Idris’ proof search mechanism and construction of a helperconstructor
JNode . This constructor would contain an instance of a
JSONDoc value and proof thatthe predicate applied to the type-level value holds. For instance
JSONDoc can be extended as fol-lows: data JSONDoc : JTy -> Type where...JNode : JSONDoc ty-> {auto prf : JPred ty}-> JSONDoc NODEJArray : List (JSONDoc NODE)
Jan de Muijnck-Hughes -> JSONDoc ARRAYJMap : List (String, JSONDoc NODE)-> JSONDoc MAP
However, this approach requires that we needless increase the verbosity of our documents rep-resentation using
JNode , that adds a layer of indirection to access values. We will see in the nextsection how we can remove the need for an explicit constructor to contain items. data PList : (aTy : Type)-> (elemTy : aTy -> Type)-> (predTy : aTy -> Type)-> (as : List aTy)-> (prf : DList aTy pred as)-> TypewhereNil : PList aTy elemTy predTy Nil Nil(::) : (elem : elemTy x)-> {prf : predTy x}-> (rest : PList aTy elemTy predTy xs ps)-> PList aTy elemTy predTy (x::xs) (prf::ps)
Fig. 3. Definition for a Predicated List.
Figure 3 presents the definition of
PList . PList is a variant of
DList that constrains the elementswithin a list by reasoning about the allowed values in each element’s type. Much of the definitionof
PList follows that of
DList . At the type level, we collect the values that index the list’s elements.
PList differs such that the ‘cons’ constructor, (::) , requires implicit proof that the element to beadded also satisfies the given predicate. Proof of predicate satisfaction is collected in the type foreach element in prf using a
DList instance. Predicates are themselves dependently typed values.To illustrate how
PList works, let us revist the introductory example of modelling a list ofTODO items, and how to model a list of complete items. First we define a predicate
IsComplete as: data IsComplete : Status -> Type whereIsDone : IsComplete DONE
With
IsComplete we can now specify a list of completed items: items : PList Status Item IsComplete[DONE,DONE] [IsDone,IsDone]items = MkItem DONE "Write Paper":: MkItem DONE "Proof Read":: Nil
For each element in items , the value parameterising the type, and proof that the collected valuesatisfies
IsComplete are collected. If we were to add a non-complete item (i.e.
TODO or STARTED )Idris’ will fail to compile as neither item’s status satisfies
IsComplete .The type of the ‘cons’ constructor for
PList uses an implicit argument ( prf ) to establish proofthat the element to be added satisfies the list’s predicate. Here use of Idris’ proof search mechanismis not suitable to construct the proof. If we wish to construct arbitrary predicates over
PList instances Idris’ proof search will not be able to construct the proof. There is not enough concreteinformation. Take for example, the definition of
NonEmpty presented in Figure 4.
Short Note on Collecting Dependently Typed Values 7 data NonEmpty : PList aTy eTy pTy as prfs-> TypewhereIsNonEmpty : NonEmpty (x::rest)
Fig. 4. The
NonEmpty
Predicate for
PList . head : (xs : PList aTy eTy pTy (a::as) prfs)-> {auto ok : NonEmpty xs}-> {auto ok' : NonEmpty (a::as)}-> {auto ok'' : NonEmpty prfs}-> eTy ahead (elem :: rest) = elem (a) Head tail : (xs : PList aTy eTy pTy (a::as) (p::prfs))-> {auto ok : NonEmpty xs}-> {auto ok' : NonEmpty (a::as)}-> {auto ok'' : NonEmpty (p::prfs)}-> PList aTy eTy pTy as prfstail (elem::rest) = rest (b) Tail take : (n : Nat)-> (xs : PList aTy elemTy predTy as prfs)-> PList aTy elemTy predTy (take n as)(take n prfs)take Z xs = Niltake (S k) [] = Niltake (S k) (elem ::rest) = elem :: take k rest (c) Take drop : (n : Nat)-> (xs : PList aTy elemTy predTy as prfs)-> PList aTy elemTy predTy (drop n as)(drop n prfs)drop Z rest = restdrop (S k) [] = Nildrop (S k) (elem::rest) = drop k rest (d) DropFig. 5. Example functions operating on PList instances.
Although the implicit argument prf ensures that the value in the list is only the value presentedby the element, it does make adding elements to the list harder. Unless the implicit argument isexplicitly mentioned, Idris will not search for the proof. We address this through provision of asecondary ‘cons’ function ( add ) that will have the necessary information for Idris’ proof search toconstruct the proof. add : (elem : elemTy x)-> {auto prf : predTy x}-> (rest : PList aTy elemTy predTy xs ps)-> PList aTy elemTy predTy (x::xs) (prf::ps)add x prf rest = x::rest
Figure 5 illustrates how common list operations can be defined for
PList are constructed. Notetheir similarity to the implementations for
DList in Figure 2, and those for standard lists. Noteagain that the operations performed at the value level for each element are also mirrored in thevalues collected at the type level.We can now use
PList to complete our model of a
Json document as an dependently typedADT. Using the the predicate
JPred from Section 3 we can now rewrite the data constructors for
JArray and
JMap to use
PList . Figure 6 presents the complete and final definition for our
Json document.The benefit of using
PList is that no secondary data types are required to describe the structureof a
Json document, the inductive structure of ADT is kept.
There are several alternative methods with which dependently typed values can be collected.
Jan de Muijnck-Hughes data JSONDoc : JTy -> Type whereJStr : String -> JSONDoc VALUEJNum : Float -> JSONDoc VALUEJBool : Bool -> JSONDoc VALUEJNull : JSONDoc VALUEJArray : PList JTy JSONDoc JPred as prfs -> JSONDoc ARRAYJMap : PList JTy (\ty => (String, JSONDoc ty)) JPred as prfs -> JSONDoc MAPJDoc : JSONDoc MAP -> JSONDoc DOC
Fig. 6. Complete Implementation of an ADT for
Json . Dependent pairs allow one to specify a dependency between thesecond element in the pair to the value presented as the first element. Dependent Pairs wouldallow us to collect dependently typed values much the same as
DList . However, this requires thatthe value in the type is presented at the value level, making programming with such lists morecumbersome due to extra information.
DList is a formulation of a list of dependent pairs in whichthe depended upon value is hidden away at the type level.Further, one can constrain the elements in the list of dependent pairs using a nested tuple. Usingthe example of a predicated list for TODO items from Section 4 an alternative construction wouldbe: items : List (ty : Status ** IsComplete ty** Item ty)items = (DONE ** IsDone** MkItem DONE "Writing Paper"):: Nil the contents of the list are not constrained. One will need to introduce a predicate to constrainthe contents.
Another approach would be to use Heterogeneous vectors: Listswith a prescribed length whose elements can be of any type. However, there are no restrictions onthe types that can be listed within such vectors.
Idris allows list syntax to be provided for data structures that overrivethe
Nil and (::) constructors. A common idiom within Idris is the creation of bespoke lists usingthis syntax. However, a custom list is required to collect each different dependent type. Operationson lists are not generic and for each dependent type all operations on list like structures haveto be written for each list.
PList and
DList provide generic structures and operations on thosestructures.
Dependently typed languages provide a means to existentially quantify proof that a predicateholds over a list of values using parameterised types. Two such examples are the
All and
Any datatypes.
DList and
PList are two comparable structures. However,
Any and
All are concerned withpresenting proofs that a list of homogeneously typed values satisfy some predicate. Further, thesedata structures present data structure that are a collection of proofs that the values satisfy thepredicate. With
DList and
PList , the proofs are the values in the type.
Short Note on Collecting Dependently Typed Values 9
The variant of
Json , used as a running example, provides an exemplar of the limitations of simpletypes to accurately capture the inductive structure of some real world data structures. Modellingthese documents using a dependent type and dependently typed containers shows how succinctand accurate data structures can be constructed.
DList and
PList have been used in several exist-ing Idris packages to provide such succinct data structures. idris-xml . Presents a library for working with XML documents, and allows for simple queriesusing an XPath like language de Muijnck-Hughes [7]. Here
PList is used to capture the list ofelements presented at each node in the document. Using
PList facilitates the construction of asingle ADT to represent the structure of an XML document in its entirety. idris-commons . Presents a library collecting ‘common’ modules for Idris whose size does notmerit distinct Idris packages [5]. Within idris-commons is a module for working with
Json . TheADTs for the
Json format utilises our dependent list structure (
PList ) as a proof-of-concept . Futurework will be to include data types for
Yaml , INI , Toml , and
Conf that also use
PList . DList and
PList are dependently typed containers to collect dependently typed values as a ‘cons’-style list. For both of these data structures a library of generic operations can be defined and reused,where once bespoke structures and operations were created.These structures are useful when constructing inductive ADTs that are dependently typed. Thiswas demonstrated through specification of a data structure for
Json documents, and links to otherreal-world uses.Both
DList and
PList have been made available online for use by others when programmingin Idris [6].
REFERENCES [1] Lennart Augustsson and Magnus Carlsson. 1999. An Exercise in Dependent Types: A Well-Typed Interpreter. In
InWorkshop on Dependent Types in Programming, Gothenburg .[2] Edwin Brady. 2013. Idris, a general-purpose dependently typed programming language: Design and implementation.
Journal of Functional Programming
23 (Sept. 2013), 552–593. Issue 05. https://doi.org/10.1017/S095679681300018X[3] David Raymond Christiansen. 2013.
Looking Outward: When Dependent Types Meet I/O . Master’s thesis. IT Universityof Copenhagen.[4] Jan de Muijnck-Hughes. 2015. Machine Checkable Design Patterns using Dependent Types and Domain Specific Goal-Oriented Modelling Languages. (10 2015).[5] J. de Muijnck-Hughes. 2018. Commons: Utilities for processing configuration file formats in Idris. (2018).https://github.com/jfdm/idris-config[6] J. de Muijnck-Hughes. 2018. Containers: Containers for Idris. (2018). https://github.com/jfdm/idris-containers[7] J. de Muijnck-Hughes. 2018. XML: Modelling and processing XML using DOM and XPath in Idris. (2018).https://github.com/jfdm/idris-xml[8] Ulf Norell. 2009. Dependently Typed Programming in Agda. In