Scala 3 Data Transformation Library: ducktape 0.2.0.
Introduction: Is ducktape still all duct tape under the hood? Or, why are macros so cool that I’m basically rewriting it for the third time?
Before I go off talking about the insides of the library, let’s first touch base on what ducktape actually is, its Github page describes it as this:
Automatic and customizable compile time transformations between similar case classes and sealed traits/enums, essentially a thing that glues your code.
The last part of that sentence also shines some light on why the name is what it is. It was originally supposed to be called ‘ducttape’ to signify its potential of generating glue code for you. However, the idea of joining two words that end and start with the same letter wouldn’t let me sleep, so I went with what I considered an absolute kino of a name at the time – ducktape, but I digress.
You can also check the first blogpost about ducktape: Scala 3 Data Transformation Library: Automating Data Transformations with ducktape
Motivating example
For the purposes of showing what the library is capable of, let’s consider two nearly identical models, starting with a wire model:
import io.github.arainko.ducktape.*
import java.time.Instant
object wire {
final case class Person(
firstName: String,
lastName: String,
paymentMethods: List[wire.PaymentMethod],
status: wire.Status,
updatedAt: Option[Instant],
)
enum Status:
case Registered, PendingRegistration, Removed
enum PaymentMethod:
case Card(name: String, digits: Long, expires: Instant)
case PayPal(email: String)
case Cash
}
…and a domain model:
object domain {
final case class Person( // <-- fields reshuffled
lastName: String,
firstName: String,
status: Option[domain.Status], // <-- 'status' in the domain model is optional
paymentMethods: Vector[domain.Payment], // <-- collection type changed from a List to a Vector
updatedAt: Option[Instant],
)
enum Status:
case Registered, PendingRegistration, Removed
case PendingRemoval // <-- additional enum case
enum Payment:
case Card(name: String, digits: Long, expires: Instant)
case PayPal(email: String)
case Cash
}
So, we can imagine having to somehow map between these two since the wire model is something that our HTTP layer spits out, eg. given a wire.Person
defined as such:
val wirePerson: wire.Person = wire.Person(
"John",
"Doe",
List(
wire.PaymentMethod.Cash,
wire.PaymentMethod.PayPal("[email protected]"),
wire.PaymentMethod.Card("J. Doe", 12345, Instant.now)
),
wire.Status.PendingRegistration,
Some(Instant.ofEpochSecond(0))
)
We can turn it into a domain.Person
in a single, yet wonderful, line of code:
val domainPerson = wirePerson.to[domain.Person]
// domainPerson: Person = Person(
// lastName = "Doe",
// firstName = "John",
// status = Some(value = PendingRegistration),
// paymentMethods = Vector(
// Cash,
// PayPal(email = "[email protected]"),
// Card(
// name = "J. Doe",
// digits = 12345L,
// expires = 2024-02-15T21:23:32.423498Z
// )
// ),
// updatedAt = Some(value = 1970-01-01T00:00:00Z)
// )
Can you imagine writing all that by hand?
(({
val paymentMethods$2: Vector[Payment] =
MdocApp.this.wirePerson.paymentMethods
.map[Payment]((src: PaymentMethod) =>
if (src.isInstanceOf[Card])
new Card(
name = src.asInstanceOf[Card].name,
digits = src.asInstanceOf[Card].digits,
expires = src.asInstanceOf[Card].expires
)
else if (src.isInstanceOf[PayPal])
new PayPal(email = src.asInstanceOf[PayPal].email)
else if (src.isInstanceOf[Cash.type]) MdocApp.this.domain.Payment.Cash
else
throw new RuntimeException(
"Unhandled condition encountered during Coproduct Transformer derivation"
)
)
.to[Vector[Payment]](iterableFactory[Payment])
val status$2: Some[Status] = Some.apply[Status](
if (MdocApp.this.wirePerson.status.isInstanceOf[Registered.type])
MdocApp.this.domain.Status.Registered
else if (
MdocApp.this.wirePerson.status.isInstanceOf[PendingRegistration.type]
) MdocApp.this.domain.Status.PendingRegistration
else if (MdocApp.this.wirePerson.status.isInstanceOf[Removed.type])
MdocApp.this.domain.Status.Removed
else
throw new RuntimeException(
"Unhandled condition encountered during Coproduct Transformer derivation"
)
)
new Person(
lastName = MdocApp.this.wirePerson.lastName,
firstName = MdocApp.this.wirePerson.firstName,
status = status$2,
paymentMethods = paymentMethods$2,
updatedAt = MdocApp.this.wirePerson.updatedAt
)
}: Person): Person)
I’d rather stare at a CI pipeline that fails after 18 minutes because I forgot to format my code than write this piece of code by hand.
Previous ‘art’
As hinted in the overlong title, which is itself inspired by the title of this album (go listen to it, it’s got a cool lizard on the cover)…
…the library has been through 2 phases already.
The first of which is documented in this blogpost and can be summarized as match type abuse (aka. the 0.0.x line of releases).
The second one is the 0.1.x line of releases which pretty much scrapped all traces of its predecessor and replaced it with those pesky macros and developed an over-reliance on automatic typeclass derivation which then had to be unpacked in a process I can only call ‘beta-reduction at home’ to not generate unnecessary Transformer
instances (the typeclass being automatically derived) at runtime. All in all, a pretty fun piece of code.
The third and newest iteration is the 0.2.x line – this time I took a more thought-through approach to structuring the library than constantly telling the compiler to derive that good-good.
The main motivation was being able to support stuff like nested configuration of fields and cases (which IMO were the worst offenders to usability of the library itself since even if your transformation was aaaaalmost there but a single field was missing in a nested case class you were done for and had to create a new Transformer instance and put it in implicit scope) and being able to show the user all of the failures that occurred all at once in addition to being more actionable than just ‘Yeah, the field 'chips' is missing in Diner
‘.
Reifying all the stuff
Most of the issues of 0.1.x came from relying on automatic derivation of Transformers
to do everything, which resulted in the library not being in control of anything since it gave away control to the compiler right after pulling out of the driveway, so to be able to do all of the things listed above I had to find a way of introspecting and somehow transforming the transformations, which came down to data-fying each and every step.
Let’s take a look at a high-level overview of the new architecture:
So, what is all this stuff you may ask and am I even showing you this… Let’s try to peel it back layer by layer starting from the topmost piece of the graph, i.e:
The structure of a Structure
A Structure
is meant to capture stuff like fields of a case class (their names and Structures
that correspond to them), children of an enum/sealed trait and some more specialized stuff like optional types, collections, value classes, singletons etc. (the implementation itself looks like this), so going back to our Motivating Example, a Structure
of a wire.Person
looks like this:
Product(
tpe = Type.of[Person],
path = Person,
fields = Map(
"updatedAt" -> Lazy(
tpe = Type.of[Option[Instant]],
path = Person.updatedAt
),
"paymentMethods" -> Lazy(
tpe = Type.of[List[PaymentMethod]],
path = Person.paymentMethods
),
"lastName" -> Lazy(tpe = Type.of[String], path = Person.lastName),
"firstName" -> Lazy(tpe = Type.of[String], path = Person.firstName),
"status" -> Lazy(tpe = Type.of[Status], path = Person.status)
)
)
That doesn’t tell us everything since most of that stuff is Lazy
, but it’s lazy for a reason not because it doesn’t want to do anything, that reason being recursive types, which are suspiciously prevalent in our day-to-day life, looking no further than Scala’s List
.
See, if it wasn’t for those Lazy
nodes we’d be sending the compiler on a trip it wouldn’t be coming back from every time we encounter a recursive type, but if we encode the notion of laziness we can expand those calls later on when we’ll be taking special care to not overflow the stack.
You may ask, how is a Structure
actually constructed – the answer to that, and most other things, is a big pattern match!
Click here to see it in all of its glory
object Structure {
def of[A: Type](path: Path)(using Quotes): Structure = {
import quotes.reflect.*
Type.of[A] match {
case tpe @ '[Nothing] =>
Structure.Ordinary(tpe, path)
case tpe @ '[Option[param]] =>
Structure.Optional(tpe, path, Structure.of[param](path.appended(Path.Segment.Element(Type.of[param]))))
case tpe @ '[Iterable[param]] =>
Structure.Collection(tpe, path, Structure.of[param](path.appended(Path.Segment.Element(Type.of[param]))))
case tpe @ '[AnyVal] if tpe.repr.typeSymbol.flags.is(Flags.Case) =>
val repr = tpe.repr
val param = repr.typeSymbol.caseFields.head
val paramTpe = repr.memberType(param)
Structure.ValueClass(tpe, path, paramTpe.asType, param.name)
case _ =>
Expr.summon[Mirror.Of[A]] match {
case None =>
Structure.Ordinary(Type.of[A], path)
case Some(value) =>
value match {
case '{
type label <: String
$m: Mirror.Singleton {
type MirroredLabel = `label`
}
} =>
val value = materializeSingleton[A]
Structure.Singleton(Type.of[A], path, constantString[label], value.asExpr)
case '{
type label <: String
$m: Mirror.SingletonProxy {
type MirroredLabel = `label`
}
} =>
val value = materializeSingleton[A]
Structure.Singleton(Type.of[A], path, constantString[label], value.asExpr)
case '{
$m: Mirror.Product {
type MirroredElemLabels = labels
type MirroredElemTypes = types
}
} =>
val structures =
tupleTypeElements(TypeRepr.of[types])
.zip(constStringTuple(TypeRepr.of[labels]))
.map((tpe, name) =>
name -> (tpe.asType match {
case '[tpe] => Lazy.of[tpe](path.appended(Path.Segment.Field(Type.of[tpe], name)))
})
)
.toMap
Structure.Product(Type.of[A], path, structures)
case '{
$m: Mirror.Sum {
type MirroredElemLabels = labels
type MirroredElemTypes = types
}
} =>
val structures =
tupleTypeElements(TypeRepr.of[types])
.zip(constStringTuple(TypeRepr.of[labels]))
.map((tpe, name) =>
name -> (tpe.asType match { case '[tpe] => Lazy.of[tpe](path.appended(Path.Segment.Case(Type.of[tpe]))) })
)
.toMap
Structure.Coproduct(Type.of[A], path, structures)
}
}
}
}
}
Here’s a link to the full implementation.
In short, it matches on a type given to it in the type parameter of Structure.of[A]
, gets the first few special cases out of the way:
Nothing
since it’s a subtype of all other types,Option[param]
,Iterable[param]
,AnyVal
that’s also a case class (also known as a value class)
and then proceeds to roll up its sleeves by trying to summon an instance of a Mirror
and matches on its subtypes:
Mirror.Singleton
for Scala 3 singletons,Mirror.SingletonProxy
for Scala 2 singletons,Mirror.Product
for case classes,Mirror.Sum
for sealed traits/enums
to recursively derive Structures
for fields/known subtypes. This gives us just enough information to be able to collate two Structures
together in order to create a transformation plan.
The makings of a Plan
When I said that it’s just enough information to create a transformation plan I meant that literally, the next step in the diagram is creating a Plan[Plan.Error]
.
Before diving into it let’s first see what the final product looks like for a transformation between wire.PaymentMethod
and domain.Payment
:
BetweenCoproducts(
source = Structure.of[PaymentMethod],
dest = Structure.of[Payment],
casePlans = Vector(
BetweenProducts(
source = Structure.of[Card],
dest = Structure.of[Card],
fieldPlans = Map(
"name" -> Upcast(
source = Structure.of[String],
dest = Structure.of[String]
),
"digits" -> Upcast(
source = Structure.of[Long],
dest = Structure.of[Long]
),
"expires" -> Upcast(
source = Structure.of[Instant],
dest = Structure.of[Instant]
)
)
),
BetweenProducts(
source = Structure.of[PayPal],
dest = Structure.of[PayPal],
fieldPlans = Map(
"email" -> Upcast(
source = Structure.of[String],
dest = Structure.of[String]
)
)
),
BetweenSingletons(source = Structure.of[Cash], dest = Structure.of[Cash])
)
)
The above can be roughly read as ‘this a transformation between coproducts (source being wire.PaymentMethod
and the destination domain.Payment
), for which the transformations between the cases of that coproduct is as follows:
wire.PaymentMethod.Card
maps todomain.Payment.Card
which itself is a product transformation for fields:- “name” which is just an upcast from the source field,
- “digits” which is just an upcast from the source field,
- “expires” which is just an upcast from the source field
wire.PaymentMethod.PayPal
maps todomain.Payment.PayPal
which itself is a product transformation for fields:- “name” which is just an upcast from the source field
wire.PaymentMethod.Cash
maps todomain.Payment.Cash
which is a singleton transformation (i.e. the value of the destination singleton is just inserted)’.
Plans
are meant to represent a higher level representation of a transformation specifically tailored to be modifiable in a variety of ways, eg. by changing one of the nodes to a constant value. Its declaration roughly cut down to only nodes visible in the example looks like this (for the real deal take a look here):
sealed trait Plan[+E <: Plan.Error]
object Plan {
case class Upcast(
source: Structure,
dest: Structure
) extends Plan[Nothing]
case class BetweenSingletons(
source: Structure.Singleton,
dest: Structure.Singleton
) extends Plan[Nothing]
case class BetweenProducts[+E <: Plan.Error](
source: Structure.Product,
dest: Structure.Product,
fieldPlans: Map[String, Plan[E]]
) extends Plan[E]
case class BetweenCoproducts[+E <: Plan.Error](
source: Structure.Coproduct,
dest: Structure.Coproduct,
casePlans: Vector[Plan[E]]
) extends Plan[E]
case class Error(
source: Structure,
dest: Structure,
message: ErrorMessage,
suppressed: Option[Plan.Error]
) extends Plan[Plan.Error]
}
… more cases elided for readability.
There isn’t that much going on here, besides that weird +E <: Plan.Error
– why is it there exactly?
To answer that question let me pull up an another example, let’s examine what happens when we try to create a transformation plan between case classes that don’t fit each other:
case class Car(brand: String, age: Int, noOfSeats: Long)
case class Plane(brand: String, noOfSeats: Long, age: Int, wingColor: String)
BetweenProducts(
source = Structure.of[Car],
dest = Structure.of[Plane],
fieldPlans = Map(
"brand" -> Upcast(
source = Structure.of[String],
dest = Structure.of[String]
),
"noOfSeats" -> Upcast(
source = Structure.of[Long],
dest = Structure.of[Long]
),
"age" -> Upcast(source = Structure.of[Int], dest = Structure.of[Int]),
"wingColor" -> Error(
source = Structure.of[Nothing],
dest = Structure.of[String],
message = NoFieldFound(
fieldName = "wingColor",
fieldTpe = Type.of[String],
sourceTpe = Type.of[Car]
),
suppressed = None
)
)
)
We can see that in case something doesn’t fully line up a Plan
is still created but with a Plan.Error
node somewhere inside, this is what the E
type parameter of Plan
is meant to represent (namely, the possibility of being erroneous), as for why is it declared as covariant, it’s due to one of Scala 3’s new features.
That feature being a proper support for GADTs in pattern matching, generally this means that for an enum declared as such:
enum Data[+E <: Throwable] {
case NonFallible extends Data[Nothing]
case SomeOtherStuff(value: String) extends Data[Nothing]
case Error(error: Throwable) extends Data[Throwable]
}
…if we end up with a value of Data[Nothing]
and pattern match on it the compiler will yell at us when we try to put Data.Error
as one of the cases:
val dataNonFallible: Data[Nothing] = Data.NonFallible
dataNonFallible match {
case Data.NonFallible => "non fallible"
case Data.SomeOtherStuff(value) => value
case Data.Error(error) => "a warning will be issued on this line"
// ^ Unreachable case
}
But if we end up with a value of Data[Throwable]
we will be forced to pattern match on all of the cases:
val dataFallible: Data[Throwable] = Data.Error(Exception("woops"))
dataFallible match {
case Data.NonFallible => "non fallible"
case Data.SomeOtherStuff(value) => value
// ^ match may not be exhaustive.
// It would fail on pattern case: Data.Error(_, _)
}
This brings us back to the usage of +E <: Plan.Error
as a way to keep track of possibly erroneous plans at compile time (later on we will find out that to interpret a Plan
into an actual transformation we need to feed the interpreter a Plan[Nothing]
that is, a Plan
without any Plan.Error
nodes).
Now onto how a Plan
is actually constructed, the very first two things you need are two Structures which are then plopped into a method called Planner.create
which then does a big pattern match on those two Structures
trying to extract information by matching on its subtypes and constructing plans accordingly.
For example, given two Structure.Products
a Plan.BetweenProducts
will be constructed unless a user-supplied instance of a Transformer
is defined in the current implicit scope, in which case it takes precedence over any automatically constructed transformations.
Click here to see more pattern matching
object Planner {
import Structure.*
def between(source: Structure, dest: Structure)(using Quotes, TransformationSite) = {
given Depth = Depth.zero
recurse(source, dest)
}
private def recurse(
source: Structure,
dest: Structure
)(using quotes: Quotes, depth: Depth, transformationSite: TransformationSite): Plan[Plan.Error] = {
import quotes.reflect.*
given Depth = Depth.incremented(using depth)
(source.force -> dest.force) match {
case _ if Depth.current > 64 =>
Plan.Error(source, dest, ErrorMessage.RecursionSuspected, None)
case (source: Product, dest: Function) =>
planProductFunctionTransformation(source, dest)
case UserDefinedTransformation(transformer) =>
Plan.UserDefined(source, dest, transformer)
case (source, dest) if source.tpe.repr <:< dest.tpe.repr =>
Plan.Upcast(source, dest)
case (source @ Optional(_, _, srcParamStruct)) -> (dest @ Optional(_, _, destParamStruct)) =>
Plan.BetweenOptions(
source,
dest,
recurse(srcParamStruct, destParamStruct)
)
case source -> (dest @ Optional(_, _, paramStruct)) =>
Plan.BetweenNonOptionOption(
source,
dest,
recurse(source, paramStruct)
)
case (source @ Collection(_, _, srcParamStruct)) -> (dest @ Collection(_, _, destParamStruct)) =>
Plan.BetweenCollections(
source,
dest,
recurse(srcParamStruct, destParamStruct)
)
case (source: Product, dest: Product) =>
planProductTransformation(source, dest)
case (source: Coproduct, dest: Coproduct) =>
planCoproductTransformation(source, dest)
case (source: Structure.Singleton, dest: Structure.Singleton) if source.name == dest.name =>
Plan.BetweenSingletons(source, dest)
case (source: ValueClass, dest) if source.paramTpe.repr <:< dest.tpe.repr =>
Plan.BetweenWrappedUnwrapped(source, dest, source.paramFieldName)
case (source, dest: ValueClass) if source.tpe.repr <:< dest.paramTpe.repr =>
Plan.BetweenUnwrappedWrapped(source, dest)
case DerivedTransformation(transformer) =>
Plan.Derived(source, dest, transformer)
case (source, dest) =>
Plan.Error(
source,
dest,
ErrorMessage.CouldntBuildTransformation(source.tpe, dest.tpe),
None
)
}
}
}
See? I told you everything is just a big pattern match. If you require even more context head on over to here to see the code in an even nittier and grittier detail.
The rest of the owl
Now onto the lower part of the graph.
Going back to my previous tangent about Plans
being possibly erroneous, having Plan.Error
nodes inside our transformation Plan
is a big no-no from the translate-to-actual-Scala-code point of view. What can you even do with an error node? Put a ???
in its place? Throw an exception?
There’s no good answer besides aborting the compilation but that would disallow us from reporting all of the errors at once. The solution to that problem is refining a possibly erroneous Plan[Plan.Error]
into a Plan[Nothing]
(that enforces no Plan.Error
nodes at compiletime!) while also collecting all of the Plan.Error
nodes to report to the user.
The whole implementation comes down to recursively diving into the Plan
tree and accumulating Plan.Error
nodes we’ve encountered. At the very end if there aren’t any error nodes we employ some dirty tricks and cast the input Plan
to Plan[Nothing]
(since the E param of Plan is effectively a phantom type) and pat ourselves on a back for doing such a good job.
Click here if you want to see some pattern-matching
object PlanRefiner {
def run(plan: Plan[Plan.Error]): Either[NonEmptyList[Plan.Error], Plan[Nothing]] = {
@tailrec
def recurse(stack: List[Plan[Plan.Error]], errors: List[Plan.Error]): List[Plan.Error] =
stack match {
case head :: next =>
head match {
case plan: Plan.Upcast =>
recurse(next, errors)
case Plan.BetweenProducts(_, _, fieldPlans) =>
recurse(fieldPlans.values.toList ::: next, errors)
case Plan.BetweenCoproducts(_, _, casePlans) =>
recurse(casePlans.toList ::: next, errors)
case Plan.BetweenProductFunction(_, _, argPlans) =>
recurse(argPlans.values.toList ::: next, errors)
case Plan.BetweenOptions(_, _, plan) =>
recurse(plan :: next, errors)
case Plan.BetweenNonOptionOption(_, _, plan) =>
recurse(plan :: next, errors)
case Plan.BetweenCollections(_, _, plan) =>
recurse(plan :: next, errors)
case plan: Plan.BetweenSingletons =>
recurse(next, errors)
case plan: Plan.UserDefined =>
recurse(next, errors)
case plan: Plan.Derived =>
recurse(next, errors)
case plan: Plan.Configured =>
recurse(next, errors)
case plan: Plan.BetweenWrappedUnwrapped =>
recurse(next, errors)
case plan: Plan.BetweenUnwrappedWrapped =>
recurse(next, errors)
case error: Plan.Error =>
recurse(next, error :: errors)
}
case Nil => errors
}
val errors = recurse(plan :: Nil, Nil)
// if no errors were accumulated that means there are no Plan.Error nodes which means we operate on a Plan[Nothing]
NonEmptyList.fromList(errors).toLeft(plan.asInstanceOf[Plan[Nothing]])
}
}
After the refinement is done we find ourselves with an Either[NonEmptyList[Plan.Error], Plan[Nothing]]
in hand, we eliminate the left side by doing 🪄 magical 🪄 things with the errors (like lining up their positions, deduping, reporting and all that good stuff, very boring tho) – after that’s done we’re left with a Plan[Nothing]
which means it’s time to grind that plan into an AST with a PlanInterpreter
.
Going back to an example from a previous chapter, the Plan
showcased there will be expanded into this code:
{
val source$proxy1: PaymentMethod =
MdocApp.this.wirePerson.paymentMethods.head
(if (source$proxy1.isInstanceOf[Card])
new Card(
name = source$proxy1.asInstanceOf[Card].name,
digits = source$proxy1.asInstanceOf[Card].digits,
expires = source$proxy1.asInstanceOf[Card].expires
)
else if (source$proxy1.isInstanceOf[PayPal])
new PayPal(email = source$proxy1.asInstanceOf[PayPal].email)
else if (source$proxy1.isInstanceOf[Cash.type])
MdocApp.this.domain.Payment.Cash
else
throw new RuntimeException(
"Unhandled condition encountered during Coproduct Transformer derivation"
): Payment): Payment
}
If we squint hard enough we should be able to kind of see what each Plan
maps to in terms of actual code, namely:
Plan.BetweenCoproducts
is expanded into an if expression with.isInstanceOf
calls to determine the subtype and an expansion of the plan attached to a given case (along with a cast thrown in there for good measure),Plan.BetweenProducts
is expanded into an invocation of the primary constructor with the fields recursively expanded under their respective names,Plan.Upcast
just forwards the parameter it is given (since it implies that value fits as is),Plan.BetweenSingletons
inserts the value of the destination singleton right then and there.
To quickly showcase the translation of other Plan
nodes let’s come up with another example
final case class TwoWheeler(
colorPalette: List[String],
numberOfGears: Option[Long],
seatColor: Option[String]
)
final case class Bike(
numberOfGears: Long,
colorPalette: Vector[Color],
seatColor: Option[Color]
)
final case class Color(value: String) extends AnyVal
Now, the Plan
`for a transformation between a Bike
and a TwoWheeler
would look as such:
BetweenProducts(
source = Structure.of[Bike],
dest = Structure.of[TwoWheeler],
fieldPlans = Map(
"colorPalette" -> BetweenCollections(
source = Structure.of[Vector[Color]],
dest = Structure.of[List[String]],
plan = BetweenWrappedUnwrapped(
source = Structure.of[Color],
dest = Structure.of[String],
fieldName = "value"
)
),
"numberOfGears" -> BetweenNonOptionOption(
source = Structure.of[Long],
dest = Structure.of[Option[Long]],
plan = Upcast(source = Structure.of[Long], dest = Structure.of[Long])
),
"seatColor" -> BetweenOptions(
source = Structure.of[Option[Color]],
dest = Structure.of[Option[String]],
plan = BetweenWrappedUnwrapped(
source = Structure.of[Color],
dest = Structure.of[String],
fieldName = "value"
)
)
)
)
There are a couple of new faces here, like BetweenOptions
, BetweenNonOptionOption
, BetweenCollections
and BetweenWrappedUnwrapped
, let’s find out what they do from looking at the derived code:
((new TwoWheeler(
colorPalette = bike.colorPalette
.map[String]((src: Color) => src.value)
.to[List[String]](iterableFactory[String]),
numberOfGears = Some.apply[Long](bike.numberOfGears),
seatColor = bike.seatColor.map[String]((`src₂`: Color) => `src₂`.value)
): TwoWheeler): TwoWheeler)
Having read all this we can conclude that:
Plan.BetweenCollection
expands into a.map
call with a recursive expansion of a plan of the parameter and a.to(DestCollectionFactory)
call at the end,Plan.BetweenWrappedUnwrapped
unwraps a value class by getting the value of its single field,Plan.BetweenNonOptionOption
wraps the expansion of the plan inside with aSome.apply
(effectively just wrapping it),Plan.BetweenOptions
maps over the option while also expanding the plan inside the lambda.
The implementation itself is once again a giant pattern match
object PlanInterpreter {
def run[A: Type](plan: Plan[Nothing], sourceValue: Expr[A])(using Quotes): Expr[Any] =
recurse(plan, sourceValue)(using sourceValue)
private def recurse[A: Type](plan: Plan[Nothing], value: Expr[Any])(using toplevelValue: Expr[A])(using Quotes): Expr[Any] = {
import quotes.reflect.*
plan match {
case Plan.Upcast(_, _) => value
case Plan.Configured(_, _, config) =>
config match {
case Configuration.Const(value, _) =>
value
case Configuration.CaseComputed(_, function) =>
'{ $function.apply($value) }
case Configuration.FieldComputed(_, function) =>
'{ $function.apply($toplevelValue) }
case Configuration.FieldReplacement(source, name, tpe) =>
source.accessFieldByName(name).asExpr
}
case Plan.BetweenProducts(sourceTpe, destTpe, fieldPlans) =>
val args = fieldPlans.map {
case (fieldName, p: Plan.Configured) =>
NamedArg(fieldName, recurse(p, value).asTerm)
case (fieldName, plan) =>
val fieldValue = value.accessFieldByName(fieldName).asExpr
NamedArg(fieldName, recurse(plan, fieldValue).asTerm)
}
Constructor(destTpe.tpe.repr).appliedToArgs(args.toList).asExpr
case Plan.BetweenCoproducts(sourceTpe, destTpe, casePlans) =>
val branches = casePlans.map { plan =>
(plan.source.tpe -> plan.dest.tpe) match {
case '[src] -> '[dest] =>
val sourceValue = '{ $value.asInstanceOf[src] }
IfBranch(IsInstanceOf(value, plan.source.tpe), recurse(plan, sourceValue))
}
}.toList
ifStatement(branches).asExpr
case Plan.BetweenProductFunction(sourceTpe, destTpe, argPlans) =>
val args = argPlans.map {
case (fieldName, p: Plan.Configured) =>
recurse(p, value).asTerm
case (fieldName, plan) =>
val fieldValue = value.accessFieldByName(fieldName).asExpr
recurse(plan, fieldValue).asTerm
}
destTpe.function.appliedTo(args.toList)
case Plan.BetweenOptions(sourceTpe, destTpe, plan) =>
(sourceTpe.paramStruct.tpe -> destTpe.paramStruct.tpe) match {
case '[src] -> '[dest] =>
val optionValue = value.asExprOf[Option[src]]
def transformation(value: Expr[src])(using Quotes): Expr[dest] = recurse(plan, value).asExprOf[dest]
'{ $optionValue.map(src => ${ transformation('src) }) }
}
case Plan.BetweenNonOptionOption(sourceTpe, destTpe, plan) =>
(sourceTpe.tpe -> destTpe.paramStruct.tpe) match {
case '[src] -> '[dest] =>
val sourceValue = value.asExprOf[src]
def transformation(value: Expr[src])(using Quotes): Expr[dest] = recurse(plan, value).asExprOf[dest]
'{ Some(${ transformation(sourceValue) }) }
}
case Plan.BetweenCollections(source, dest, plan) =>
(dest.tpe, source.paramStruct.tpe, dest.paramStruct.tpe) match {
case ('[destCollTpe], '[srcElem], '[destElem]) =>
val sourceValue = value.asExprOf[Iterable[srcElem]]
// TODO: Make it nicer, move this into Planner since we cannot be sure that a factory exists
val factory = Expr.summon[Factory[destElem, destCollTpe]].get
def transformation(value: Expr[srcElem])(using Quotes): Expr[destElem] = recurse(plan, value).asExprOf[destElem]
'{ $sourceValue.map(src => ${ transformation('src) }).to($factory) }
}
case Plan.BetweenSingletons(sourceTpe, destTpe) => destTpe.value
case Plan.BetweenWrappedUnwrapped(sourceTpe, destTpe, fieldName) =>
value.accessFieldByName(fieldName).asExpr
case Plan.BetweenUnwrappedWrapped(sourceTpe, destTpe) =>
Constructor(destTpe.tpe.repr).appliedTo(value.asTerm).asExpr
case Plan.UserDefined(source, dest, transformer) =>
transformer match {
case '{ $t: Transformer[src, dest] } =>
val sourceValue = value.asExprOf[src]
'{ $t.transform($sourceValue) }
}
case Plan.Derived(source, dest, transformer) =>
transformer match {
case '{ $t: Transformer.Derived[src, dest] } =>
val sourceValue = value.asExprOf[src]
'{ $t.transform($sourceValue) }
}
}
}
}
If you’re hungry for more here’s a link to the whole thing.
That’s it?
No, not really – I went past at least a single crucial step – configuration, that is, enabling the user to ‘fix’ broken Plans
with a slew of config options like Field.const
, Case.computed
etc., so the graph shown in Reifying all the stuff actually looks more like this:
Then there’s also Fallible Transformers
which enable things like automatic validations and transformations from incomplete models (in the meantime you can see more on that here).
I hope to try and touch upon those things in a future blogpost, but in the meantime give ducktape 0.2.x a try in your project!
Read more on our blog