Weird behavior trying to convert case classes to heterogeneous lists recursively with Shapeless

Question

I stayed up way too late last night trying to figure out this Shapeless issue and I\'m afraid it\'s going to eat my evening if I don\'t get it off my chest, so here goes.

Answer 1

This now works more or less as written using recent shapeless-2.1.0-SNAPSHOT builds, and a close relative of the sample in this question has been added there as an example.

The problem with the original is that each expansion of a Generic introduces a new HList type into the implicit resolution of the DeepHLister type class instances and, in principle, could produce an HList type that is related to but more complex than some type seen previously during the same resolution. This condition trips the divergence checker and aborts the resolution process.

The exact details of why this happens for D but not for C is lurking in the details of the implementation of Scala's typechecker but, to a rough approximation, the differentiator is that during the resolution for C we see the B (larger) before the A (smaller) so the divergence checker is happy that our types are converging; conversely during the resolution for D we see the A (smaller) before the B (larger) so the divergence checker (conservatively) bails.

The fix for this in shapeless 2.1.0 is the recently enhanced Lazy type constructor and associated implicit macro infrastructure. This allows much more user control over divergence and supports the use of implicit resolution to construct the recursive implicit values which are crucial to the ability to automatically derive type class instances for recursive types. Many examples of this can be found in the shapeless code base, in particular the reworked type class derivation infrastructure and Scrap Your Boilerplate implementation, which no longer require dedicated macro support, but are implemented entirely in terms of the Generic and Lazy primitives. Various applications of these mechanisms can be found in the shapeless examples sub-project.

Answer 2

I took a slightly different approach.

trait CaseClassToHList[X] {
  type Out <: HList
}

trait LowerPriorityCaseClassToHList {
  implicit def caseClass[X](implicit gen: Generic[X]): CaseClassToHList[X] {
    type Out = generic.Repr
  } = null
}

object CaseClassToHList extends LowerPriorityCaseClassToHList {
  type Aux[X, R <: HList] = CaseClassToHList[X] { type Out = R }

  implicit def caseClassWithCaseClasses[X, R <: HList](
    implicit toHList: CaseClassToHList.Aux[X, R],
    nested: DeepHLister[R]): CaseClassToHList[X] {
    type Out = nested.Out
  } = null
}

trait DeepHLister[R <: HList] {
  type Out <: HList
}

object DeepHLister {

  implicit def hnil: DeepHLister[HNil] { type Out = HNil } = null

  implicit def caseClassAtHead[H, T <: HList](
    implicit head: CaseClassToHList[H],
    tail: DeepHLister[T]): DeepHLister[H :: T] {
    type Out = head.Out :: tail.Out
  } = null

  def apply[X <: HList](implicit d: DeepHLister[X]): d.type = null
}

Tested with the following code:

case class A(x: Int, y: String)
case class B(x: A, y: A)
case class C(b: B, a: A)
case class D(a: A, b: B)

object Test {

  val z = DeepHLister[HNil]
  val typedZ: DeepHLister[HNil] {
    type Out = HNil
  } = z

  val a = DeepHLister[A :: HNil]
  val typedA: DeepHLister[A :: HNil] {
    type Out = (Int :: String :: HNil) :: HNil
  } = a

  val b = DeepHLister[B :: HNil]
  val typedB: DeepHLister[B :: HNil] {
    type Out = ((Int :: String :: HNil) :: (Int :: String :: HNil) :: HNil) :: HNil
  } = b

  val c = DeepHLister[C :: HNil]
  val typedC: DeepHLister[C :: HNil] {
    type Out = (((Int :: String :: HNil) :: (Int :: String :: HNil) :: HNil) :: (Int :: String :: HNil) :: HNil) :: HNil 
  } = c

  val d = DeepHLister[D :: HNil]
  val typedD: DeepHLister[D :: HNil] {
    type Out = ((Int :: String :: HNil) :: ((Int :: String :: HNil) :: (Int :: String :: HNil) :: HNil) :: HNil) :: HNil
  } = d
}