Dependent type seems to “not work” when generated by Scala macro

Question

Apologies for the handwavey title. I’m not entirely sure how to phrase the question succinctly, since I’ve never encountered something like this before.

Background info:

I have the following trait, where the type U is meant to hold a Shapeless extensible record type:

trait Flattened[T] {
  type U <: shapeless.HList
  def fields: U
}

I’m using a blackbox macro (for reasons outside the scope of this question) to create new instances of the trait:

object flatten {

  import scala.language.experimental.macros
  import scala.reflect.macros.blackbox.Context

  def apply[T](struct: T): Flattened[T] =
    macro impl[T]

  def impl[T : c.WeakTypeTag](c: Context)(struct: c.Tree): c.Tree = {
    import c.universe._

    // AST representing a Shapeless extensible record (actual
    // implementation uses values in `struct` to construct this
    // AST, but I've simplified it for this example).
    val fields: Tree = q"""("a" ->> 1) :: ("b" ->> "two") :: ("c" ->> true) :: HNil"""

    // Result type of the above AST
    val tpe: TypeTree = TypeTree(c.typecheck(q"$fields").tpe)

    q"""
    new Flattened[${weakTypeOf[T]}] {
      import shapeless._
      import syntax.singleton._
      import record._

      type U = $tpe
      val fields = $fields
    }
    """
  }
}

The problem:

The problem is, when I use this macro to create a new instance of Flattened, the type of fields is no longer an extensible record:

import shapeless._
import syntax.singleton._
import record._

val s = "some value... it doesn't matter for this example, since it isn't used. I'm just putting something here so the example compiles and runs in a REPL."
val t = flatten(s)

val fields = t.fields
// fields: t.U = 1 :: "two" :: true :: HNil

fields("a")  // compile error!

// The compile error is:
//     cmd5.sc:1: No field String("a") in record ammonite.$sess.cmd4.t.U
//     val res5 = fields("a")
//                      ^
//     Compilation Failed

Side note:

Oddly, if I do by hand what the macro does, it works:

// I can't actually instantiate a new `Flattened[_]` manually, since
// defining the type `U` would be convoluted (not even sure it can be
// done), so an object with the same field is the next best thing.
object Foo {
  import shapeless._
  import syntax.singleton._
  import record._

  val fields = ("a" ->> 1) :: ("b" ->> "two") :: ("c" ->> true) :: HNil
}

val a = Foo.fields("a")
// a: Int = 1

val b = Foo.fields("b")
// b: String = "two"

val c = Foo.fields("c")
// c: Boolean = true

Why is there this discrepancy, and how can I get the macro version to behave the same as the manual version?

Answer 1

Nothing's wrong with your macro, probably. It's the type signatures:

def apply[T](struct: T): Flattened[T] = macro impl[T]
def impl[T : c.WeakTypeTag](c: Context)(struct: c.Tree): c.Tree

You are using a blackbox macro, and, according to the documentation, blackbox macros are true to their signatures. That is, even though impl produces a Flattened[T] { type U = ... }, the fact that it is a blackbox macro means that scalac always wraps it in (_: Flattened[T]), forgetting the definition of U in the refinement. Make it a whitebox macro:

// import scala.reflect.macros.blackbox.context // NO!
import scala.reflect.macros.whitebox.context
def impl[T: c.WeakTypeTag](c: Context)(struct: c.Tree): c.Tree