Why do compile-time generative techniques for structural typing prevent separate compilation?

Question

I was reading (ok, skimming) Dubochet and Odersky's Compiling Structural Types on the JVM and was confused by the following claim:

Generative techniques create Java interfaces to stand in for structural types on the JVM. The complexity of such techniques lies in that all classes that are to be used as structural types anywhere in the program must implement the right interfaces. When this is done at compile time, it prevents separate compilation.

(emphasis added)

Consider the autoclose example from the paper:

type Closeable = Any { def close(): Unit }

def autoclose(t: Closeable)(run: Closeable => Unit): Unit = {
   try { run(t) }
   finally { t.close }
}

Couldn't we generate an interface for the Closeable type as follows:

public interface AnonymousInterface1 {
   public void close();
}

and transform our definition of autoclose to

// UPDATE: using a view bound here, so implicit conversion is applied on-demand
def autoclose[T <% AnonymousInterface1](t: T)(run: T => Unit): Unit = {
   try { run(t) }
   finally { t.close }
}

Then consider a call-site for autoclose:

val fis = new FileInputStream(new File("f.txt"))
autoclose(fis) { ... }

Since fis is a FileInputStream, which does not implement AnonymousInterface1, we need to generate a wrapper:

class FileInputStreamAnonymousInterface1Proxy(val self: FileInputStream) 
      extends AnonymousInterface1 {
   def close() = self.close();
}

object FileInputStreamAnonymousInterface1Proxy {
   implicit def fis2proxy(fis: FileInputStream): FileInputStreamAnonymousInterface1Proxy =
      new FileInputStreamAnonymousInterface1Proxy(fis)
}

I must be missing something, but it's unclear to me what it is. Why would this approach prevent separate compilation?

Answer 1

I actually use the implicit approach (using typeclasses) you describe in the Scala ARM library. Remember that this is a hand-coded solution to the problem.

The biggest issue here is implicit resolution. The compiler will not generate wrappers for you on the fly, you must do so ahead of time and make sure they are one the implicit scope. This means (for Scala-ARM) that we provide "common" wrapper for whatever resources we can, and fall back to reflection-based types when we can't find the appropriate wrapper. This gives the advantage of allowing the user to specify their own wrapper using normal implicit rules.

See: The Resource Type-trait and all of it's predefined wrappers.

Also, I blogged about this technique describing the implicit resolution magic in more detail: Monkey Patching, Duck Typing and Type Classes.

In any case, you probably don't want to hand-encode a type class everytime you use structural types. If you actually wanted the compiler to automatically create an interface and do the magic for you, it could get messy. Everytime you define a structural type, the compiler will have to create an interface for it (somewhere in the ether perhaps?). We now need to add namespaces for these things. Also, with every call the compiler will have to generate some kind of a wrapper-implementation class (again with the namespace issue). Finally, if we have two different methods with the same structural type that are compiled separately, we've just exploded the number of interfaces we require.

Not that the hurdle couldn't be overcome, but if you want to have structural typing with "direct" access for particular types the type-trait pattern seems to be your best bet today.

Answer 2

As I recall from a discussion on the Scala-Inernals mailing list, the problem with this is object identity, which is preserved by the current approach to compiling, is lost when you wrap values.

Answer 3

Think about it. Consider class A

class A { def a1(i: Int): String = { ... }; def a2(s: String): Boolean = { ... }

Some place in the program, possibly in a separately compiled library, this structural type is used:

{ def a1(i: Int): String }

and elsewhere, this one is used:

{ def a2(s: String): Boolean }

How, apart from global analysis, is class A to be decorated with the interfaces necessary to allow it to be used where those far-flung structural types are specified?

If every possible structural type that a given class could conform to is used to generate an interface capturing that structural type, there's an explosion of such interfaces. Remember, that a structural type may mention more than one required member, so for a class with N public elements (vals or defs) all the possible subsets of those N are required, and that's the powerset of N whose cardinality is 2^N.