Scala pickling: Simple custom pickler for my own class?

问题

I am trying to pickle some relatively-simple-structured but large-and-slow-to-create classes in a Scala NLP (natural language processing) app of mine. Because there's lots of data, it needs to pickle and esp. unpickle quickly and without bloat. Java serialization evidently sucks in this regard. I know about Kryo but I've never used it. I've also run into Apache Avro, which seems similar although I'm not quite sure why it's not normally mentioned as a suitable solution. Neither is Scala-specific and I see there's a Scala-specific package called Scala Pickling. Unfortunately it lacks almost all documentation and I'm not sure how to create a custom pickler.

I see a question here:

Scala Pickling: Writing a custom pickler / unpickler for nested structures

There's still some context lacking in that question, and also it looks like an awful lot of boilerplate to create a custom pickler, compared with the examples given for Kryo or Avro.

Here's some of the classes I need to serialize:

trait ToIntMemoizer[T] {
  protected val minimum_raw_index: Int = 1
  protected var next_raw_index: Int = minimum_raw_index

  // For replacing items with ints. This is a wrapper around
  // gnu.trove.map.TObjectIntMap to make it look like mutable.Map[T, Int].
  // It behaves the same way.
  protected val value_id_map = trovescala.ObjectIntMap[T]()

  // Map in the opposite direction. This is a wrapper around
  // gnu.trove.map.TIntObjectMap to make it look like mutable.Map[Int, T].
  // It behaves the same way.
  protected val id_value_map = trovescala.IntObjectMap[T]()

  ...
}

class FeatureMapper extends ToIntMemoizer[String] {
  val features_to_standardize = mutable.BitSet()
  ...
}

class LabelMapper extends ToIntMemoizer[String] {
}

case class FeatureLabelMapper(
  feature_mapper: FeatureMapper = new FeatureMapper,
  label_mapper: LabelMapper = new LabelMapper
)

class DoubleCompressedSparseFeatureVector(
  var keys: Array[Int], var values: Array[Double],
  val mappers: FeatureLabelMapper
) { ... }

How would I create custom pickers/unpicklers in way that uses as little boilerplate as possible (since I have a number of other classes that need similar treatment)?

Thanks!

来源：https://stackoverflow.com/questions/20832563/scala-pickling-simple-custom-pickler-for-my-own-class

标签

scala

serialization

avro

kryo

scala-pickling