Why can't there be an instance of MonadFix for the continuation monad?

Question

How can we prove that the continuation monad has no valid instance of MonadFix?

Answer 1

Well actually, it's not that there can't be a MonadFix instance, just that the library's type is a bit too constrained. If you define ContT over all possible rs, then not only does MonadFix become possible, but all instances up to Monad require nothing of the underlying functor :

newtype ContT m a = ContT { runContT :: forall r. (a -> m r) -> m r }
instance Functor (ContT m) where
  fmap f (ContT k) = ContT (\kb -> k (kb . f))
instance Monad (ContT m) where
  return a = ContT ($a)
  join (ContT kk) = ContT (\ka -> kk (\(ContT k) -> k ka))
instance MonadFix m => MonadFix (ContT m) where
  mfix f = ContT (\ka -> mfixing (\a -> runContT (f a) ka<&>(,a)))
    where mfixing f = fst <$> mfix (\ ~(_,a) -> f a )