Hyperparameter optimization for Pytorch model

Question

What is the best way to perform hyperparameter optimization for a Pytorch model? Implement e.g. Random Search myself? Use Skicit Learn? Or is there anything else I am not aware

Answer 1

What I found is following:

• HyperOpt
• Optuna
• tune

More young projects:

• hypersearch limited only to FC layers.
• skorch Just grid search available
• Auto-PyTorch

UPDATE something new:

• Ax: Adaptive Experimentation Platform by facebook

• BoTorch: Bayesian Optimization in PyTorch

Also, I found a useful table at post by @Richard Liaw: