Generally you should expose as little as possible and make everything private that is possible.
If you make a mistake and hide something you should be exposing, no problem, just make it public. However, if you make something public and then later decide it should be private, then you may have problems because now the public method may be used by many other classes.
You are free to change implementation of your private methods without any external effects. If you have all public classes, that may not be possible, because those methods could be used by somethings outside your classes.