Here is an article which briefly and concisely sums up why folding is cool. The one line reason is that folding makes navigating very large files a breeze.
If you want to leave folding enabled, and simply always start with all folds open, the vim wiki tells how. The method of interest to you would probably be the autocommand method.
" Tweak the event and filetypes matched to your liking.
" Note, perl automatically sets foldmethod in the syntax file
autocmd Syntax c,cpp,vim,xml,html,xhtml setlocal foldmethod=syntax
autocmd Syntax c,cpp,vim,xml,html,xhtml,perl normal zR
I would also recommend searching for custom folding methods for the language you use. Simply googling "vim