How do i create this with $this->formSelect() ?
For the Zend_Form_Element_Select() it goes like this
$multiOptions = array(
'Group A' => array(1 => 'First Value',2 => 'Second Value A),
'Group B' => array(3 => 'Third Value'),
);
$element->setMultiOptions($multiOptions);
Note that you also have addMultiOption($option,$value) and addMultiOptions($options). Simply include the value or options in an additional array.