So I started with
brew reinstall php56 --with-apache
This worked fine I then amended httpd.conf to include
httpd.conf
LoadM
I was restarting apache, but it was not working it turns out.
Restarting my whole machine did work and apache then used the correct version of PHP.
Apparently killall httpd has the same effect too. Although I have not tried that.
killall httpd