EDIT 2: TL;DR: the answer was yes in 2013, but this flaw has been fixed
By following the Getting Started instructions on vagrantup.
I wrote this simple inline shell provisioner to swap out the authorized_keys with my id_rsa.pub. Once provisioned the insecure_private_key cannot be used to authenticate.
VAGRANTFILE_API_VERSION = "2"
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
# ...
config.ssh.shell = "bash -c 'BASH_ENV=/etc/profile exec bash'" # avoids 'stdin: is not a tty' error.
config.ssh.private_key_path = ["#{ENV['HOME']}/.ssh/id_rsa","#{ENV['HOME']}/.vagrant.d/insecure_private_key"]
config.vm.provision "shell", inline: <<-SCRIPT
printf "%s\n" "#{File.read("#{ENV['HOME']}/.ssh/id_rsa.pub")}" > /home/vagrant/.ssh/authorized_keys
chown -R vagrant:vagrant /home/vagrant/.ssh
SCRIPT
end