On Gentoo, simply look in the ebuild you compiled the package with :D.
If you're not sure?
# which factor
/usr/bin/factor
# grep '/usr/bin/factor' /var/db/pkg/*/*/CONTENTS
/var/db/pkg/sys-apps/coreutils-6.12-r2/CONTENTS:obj /usr/bin/factor 5aaf903daa4345efb11618b3cb47e9a5 1224224574
/var/db/pkg/sys-apps/coreutils-6.12-r2/CONTENTS:obj /usr/lib64/debug/usr/bin/factor.debug 517d965636850633e9b15926dde8c222 1224224575
# cat /var/db/pkg/sys-apps/coreutils-6.12-r2/SRC_URI
ftp://alpha.gnu.org/gnu/coreutils/coreutils-6.12.tar.lzma mirror://gnu/coreutils/coreutils-6.12.tar.lzma mirror://gentoo/coreutils-6.12.tar.lzma mirror://gentoo/coreutils-6.12-patches-1.0.tar.lzma http://dev.gentoo.org/~vapier/dist/coreutils-6.12-patches-1.0.tar.lzma
# cat /var/db/pkg/sys-apps/coreutils-6.12-r2/HOMEPAGE
http://www.gnu.org/software/coreutils/
But of course, the source code is probably still available in /usr/portage/distfiles
.