So lets say I have 10,000 points in A and 10,000 points in B and want to find out the closest point in A for every B point.
Currently, I simply loop through every point
You could use some spatial lookup structure. A simple option is an octree; fancier ones include the BSP tree.