You can use advanced indexing. You need to define an indexing array across the first axis, so that both indexing arrays are broadcast together and each column index refers to a specific row. In this case you just want an np.arange to index on the rows:
A[np.arange(A.shape[0]), b]
# array([ 7, 5, 3, 10, 5])
