How to convert object columns to int and float columns without converting datetime columns

Question

I am loading the table into pandas

import pandas as pd
df=pd.read_excel(file.xlsx,dtype=object)
A   B         C        D
1   2.0   1-02-1997   sam
Nan 2.3   3