I needed to set both baseUrl and rootDir to point to my src folder in tsconfig.json
{
"compilerOptions": {
"baseUrl": "./src",
"rootDir": "./src",
...
}
I could then import .tsx files without needing a prefixed '/' or a relative path.
e.g.
import BreadCrumb from 'components/BreadCrumb'