Power BI - Can't Display Fields if One isn't Aggregated

Question

My relationships in Power BI, look like this:

Table A has a 1 to 1 relationship with table B, cross filtering in both directions. Table B (my bridge table) has a many