Complex query. Averageifs with a few intricacies. My current formula is shown, but not sure how to add another layer to the query

Question

TLDR: Formula and summary of question/problem at the bottom if you want to skip ahead.

I have a sheet where names are in column A and B. There are numerical values in