You are right - this happens beacause you used lazy loading in EF (virtual property). You may remove virtual (but this may be impossible for you). Other way you described in your question - call property, and set this to null.
Also you could read another topic about this problem on SO.