I had somewhat similar problem and I did not use Authorize attribute.
Instead I decided to to take the different -2 Layout for the Different user .
if the User is Admin then he will go to the different view and if the user is guest he will go to the different view through different layout.