Azure Active Directory Graph Api allows you to perform operations on the signed in user.
https://msdn.microsoft.com/en-us/library/azure/ad/graph/api/signed-in-user-o
What you have done there is client credentials grant flow. It is used by apps to access APIs not on behalf of users, but the app itself. So if your app has rights to read data from the API without user context, this is what you would use for that.
However, to make calls in a user context, you need to do something differently.
If this is a web app, you would probably use authorization code grant flow, in which the user's browser gets redirected to Azure AD. After giving consent to your app, they get forwarded back to your app with an authorization code. You can then exchange this code for an access token using the code and your client id and secret. Here is a sample app that does that: https://github.com/Azure-Samples/active-directory-dotnet-webapp-webapi-openidconnect, focus especially here: https://github.com/Azure-Samples/active-directory-dotnet-webapp-webapi-openidconnect/blob/master/TodoListWebApp/App_Start/Startup.Auth.cs.
If this is a native app, you will have to pop up a browser to do authentication. This is automated by ADAL, so you don't need to do much. Here is a sample UWP app for that: https://github.com/Azure-Samples/active-directory-dotnet-windows-store. Focus especially here: https://github.com/Azure-Samples/active-directory-dotnet-windows-store/blob/master/TodoListClient/MainPage.xaml.cs#L108. There is also a sample WPF app here: https://github.com/Azure-Samples/active-directory-dotnet-native-desktop. What the WPF app does can also be applied to console apps, winforms apps etc.
You can find all the samples here: https://docs.microsoft.com/en-us/azure/active-directory/develop/active-directory-code-samples.
Feel free to ask in a comment if you have more questions :)
P.S. Another choice would be the password grant flow, in which you send a username, password, client id and secret to Azure AD to get an access token. There are numerous problems that can come up with this approach so I don't recommend it. If you can use the ones where you present the login screen to the user in a browser window, use them.
The problem with the second call with the authorization code was that it didn't specify a resource URI. Making the call like this fixed the problem:
var result = await authContext.AcquireTokenByAuthorizationCodeAsync
(
authorizationCode,
redirectUri, // eg http://localhost:56950/
clientCredential, // Application ID, application secret
"https://graph.windows.net/"
);