I want to access my own google drive on web page but allow anyone to upload files and restrict download access or show only limited file to user for downloading.
Drive A
You say "Drive API assumes that I will access another users credentials" but this isn't correct. Calls to Drive include an Access Token, and it is that which determines which Drive account the call will access. Drive makes no assumptions about how that Access Token was obtained or which account it refers to.
All of the examples show how an Access Token is obtained for an end user, since that is the more complex flow. Your requirement is very much simpler.
All you need to do is:-