Javascript user-agent (ajax) different to sent user-agent when requesting website

Question

I noticed that Chrome (64.0.3282.137) on my phone (OnePlus 3, Android 8.0.0) sends slightly different user-agents when requesting a webpage in contrast to requesting via ajax.

Answer 1

I analyzed the chromium source code to get some insights. I was able to get only to some level with my novice abilities in c++.

User agent of the client or the platform is detected in this code block (File: useragent.cc).

std::string BuildUserAgentFromProduct(const std::string& product) {
  std::string os_info;
  base::StringAppendF(
      &os_info,
      "%s%s",
      getUserAgentPlatform().c_str(),
      BuildOSCpuInfo().c_str());
  return BuildUserAgentFromOSAndProduct(os_info, product);
}

You can see BuildOSCpuInfo() in the code block which takes care of adding the os realted informations based on platforms which can be found here

std::string android_build_codename = base::SysInfo::GetAndroidBuildCodename();
std::string android_device_name = base::SysInfo::HardwareModelName(); // this line in particular adds the ONEPLUS A3003

But this function(BuildUserAgentFromProduct()) is not used directly in the net module which takes care of sending the http requests.

When I investigated the code for the net(http) module I see that they are getting the useragent* and processing it through a series of string manipulations and white space trimming functionalities. AddHeadersFromString() in http_request_headers.cc is the interface through which the useragent string is added to the request header.

Note*: But I think the header data is not from useragent.cc, because I am not able to find the calls for this function anywhere. But I might be wrong here.

**I believe that this is the place the value for the OSInfo is getting modified. Any whitespace character that is not recognized or in a wrong format then originally intended can give this result.

Note**: I couldn't test the above statement and prove it, because the String that is used in Chromium has a wrapper around it in the name of StringPiece( *wrapper is just a term that I am using, technically it can be called in a different way which I don't know.). And I don't know how to write the code in c++ for StringPiece.

But a very simple example of how it can go wrong is given below.

int main()
{
   std::string s = " ONEPLUS\rA3003\rBuild/OPR6.170623.013";
   std::string delimiter = "\r\n"; //this is the delimeter used in chromium source code.
   std::string token = s.substr(0, s.find(delimiter,0));
   std::cout << token << std::endl;
   return 0;
}

https://www.onlinegdb.com/SkTrbFJDz

Coming to the reason why the initial user agent string is having the value and the subsequent http request doesnt have the value, lies with the architecture of chrome app in android. When the page is loaded initially the values are actually set by the chrome app (a very big java code base But I think the core file that we need to see is LoadUrlParams.java) which has a different implementation of sending the http request (here the useragent is not trimmed by the same net(http) module instead its taken care by the Java Implementation), this happens only during the very first load. But any other subsequent calls uses the browser's net(http) module.

File Reference links: https://cs.chromium.org/chromium/src/content/common/user_agent.cc?sq=package:chromium&dr=CSs&l=80

https://cs.chromium.org/chromium/src/net/http/http_request_headers.cc?type=cs&q=AddHeadersFromString&l=155

https://cs.chromium.org/chromium/src/content/public/android/java/src/org/chromium/content_public/browser/LoadUrlParams.java?q=createLoadDataParamsWithBaseUrl&dr=CSs

I am just including this answer to give one of the reasons where the problem might have occurred. If I have some more time I will see if I can run a test somehow and prove this. One Final Note this answer doesn't give any solution to fix the problem. It just gives the reason for the cause.

[Update]

One very cheap trick is to see if navigator.useragent has the oneplus value and set ajax headers on the request and send it. This will override the browser's mechanism of adding the user agent header.

XMLHttpRequest.setRequestHeader(header, value)

Answer 2

In the first userAgent, the browser identifies the device as a mobile device by modifying the userAgent before making the request; hence the ONEPLUS A3003. In the second however, due w3 specification (Find it here), you cannot modify the userAgent; hence the omission of ONEPLUS A3003.

When you use the "Request desktop site" feature, there is no need for modification of userAgent by the browser, hence you get the same userAgent.

NOTE: That the default userAgent for that Chrome browser is: Mozilla/5.0 (Linux; Android 8.0.0; Build/OPR6.170623.013) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.137 Mobile Safari/537.36