I am totally new to ASP.NET MVC 4 as I was developing into PHP (Laravel) and the way to organize things is usually something like a "public" folder which would the
Frankly, you can organize your MVC project however you see fit. If there's anything close to an "official" way to organize your web files, it would be how the default Visual Studio MVC4 template organizes them:
If you're more familiar with another sort of organization or are part of a team that would prefer another organization, then go for it. (Though in that team scenario, make sure all the team members follow the same organization rules!).
Edit:
To alter the folder structure that the MVC template provides using the Solution Explorer, right-click on the folder in which you want to add a subfolder (this includes the project name, for project-level folders), and then select Add and then Add Folder from the pop-up menu that appears.
If you want to move around files that are already in your solution to another location within the solution, you should move them around using the Solution Explorer, not Windows Explorer, as you'll otherwise get the behavior you are experiencing, where Visual Studio does not know where you moved them. You can click and drag files around, right-click and copy and paste, etc.
If you need to add pre-existing files to your solution (say, to include a set of custom scripts), you can copy the files to the appropriate project folder using Windows Explorer. Then, in Visual Studio, highlight the project that corresponds to where you moved them, and press the Show All Files button in the Solution Explorer toolbar - this will show the files you copied in Windows Explorer, which otherwise won't be listed in Solution Explorer tab. Lastly, highlight the new files, right-click, and select the Include in Project option from the pop-up menu.
One final pointer: if you need to add a specific JavaScript library to your project, the easiest way to do so would be to use the NuGet package manager, rather than to download and add the files in Windows Explorer. This option is found in Visual Studio in the Tools menu, under Library Project Manager --> Manage NuGet Packages for Solution. Not all JavaScript libraries will be available this way, but the most popular ones are.