403 rate limit on insert sometimes succeeds

Question

I insert 100 files in a loop. For this test I have DISABLED backoff and retry, so if an insert fails with a 403, I ignore it and proceed with the next file. Out of 100 files

Answer 1

I'm assuming you're trying to do Parallel downloads...

This may not be an answer you're looking for, but this is what I've experienced in my interactions with google drive api. I use C#, so it's a bit different, but maybe it'll help.

I had to set a specific amount of threads to run at one time. If I let my program run all 100 entries at one time as separate threads, I run into the rate limit error as well.

I don't know well at all, but in my C# program, I run 3 threads (definable by the user, 3 is default)

opts = new ParallelOptions { MaxDegreeOfParallelism = 3 };
var checkforfinished = 
Parallel.ForEach(lstBackupUsers.Items.Cast<ListViewItem>(), opts, name => {
{ // my logic code here }

I did a quick search and found that Java 8 (not sure if that's what you're using) supports Parallel().forEach(), maybe that'd help you. The resource I found for this is at: http://radar.oreilly.com/2015/02/java-8-streams-api-and-parallelism.html

Hope this helps, taking my turns trying to help others on SO as people have helped me!