Short answer: almost no difference except stylistically.
Short blurb: If you don't want to escape the quote characters inside your string, use the other type. eg:
string1 = "He turned to me and said, \"Hello there\""
would be slightly more unsightly than saying
string2 = 'He turned to me and said, "Hello there"'
The same applies to single quotes/apostrophes.