How do I fix a ValueError: read of closed file exception?

Question

This simple Python 3 script:

import urllib.request

host = \"scholar.google.com\"
link = \"/scholar.bib?q=info:K7uZdMSvdQ0J:scholar.google.c         


        

Answer 1

If you run your app over managed cloud infrastructure, or managed security service, check for limitations that may come from them. Happened to me. Cloud providers sometimes impose a white list on accessible sites.

Answer 2

Your URL return a 403 code error and apparently urllib.request.urlretrieve is not good at detecting all the HTTP errors, because it's using urllib.request.FancyURLopener and this latest try to swallow error by returning an urlinfo instead of raising an error.

About the fix if you still want to use urlretrieve you can override FancyURLopener like this (code included to also show the error):

import urllib.request
from urllib.request import FancyURLopener


class FixFancyURLOpener(FancyURLopener):

    def http_error_default(self, url, fp, errcode, errmsg, headers):
        if errcode == 403:
            raise ValueError("403")
        return super(FixFancyURLOpener, self).http_error_default(
            url, fp, errcode, errmsg, headers
        )

# Monkey Patch
urllib.request.FancyURLopener = FixFancyURLOpener

url = "http://scholar.google.com/scholar.bib?q=info:K7uZdMSvdQ0J:scholar.google.com/&output=citation&hl=en&as_sdt=1,14&ct=citation&cd=0"
urllib.request.urlretrieve(url, "cite0.bib")

Else and this is what i recommend you can use urllib.request.urlopen like so:

fp = urllib.request.urlopen('http://scholar.google.com/scholar.bib?q=info:K7uZdMSvdQ0J:scholar.google.com/&output=citation&hl=en&as_sdt=1,14&ct=citation&cd=0')
with open("citi0.bib", "w") as fo:
    fo.write(fp.read())