Cancellation not returning past whenall using httpclient

问题

Here is an updated version of the code posted under httpclient.GetStringAsync blocking

The question is when cancel is done, though the tasks are cancelled, I am expecting Await Task.WhenAll(tasks) to return and print whats in finally, but its not. I can see task cancellations when pressing cancel and I also see the connections reduce to 0, but the finally guessing WhenAll still thinks some tasks are being executed.

Here is the code:

Private concurrencySemaphore As New SemaphoreSlim(10)
Private cts As CancellationTokenSource
Private Async Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
    Try
        cts = New CancellationTokenSource
        Dim urls As List(Of String) = SetUpURLList()
        ServicePointManager.DefaultConnectionLimit = 10
        Dim tasks As List(Of Task) = New List(Of Task)()
        For Each url In urls
            cts.Token.ThrowIfCancellationRequested()
            tasks.Add(GetUrl(url, cts.Token))
        Next
        Await Task.WhenAll(tasks)
    Catch tx As TaskCanceledException
        Console.WriteLine("Task was cancelled")
    Catch ox As OperationCanceledException
        Console.WriteLine("Operation was cancelled")
    Catch ex As Exception
        Console.WriteLine(ex.Message)
    Finally
        Console.WriteLine("Done")
    End Try
End Sub
Async Function GetUrl(url As String, ByVal ct As CancellationToken) As Task
    Try
        ct.ThrowIfCancellationRequested()
        Await concurrencySemaphore.WaitAsync()
        Dim baseAddress = New Uri("http://www.amazon.com")
        Dim cookies As New CookieContainer()
        Dim handler As New HttpClientHandler With {.CookieContainer = cookies, _
                                                   .UseCookies = True}
        Dim httpClient = New HttpClient(handler) With {.BaseAddress = baseAddress}
        ct.ThrowIfCancellationRequested()
        Dim responseMessage As HttpResponseMessage = Await httpClient.GetAsync(url, ct).ConfigureAwait(False)
        Dim response As String = Await responseMessage.Content.ReadAsStringAsync()
        For Each cook As Cookie In cookies.GetCookies(baseAddress)
            Console.WriteLine(cook.Name & "=" & cook.Value)
        Next
        httpClient.Dispose()
        concurrencySemaphore.Release()
    Catch tx As TaskCanceledException
        Console.WriteLine("Task Cancelled Exception")
    Catch ox As OperationCanceledException
        Console.WriteLine("Operation Cancelled Exception")
    End Try
End Function

Private Sub btnCancel_Click(sender As Object, e As EventArgs) Handles btnCancel.Click
    If cts IsNot Nothing Then
        cts.Cancel()
    End If
End Sub

UPDATE: Have changed the question based on @I3arnon suggestion.

But 3 problems in the code below now:

 Dim downLoader As TransformBlock(Of String, Task(Of String))
 Dim cts As CancellationTokenSource
 Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
    cts = New CancellationTokenSource
    Dim urls As List(Of String) = SetUpURLList()
    Dim tasks As List(Of Task) = New List(Of Task)()
    downLoader.Post(urls(0))
    downLoader.Post(urls(1))
    'For Each url In urls
    '    tasks.Add(downLoader.Post(url))
    'Next
End Sub

Private Sub TPL_Load(sender As Object, e As EventArgs) Handles Me.Load
    downLoader = New TransformBlock(Of String, Task(Of String))(
    Async Function(url) As Task(Of String)
        Console.WriteLine("Downloading URL:{0}", url)
        Dim httpClient = New HttpClient()
        Using responseMessage As HttpResponseMessage = Await httpClient.GetAsync(url).ConfigureAwait(False)
            Dim response As String = Await responseMessage.Content.ReadAsStringAsync()
            Return response
            Console.WriteLine("Downloaded, URL:{0}, length:{1}", url, response.Length)
        End Using
    End Function, New ExecutionDataflowBlockOptions With {.MaxDegreeOfParallelism = 4, .CancellationToken = cts.Token})
End Sub

1. Code doesnt compile as I am not sure where to the TransformBlock signature and body. Putting it inside load is resulting in a null exception when it is being called since the object is not created.

2. How to we now fire the multiple URLs in parallel

3. How do we ensure cancellations, now that the CancellationToken is passed to the extension but not the inner method that does the job.

回答1:

The problem is that you don't release the semaphore when there's an exception (or cancellation in your case). That means Task.WhenAll is waiting for tasks that will never complete since they are waiting for that semaphore.

Move the semaphore's release to a Finally block to make sure it's never skipped:

Async Function GetUrl(url As String, ByVal ct As CancellationToken) As Task
    Try
        ct.ThrowIfCancellationRequested()
        Await concurrencySemaphore.WaitAsync()
        Dim baseAddress = New Uri("http://www.amazon.com")
        Dim cookies As New CookieContainer()
        Dim handler As New HttpClientHandler With {.CookieContainer = cookies, _
                                                   .UseCookies = True}
        Dim httpClient = New HttpClient(handler) With {.BaseAddress = baseAddress}
        ct.ThrowIfCancellationRequested()
        Dim responseMessage As HttpResponseMessage = Await httpClient.GetAsync(url, ct).ConfigureAwait(False)
        Dim response As String = Await responseMessage.Content.ReadAsStringAsync()
        For Each cook As Cookie In cookies.GetCookies(baseAddress)
            Console.WriteLine(cook.Name & "=" & cook.Value)
        Next
        httpClient.Dispose()
    Catch tx As TaskCanceledException
        Console.WriteLine("Task Cancelled Exception")
    Catch ox As OperationCanceledException
        Console.WriteLine("Operation Cancelled Exception")
    Finally
        concurrencySemaphore.Release()
    End Try
End Function

来源：https://stackoverflow.com/questions/27726835/cancellation-not-returning-past-whenall-using-httpclient