Asp.Net Core 3 graceful shutdown throws OperationCanceledException

问题

I'm trying to gracefully terminate a ASP.Net Core 3.1 service (which will run in Kubernetes). When Kubernetes stops a service, it will send a SIGTERM event to the application, at which point I want in-flight requests to complete (which may take several seconds) before terminating... I think I can catch this in a hostedservice, as below, and hence not stop immediately.

The following works, but with a timeout of 5 seconds or longer, I receive an OperationCanceledException. Could anyone shed any light on why I get an OperationCanceledException or how shed any light on an alternative way to delay a SIGTERM event, to allow a graceful shutdown?

    public static int Main(string[] args)
    {
        var logger = NLogBuilder
            .ConfigureNLog("nlog.config")
            .GetCurrentClassLogger();
        try
        {
            CreateHostBuilder(args)
                .ConfigureServices((hostBuilderContext, services) => { services.AddHostedService<LifetimeEventsHostedService>(); })
                .Build()
                .Run();

            return 0;
        }
        catch (Exception e)
        {
            logger.Fatal(e, "Stopping due to exception");

            return -1;
        }
        finally
        {
            LogManager.Shutdown();
        }
    }

This is the hosted service...

    internal class LifetimeEventsHostedService : IHostedService
    {
        private readonly Microsoft.Extensions.Logging.ILogger _logger;
        private readonly IHostApplicationLifetime _appLifetime;

        public LifetimeEventsHostedService(
            ILogger<LifetimeEventsHostedService> logger,
            IHostApplicationLifetime appLifetime)
        {
            _logger = logger;
            _appLifetime = appLifetime;
        }

        public Task StartAsync(CancellationToken cancellationToken)
        {
            _appLifetime.ApplicationStarted.Register(OnStarted);
            _appLifetime.ApplicationStopping.Register(OnStopping);
            _appLifetime.ApplicationStopped.Register(OnStopped);

            return Task.CompletedTask;
        }

        public Task StopAsync(CancellationToken cancellationToken)
        {
            return Task.CompletedTask;
        }

        private void OnStarted()
        {
            _logger.LogInformation("OnStarted has been called.");

            // Perform post-startup activities here
        }

        private void OnStopping()
        {
            _logger.LogInformation("OnStopping has been called.");
            // Perform on-stopping activities here


            // This works, but a timeout of 5 seconds or more subsequently causes an OperationCanceledException
            Thread.Sleep(5000);
        }

        private void OnStopped()
        {
            _logger.LogInformation("OnStopped has been called.");

            // Perform post-stopped activities here
        }
    }

回答1:

I'm open to alternative approaches to graceful shutdown with ASP.Net Core 3.1, as it stands, I'm using a hosted service.

Within the .Net Core app, I was setting ShutdownTimeout on the webhost, however, setting the ShutdownTimeout on the generic host, does allow me to gracefully wait a number of seconds (more than the default sigterm, which is 5 seconds) prior to shutdown. The hint from @PmanAce helped me work that out.

As such, the following codes allow me to gracefully terminate. One caveat, the Thread.Sleep in LifetimeEventsHostedService must be less than option.ShutdownTimeout.

public class Program
{
    public static void Main(string[] args)
    {
        CreateHostBuilder(args).Build().Run();
    }

    public static IHostBuilder CreateHostBuilder(string[] args) =>
        Host.CreateDefaultBuilder(args)
            .ConfigureServices((hostBuilderContext, services) =>
            {
                services.AddHostedService<LifetimeEventsHostedService>();
                services.Configure<HostOptions>(option =>
                {
                    option.ShutdownTimeout = TimeSpan.FromSeconds(30);
                });
            })
            .ConfigureWebHostDefaults(webBuilder =>
            {
                webBuilder.UseKestrel();
                webBuilder.UseStartup<Startup>();
            });
}

The following LifetimeEventsHostedService

public class LifetimeEventsHostedService : IHostedService
{
    private readonly IHostApplicationLifetime _hostApplicationLifetime;

    public LifetimeEventsHostedService(IHostApplicationLifetime hostApplicationLifetime)
    {
        _hostApplicationLifetime = hostApplicationLifetime;
    }

    public Task StartAsync(CancellationToken cancellationToken)
    {
        _hostApplicationLifetime.ApplicationStarted.Register(OnStarted);
        _hostApplicationLifetime.ApplicationStopping.Register(OnStopping);
        _hostApplicationLifetime.ApplicationStopped.Register(OnStopped);

        return Task.CompletedTask;
    }

    public Task StopAsync(CancellationToken cancellationToken)
    {
        return Task.CompletedTask;
    }

    private void OnStopped()
    {
        Console.WriteLine("OnStopped");
    }

    private void OnStopping()
    {
        Console.WriteLine("OnStopping");
        Console.WriteLine(DateTime.Now.ToLongTimeString());

        Thread.Sleep(15000);
        Console.WriteLine("Sleep finished");
        Console.WriteLine(DateTime.Now.ToLongTimeString());
    }

    private void OnStarted()
    {
        Console.WriteLine("OnStarted");
    }
}

来源：https://stackoverflow.com/questions/60228641/asp-net-core-3-graceful-shutdown-throws-operationcanceledexception