Azure Function deployments cause brief 503 errors even when auto swap is turned on

ε祈祈猫儿з 提交于 2021-01-27 05:26:24

问题


I'm deploying updates to my Function app through the VS publish window. I set up a deployment slot with auto swap turned on. My updates through VS are going to the slot. The problem is, right after the publish is successful and when I test my API endpoints, I briefly receive 503 errors. I was under the impression that auto swap was seamless and end-users would not experience such interruptions. Am I missing something? How can I make my deployments unnoticeable to the users?


回答1:


Switching to something like API Management or Traffic Manager is obviously an option, but slots are designed to do exactly what you want, and they should work the way you expect.

I looked into this a bit. Unfortunately, I can reproduce your issue, which suprised me. A few things feel a bit off when using Azure Functions with slots, so maybe there is some weirdness under the covers.

The official documentation does not mention anything about this however, quite the opposite:

Traffic redirection is seamless; no requests are dropped because of a swap.

If a function is running during a swap, execution continues and the next triggers are routed to the swapped app instance.

You don't even need to use Auto Swap. Just publish to both slots and swap the slots manually. When observing the responses, the following pattern can be seen:

  1. Responses of old code
  2. Responses of new code
  3. 503 errors for ~10 seconds
  4. Request slowdown
  5. Responses of new code

I tried:

  • AppService Plan & Consumption Plan
  • AAR Affinity On/Off
  • Azure Function V2 and V3 runtime

This seems like a bug to me. I would suggest you create a support case and maybe an issue at Github. I might do so myself if I find the time in the next few days. See also this Issue:

https://github.com/Azure/Azure-Functions/issues/862

edit: the linked GitHub issue and also the medium article mentioned by Ron point out that you can set WEBSITE_ADD_SITENAME_BINDINGS_IN_APPHOST_CONFIG to 1 and this should help with the 503 errors. It is a documented behavior very deep in the AppService docs. Why it is not mentioned for Azure Functions eludes me.




回答2:


Did you see this Does Azure Functions throw 503 errors while app settings are being updated?

Depending on how you are doing the swap it could be triggering a restart because the app settings are "changing"

There is also this that probably would help but its only a prem feature https://docs.microsoft.com/en-us/azure/azure-functions/functions-bindings-warmup?tabs=csharp

I would also check out https://medium.com/@yapaxinl/azure-deployment-slots-how-not-to-make-deployment-worse-23c5819d1a17




回答3:


I believe the solution would be adding an API Management in front of your Azure Functions, then implement a retry policy in it. This error seems to be related to the DNS swap between the slots.




回答4:


The general practice a lot of people follow is maintaining two hosts(Az function/App services) in two different regions behind azure traffic manager and deployment goes as follows:

  1. disable first region in traffic manager
  2. swap functions in the first region
  3. enable first region in traffic manager
  4. disable second region in traffic manager
  5. swap functions in the second region
  6. enable second region in traffic manager

Although it does not solve the issue of Az functions returning 503, but it does make it unnoticable to the user as you always route to the stable endpoint.

Having two regions also help handle other issues like azure outage in specific regions



来源:https://stackoverflow.com/questions/62767337/azure-function-deployments-cause-brief-503-errors-even-when-auto-swap-is-turned

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!