问题
Here is a description of the application. I have a console application that runs Hangfire. There are several jobs that are recurring. One job, in particular, parses several large XML files. I deploy this console app as an AWS ECS Fargate ontainer.
Every time the recurring job runs, the container memory usage keeps rising and rising until either the application becomes unresponsive or AWS kills the container. This is a 16GB container on ECS (big) and so it should be able to handle anything I throw at it. I can tell as soon as the memory usage gets up toward ~13GB it is doomed.
The kicker is that I run this application locally and it never gets very high in memory usage. I will post a couple of example logs from both my workstation and on the container below.
THANKS FOR ANY INSIGHT!
Parsing the XML:
- Use XmlSerializer to drop into a JSON formatted object
- Use XDocument to just go through the document
- Use XSD file to generate classes, and then XmlSerializer to populate
All three work locally, but use a ton of memory on the server. These are unchanging XML files, and I run the exact same files on both local and in the container.
FYI: I have turned off server garbage collection on the container. That didn't work.
EDIT 3/5/2019
The logs below are from the following code. It doesn't have the full breath of the application which is quite complex. This code is in a method registered with Hangfire to run once per day. It pulls XML files from an FTP site, and then runs essentially the code below to process those files. Each file is at most 10 MB in size.
XmlSerializer serializer = new XmlSerializer(Model);
foreach (var file in files)
{
Console.WriteLine($"Processing file: {file.Name}");
Model model;
using (XmlReader reader = XmlReader.Create(file.FullName, new XmlReaderSettings { CheckCharacters = false }))
{
model = (Model)serializer.Deserialize(reader);
}
SaveToDatabase(model);
Console.WriteLine($"Application Memory Usage - Working: {Process.GetCurrentProcess()?.WorkingSet64 ?? 0}, Private: {Process.GetCurrentProcess()?.PrivateMemorySize64 ?? 0}");
}
Workstation log:
[13:52:58 INF] Processing file: price_feed_supplemental_002981.xml
[13:52:58 INF] Application Memory Usage - Working: 1.4 GB, Private: 1.6 GB
[13:52:58 INF] Processing file: price_feed_002982.xml
[13:53:01 INF] Application Memory Usage - Working: 1.4 GB, Private: 1.6 GB
[13:53:01 INF] Processing file: price_feed_supplemental_002982.xml
[13:53:01 INF] Application Memory Usage - Working: 1.4 GB, Private: 1.6 GB
[13:53:01 INF] Processing file: price_feed_002983.xml
[13:53:01 INF] Application Memory Usage - Working: 1.4 GB, Private: 1.6 GB
[13:53:01 INF] Processing file: price_feed_supplemental_002983.xml
[13:53:02 INF] Application Memory Usage - Working: 1.4 GB, Private: 1.6 GB
[13:53:02 INF] Processing file: price_feed_002984.xml
Container log:
[19:05:35 INF] Application Memory Usage - Working: 1.9 GB, Private: 0.0 bytes
[19:05:35 INF] Processing file: price_feed_supplemental_002987.xml
[19:05:35 INF] Application Memory Usage - Working: 1.9 GB, Private: 0.0 bytes
[19:05:35 INF] Processing file: price_feed_supplemental_002988.xml
[19:05:35 INF] Application Memory Usage - Working: 1.9 GB, Private: 0.0 bytes
[19:05:35 INF] Processing file: price_feed_002988.xml
[19:05:40 INF] Application Memory Usage - Working: 1.9 GB, Private: 0.0 bytes
[19:05:40 INF] Processing file: price_feed_002989.xml
[19:05:43 INF] Application Memory Usage - Working: 2.1 GB, Private: 0.0 bytes
[19:05:43 INF] Processing file: price_feed_supplemental_002989.xml
[19:05:43 INF] Application Memory Usage - Working: 2.1 GB, Private: 0.0 bytes
[19:05:43 INF] Processing file: price_feed_supplemental_002990.xml
[19:05:43 INF] Application Memory Usage - Working: 2.1 GB, Private: 0.0 bytes
[19:05:43 INF] Processing file: price_feed_002990.xml
[19:05:51 INF] Application Memory Usage - Working: 2.6 GB, Private: 0.0 bytes
[19:05:51 INF] Processing file: price_feed_002991.xml
[19:05:57 INF] Application Memory Usage - Working: 2.2 GB, Private: 0.0 bytes
[19:05:57 INF] Processing file: price_feed_supplemental_002991.xml
[19:06:00 INF] Application Memory Usage - Working: 2.2 GB, Private: 0.0 bytes
[19:06:00 INF] Processing file: price_feed_002992.xml
[19:06:19 INF] Application Memory Usage - Working: 2.1 GB, Private: 0.0 bytes
[19:06:19 INF] Processing file: price_feed_supplemental_002992.xml
[19:06:21 INF] Application Memory Usage - Working: 2.3 GB, Private: 0.0 bytes
[19:06:21 INF] Uncompressing file: usa_LTSF_20190228_002993_001_INC.zip
[19:06:22 INF] Source file: price_feed_002993_1.xml
[19:06:22 INF] Source file: price_feed_supplemental_002993_8.xml
[19:06:23 INF] Source file: price_feed_supplemental_002993_11.xml
[19:06:23 INF] Source file: price_feed_002993_2.xml
[19:06:23 INF] Source file: price_feed_supplemental_002993_5.xml
[19:06:23 INF] Source file: price_feed_supplemental_002993_4.xml
[19:06:24 INF] Source file: price_feed_supplemental_002993_17.xml
[19:06:24 INF] Source file: price_feed_supplemental_002993_15.xml
[19:06:24 INF] Source file: price_feed_supplemental_002993_3.xml
[19:06:25 INF] Source file: price_feed_supplemental_002993_1.xml
[19:06:25 INF] Source file: price_feed_supplemental_002993_9.xml
[19:06:25 INF] Source file: price_feed_supplemental_002993_7.xml
[19:06:26 INF] Source file: price_feed_supplemental_002993_6.xml
[19:06:26 INF] Source file: price_feed_supplemental_002993_14.xml
[19:06:26 INF] Source file: price_feed_supplemental_002993_13.xml
[19:06:27 INF] Source file: price_feed_supplemental_002993_16.xml
[19:06:27 INF] Source file: price_feed_supplemental_002993_12.xml
[19:06:27 INF] Source file: price_feed_supplemental_002993_2.xml
[19:06:28 INF] Source file: price_feed_supplemental_002993_10.xml
[19:06:28 INF] Processing file: price_feed_002984.xml
[19:06:28 INF] Application Memory Usage - Working: 2.5 GB, Private: 0.0 bytes
[19:06:28 INF] Processing file: price_feed_supplemental_002984.xml
[19:06:28 INF] Application Memory Usage - Working: 2.5 GB, Private: 0.0 bytes
[19:06:28 INF] Processing file: price_feed_supplemental_002985.xml
[19:06:28 INF] Application Memory Usage - Working: 2.5 GB, Private: 0.0 bytes
[19:06:28 INF] Processing file: price_feed_002985.xml
[19:06:28 INF] Application Memory Usage - Working: 2.5 GB, Private: 0.0 bytes
[19:06:28 INF] Processing file: price_feed_supplemental_002986.xml
[19:06:28 INF] Application Memory Usage - Working: 2.5 GB, Private: 0.0 bytes
[19:06:28 INF] Processing file: price_feed_002986.xml
[19:06:28 INF] Application Memory Usage - Working: 2.5 GB, Private: 0.0 bytes
[19:06:28 INF] Processing file: price_feed_002987.xml
[19:06:30 INF] Application Memory Usage - Working: 2.6 GB, Private: 0.0 bytes
[19:06:30 INF] Processing file: price_feed_supplemental_002987.xml
[19:06:30 INF] Application Memory Usage - Working: 2.6 GB, Private: 0.0 bytes
[19:06:30 INF] Processing file: price_feed_supplemental_002988.xml
[19:06:30 INF] Application Memory Usage - Working: 2.6 GB, Private: 0.0 bytes
[19:06:30 INF] Processing file: price_feed_002988.xml
[19:06:35 INF] Application Memory Usage - Working: 2.9 GB, Private: 0.0 bytes
[19:06:35 INF] Processing file: price_feed_002989.xml
[19:06:38 INF] Application Memory Usage - Working: 3.1 GB, Private: 0.0 bytes
[19:06:38 INF] Processing file: price_feed_supplemental_002989.xml
[19:06:38 INF] Application Memory Usage - Working: 3.1 GB, Private: 0.0 bytes
[19:06:38 INF] Processing file: price_feed_supplemental_002990.xml
[19:06:38 INF] Application Memory Usage - Working: 3.1 GB, Private: 0.0 bytes
[19:06:38 INF] Processing file: price_feed_002990.xml
[19:06:47 INF] Application Memory Usage - Working: 3.7 GB, Private: 0.0 bytes
[19:06:47 INF] Processing file: price_feed_002991.xml
[19:06:53 INF] Application Memory Usage - Working: 4.1 GB, Private: 0.0 bytes
[19:06:53 INF] Processing file: price_feed_supplemental_002991.xml
[19:06:55 INF] Application Memory Usage - Working: 4.3 GB, Private: 0.0 bytes
[19:06:55 INF] Processing file: price_feed_002992.xml
[19:07:05 INF] Application Memory Usage - Working: 4.9 GB, Private: 0.0 bytes
[19:07:05 INF] Processing file: price_feed_supplemental_002992.xml
[19:07:07 INF] Application Memory Usage - Working: 5.0 GB, Private: 0.0 bytes
[19:07:07 INF] Processing file: price_feed_002993_1.xml
[19:09:40 INF] 1 servers were removed due to timeout
[19:10:49 INF] Application Memory Usage - Working: 10.0 GB, Private: 0.0 bytes
[19:10:49 INF] Processing file: price_feed_supplemental_002993_8.xml
[19:12:10 INF] Application Memory Usage - Working: 12.5 GB, Private: 0.0 bytes
[19:12:10 INF] Processing file: price_feed_supplemental_002993_11.xml
[19:13:08 INF] Application Memory Usage - Working: 13.5 GB, Private: 0.0 bytes
[19:13:08 INF] Processing file: price_feed_002993_2.xml
[19:14:06 INF] Application Memory Usage - Working: 13.8 GB, Private: 0.0 bytes
[19:14:06 INF] Processing file: price_feed_supplemental_002993_5.xml
来源：https://stackoverflow.com/questions/54992771/net-core-high-memory-usage-in-docker-aws-ecs-fargate