Parallel.ForEach throws exception when processing extremely large sets of data

怎甘沉沦 提交于 2020-01-14 03:59:14

问题


My question centers on some Parallel.ForEach code that used to work without fail, and now that our database has grown to 5 times as large, it breaks almost regularly.

Parallel.ForEach<Stock_ListAllResult>( lbStockList.SelectedItems.Cast<Stock_ListAllResult>(), SelectedStock =>
{
    ComputeTipDown( SelectedStock.Symbol );
} );

The ComputeTipDown() method gets all daily stock tic data for the symbol, and iterates through each day, gets yesterday's data and does a few calculations and then inserts them into the database for each day.

We use this rarely to recalculate static data values when a formula changes.

The exception is this:

The database we are hitting has 16 gigs of ram and is a dual quad-core and nobody was using the system while I was recalculating. The machine running the application to regenerate the code is a laptop with 12 gigs of ram with a hyper-threaded octal-core. So there was no obvious resource contention.

This is my foray into using .NET 4 and parallel processing, so I am wondering if there is something I am missing. Any thoughts would be welcomed.


回答1:


This looks like you received an AggregateException, which is what Parallel.ForEach will raise if any of the loop body methods raise an exception.

If you debug this, you should be able to look at the InnerExceptions to see the actual exceptions thrown. It looks like ComputeTipDown raised an exception in one or more of your iterations, causing this to occur.



来源:https://stackoverflow.com/questions/3832852/parallel-foreach-throws-exception-when-processing-extremely-large-sets-of-data

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!