Most Effective Dynamic Query or Predicate Builder in ASP.NET MVC 5 with C#, Entity Framework, SQL Server

问题

In my business application I'm finding it necessary to use a list of results generated from Table 2 with which to query Table 1 for more data, where there is a many to one relationship between Table 2 and Table 1.

For the sake of example please consider the following:

Say Table 2 contains Order Information and two fields in this table are Product ID and Total Sale Value. Say Table 1 contains Product Information where Product ID is unique and details more information about the product. In this example I would like to query Table 2 for a list of all sales over $1000 and return the Product ID. Then I would like to use that list to query Table 1 to return some more product details of these items.

To accomplish this I've discovered the albahari PredicateBuilder, found here: http://www.albahari.com/nutshell/predicatebuilder.aspx.

This is an excellent solution to my problem however I am finding that my data sets are too large for it. I'm getting a stack overflow error at anything over 40,000 predicates. Apparently the PredicateBuilder must recursively step through the expression tree and because the stack size is limited, the StackOverflowException will be thrown.

I have done some digging and this problem including some solutions are detailed here: https://kalcik.net/2014/01/05/joining-data-in-memory-with-data-in-database-table/

To overcome the StackOverflowException I have implemented the following from the above link:

        IQueryable<Asset> query = DbContext.ProductInformation;

        IEnumerable<OrderInformation> query_order = DbContext.OrderInformation;             
        query_order = query_order.Where(x => x.SalesValue > 1000);
        query = FoundProductsWithFragments(query_order, 10);

        private static IQueryable<ProductInformation> FoundProductsWithFragments(IEnumerable<OrderInformation> productsLookupTable, int chunkSize)
    {

        var productsLookupTableFragmented = productsLookupTable.Select((productToSearch, index) =>
                                                                new { ProductToSearch = productToSearch.ProductID, Index = index })
                                                                //.Distinct()
                                                                .GroupBy(productToSearch => productToSearch.Index / chunkSize);

        System.Diagnostics.Debug.WriteLine("productsLookupTableFragmented_cnt: " + productsLookupTableFragmented.Count());


        var foundProducts = new List<ProductInformation>();
        int ii = 0;

        foreach (var productLookupTableFragmentEntry in productsLookupTableFragmented)
        {
            var productsFromLookupTable = productLookupTableFragmentEntry.Select(e => e.ProductToSearch);

            var predicate = PredicateBuilder.New<ProductInformation>();

            foreach (var productFromLookupTable in productsFromLookupTable)
            {
                predicate = predicate.Or(searchedProduct => searchedProduct.ProductID == productFromLookupTable);
                System.Diagnostics.Debug.WriteLine(ii + " employeeFromLookupTable_id: " + productFromLookupTable);
                ii++;
            }

            using (var companyDbContext = new ApplicationDbContext())
            {
                foundProducts.AddRange(companyDbContext.Assets.AsExpandable().Where(predicate).ToList());

            }

        }


        return foundProducts.AsQueryable();


    }

Currently it's working but is generating a list of 100,000 predicates. As of writing this my function is 30 minutes into run time and still going.

In the above link by Anton Kalcik in Test Case 3 (TEST CASE 3 – Using PredicateBuilder with 10 fragments) the run time is only 2 minutes with 100,000 predicates. I'm not sure what the huge difference is in my implementation.

In any case, my question is the same - what is the most effective means to build a dynamic query for the type of use case I have described?

Any help would be greatly appreciated

来源：https://stackoverflow.com/questions/52324549/most-effective-dynamic-query-or-predicate-builder-in-asp-net-mvc-5-with-c-enti

标签

sql-server

asp.net-mvc

entity-framework

predicatebuilder