Create LINQ to entities OrderBy expression on the fly

Question

I\'m trying to add the orderby expression on the fly. But when the query below is executed I get the following exception:

System.NotSupportedException

Answer 1

I wanted to share my implementation using Jon's answer above as a starting point. In this case, rather than sorting by a string property name coming from the presentation layer (since the title of this question isn't specific about that), I am building out an Entity Framework data layer and want to allow its consumer to specify order by properties as lambda expressions. I.E. Rather than passing "sidx", I wanted to be able to use p => p.sidx. I also wanted to be able to pass an unlimited number of order by properties and be able to specify ascending or descending order.

Well my method can accept such a lambda expression as type Expression<Func<T, object>>. That let's me call it the way I want but the problem is, Entity Framework can't translate the expression to SQL unless the second generic parameter is strongly typed. The OrderBy extension method requires two generic parameters: T - the type the property belongs to, and TKey - the type the property returns. So the first step was to modify Jon's example to convert a given Expression<Func<T, object>> to a Expression<Func<T, Tkey>> (once we're working within the context of a query, we can determine the type TKey):

internal static IQueryable<T> OrderByDynamic<T>(this IQueryable<T> source, Expression<Func<T, object>> sortExp)
{
    //We need to convert the key selector from Expression<Func<T, object>> to a strongly typed Expression<Func<T, TKey>>
    //in order for Entity Framework to be able to translate it to SQL
    MemberExpression orderByMemExp = ExpressionHelpers.RemoveConvert(sortExp.Body) as MemberExpression;
    ParameterExpression sourceParam = sortExp.Parameters[0];

    LambdaExpression orderByLamda = Expression.Lambda(orderByMemExp, new[] { sourceParam });

    MethodInfo orderByMethod = OrderByMethod.MakeGenericMethod(new[] { typeof(T), orderByMemExp.Type });

    //Call OrderBy or OrderByDescending on the source IQueryable<T>
    return (IQueryable<T>)orderByMethod.Invoke(null, new object[] { source, orderByLamda });
}

As I mentioned, I want to accept an unlimited number of order by key selectors and also have the ability to specify the ascending or descending direction so I made a wrapper class for the Expression<Func<T, object>> which I named DynamicSortExpression:

public class DynamicSortExpression<T>
{
    /// <summary>
    /// Creates a new ascending DynamicSortExpression 
    /// </summary>
    /// <param name="keySelector">A MemberExpression identifying the property to sort on</param>
    public DynamicSortExpression(Expression<Func<T, object>> keySelector) : this(keySelector, false)
    {
    }

    public DynamicSortExpression(Expression<Func<T, object>> keySelector, bool descending)
    {
        this.KeySelector = keySelector;
        this.Desc = descending;
    }

    /// <summary>
    /// Gets the expression that selects the property of T to sort on
    /// </summary>
    public Expression<Func<T, object>> KeySelector { get; }

    /// <summary>
    /// Gets sort expression is in ascending or descending order
    /// </summary>
    public bool Desc { get; }
}

Then I updated the extension method to accept this type and created an overload for OrderBy that receives a List<DynamicSortExpression<T>> and adds them to the query using the extension method(s) one by one. Here is the final result:

public static class Extensions
{
    private static readonly MethodInfo OrderByMethod = typeof(Queryable).GetMethods()
                                                        .Where(method => method.Name == "OrderBy")
                                                        .Where(method => method.GetParameters().Length == 2)
                                                        .Single();

    private static readonly MethodInfo OrderByDescMethod = typeof(Queryable).GetMethods()
                                                            .Where(method => method.Name == "OrderByDescending")
                                                            .Where(method => method.GetParameters().Length == 2)
                                                            .Single();

    private static readonly MethodInfo ThenByMethod = typeof(Queryable).GetMethods()
                                                    .Where(method => method.Name == "ThenBy")
                                                    .Where(method => method.GetParameters().Length == 2)
                                                    .Single();

    private static readonly MethodInfo ThenByDescMethod = typeof(Queryable).GetMethods()
                                                    .Where(method => method.Name == "ThenByDescending")
                                                    .Where(method => method.GetParameters().Length == 2)
                                                    .Single();

    internal static IQueryable<T> OrderBy<T>(this IQueryable<T> sourceQuery, List<DynamicSortExpression<T>> orderBy)
    {
        bool isFirst = true;
        foreach (var sortExpression in orderBy)
        {
            if (isFirst)
            {
                sourceQuery = sourceQuery.OrderByDynamic(sortExpression);
                isFirst = false;
            }
            else
                sourceQuery = sourceQuery.ThenByDynamic(sortExpression);
        }

        return sourceQuery;
    }

    internal static IQueryable<T> OrderByDynamic<T>(this IQueryable<T> source, DynamicSortExpression<T> sortExpression)
    {
        //We need to convert the key selector from Expression<Func<T, object>> to a strongly typed Expression<Func<T, TKey>>
        //in order for Entity Framework to be able to translate it to SQL
        MemberExpression orderByMemExp = ExpressionHelpers.RemoveConvert(sortExpression.KeySelector.Body) as MemberExpression;
        ParameterExpression sourceParam = sortExpression.KeySelector.Parameters[0];

        LambdaExpression orderByLamda = Expression.Lambda(orderByMemExp, new[] { sourceParam });

        MethodInfo orderByMethod = sortExpression.Desc ?
                                        OrderByDescMethod.MakeGenericMethod(new[] { typeof(T), orderByMemExp.Type }) :
                                        OrderByMethod.MakeGenericMethod(new[] { typeof(T), orderByMemExp.Type });

        //Call OrderBy or OrderByDescending on the source IQueryable<T>
        return (IQueryable<T>)orderByMethod.Invoke(null, new object[] { source, orderByLamda });
    }

    internal static IQueryable<T> ThenByDynamic<T>(this IQueryable<T> source, DynamicSortExpression<T> sortExpression)
    {
        //We need to convert the key selector from Expression<Func<T, object>> to a strongly typed Expression<Func<T, TKey>>
        //in order for Entity Framework to be able to translate it to SQL
        Expression orderByMemExp = ExpressionHelpers.RemoveConvert(sortExpression.KeySelector.Body) as MemberExpression;
        ParameterExpression sourceParam = sortExpression.KeySelector.Parameters[0];

        LambdaExpression orderByLamda = Expression.Lambda(orderByMemExp, new[] { sourceParam });

        MethodInfo orderByMethod = sortExpression.Desc ?
                                        ThenByDescMethod.MakeGenericMethod(new[] { typeof(T), orderByMemExp.Type }) :
                                        ThenByMethod.MakeGenericMethod(new[] { typeof(T), orderByMemExp.Type });

        //Call OrderBy or OrderByDescending on the source IQueryable<T>
        return (IQueryable<T>)orderByMethod.Invoke(null, new object[] { source, orderByLamda });
    }
}

Now my data layer can have a method like List<T> GetList(Expression<Func<T, bool>> where, params DynamicSortExpression<T>[] orderBy) that can be called like

new MyClass<Person>().GetList(p => p.FirstName == "Billy", //where clause
                              new DynamicSortExpression<Person>(p => p.FirstName),
                              new DynamicSortExpression<Person>(p => p.LastName, true));

Where internally, MyClass<T>.GetList builds the query which calls my OrderBy extension method. Something like:

DBContext.Set<T>().Where(whereParam).OrderBy(orderByParams);

The RemoveConvert method is something I yanked out of the EntityFramework source code to recursively remove Convert calls from a MemberExpression:

internal static Expression RemoveConvert(Expression expression)
{
    System.Diagnostics.Debug.Assert(expression != null);

    while ((expression != null)
            && (expression.NodeType == ExpressionType.Convert
                || expression.NodeType == ExpressionType.ConvertChecked))
    {
        expression = RemoveConvert(((UnaryExpression)expression).Operand);
    }

    return expression;
}

I hope this is helpful! Thanks Jon!

Answer 2

You basically can't use query expressions like this, due to the way they're translated. However, you can do it explicitly with extension methods:

string sortBy = HttpContext.Current.Request.QueryString["sidx"];
ParameterExpression prm = Expression.Parameter(typeof(buskerPosting), "posting");
Expression orderByProperty = Expression.Property(prm, sortBy);

// get the paged records
IQueryable<PostingListItemDto> query = be.buskerPosting
    .Where(posting => posting.buskerAccount.cmsMember.nodeId == m.Id)
    .OrderBy(orderByExpression)
    .Select(posting => new PostingListItemDto { Set = posting })
    .Skip<PostingListItemDto>((page -   1) * pageSize)
    .Take<PostingListItemDto>(pageSize);

The tricky bit is getting the right expression tree type - that'll come in an edit :)

EDIT: The edit will be somewhat delayed for various reasons. Basically you may need to call a generic method using reflection, as Queryable.OrderBy needs a generic Expression<Func<TSource, TKey>> and although it looks like you know the source type at compile-time, you may not know the key type. If you do know it'll always be ordering by (say) an int, you can use:

Expression orderByProperty = Expression.Property(prm, sortBy);
var orderByExpression = Expression.Lambda<Func<buskerPosting, int>>
    (orderByProperty, new[] { prm });

EDIT: Okay, it looks like I had time after all. Here's a short example of calling OrderBy using reflection:

using System;
using System.Reflection;
using System.Linq;
using System.Linq.Expressions;

public class Test
{
    static void Main()
    {
        string[] names = { "Jon", "Holly", "Tom", "Robin", "Will" };
        var query = names.AsQueryable();
        query = CallOrderBy(query, "Length");
        foreach (var name in query)
        {
            Console.WriteLine(name);
        }
    }

    private static readonly MethodInfo OrderByMethod =
        typeof(Queryable).GetMethods()
            .Where(method => method.Name == "OrderBy")
            .Where(method => method.GetParameters().Length == 2)
            .Single();

    public static IQueryable<TSource> CallOrderBy<TSource>
        (IQueryable<TSource> source, string propertyName)
    {
        ParameterExpression parameter = Expression.Parameter(typeof(TSource), "posting");
        Expression orderByProperty = Expression.Property(parameter, propertyName);

        LambdaExpression lambda = Expression.Lambda(orderByProperty, new[] { parameter });
        Console.WriteLine(lambda);
        MethodInfo genericMethod = OrderByMethod.MakeGenericMethod
            (new[] { typeof(TSource), orderByProperty.Type });
        object ret = genericMethod.Invoke(null, new object[] {source, lambda});
        return (IQueryable<TSource>) ret;
    }
}

You could easily refactor CallOrderBy into an extension method (e.g. OrderByProperty) like this:

public static class ReflectionQueryable
{
    private static readonly MethodInfo OrderByMethod =
        typeof(Queryable).GetMethods()
            .Where(method => method.Name == "OrderBy")
            .Where(method => method.GetParameters().Length == 2)
            .Single();

    public static IQueryable<TSource> OrderByProperty<TSource>
        (this IQueryable<TSource> source, string propertyName)
    {
        ParameterExpression parameter = Expression.Parameter(typeof(TSource), "posting");
        Expression orderByProperty = Expression.Property(parameter, propertyName);

        LambdaExpression lambda = Expression.Lambda(orderByProperty, new[] { parameter });
        Console.WriteLine(lambda);
        MethodInfo genericMethod = OrderByMethod.MakeGenericMethod
            (new[] { typeof(TSource), orderByProperty.Type });
        object ret = genericMethod.Invoke(null, new object[] {source, lambda});
        return (IQueryable<TSource>) ret;
    }    
}

Your original code then becomes:

string sortBy = HttpContext.Current.Request.QueryString["sidx"];
// get the paged records
IQueryable<PostingListItemDto> query = be.buskerPosting
    .Where(posting => posting.buskerAccount.cmsMember.nodeId == m.Id)
    .OrderByProperty(sortBy)
    .Select(posting => new PostingListItemDto { Set = posting })
    .Skip<PostingListItemDto>((page -   1) * pageSize)
    .Take<PostingListItemDto>(pageSize);

(Apologies for the formatting involving horizontal scrollbars... I'll reformat later if anyone cares. Or you could do it for me if you have enough rep ;)