Retrieve an object from entityframework without ONE field

Question

I\'m using entity framework to connect with the database. I\'ve one little problem:

I\'ve one table which have one varbinary(MAX) column(with filestream).

I\

Answer 1

I had this requirement because I have a Document entity which has a Content field with the content of the file, i.e. some 100MB in size, and I have a Search function that I wanted to return the rest of the columns.

I chose to use projection:

IQueryable<Document> results = dbContext.Documents.Include(o => o.UploadedBy).Select(o => new {
    Content = (string)null,
    ContentType = o.ContentType,
    DocumentTypeId = o.DocumentTypeId,
    FileName = o.FileName,
    Id = o.Id,
    // etc. even with related entities here like:
    UploadedBy = o.UploadedBy
});

Then my WebApi controller passes this results object to a common Pagination function, which applies a .Skip, .Take and a .ToList.

This means that when the query gets executed, it doesn't access the Content column, so the 100MB data is not being touched, and the query is as fast as you'd want/expect it to be.

Next, I cast it back to my DTO class, which in this case is pretty much exactly the same as the entity class, so this might not be a step you need to implement, but it's follows my typical WebApi coding pattern, so:

var dtos = paginated.Select(o => new DocumentDTO
{
    Content = o.Content,
    ContentType = o.ContentType,
    DocumentTypeId = o.DocumentTypeId,
    FileName = o.FileName,
    Id = o.Id,
    UploadedBy = o.UploadedBy == null ? null : ModelFactory.Create(o.UploadedBy)
});

Then I return the DTO list:

return Ok(dtos);

So it uses projection, which might not fit the original poster's requirements, but if you're using DTO classes, you're converting anyway. You could just as easily do the following to return them as your actual entities:

var dtos = paginated.Select(o => new Document
{
    Content = o.Content,
    ContentType = o.ContentType,
    DocumentTypeId = o.DocumentTypeId,
    //...

Just a few extra steps but this is working nicely for me.

Answer 2

I'd like to share my attempts to workaround this problem in case somebody else is in the same situation.

I started with what Jeremy Danyow suggested, which to me is the less painful option.

// You need to include all fields in the query, just make null the ones you don't want.
var results = context.Database.SqlQuery<myEntity>("SELECT Field1, Field2, Field3, HugeField4 = NULL, Field5 FROM TableName");

In my case, I needed a IQueryable<> result object so I added AsQueryable() at the end. This of course let me add calls to .Where, .Take, and the other commands we all know, and they worked fine. But there's a caveat:

The normal code (basically context.myEntity.AsQueryable()) returned a System.Data.Entity.DbSet<Data.DataModel.myEntity>, while this approach returned System.Linq.EnumerableQuery<Data.DataModel.myEntity>.

Apparently this means that my custom query gets executed "as is" as soon as needed and the filtering I added later is done afterwards and not in the database.

Therefore I tried to mimic Entity Framework's object by using the exact query EF creates, even with those [Extent1] aliases, but it didn't work. When analyzing the resulting object, its query ended like

FROM [dbo].[TableName] AS [Extent1].Where(c => ...

instead of the expected

FROM [dbo].[TableName] AS [Extent1] WHERE ([Extent1]...

Anyway, this works, and as long as the table is not huge, this method will be fast enough. Otherwise you have no option than to manually add the conditions by concatenating strings, like classic dynamic SQL. A very basic example in case you don't know what I'm talking about:

string query = "SELECT Field1, Field2, Field3, HugeField4 = NULL, Field5 FROM TableName";
if (parameterId.HasValue)
    query += " WHERE Field1 = " + parameterId.Value.ToString();
var results = context.Database.SqlQuery<myEntity>(query);

In case your method sometimes needs this field, you can add a bool parameter and then do something like this:

IQueryable<myEntity> results;
if (excludeBigData)
    results = context.Database.SqlQuery<myEntity>("SELECT Field1, Field2, Field3, HugeField4 = NULL, Field5 FROM TableName").AsQueryable();
else
    results = context.myEntity.AsQueryable();

If anyone manages to make the Linq extensions work properly like if it was the original EF object, please comment so I can update the answer.

Answer 3

you can do this:

var files = dbContext.Database.SqlQuery<File>("select FileId, DataType, MimeType from Files");

or this:

var files = objectContext.ExecuteStoreQuery<File>("select FileId, DataType, MimeType from Files");

depending on your version of EF

Answer 4

I'm using here the anonymous type because otherwise you will get a NotSupportedException: The entity or complex type 'ProjectName.File' cannot be constructed in a LINQ to Entities query.

var file = context.Files
        .Where(f => f.Id == idFile)
        .FirstOrDefault() // You need to exeucte the query if you want to reuse the type
        .Select(f => new {
            f.Id, f.MimeType, f.Size, f.FileName, f.DataType,
            f.DateModification, f.FileId
        }).FirstOrDefault();

And also its not a bad practice to de-normalize the table into further, i.e one with metadata and one with payload to avoid projection. Projection would work, the only issue is, need to edit any time a new column is added to the table.

Answer 5

For EF Core 2 I implemented a solution like this:

var files = context.Files.AsNoTracking()
                         .IgnoreProperty(f => f.Report)
                         .ToList();

The base idea is to turn for example this query:

SELECT [f].[Id], [f].[Report], [f].[CreationDate]
FROM [File] AS [f]

into this:

SELECT [f].[Id], '' as [Report], [f].[CreationDate]
FROM [File] AS [f]

you can see the full source code in here: https://github.com/aspnet/EntityFrameworkCore/issues/1387#issuecomment-495630292

Answer 6

I tried this:

From the edmx diagram (EF 6), I clicked the column I wanted to hide from EF and on their properties you can set their getter and setter to private. That way, for me it works.

I return some data which includes a User reference, so I wanted to hide the Password field even though it's encrypted and salted, I just didn't want it on my json, and I didn't want to do a:

Select(col => new {}) 

because that's a pain to create and maintain, especially for big tables with a lot of relationships.

The downside with this method is that if you ever regenerate your model, you would need to modify their getter and setter again.