Is there a .NET equivalent to Apache Hadoop? [closed]

Answer 1

You can look into something like RavenDb it provides very decent support for MapReduce for a fairly large size of data. as it is built in .Net so a proper LINQ client API is available.

http://ravendb.net/

To get you started you can read my blog entery.

Answer 2

As others have mentioned, DryadLINQ is a programming framework that allows developers to write LINQ queries and execute them on a cluster, in a similar manner to MapReduce. The DryadLINQ project has recently been released under the Apache license on GitHub, and the release includes support for running on YARN clusters (including Azure HDInsight clusters).

Answer 3

Microsoft Research has project Daytona http://research.microsoft.com/en-us/projects/daytona/

You can download it. There's a WordCount sample in C#.

Answer 4

Have you looked at using Hadoop's streaming?

I use it in python all the time :-).

I'm starting to see that the heterogeneous approach is often the best and it looks like other folks are doing the same.

If you look at projects like protocol-buffers or facebook's thrift you see that sometimes it's just best to use an app written in another language and build the glue in the language of your preference.

Answer 5

There's a pretty cute MapReduce implementation for .NET at: http://mapsharp.codeplex.com/

Answer 6

dryad/linq is being productized and will be released soon: http://blogs.technet.com/b/windowshpc/archive/2011/07/07/announcing-linq-to-hpc-beta-2.aspx use in conjunction with Microsoft HPC for a powerful, cluster based solution for quering unstructured data