how to connect to HBase / Hadoop Database using C#

Question

Recently, Exploring Microsoft HDInsight Hadoop for Windows.But don\'t know where to began and start using apache hadoop with c# / asp.net mvc.

i know http://hadoopsdk.co

Answer 1

The easiest way to get started is to use the HDInsight service on Azure (which is still in preview, but works well). That way you can just log into your azure portal and spin up a cluster, which will be linked to your azure storage.

If you really don't want to go to the cloud, then you can download the HDInsight developer preview package through the web platform installer, which will create a single node local cluster. Alternatively, if you're interested in 'just hadoop' then you could try using the VMs provided by Hortonworks or Cloudera.

After that, it's probably worth starting with Hive if you are familiar with SQL. The O'Reilly Definitive Guide to Hadoop is pretty good on that.

From the .NET point of view, the hadoopsdk on codeplex is a good place to start.

As far as very basic connection examples go, try this blog for an example, but note that the connection for HDInsight is slightly different now it's all using the templeton interface, so this will get you going:

var db = new HiveConnection(
        webHCatUri: new Uri("http://localhost:50111"),
        userName: (string) "hadoop", password: (string) null);
var result = db.ExecuteHiveQuery("select * from w3c");

If you are looking to do full on MapReduce on HDInsight, then you probably want to take a look at the C# MapReduce examples with the sdk on codeplex.

Note that the default HDInsight install also comes with some good samples, which include a bit of data to play with and some powershell scripts and .NET code to get you started.

Enjoy!