问题
I am constantly getting an error "404 not found".
I have created cluster and storage account and container.
Detailed error that I get is:
Unhandled Exception: System.AggregateException: One or more errors occurred. ---
Microsoft.WindowsAzure.Storage.StorageException: The remote server returned an error: (404) Not Found.System.Net.WebException: The remote server returned an error: (404) Not Found.
This is my code:
public static void ConnectToAzureCloudServer()
{
HadoopJobConfiguration myConfig = new HadoopJobConfiguration();
myConfig.DeleteOutputFolder = true;
myConfig.InputPath = "wasb://hdstore12345687@hdstore12345678.blob.core.windows.net/example/data/gutenberg/";
myConfig.OutputFolder = "wasb://hdstore12345687@hdstore12345678.blob.core.windows.net/out/o";
Environment.SetEnvironmentVariable("HADOOP_HOME", @"C:\apps\dist\hadoop-2.7.1.2.3.3.1-25\bin");
Environment.SetEnvironmentVariable("JAVA_HOME", @"C:\Program Files\Java\jre1.8.0_101\bin\javaw.exe");
Uri azureCluster = new Uri("https://HDInsight-My-Demo.azurehdinsight.net");
string clusterUserName = "hduser";
string clusterPassword = "Hdinsight@1234";
// This is the name of the account under which Hadoop will execute jobs.
// Normally this is just "Hadoop".
string hadoopUserName = "Hadoop";
// Azure Storage Information.
string azureStorageAccount = "hdstore12345678.blob.core.windows.net";
string azureStorageKey = @"blablabalbNQNUqtSOMEStorGEKeyssbhIvFjfHTg==";
string azureStorageContainer = "hdstore12345687";
//string azureStorageContainer = "https://hdstore12345678.blob.core.windows.net/hdstore12345678";
bool createContinerIfNotExist = true;
Microsoft.Hadoop.MapReduce.IHadoop myCluster = Microsoft.Hadoop.MapReduce.Hadoop.Connect(azureCluster, clusterUserName, hadoopUserName, clusterPassword, azureStorageAccount, azureStorageKey, azureStorageContainer, createContainerIfNotExist);
//execute mapreduce job
MapReduceResult jobResult = myCluster.MapReduceJob.Execute<MySimpleMapper, MySimpleReducer>(myConfig);
int exitCode = jobResult.Info.ExitCode;
string exitStatus = "Failure";
if (exitCode == 0)
exitStatus = "Success";
exitStatus = exitCode + " (" + exitStatus + ")";
Console.WriteLine();
Console.Write("Exit Code = " + exitStatus);
Console.Read();
}
I created a output folder in as user/out. I the HadoopJobconfiguration I am setting it to true and it gets deleted!!. so that means it is able to find storage. Then why do I get that error.
来源:https://stackoverflow.com/questions/39419574/microsoft-windows-azure-storage-the-remote-server-returned-an-error-404-not-fou