Multiples Hadoop FileSystem instances

Question

I have a class (I removes try/catch for readability):

public class HadoopFileSystem {

    private FileSystem m_fileSystem = null;

    public HadoopFileSystem()         


        

Answer 1

You are creating an object based on a configuration which is hard-coded. This basically means that you're creating 2 identical objects. Because these objects are identical, the JVM will reference to the same object. So h1 and h2 are referencing the same object.

The reason for this is because you're getting an existing instance of an object based on a configuration file. If the configuration is different for h1 and h2, it will not be the same instance anymore.

Answer 2

m_fileSystem = FileSystem.get(l_configuration ); is a static call even if you have two different objects created. You need to find a way to not make this call static for two different objects.

Try this out to do away with the problem,

conf.setBoolean("fs.hdfs.impl.disable.cache", true);