How to stop INFO messages displaying on spark console?

Question

I\'d like to stop various messages that are coming on spark shell.

I tried to edit the log4j.properties file in order to stop these message.

Her

Answer 1

1. Adjust conf/log4j.properties as described by other log4j.rootCategory=ERROR, console
2. Make sure while executing your spark job you pass --file flag with log4j.properties file path
3. If it still doesn't work you might have a jar that has log4j.properties that is being called before your new log4j.properties. Remove that log4j.properties from jar (if appropriate)

Answer 2

Another way of stopping logs completely is:

    import org.apache.log4j.Appender;
    import org.apache.log4j.BasicConfigurator;
    import org.apache.log4j.varia.NullAppender;

    public class SomeClass {

        public static void main(String[] args) {
            Appender nullAppender = new NullAppender();
            BasicConfigurator.configure(nullAppender);

            {...more code here...}

        }
    }

This worked for me. An NullAppender is

An Appender that ignores log events. (https://logging.apache.org/log4j/2.x/log4j-core/apidocs/org/apache/logging/log4j/core/appender/NullAppender.html)