Elasticsearch can't write to log files

Question

I would like to activate logs of elasticsearch .

When I run elasticsearch binaries, I realize that I have problems with logging : the configuration cannot be loaded !

Here is the output :

~ $ sudo /usr/share/elasticsearch/bin/elasticsearch
Failed to configure logging...
org.elasticsearch.ElasticsearchException: Failed to load logging configuration
    at org.elasticsearch.common.logging.log4j.LogConfigurator.resolveConfig(LogConfigurator.java:117)
    at org.elasticsearch.common.logging.log4j.LogConfigurator.configure(LogConfigurator.java:81)
    at org.elasticsearch.bootstrap.Bootstrap.setupLogging(Bootstrap.java:94)
    at org.elasticsearch.bootstrap.Bootstrap.main(Bootstrap.java:178)
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:32)
Caused by: java.nio.file.NoSuchFileException: /usr/share/elasticsearch/config
    at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
    at sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55)
    at sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:144)
    at sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:97)
    at java.nio.file.Files.readAttributes(Files.java:1686)
    at java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:109)
    at java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:69)
    at java.nio.file.Files.walkFileTree(Files.java:2602)
    at org.elasticsearch.common.logging.log4j.LogConfigurator.resolveConfig(LogConfigurator.java:107)
    ... 4 more
log4j:WARN No appenders could be found for logger (node).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Java HotSpot(TM) Server VM warning: You have loaded library /usr/share/elasticsearch/lib/sigar/libsigar-x86-linux.so which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.

I am running under linuxmint 17 and using latest ES.

Answer 1

If you installed elasticsearch using a package manager like yum or apt-get you should not start elasticsearch this way. Try to use the service: for instance /etc/init.d/elasticsearch or using the command sudo service elasticsearch start

You should also check if the logging.yml file is at the location mentioned in the stack trace:/usr/share/elasticsearch/config

Using sudo to start elasticsearch is not good, you could do sudo elasticsearch to run as the elasticsearch user, but I prefer to use the service call as described in the second sentence.

Hope that helps

Answer 2

Your elasticsearch.yml file as well as logging.yml file will be in the /etc/elasticsearch folder.

Create a config folder in your elasticsearch folder in /usr/share and move the .yml files to the config folder

Now run /bin/elasticsearch start and it will work.

Answer 3

Following this worked for me.

Your elasticsearch.yml file as well as logging.yml file will be in the /etc/elasticsearch folder.

Create a config folder in your elasticsearch folder in /usr/share and move the .yml files to the config folder

Answer 4

You can do symlinks, but if you simply want to point elasticsearch to the desired config directory, you can use the following:

/usr/share/elasticsearch/bin/elasticsearch --default.path.conf=/etc/elasticsearch/

Answer 5

If you have installed elasticsearch in Debian systems using dpkg, the default configuration can be found at /etc/elasticsearch/

You will have 2 yml files:

• elasticsearch.yml
• logging.yml

You can edit the elasticsearch file (changing access right or copying file into your main elasticsearch directory). To configure your log path (l.167).

    #path.logs: /path/to/logs
    #e.g:        
    path.logs: /usr/share/elasticsearch/logs

Make sure you have right access on it. Then try

    cd /usr/share/elasticsearch/ 
    bin/elasticsearch start

and check the publish_adress given in terminal

You can also define more parameters. Hope this help.

Answer 6

If you have installed a .deb and want to start sudo /usr/share/elasticsearch/bin/elasticsearch

You have to do :

cd /usr/share/elasticsearch

sudo ln -s /etc/elasticsearch config

Answer 7

I think that you have installed your elasticsearch using yum or apt-get The best way of doing this is by typing:

$ sudo service elasticsearch start

or anything similar, like the way you would start services like nginx, etc.

It works for me.

Answer 8

If you need to run it in foreground, but not as service just specify configuration directory explicitly.

$ /usr/share/elasticsearch/bin/elasticsearch -Des.path.conf=/etc/elasticsearch

In this case elasticsearch will be able to find all your config files.

Answer 9

In my case I was trying to start elasticsearch from the tar download so that I can run multiple separate instances easily without service definitions.

What worked for me was to go up a directory and launch elasticsearch using bin/elasticsearch instead of just elasticsearch while being in the bin directory.

Now it started up no problem without any symlinks, service files, or other changes.

Answer 10

I'm running elasicsearch with the user elasticsearch, so I used this command:

sudo su elasticsearch -c './bin/elasticsearch -d --default.path.conf=/etc/elasticsearch'

from the dir where elasticsearch is installed. Had to do:

sudo chown -R elasticsearch:elasticsearch .

in there though because otherwise it won't have write access to the log files. I don't know how this would work via the debian startup scripts, but this seems to work fine like this.

Answer 11

I got into the same issue and was wondering how they could miss it in the installation script of elastic search. So, after some digging here is what I found:

1. It looks like you installed elastic search using yum or apt-get or else you would have had the 'config' directory under ES_HOME.

2. Checking the init.d/elasticsearch shows the following:

   DAEMON_OPTS="-d -p $PID_FILE --default.path.home=$ES_HOME --default.path.logs=$LOG_DIR --default.path.data=$DATA_DIR --default.path.conf=$CONF_DIR"

   :

   # Start Daemon

   start-stop-daemon -d $ES_HOME --start -b --user "$ES_USER" -c "$ES_USER" --pidfile "$PID_FILE" --exec $DAEMON -- $DAEMON_OPTS

3. So, all the variables are properly initialized and then passed as command line parameters.

4. So, the best way for you to start is using the service:

   >sudo service elasticsearch start

5. It will make sure that all the config and log parameters are passed to the main command.

6. Check logs under /var/log/$NAME that would be '/var/log/elasticsearch`