Setting up Redis on Webfaction

Question

What are the steps required to set up Redis database on Webfaction shared hosting account?

Answer 1

Introduction

Because of the special environment restrictions of Webfaction servers the installation instructions are not as straightforward as they would be. Nevertheless at the end you will have a fully functioning Redis server that stays up even after a reboot. I personally installed Redis by the following procedure about a half a year ago and it has been running flawlessy since. A little word of a warning though, half a year is not a long time, especially because the server have not been under a heavy use.

The instructions consists of five parts: Installation, Testing, Starting the Server, Managing the Server and Keeping the Server Running.

Installation

Login to your Webfaction shell

ssh foouser@foouser.webfactional.com

Download latest Redis from Redis download site.

> mkdir -p ~/src/
> cd ~/src/
> wget http://download.redis.io/releases/redis-2.6.16.tar.gz
> tar -xzf redis-2.6.16.tar.gz
> cd redis-2.6.16/

Before the make, see is your server Linux 32 or 64 bit. The installation script does not handle 32 bit environments well, at least on Webfaction's CentOS 5 machines. The command for bits is uname -m. If Linux is 32 bit the result will be i686, if 64 bit then x86_64. See this answer for details.

> uname -m
i686

If your server is 64 bit (x86_64) then just simply make.

> make

But if your server is 32 bit (i686) then you must do little extra stuff. There is a command make 32bit but it produces an error. Edit a line in the installation script to make make 32bit to work.

> nano ~/src/redis-2.6.16/src/Makefile

Change the line 214 from this

$(MAKE) CFLAGS="-m32" LDFLAGS="-m32"

to this

$(MAKE) CFLAGS="-m32 -march=i686" LDFLAGS="-m32 -march=i686"

and save. Then run the make with 32bit flag.

> cd ~/src/redis-2.6.16/  ## Note the dir, no trailing src/
> make 32bit

The executables were created into directory ~/src/redis-2.6.16/src/. The executables include redis-cli, redis-server, redis-benchmark and redis-sentinel.

Testing (optional)

As the output of the installation suggests, it would be nice to ensure that everything works as expected by running tests.

Hint: To run 'make test' is a good idea ;)

Unfortunately the testing requires tlc8.6.0 to be installed which is not the default at least on the machine web223. So you must install it first, from source. See Tcl/Tk installation notes and compiling notes.

> cd ~/src/
> wget http://prdownloads.sourceforge.net/tcl/tcl8.6.0-src.tar.gz
> tar -xzf tcl8.6.0-src.tar.gz
> cd tcl8.6.0-src/unix/
> ./configure --prefix=$HOME
> make
> make test # Optional, see notes below
> make install

Testing Tcl with make test will take time and will also fail due to WebFaction's environment restrictions. I suggest you skip this.

Now that we have Tlc installed we can run Redis tests. The tests will take a long time and also temporarily uses a quite large amount of memory.

> cd ~/src/redis-2.6.16/
> make test

After the tests you are ready to continue.

Starting the Server

First, create a custom application via Webfaction Control Panel (Custom app (listening on port)). Name it for example fooredis. Note that you do not have to create a domain or website for the app if Redis is used only locally i.e. from the same host.

Second, make a note about the socket port number that was given for the app. Let the example be 23015.

Copy the previously compiled executables to the app's directory. You may choose to copy all or only the ones you need.

> cd ~/webapps/fooredis/
> cp ~/src/redis-2.6.16/src/redis-server .
> cp ~/src/redis-2.6.16/src/redis-cli .

Copy also the sample configuration file. You will soon modify that.

> cp ~/src/redis-2.6.16/redis.conf .

Now Redis is already runnable. There is couple problems though. First the default Redis port 6379 might be already in use. Second, even if the port was free, yes, you could start the server but it stops running at the same moment you exit the shell. For the first the redis.conf must be edited and for the second, you need a daemon which is also solved by editing redis.conf.

Redis is able to run itself in the daemon mode. For that you need to set up a place where the daemon stores its process ids, PIDs. Usually pidfiles are stored in /var/run/ but because the environment restrictions you must select a place for them in your home directory. Because a reason explained later in the part Managing the Server, a good choice is to put the pidfile under the same directory as the executables. You do not have to create the file yourself, Redis creates it for you automatically.

Now open the redis.conf for editing.

> cd ~/webapps/fooredis/
> nano redis.conf

Change the configurations in the following manner.

• daemonize no -> daemonize yes
• pidfile /var/run/redis.pid -> pidfile /home/foouser/webapps/fooredis/redis.pid
• port 6379 -> port 23015

Now finally, start Redis server. Specify the conf-file so Redis listens the right port and runs as a daemon.

> cd ~/webapps/fooredis/
> ./redis-server redis.conf
> 

See it running.

> cd ~/webapps/fooredis/
> ./redis-cli -p 23015
redis 127.0.0.1:23015> SET myfeeling Phew.
OK
redis 127.0.0.1:23015> GET myfeeling
"Phew."
redis 127.0.0.1:23015> (ctrl-d)
>

Stop the server if you want to.

> ps -u $USER -o pid,command | grep redis
  718 grep redis
10735 ./redis-server redis.conf
> kill 10735

or

> cat redis.pid | xargs kill

Managing the Server

For the ease of use and as a preparatory work for the next part, make a script that helps to open the client and start, restart and stop the server. An easy solution is to write a makefile. When writing a makefile, remember to use tabs instead of spaces.

> cd ~/webapps/fooredis/
> nano Makefile

# Redis Makefile
client cli:
    ./redis-cli -p 23015

start restart:
    ./redis-server redis.conf

stop:
    cat redis.pid | xargs kill

The rules are quite self-explanatory. The special about the second rule is that while in daemon mode, calling the ./redis-server does not create a new process if there is a one running already.

The third rule has some quiet wisdom in it. If redis.pid was not stored under the directory of fooredis but for example to /var/run/redis.pid then it would not be so easy to stop the server. This is especially true if you run multiple Redis instances concurrently.

To execute a rule:

> make start

Keeping the Server Running

You now have an instance of Redis running in daemon mode which allows you to quit the shell without stopping it. This is still not enough. What if the process crashes? What if the server machine is rebooted? To cover these you have to create two cronjobs.

> export EDITOR=nano
> crontab -e

Add the following two lines and save.

*/5 * * * * make -C ~/webapps/fooredis/ -f ~/webapps/fooredis/Makefile start
@reboot make -C ~/webapps/fooredis/ -f ~/webapps/fooredis/Makefile start

The first one ensures each five minutes that fooredis is running. As said above this does not start new process if one is already running. The second one ensures that fooredis is started immediately after the server machine reboot and long before the first rule kicks in.

Some more deligate methods for this could be used, for example forever. See also this Webfaction Community thread for more about the topic.

Conclusion

Now you have it. Lots of things done but maybe more will come. Things you may like to do in the future which were not covered here includes the following.

• Setting a password, preventing other users flushing your databases. (See redis.conf)
• Limiting the memory usage (See redis.conf)
• Logging the usage and errors (See redis.conf)
• Backupping the data once in a while.

Any ideas, comments or corrections?

Answer 2

To summarize Akseli's excellent answer:

assume your user is named "magic_r_user"

cd ~
wget "http://download.redis.io/releases/redis-3.0.0.tar.gz"
tar -xzf redis-3.0.0.tar.gz
mv redis-3.0.0 redis
cd redis
make
make test
create a custom app "listening on port" through the Webfaction management website
    assume we named it magic_r_app
    assume it was assigned port 18932
cp ~/redis/redis.conf ~/webapps/magic_r_app/
vi ~/webapps/magic_r_app/redis.conf
    daemonize yes   
    pidfile ~/webapps/magic_r_app/redis.pid
    port 18932
test it
    ~/redis/src/redis-server ~/webapps/magic_r_app/redis.conf
    ~/redis/src/redis-cli -p 18932
    ctrl-d
    cat ~/webapps/magic_r_app/redis.pid | xargs kill
crontab -e
    */1 * * * * /home/magic_r_user/redis/src/redis-server /home/magic_r_user/webapps/magic_r_app/redis.conf &>> /home/magic_r_user/logs/user/cron.log
don't forget to set a password!

Answer 3

FYI, if you are installing redis 2.8.8+ you may get an error, undefined reference to __sync_add_and_fetch_4 when compiling. See http://www.eschrade.com/page/undefined-reference-to-__sync_add_and_fetch_4/ for information.

I've pasted the relevant portion from that page below in case the page ever goes offline. Essentially you need to export the CFLAGS variable and restart the build process.

[root@devvm1 redis-2.6.7]# export CFLAGS=-march=i686
[root@devvm1 redis-2.6.7]# make distclean
[root@devvm1 redis-2.6.7]# make