Dies Anleitung soll dafür Dienen wie man Redis-Cluster auf Ubuntu installiert.

Table of Contents

Voraussetzung

...

Introduction

Redis is an in-memory data storage that is used by the CoreOne Suite. For supported versions see: Next-Gen Workflows - Redis

Step 1 - Check your prerequisits

Redis has to be installed on a Linux OS (Ubuntu, Redhat, Centos, etc.)
Für die Installation braucht es sicherlich 3 Redis Instanzen. Diese können auf unterschiedlichen Server installiert werden, oder mehrere Instanzen auf einem Server. (best practice, anhand untenstehendem Beispiel.)
Sofern die Redis-Server in unterschiedlichen Zonen stehen, müssen sich diese gegenseitig erreichen können.
Folgende Firewall Port müssen geöffnet werden.
To install Redis, at least 3 instances are needed. The instances can be distributed across different servers, or concentrated on one (best practice will be shown in the examples below).
If the servers for Redis are in different network-zones, the servers must be able to communicate with eachother.
The following Firewall-Ports must be open:
- 6379
- 16384
- 16379
- 7001 (Falls über Cluster)
- 7002 (Falls über Cluster)
- 7003 (Falls über Cluster)
- 7004 (Falls über Cluster)
- 7006 (Falls über Cluster)

...

Example

In our example we got 3 virtual Ubuntu-Servers.

Redis HA 1 (192.168.17.134)
Redis HA 2 (192.168.17.135)
Redis HA 3 (192.168.17.136)

Auf On Redis HA1 ist sowohl eine Master Instanz 7001 sowohl auch eine Slave Instanz 7004
Auf Redis HA2 ist sowohl eine Master Instanz 7002 sowohl auch eine Slave Instanz 7005
Auf Redis HA3 ist sowohl eine Master Instanz 7003 sowohl auch eine Slave Instanz 7006
Der Slave für Master 7001 ist Slave 7006
Der Slave für Master 7002 ist Slave 7006
Der Slave für Master 7003 ist Slave 7004

Image Removed

Installation

Allgemeine Installation für alle 3 Server

Switch auf is a Master Instance 7001 and a Slave Instance 7004
On Redis HA2 is a Master Instance 7002 and a Slave Instance 7005
On Redis HA3 is a Master Instance 7003 and a Slave Instance 7006

For the Master 7001 the Slave is 7006
For the Master 7002 the Slave is 7006
For the Master 7003 the Slave is 7004

...

Step 2 - Installation

General installation for all 3 servers:

Switch to root user:

Code Block
sudo su

Erstelle eine Create a Folder in /etc/redis in welchem Redis installiert wirdwhich Redis will be installed:

Code Block
mkdir /etc/redis

Herunterladen von Redis und anschliessendes entpackenDownload Redis and unzipp the package

Info
`https://download.redis.io/releases/redis-6.2.6.tar.gz` kann auch auf Windows heruntergeladen werden und anschliessend auf dem Linux-Server via WINSCP kopiert werden can also be downloaded on windows and copied to your linux instance.

Code Block
wget https://download.redis.io/releases/redis-6.2.6.tar.gz tar xzf redis-6.2.6.tar.gz

Nun sollte in Verzeichnis This will create the folder /Files in /etc/redis folgende Folder/Files vorhanden sein: .

...

Info
Mit cd kann in das Verzeichnis Navigate with ‘cd’ to the folder /redis-6.2.6 navigiert werden

Redis benötigt, needs gcc, make und tcl. Dies kann mit folgendem Befehl installiert werden.and tcl. Those can be installed with the following command:

Code Block
sudo apt-get install -y tcl tk sudo apt install gcc sudo apt-get install make

Überprüfen, ob die Lokale Firewall aktiv ist Check if the local firewall is active:

Code Block
sudo ufw status

Ansonsten die erwähnten Ports aktivierenOpen the needed Ports on the firewall:

Code Block
sudo ufw allow 6379 sudo ufw allow 16384 sudo ufw allow 16379 sudo ufw allow 7001 sudo ufw allow 7002 sudo ufw allow 7003 sudo ufw allow 7004 sudo ufw allow 7005 sudo ufw allow 7006

Da wir make installiert haben, müssen wir das Since we installed ‘make’, we have to exexute the File in etc/redis/redis-6.2.6/makefile noch ausführen

Code Block
cd /etc/redis/redis-6.2.6/ sudo make test

...

Step 3 - Configuration VM 1

Da wir wie in unserem Beispiel mehrere Redis-Instanzen pro Server haben. Brauchen wir mehrere konfig-Files pro Server.
Das heisst wir erstellen auf der VM01 folgende In our example we have multiple redis instanced on one server. This means we need multiple config-files on the server as well.

We will create on VM01 the following config-Files →node1.conf / node4.conf . Mit welchen wir anschliessend Redis starten. Config-File Those will be used to start redis.

Create config-file 1 & 4 erstellen

Info
Falls Vim noch nicht installiert ist. Kann dies mit If Vim is not installed yet, use the command “apt install vim” installiert werden

...

to install it.

Create the config-Files im in the folder /etc/redis - sprich noch in das Verzeichnis navigierento navigate there, use the following command:

Code Block
cd /etc/redis

Code Block
sudo vim node1.conf

Folgender Text kopieren;Copy the following text to the config file:

Code Block
port 7001 cluster-enabled yes cluster-config-file cluster-node-1.conf cluster-node-timeout 5000 appendonly yes appendfilename node-1.aof dbfilename dump-1.rdb

Info
vim

...

: Save file ctrl+c | wq!
vim

...

: Close file without saving ctrl+c | q!
vim

...

: Search in file esc /

...

search
vim

...

: Edit File: i

Code Block
sudo vim node4.conf

Folgender Text kopieren;Copy the following text to the config file:

Code Block
port 7004 cluster-enabled yes cluster-config-file cluster-node-4.conf cluster-node-timeout 5000 appendonly yes appendfilename node-4.aof dbfilename dump-4.rdb

...

Step 4 - Configuration VM 2

Da wir wie in unserem Beispiel mehrere Redis-Instanzen pro Server haben. Brauchen wir mehrere konfig-Files pro Server.
Das heisst wir erstellen auf der VM02 folgende In our example we have multiple redis instanced on one server. This means we need multiple config-files on the server as well.

We will create on VM02 the following config-Files →node2.conf / node5.conf . Mit welchen wir anschliessend Redis starten. Config-File Those will be used to start redis.

Create config-file 2 & 5 erstellen

Info
Falls Vim noch nicht installiert ist. Kann dies mit If Vim is not installed yet, use the command “apt install vim” installiert werden

...

to install it.

Create the config-Files im in the folder /etc/redis - sprich noch in das Verzeichnis navigierento navigate there, use the following command:

Code Block
cd /etc/redis sudo vim node2.conf

Folgender Text kopieren;Copy the following text to the config file:

Code Block
port 7002 cluster-enabled yes cluster-config-file cluster-node-2.conf cluster-node-timeout 5000 appendonly yes appendfilename node-2.aof dbfilename dump-2.rdb

Info
vim

...

: Save file ctrl+c | wq!
vim

...

: Close file without saving ctrl+c | q!
vim

...

: Search in file esc /

...

search
vim

...

: Edit File: i

Code Block
sudo vim node5.conf

Folgender Text kopieren;Copy the following text to the config file:

Code Block
port 7005 cluster-enabled yes cluster-config-file cluster-node-5.conf cluster-node-timeout 5000 appendonly yes appendfilename node-5.aof dbfilename dump-5.rdb

...

Step 5 - Configuration VM 3

Da wir wie in unserem Beispiel mehrere Redis-Instanzen pro Server haben. Brauchen wir mehrere konfig-Files pro Server.
Das heisst wir erstellen auf der VM03 folgende In our example we have multiple redis instanced on one server. This means we need multiple config-files on the server as well.

We will create on VM03 the following config-Files →node3.conf / node6.conf . Mit welchen wir anschliessend Redis starten. Config-File Those will be used to start redis.

Create config-file 3 & 6 erstellen

Info
Falls Vim noch nicht installiert ist. Kann dies mit If Vim is not installed yet, use the command “apt install vim” installiert werden

...

to install it.

Create the config-Files im in the folder /etc/redis - sprich noch in das Verzeichnis navigierento navigate there, use the following command:

Code Block
cd /etc/redis sudo vim node3.conf

Folgender Text kopieren;Copy the following text:

Code Block
port 7003 cluster-enabled yes cluster-config-file cluster-node-3.conf cluster-node-timeout 5000 appendonly yes appendfilename node-3.aof dbfilename dump-3.rdb

Info
vim

...

: Save file ctrl+c | wq!
vim

...

: Close file without saving ctrl+c | q!
vim

...

: Search in file esc /

...

search
vim

...

: Edit File: i

Code Block
sudo vim node6.conf

Folgender Text kopieren;Copy the following text:

Code Block
port 7006 cluster-enabled yes cluster-config-file cluster-node-6.conf cluster-node-timeout 5000 appendonly yes appendfilename node-6.aof dbfilename dump-6.rdb

Step 6 - Start Redis

...

Auf jeder VM kann nun mit folgeden Befehl Redis mit einem Config-File gestartet werden.

...

We can start Redis now on each VM through the config-files we created.

Info
When a server shuts down, the application does not automatically restart. So you have to start the redis instances again!

To do this, navigate to the folder /etc/redis and execute the following commands:

Code Block
cd /etc/redis

Server

Instanz

Befehl

HA1 (192.168.17.134)

Node 01 - Port 7001

Node 04 - Port 7004

Code Block
./redis-6.2.6/src/redis-server node1.conf &

Code Block
./redis-6.2.6/src/redis-server node4.conf &

HA 2 (192.168.17.135)

Node 02 - Port 7002

Node 05 - Port 7005

Code Block
./redis-6.2.6/src/redis-server node2.conf &

Code Block
./redis-6.2.6/src/redis-server node5.conf &

HA 3 (192.168.17.136)

Node 03 - Port 7003

Node 06 - Port 7006

Code Block
./redis-6.2.6/src/redis-server node3.conf &

Code Block
./redis-6.2.6/src/redis-server node6.conf &

Step 7 - Connect Redis

...

We can now connect on each instace with the Redis-CLI zugegriffen werden.

Server

Instanz

Befehl

HA1 (192.168.17.134)

Node 01 - Port 7001

Node 04 - Port 7004

Code Block
./redis-6.2.6/src/redis-cli -p 7001 ./redis-6.2.6/src/redis-cli -p 7004

HA 2 (192.168.17.135)

Node 02 - Port 7002

Node 05 - Port 7005

Code Block
./redis-6.2.6/src/redis-cli -p 7002 ./redis-6.2.6/src/redis-cli -p 7005

HA 3 (192.168.17.136)

Node 03 - Port 7003

Node 06 - Port 7006

Code Block
./redis-6.2.6/src/redis-cli -p 7003 ./redis-6.2.6/src/redis-cli -p 7006

Step 8 - Configure Redis Cluster

...

Da die Instanzen jedoch noch Lokal laufen, müssen diese dem Cluster hinzugefügt werden.

...

Since the instaces are local at the moment, we need to connect them to the cluster.

Navigate to /etc/redis

Code Block
cd /etc/redis

Info
`./redis-6.2.6/src/redis-cli --cluster` + alle Instanzen. Die zweite Instanz ist in jedem Fall der Slaveall instances. The second instance will always be the slave. --cluster-replicas gibt an wie viele slave pro Master erstellt wirdshows how many slaves will be created for each master.

Code Block
./redis-6.2.6/src/redis-cli --cluster create 192.168.17.134:7001 192.168.17.134:7004 192.168.17.135:7002 192.168.17.135:7005 192.168.17.136:7003 192.168.17.136:7006 --cluster-replicas 1

...

Überprüfen und mit Y bestätigen. Check the config and confirm with: Y

Step 9 - Redis-CLI

Auf die Open Redis-CLI kommt man wie folgt:

Info
-p = Port -c = Cluster

Code Block
./redis-6.2.6/src/redis-cli -c -p 7001

ping

Code Block
ping

Können andere Nodes gepingt werden.The different nodes can be pinged:

info

Code Block
info

Zeigt alle Informationen über den Cluster anShows all information on the cluster

...

cluster nodes

Code Block
cluster nodes

Zeigt alle Nodes an, und ob diese Master oder Slave sindShows all nodes and if they are master or slave.

...

info replication

Code Block
info replication

Zeigt an, ob der Server Master oder Slave ist und wer dessen Slave istShows if the server is master or slave. Shows the depending slaves.

...

Step 10 - Failover

Wenn wir nun als Beispiel den We will terminate Master 7003 beenden.
Müsste der Slave 7004 als neuen Master fungieren.

...

Für dieses Beispiel fahren wir den Master 7003 herunter. for this example. This will prompt the Slave 7004 to become the new master:

...

Shutdown the instance

Code Block
./redis-6.2.6/src/redis-cli -c -p 7003 shutdown

...

Auf dem Master von 7001 und 7002 erhält man nun folgende Meldung:

Code Block

2074:S 26 Apr 2022 13:56:22.765 # Error condition on socket for SYNC: Connection refused
2074:S 26 Apr 2022 13:56:23.773 * Connecting to MASTER 192.168.17.136:7003
2074:S 26 Apr 2022 13:56:23.773 * MASTER <-> REPLICA sync started
2074:S 26 Apr 2022 13:56:23.773 # Error condition on socket for SYNC: Connection refused
3904:M 26 Apr 2022 13:56:24.099 * FAIL message received from 9b989e13f6e83d661c072853f48d89d1a995f69c about 88ff5fc7513281e33e5397175cba7dbec2d01265
3904:M 26 Apr 2022 13:56:24.099 # Cluster state changed: fail
2074:S 26 Apr 2022 13:56:24.100 * FAIL message received from 9b989e13f6e83d661c072853f48d89d1a995f69c about 88ff5fc7513281e33e5397175cba7dbec2d01265
2074:S 26 Apr 2022 13:56:24.100 # Cluster state changed: fail
2074:S 26 Apr 2022 13:56:24.177 # Start of election delayed for 711 milliseconds (rank #0, offset 589126).
2074:S 26 Apr 2022 13:56:24.781 * Connecting to MASTER 192.168.17.136:7003
2074:S 26 Apr 2022 13:56:24.781 * MASTER <-> REPLICA sync started
2074:S 26 Apr 2022 13:56:24.781 # Error condition on socket for SYNC: Connection refused
2074:S 26 Apr 2022 13:56:24.983 # Starting a failover election for epoch 28.
3904:M 26 Apr 2022 13:56:24.984 # Failover auth granted to cdfafbc0721ba7774a6e9412e51e7369bd150c1a for epoch 28
2074:S 26 Apr 2022 13:56:25.006 # Failover election won: I'm the new master.
2074:S 26 Apr 2022 13:56:25.006 # configEpoch set to 28 after successful failover
2074:M 26 Apr 2022 13:56:25.006 * Discarding previously cached master state.
2074:M 26 Apr 2022 13:56:25.006 # Setting secondary replication ID to ddd94fd4c80c068c06c03fc5113ce9f33cbbe780, valid up to offset: 589127. New replication ID is 897d8ffc57482c31a6d37db92136073b713e813e
 2074:M 26 Apr 2022 13:56:25.006 # Cluster state changed: ok
 3904:M 26 Apr 2022 13:56:25.024 # Cluster state changed: ok

...

Wenn der ehmalige Slave 7004 nun wieder gestartet wird, ist dieser Standardmässig noch als Slave hinterlegtOn the masters 7001 and 7002 we will get the following message:

We can now check with the Redis-CLI. With the command “Cluster nodes”, we can see, Slave 7004 is now a Master.
We can also see that Master 7003 can’t be reached.

...

If we start the Slave 7004, who is now a Master, the server will still be described as Slave.

Code Block
./redis-6.2.6/src/redis-server node3.conf &

Meldung nach dem AufstartenMessage after restart:

Code Block

17830:S 26 Apr 2022 14:05:46.962 * MASTER <-> REPLICA sync: receiving 193 bytes from master to disk
17830:S 26 Apr 2022 14:05:46.962 * MASTER <-> REPLICA sync: Flushing old data
17830:S 26 Apr 2022 14:05:46.962 * MASTER <-> REPLICA sync: Loading DB in memory
17830:S 26 Apr 2022 14:05:46.963 * Loading RDB produced by version 6.2.6
17830:S 26 Apr 2022 14:05:46.963 * RDB age 0 seconds
17830:S 26 Apr 2022 14:05:46.963 * RDB memory usage when created 2.54 Mb
17830:S 26 Apr 2022 14:05:46.963 # Done loading RDB, keys loaded: 2, keys expired: 0.
17830:S 26 Apr 2022 14:05:46.963 * MASTER <-> REPLICA sync: Finished with success
17830:S 26 Apr 2022 14:05:46.963 * Background append only file rewriting started by pid 17836
17830:S 26 Apr 2022 14:05:47.001 * AOF rewrite child asks to stop sending diffs.
17836:C 26 Apr 2022 14:05:47.001 * Parent agreed to stop sending diffs. Finalizing AOF...
17836:C 26 Apr 2022 14:05:47.001 * Concatenating 0.00 MB of AOF diff received from parent.
17836:C 26 Apr 2022 14:05:47.001 * SYNC append only file rewrite performed
17836:C 26 Apr 2022 14:05:47.001 * AOF rewrite: 0 MB of memory used by copy-on-write
17830:S 26 Apr 2022 14:05:47.075 * Background AOF rewrite terminated with success
17830:S 26 Apr 2022 14:05:47.075 * Residual parent diff successfully flushed to the rewritten AOF (0.00 MB)
17830:S 26 Apr 2022 14:05:47.075 * Background AOF rewrite finished successfully

Wenn man den 7003 wieder als Master deklarieren möchte, kann man dies mitIf you want the 7003 as Master again, use the following command:

Code Block
./redis-6.2.6/src/redis-cli -c -p 7003 cluster failover

MeldungMessage:

Code Block

127.0.0.1:7003> 17830:S 26 Apr 2022 14:07:57.685 # Received replication offset for paused master manual failover: 589308
17830:S 26 Apr 2022 14:07:57.685 # All master replication stream processed, manual failover can start.
7830:S 26 Apr 2022 14:07:57.685 # Start of election delayed for 0 milliseconds (rank #0, offset 589308).
17830:S 26 Apr 2022 14:07:57.685 # Starting a failover election for epoch 29.
17830:S 26 Apr 2022 14:07:57.686 # Currently unable to failover: Waiting for votes, but majority still not reached.
17830:S 26 Apr 2022 14:07:57.686 # Failover election won: I'm the new master.
17830:S 26 Apr 2022 14:07:57.686 # configEpoch set to 29 after successful failover
17830:M 26 Apr 2022 14:07:57.686 # Connection with master lost.
17830:M 26 Apr 2022 14:07:57.686 * Caching the disconnected master state.
17830:M 26 Apr 2022 14:07:57.686 * Discarding previously cached master state.
17830:M 26 Apr 2022 14:07:57.686 # Setting secondary replication ID to 897d8ffc57482c31a6d37db92136073b713e813e, valid up to offset: 589309. New replication ID is 17e90a8ddf3d4fd645d4ec162267629a4109ea9f

Erneute überprüfung über Cluster nodesCheck the cluster nodes:

...

Version	Old Version 1	New Version Current
Changes made by	Marco Mele (Unlicensed)	Christian Huber (Unlicensed)
Saved on	02. May 2022	14. Nov 2022

Versions Compared

Key

Voraussetzung

Introduction

Step 1 - Check your prerequisits

Example

Installation

Allgemeine Installation für alle 3 Server

Step 2 - Installation

General installation for all 3 servers:

Step 3 - Configuration VM 1

Step 4 - Configuration VM 2

Step 5 - Configuration VM 3

Step 6 - Start Redis

Step 7 - Connect Redis

Step 8 - Configure Redis Cluster

Step 9 - Redis-CLI

ping

info

cluster nodes

info replication

Step 10 - Failover

Page Comparison

Versions Compared

Key

Voraussetzung

Introduction

Step 1 - Check your prerequisits

Example

Installation

Allgemeine Installation für alle 3 Server

Step 2 - Installation

General installation for all 3 servers:

Step 3 - Configuration VM 1

Step 4 - Configuration VM 2

Step 5 - Configuration VM 3

Step 6 - Start Redis

Step 7 - Connect Redis

Step 8 - Configure Redis Cluster

Step 9 - Redis-CLI

ping

info

cluster nodes

info replication

Step 10 - Failover