MySQL-Cluster-7.5 for centos 6.9 64bit
一、安装要求
1、安装环境：
系统环境：CentOS release 6.9 64bit
软件名称：MySQL-Cluster-server-gpl-7.4.15-1.el6.x86_64.rpm，MySQL-Cluster-client-gpl-7.4.15-1.el6.x86_64.rpm
软件安装位置：/usr/bin/mysql
数据存放位置：/var/lib/mysql
日志存放位置：/var/log/mysql
二、集群设计
首先设计集群的安装分配方式，共需要5台服务器，服务器分配如下：
管理节点： 192.168.99.35
数据节点1： 192.168.99.36
数据节点2： 192.168.99.37
sql节点1： 192.168.99.38
sql节点2： 192.168.99.39
注意：目前这种设计存在的问题是管理节点是单点的。
三、安装部署
1、在五台服务器上/home创建mysql目录，下载如下两个软件包，建议下载后直接scp过去
[root@centos-6 ~]# mkdir /home/mysql
[root@centos-6 ~]# cd /home/mysql/
[root@centos-6 mysql]#wget http://mirrors.sohu.com/mysql/MySQL-Cluster-7.4/MySQL-Cluster-server-gpl-7.4.15-1.el6.x86_64.rpm
[root@centos-6 mysql]#wget http://mirrors.sohu.com/mysql/MySQL-Cluster-7.4/MySQL-Cluster-client-gpl-7.4.15-1.el6.x86_64.rpm
2、清除五台服务器上的旧版本mysql
查询遗留了哪些mysql
rpm -qa | grep mysql
通过yum方式清除mysql
yum -y remove mysql*
查询清除是否完成
rpm -qa | grep mysql
3、在五台服务器上安装MySQL-Cluster-server-gpl-7.4.15-1.el6.x86_64及MySQL-Cluster-client-gpl-7.4.15-1.el6.x86_64
yum -y install MySQL-Cluster-server-gpl-7.4.15-1.el6.x86_64.rpm MySQL-Cluster-client-gpl-7.4.15-1.el6.x86_64.rpm
备注：整个cluster安装后的初次超级账户密码存在/root/.mysql_secret这个文件当中
4、创建文件夹及进行授权
管理节点创建文件夹并授权
mkdir /var/lib/mysql-cluster /var/run/mysqld/
chown mysql.mysql -R /var/lib/mysql-cluster /var/run/mysqld/
数据节点创建文件夹并授权：
mkdir /var/run/mysqld
chown mysql.mysql -R /var/lib/mysql /var/run/mysqld/
SQL节点创建文件夹并授权
mkdir /var/run/mysqld
chown mysql.mysql -R /var/lib/mysql /var/run/mysqld/
5、配置管理节点配置文件
vim /var/lib/mysql-cluster/config.ini
[ndbd default]
NoOfReplicas=2 #数据写入数量。2表示两份
DataMemory=500M #配置数据存储可使用的内存
IndexMemory=100M #索引给100M
#manager options: #管理节点
[ndb_mgmd]
id=1
datadir=/var/lib/mysql-cluster #管理节点的日志
HostName=192.168.99.35 #管理节点的IP地址。本机IP
#data node options: #数据存储节点
[ndbd]
HostName=192.168.99.36
DataDir=/var/lib/mysql #mysql数据存储路径
id=2
[ndbd]
HostName=192.168.99.37
DataDir=/var/lib/mysql #mysql数据存储路径
id=3
#SQLnode options: #关于SQL节点
[mysqld]
HostName=192.168.99.38
id=4
[mysqld]
HostName=192.168.99.39
id=5
6、配置数据节点配置文件
vim /etc/my.cnf
[mysqld]
datadir=/var/lib/mysql #mysql数据存储路径
ndbcluster #启动ndb引擎
ndb-connectstring=192.168.99.35 # 管理节点IP地址
[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
[mysql_cluster]
ndb-connectstring=192.168.99.35 #管理节点IP地址
7、配置sql节点配置文件
vim /etc/my.cnf
[mysqld]
ndbcluster #启动ndb引擎
ndb-connectstring=192.168.99.35 # 管理节点IP地址
[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
[mysql_cluster]
ndb-connectstring=192.168.99.35 #管理节点IP地址
说明：数据节点和SQL结点配置文件区别 ，就多一行
数据结点有：datadir=/var/lib/mysql #mysql数据存储路径。
四、启动mysql集群
初次启动命令以及用户密码更改调整：（请严格按照次序启动）
先启动：管理结点服务->数据结点服务->sql结点服务
关闭：关闭管理结点服务，关闭管理结点服务后，nbdb数据结点服务会自动关闭->手动把sql结点服务关了。
执行初次启动前请先确认 将两台机器的防火墙关闭（service iptables stop 或者 设定 防火墙端口可通，两个端口即通讯端口1186、数据端口3306 )
1、在99.35上启动管理节点
ndb_mgmd -f/var/lib/mysql-cluster/config.ini # mysql cluster 后台运行进程
[root@centos-6 mysql-cluster]# ndb_mgmd -f/var/lib/mysql-cluster/config.ini
MySQL Cluster Management Server mysql-5.6.36 ndb-7.4.15
2017-06-05 20:18:55 [MgmtSrvr] INFO -- The default config directory '/usr/mysql-cluster' does not exist. Trying to create it...
2017-06-05 20:18:55 [MgmtSrvr] INFO -- Sucessfully created config directory
2017-06-05 20:18:55 [MgmtSrvr] WARNING -- at line 8: [MGM] id is deprecated, use NodeId instead
2017-06-05 20:18:55 [MgmtSrvr] WARNING -- at line 16: [DB] id is deprecated, use NodeId instead
2017-06-05 20:18:55 [MgmtSrvr] WARNING -- at line 20: [DB] id is deprecated, use NodeId instead
2017-06-05 20:18:55 [MgmtSrvr] WARNING -- at line 25: [API] id is deprecated, use NodeId instead
2017-06-05 20:18:55 [MgmtSrvr] WARNING -- at line 28: [API] id is deprecated, use NodeId instead
2017-06-05 20:18:55 [MgmtSrvr] WARNING -- at line 8: [MGM] id is deprecated, use NodeId instead
2017-06-05 20:18:55 [MgmtSrvr] WARNING -- at line 16: [DB] id is deprecated, use NodeId instead
2017-06-05 20:18:55 [MgmtSrvr] WARNING -- at line 20: [DB] id is deprecated, use NodeId instead
2017-06-05 20:18:55 [MgmtSrvr] WARNING -- at line 25: [API] id is deprecated, use NodeId instead
2017-06-05 20:18:55 [MgmtSrvr] WARNING -- at line 28: [API] id is deprecated, use NodeId instead
查看启动是否成功
[root@centos-6 mysql-cluster]# netstat -antup | grep 1186
tcp 0 0 0.0.0.0:1186 0.0.0.0:* LISTEN 5075/ndb_mgmd
tcp 0 0 127.0.0.1:47322 127.0.0.1:1186 ESTABLISHED 5075/ndb_mgmd
tcp 0 0 127.0.0.1:1186 127.0.0.1:47322 ESTABLISHED 5075/ndb_mgmd
2、在99.36及99.37上启动数据节点，第一次启动记得带上初始化参数initial
[root@Centos-6-99 mysql]# ndbd --initial
2017-06-05 20:21:28 [ndbd] INFO -- Angel connected to '192.168.99.35:1186'
2017-06-05 20:21:28 [ndbd] INFO -- Angel allocated nodeid: 2
[root@Centos-6-99 mysql]#
[root@Centos-6-99 mysql]# ndbd --initial
2017-06-05 20:21:47 [ndbd] INFO -- Angel connected to '192.168.99.35:1186'
2017-06-05 20:21:47 [ndbd] INFO -- Angel allocated nodeid: 3
[root@Centos-6-99 mysql]#
3、在99.38及99.39上启动sql节点
[root@localhost mysql]# 170605 20:23:23 mysqld_safe Logging to '/var/log/mysqld.log'.
170605 20:23:23 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
[root@localhost mysql]# 170605 20:23:32 mysqld_safe Logging to '/var/log/mysqld.log'.
170605 20:23:32 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
4、在99.35上登录管理节点查看集群状态
[root@centos-6 mysql-cluster]# ndb_mgm
-- NDB Cluster -- Management Client --
ndb_mgm> show
Connected to Management Server at: localhost:1186
Cluster Configuration
---------------------
[ndbd(NDB)] 2 node(s)
id=2 @192.168.99.36 (mysql-5.6.36 ndb-7.4.15, Nodegroup: 0, *)
id=3 @192.168.99.37 (mysql-5.6.36 ndb-7.4.15, Nodegroup: 0)
[ndb_mgmd(MGM)] 1 node(s)
id=1 @192.168.99.35 (mysql-5.6.36 ndb-7.4.15)
[mysqld(API)] 2 node(s)
id=4 @192.168.99.38 (mysql-5.6.36 ndb-7.4.15)
id=5 @192.168.99.39 (mysql-5.6.36 ndb-7.4.15)
ndb_mgm>
5、在99.38及99.39sql节点添加mysql用户
[root@localhost mysql]# cat /root/.mysql_secret
# The random password set for the root user at Mon Jun 5 19:55:57 2017 (local time): MsIBXVtxWT4He5JF
[root@localhost mysql]# mysql -u root -pMsIBXVtxWT4He5JF
Warning: Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.6.36-ndb-7.4.15-cluster-gpl
Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> show databases;
ERROR 1820 (HY000): You must SET PASSWORD before executing this statement
mysql> set password for 'root'@'localhost'=password('123456');
Query OK, 0 rows affected (0.00 sec)
mysql> quit
Bye
[root@localhost mysql]# mysql -u root -p123456
Warning: Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 3
Server version: 5.6.36-ndb-7.4.15-cluster-gpl MySQL Cluster Community Server (GPL)
Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> grant all privileges on *.* to cluster@"%" identified by "123456";
Query OK, 0 rows affected (0.00 sec)
mysql> flush privileges;
Query OK, 0 rows affected (0.01 sec)
mysql>
6、外部链接99.38sql节点进行创建数据库及数据表测试
mysql -ucluster -p123456 -h113.106.99.38
mysql> create database db;
mysql> use db;
mysql> create table test(id int) engine=ndb;
mysql> insert into test values(10);
mysql> insert into test values(11);
mysql> select * from test;
+------+
| id |
+------+
| 11 |
| 10 |
+------+
2 rows in set (0.00 sec)
mysql>
7、外部链接登录99.39进行查看及插入数据测试
mysql -ucluster -p123456 -h113.106.99.39
mysql> show databases;
mysql> use db;
mysql> insert into test values(12);
Query OK, 1 row affected (0.01 sec)
mysql> insert into test values(13);
Query OK, 1 row affected (0.00 sec)
mysql> select * from test;
+------+
| id |
+------+
| 10 |
| 12 |
| 13 |
| 11 |
+------+
4 rows in set (0.01 sec)
mysql>
五、关闭mysql集群
关闭mysql集群顺序： 关闭管理节点服务-》 关闭管理节点时，数据结点服务自动关闭 –》 需要手动关闭SQL结点服务
1、关闭管理节点
ndb_mgm
ndb_mgm> shutdown
Node 2: Cluster shutdown initiated
Node 3: Cluster shutdown initiated
Node 2: Node shutdown completed.
Node 3: Node shutdown completed.
3 NDB Cluster node(s) have shutdown.
Disconnecting to allow management server to shutdown.
查看管理节点是否已关闭
[root@centos-6 mysql-cluster]# ps -axu | grep ndbd
Warning: bad syntax, perhaps a bogus '-'? See /usr/share/doc/procps-3.2.8/FAQ
root 5127 0.0 0.0 103328 876 pts/0 S+ 20:44 0:00 grep ndbd
[root@centos-6 mysql-cluster]#
查看数据节点是否关闭
[root@Centos-6-99 mysql]# ps -axu | grep ndbd
Warning: bad syntax, perhaps a bogus '-'? See /usr/share/doc/procps-3.2.8/FAQ
root 2243 0.0 0.0 103328 872 pts/0 S+ 21:09 0:00 grep ndbd
[root@Centos-6-99 mysql]#
2、手动关闭sql节点
[root@localhost mysql]# mysqladmin -uroot -p123456 shutdown
Warning: Using a password on the command line interface can be insecure.
170605 20:45:02 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
[1]+ Done mysqld_safe --defaults-file=/etc/my.cnf
或方法二采用暴力kill方式关闭
ps -axu |grep mysql
kill -9
kill -9
ps -axu |grep mysql
六、再次启动mysql集群进行故障切换测试
1、启动管理节点
[root@centos-6 mysql-cluster]# ndb_mgm
-- NDB Cluster -- Management Client --
ndb_mgm> show
Connected to Management Server at: localhost:1186
Cluster Configuration
---------------------
[ndbd(NDB)] 2 node(s)
id=2 (not connected, accepting connect from 192.168.99.36)
id=3 (not connected, accepting connect from 192.168.99.37)
[ndb_mgmd(MGM)] 1 node(s)
id=1 @192.168.99.35 (mysql-5.6.36 ndb-7.4.15)
[mysqld(API)] 2 node(s)
id=4 (not connected, accepting connect from 192.168.99.38)
id=5 (not connected, accepting connect from 192.168.99.39)
2、启动数据节点
[root@Centos-6-99 mysql]# ndbd
2017-06-05 20:47:25 [ndbd] INFO -- Angel connected to '192.168.99.35:1186'
2017-06-05 20:47:25 [ndbd] INFO -- Angel allocated nodeid: 2
[root@Centos-6-99 mysql]#
[root@Centos-6-99 mysql]# ndbd
2017-06-05 20:47:28 [ndbd] INFO -- Angel connected to '192.168.99.35:1186'
2017-06-05 20:47:28 [ndbd] INFO -- Angel allocated nodeid: 3
[root@Centos-6-99 mysql]#
注意：非初始化启动记得不要加initial参数
3、启动sql节点
[root@localhost mysql]# mysqld_safe --defaults-file=/etc/my.cnf &
[1] 2014
[root@localhost mysql]# 170605 20:49:49 mysqld_safe Logging to '/var/log/mysqld.log'.
170605 20:49:49 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
[root@localhost mysql]#
[root@localhost mysql]# mysqld_safe --defaults-file=/etc/my.cnf &
[1] 1740
[root@localhost mysql]# 170605 20:49:57 mysqld_safe Logging to '/var/log/mysqld.log'.
170605 20:49:57 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
[root@localhost mysql]#
4、通过外部服务器分别登录两台sql节点进行查询
[coolner@localhost ~]$ mysql -ucluster -p123456 -h113.106.99.38
mysql> use db;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> select * from test;
+------+
| id |
+------+
| 12 |
| 13 |
| 11 |
| 10 |
+------+
4 rows in set (0.00 sec)
mysql>
[coolner@localhost ~]$ mysql -ucluster -p123456 -h113.106.99.39
mysql> use db;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> select * from test;
+------+
| id |
+------+
| 12 |
| 13 |
| 11 |
| 10 |
+------+
4 rows in set (0.00 sec)
mysql>
5、模拟数据节点99.36故障，在管理节点查询集群状态并在两台sql节点进行查询
[root@Centos-6-99 mysql]# ps -aux | grep ndbd
Warning: bad syntax, perhaps a bogus '-'? See /usr/share/doc/procps-3.2.8/FAQ
root 2141 0.0 0.0 27292 2640 ? Ss 20:47 0:00 ndbd
root 2142 1.0 27.5 3155544 1082592 ? Sl 20:47 0:04 ndbd
root 2187 0.0 0.0 103328 876 pts/0 S+ 20:54 0:00 grep ndbd
[root@Centos-6-99 mysql]# kill -9 2141
[root@Centos-6-99 mysql]# kill -9 2142
在管理节点查看可以看到99.36这台数据节点断开了链接
ndb_mgm> show
Cluster Configuration
---------------------
[ndbd(NDB)] 2 node(s)
id=2 (not connected, accepting connect from 192.168.99.36)
id=3 @192.168.99.37 (mysql-5.6.36 ndb-7.4.15, Nodegroup: 0, *)
[ndb_mgmd(MGM)] 1 node(s)
id=1 @192.168.99.35 (mysql-5.6.36 ndb-7.4.15)
[mysqld(API)] 2 node(s)
id=4 @192.168.99.38 (mysql-5.6.36 ndb-7.4.15)
id=5 @192.168.99.39 (mysql-5.6.36 ndb-7.4.15)
ndb_mgm>
sql节点查询显示正常
[coolner@localhost ~]$ mysql -ucluster -p123456 -h113.106.99.38
mysql> use db;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> select * from test;
+------+
| id |
+------+
| 12 |
| 13 |
| 11 |
| 10 |
+------+
4 rows in set (0.00 sec)
[coolner@localhost ~]$ mysql -ucluster -p123456 -h113.106.99.39
mysql> use db;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> select * from test;
+------+
| id |
+------+
| 12 |
| 13 |
| 11 |
| 10 |
+------+
4 rows in set (0.00 sec)
mysql>
6、重新启动数据节点99.36，并在管理节点查看状态
[root@Centos-6-99 mysql]# ndbd
2017-06-05 20:58:05 [ndbd] INFO -- Angel connected to '192.168.99.35:1186'
2017-06-05 20:58:05 [ndbd] INFO -- Angel allocated nodeid: 2
[root@Centos-6-99 mysql]#
ndb_mgm> show
Cluster Configuration
---------------------
[ndbd(NDB)] 2 node(s)
id=2 @192.168.99.36 (mysql-5.6.36 ndb-7.4.15, starting, Nodegroup: 0) #显示在初始化状态
id=3 @192.168.99.37 (mysql-5.6.36 ndb-7.4.15, Nodegroup: 0, *)
[ndb_mgmd(MGM)] 1 node(s)
id=1 @192.168.99.35 (mysql-5.6.36 ndb-7.4.15)
[mysqld(API)] 2 node(s)
id=4 @192.168.99.38 (mysql-5.6.36 ndb-7.4.15)
id=5 @192.168.99.39 (mysql-5.6.36 ndb-7.4.15)
已完成初始化的状态
ndb_mgm> show
Cluster Configuration
---------------------
[ndbd(NDB)] 2 node(s)
id=2 @192.168.99.36 (mysql-5.6.36 ndb-7.4.15, Nodegroup: 0)
id=3 @192.168.99.37 (mysql-5.6.36 ndb-7.4.15, Nodegroup: 0, *)
[ndb_mgmd(MGM)] 1 node(s)
id=1 @192.168.99.35 (mysql-5.6.36 ndb-7.4.15)
[mysqld(API)] 2 node(s)
id=4 @192.168.99.38 (mysql-5.6.36 ndb-7.4.15)
id=5 @192.168.99.39 (mysql-5.6.36 ndb-7.4.15)
ndb_mgm>
7、模拟sql节点99.38故障，在管理节点查询集群状态并在99.39sql节点进行查询
在99.38关闭sql节点
[root@localhost mysql]# mysqladmin -uroot -p123456 shutdown
Warning: Using a password on the command line interface can be insecure.
170605 21:01:02 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
[1]+ Done mysqld_safe --defaults-file=/etc/my.cnf
[root@localhost mysql]#
在管理节点查看集群状态，显示99.38sql节点断开链接
ndb_mgm> show
Cluster Configuration
---------------------
[ndbd(NDB)] 2 node(s)
id=2 @192.168.99.36 (mysql-5.6.36 ndb-7.4.15, Nodegroup: 0)
id=3 @192.168.99.37 (mysql-5.6.36 ndb-7.4.15, Nodegroup: 0, *)
[ndb_mgmd(MGM)] 1 node(s)
id=1 @192.168.99.35 (mysql-5.6.36 ndb-7.4.15)
[mysqld(API)] 2 node(s)
id=4 (not connected, accepting connect from 192.168.99.38)
id=5 @192.168.99.39 (mysql-5.6.36 ndb-7.4.15)
ndb_mgm>
远程登录99.39节点查询
[coolner@localhost ~]$ mysql -ucluster -p123456 -h192.168.99.39
mysql> use db;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> select * from test;
+------+
| id |
+------+
| 10 |
| 12 |
| 13 |
| 11 |
+------+
4 rows in set (0.00 sec)
mysql>
来源：oschina
链接：https://my.oschina.net/u/4390260/blog/4293467