MAXWELL系列(一)-利用maxwell 解析binlog 到 redis

巧了我就是萌 提交于 2020-05-01 01:06:22

今天猪脚是maxwell,zendesk公司开源 https://github.com/zendesk/maxwell

先看架构,和他竞争的有 Debezium Connector for MySQL

 

 

废话不多说,搭建目标任务 mysql的binlog 到redis (192.168.0.1 ~~~~~~~192.168.0.3)

1:下载 https://github.com/zendesk/maxwell/releases/download/v1.22.0/maxwell-1.22.0.tar.gz

2: 安装java ,配置好java环境变量 ,解压maxwell-1.22.0.tar.gz(因为是java 写的)

3:mv  maxwell-1.22.0  /usr/local   && ln -s maxwell  maxwell-1.22.0

4:  配置原库(192.168.0.1)添加账号,(后面要用).

 

mysql> select user,host from mysql.user where user='canal';
+-------+------+
| user  | host |
+-------+------+
| canal | %    |
+-------+------+
1 row in set (0.01 sec)

mysql> show grants for canal@'%';
+---------------------------------------------------------------------------+
| Grants for canal@% |
+---------------------------------------------------------------------------+
| GRANT SELECT, REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO 'canal'@'%' |
+---------------------------------------------------------------------------+

 

5:配置伪目标mysql库:(maxwell的主机)为什么叫伪目标库???因为就是模拟的从库行为。

mysql> select user,host from mysql.user where user='maxwell';
+---------+-----------+
| user    | host      |
+---------+-----------+
| maxwell | localhost |
+---------+-----------+
1 row in set (0.00 sec)

mysql> show grants for maxwell@'localhost';
+--------------------------------------------------------------+
| Grants for maxwell@localhost                                 |
+--------------------------------------------------------------+
| GRANT ALL PRIVILEGES ON *.* TO 'maxwell'@'localhost'         |
+--------------------------------------------------------------+
2 rows in set (0.00 sec)

 

 

 6:配置maxwell:就是他了!

config.properties

 

[root@redis-binlog-reportform maxwell]# pwd;ls  -la config.properties
/usr/local/maxwell
-rw-r--r-- 1 root root 10505 May  8 23:19 config.properties

 

配置redis入口

#           *** redis ***

redis_host=192.168.0.3
redis_port=xxxx
redis_auth=XXXX
redis_database=0
redis_pub_channel=maxwell
redis_list_key=maxwell
redis_type=lpush

 

 

配置源库入口

replication_host=192.168.0.2
replication_user=canal
replication_password=xxx
replication_port=3306

 

配置伪从库入口

# mysql login info
host=localhost
user=maxwell
password=xxx
port=3306

7 启动maxwell

nohup bin/maxwell --config=config.properties &

8 查看进程及日志

MaxwellConfig - disabling bootstrapping; not available when using a separate replication host.
18:23:29,734 WARN  MaxwellMetrics - Metrics will not be exposed: metricsReportingType not configured.
18:23:30,153 INFO  Maxwell - Maxwell v1.22.0 is booting (MaxwellRedisProducer), starting at Position[BinlogPosition[mysql_bin.003563:231462217], lastHeartbeat=0]
18:23:30,334 INFO  MysqlSavedSchema - Restoring schema id 1 (last modified at Position[BinlogPosition[mysql_bin.003555:753826693], lastHeartbeat=0])
18:23:30,537 INFO  BinlogConnectorReplicator - Setting initial binlog pos to: mysql_bin.003563:231462217

9:查看redis 接收情况

[root@redis-binlog-reportform maxwell]# redis-cli -p 9001
127.0.0.1:9001> auth xxxx
OK
127.0.0.1:9001> 
127.0.0.1:9001> 
127.0.0.1:9001> 
127.0.0.1:9001> type maxwell
list
127.0.0.1:9001> 
127.0.0.1:9001> llen maxwell
(integer) 53900946
127.0.0.1:9001> llen maxwell
(integer) 53901342
127.0.0.1:9001> llen maxwell
(integer) 53901028
127.0.0.1:9001> llen maxwell
(integer) 53900745
127.0.0.1:9001>

 

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!