MySQL主從復(fù)制(Master-Slave)與讀寫分離(MySQL-Proxy)實踐
Mysql作為目前世界上使用最廣泛的免費數(shù)據(jù)庫,相信所有從事系統(tǒng)運維的工程師都一定接觸過。但在實際的生產(chǎn)環(huán)境中,由單臺Mysql作為獨立的數(shù)據(jù)庫是完全不能滿足實際需求的,無論是在安全性,高可用性以及高并發(fā)等各個方面。
因此,一般來說都是通過 主從復(fù)制(Master-Slave)的方式來同步數(shù)據(jù),再通過讀寫分離(MySQL-Proxy)來提升數(shù)據(jù)庫的并發(fā)負(fù)載能力 這樣的方案來進行部署與實施的。
如下圖所示:
下面是我在實際工作過程中所整理的筆記,在此分享出來,以供大家參考。
一、MySQL的安裝與配置
具體的安裝過程,建議參考我的這一篇文章:http://heylinux.com/archives/993.html 值得一提的是,我的安裝過程都是源碼包編譯安裝的,并且所有的配置與數(shù)據(jù)等都統(tǒng)一規(guī)劃到了/opt/mysql目錄中,因此在一臺服務(wù)器上安裝完成以后,可以將整個mysql目錄打包,然后傳到其它服務(wù)器上解包,便可立即使用。二、MySQL主從復(fù)制
場景描述: 主數(shù)據(jù)庫服務(wù)器:192.168.10.130,MySQL已經(jīng)安裝,并且無應(yīng)用數(shù)據(jù)。 從數(shù)據(jù)庫服務(wù)器:192.168.10.131,MySQL已經(jīng)安裝,并且無應(yīng)用數(shù)據(jù)。2.1 主服務(wù)器上進行的操作
啟動mysql服務(wù)
/opt/mysql/init.d/mysql start
通過命令行登錄管理MySQL服務(wù)器
/opt/mysql/bin/mysql -uroot -p'new-password'
授權(quán)給從數(shù)據(jù)庫服務(wù)器192.168.10.131
mysql> GRANT REPLICATION SLAVE ON *.* to 'rep1'@'192.168.10.131' identified by ‘password’;
查詢主數(shù)據(jù)庫狀態(tài)
Mysql> show master status;
File | Position | Binlog_Do_DB | Binlog_Ignore_DB |
---|---|---|---|
mysql-bin.000005 | right-aligned | $1600 | |
col 2 is | 261 |
記錄下 FILE 及 Position 的值,在后面進行從服務(wù)器操作的時候需要用到。
2.2 配置從服務(wù)器
修改從服務(wù)器的配置文件/opt/mysql/etc/my.cnf 將 server-id = 1修改為 server-id = 10,并確保這個ID沒有被別的MySQL服務(wù)所使用。啟動mysql服務(wù)
/opt/mysql/init.d/mysql start
通過命令行登錄管理MySQL服務(wù)器
/opt/mysql/bin/mysql -uroot -p'new-password'
執(zhí)行同步SQL語句
mysql> change master to master_host=’192.168.10.130’, master_user=’rep1’, master_password=’password’, master_log_file=’mysql-bin.000005’, master_log_pos=261;
正確執(zhí)行后啟動Slave同步進程
mysql> start slave;
主從同步檢查
mysql> show slave status\G
==============================================
**************** 1. row *******************
Slave_IO_State: Master_Host: 192.168.10.130
Master_User: rep1
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: mysql-bin.000005
Read_Master_Log_Pos: 415
Relay_Log_File: localhost-relay-bin.000008
Relay_Log_Pos: 561
Relay_Master_Log_File: mysql-bin.000005
Slave_IO_Running: YES
Slave_SQL_Running: YES
Replicate_Do_DB: ……………省略若干……………
Master_Server_Id: 1
1 row in set (0.01 sec)
==============================================
其中Slave_IO_Running 與 Slave_SQL_Running 的值都必須為YES,才表明狀態(tài)正常。
如果主服務(wù)器已經(jīng)存在應(yīng)用數(shù)據(jù),則在進行主從復(fù)制時,需要做以下處理:
(1)主數(shù)據(jù)庫進行鎖表操作,不讓數(shù)據(jù)再進行寫入動作
mysql> FLUSH TABLES WITH READ LOCK;
(2)查看主數(shù)據(jù)庫狀態(tài)
mysql> show master status;
(3)記錄下 FILE 及 Position 的值。
將主服務(wù)器的數(shù)據(jù)文件(整個/opt/mysql/data目錄)復(fù)制到從服務(wù)器,建議通過tar歸檔壓縮后再傳到從服務(wù)器解壓。
(4)取消主數(shù)據(jù)庫鎖定
mysql> UNLOCK TABLES;
2.3 驗證主從復(fù)制效果
主服務(wù)器上的操作
在主服務(wù)器上創(chuàng)建數(shù)據(jù)庫first_db
mysql> create database first_db;
Query Ok, 1 row affected (0.01 sec)
在主服務(wù)器上創(chuàng)建表first_tb
mysql> create table first_tb(id int(3),name char(10));
Query Ok, 1 row affected (0.00 sec)
在主服務(wù)器上的表first_tb中插入記錄
mysql> insert into first_tb values (001,’myself’);
Query Ok, 1 row affected (0.00 sec)
在從服務(wù)器上查看
mysql> show databases;
=============================
+--------------------+ | Database |
+--------------------+ | information_schema | | first_db | | mysql | | performance_schema | | test |
+--------------------+ 5 rows in set (0.01 sec)
=============================
數(shù)據(jù)庫first_db已經(jīng)自動生成
mysql> use first_db
Database chagedmysql> show tables;
=============================
+--------------------+ | Tables_in_first_db |
+--------------------+ | first_tb |
+--------------------+
1 row in set (0.02 sec)
=============================
數(shù)據(jù)庫表first_tb也已經(jīng)自動創(chuàng)建
mysql> select * from first_tb;
=============================
+------+------+ | id | name |
+------+------+ | 1 | myself |
+------+------+ 1 rows in set (0.00 sec)
=============================
記錄也已經(jīng)存在由此,整個MySQL主從復(fù)制的過程就完成了,接下來,我們進行MySQL讀寫分離的安裝與配置。
三、MySQL讀寫分離
場景描述:
數(shù)據(jù)庫Master主服務(wù)器:192.168.10.130
數(shù)據(jù)庫Slave從服務(wù)器:192.168.10.131
MySQL-Proxy調(diào)度服務(wù)器:192.168.10.132
給調(diào)度服務(wù)器分配賬戶密碼:分別在主從庫上
mysql>
grant all on *.* to shmily@'192.168.1.%' identified by 'shmily';
以下操作,均是在192.168.10.132即MySQL-Proxy調(diào)度服務(wù)器 上進行的。
1 安裝java環(huán)境 我用的是 jdk-7u79-linux-x64.tar.gz 安裝路徑為/usr/local/java
2 配置環(huán)境
vi /etc/profile
在最下面添加
export JAVA_HOME=/usr/local/java/jdk1.7.0_79
export JRE_HOME=/usr/local/java/jdk1.7.0_79/jre
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar:$JRE_HOME/lib:$CLASSPATH
export PATH=$JAVA_HOME/bin:$PATH
保存后執(zhí)行
source /etc/profile
3 安裝amoeba 這里我下載的amoeba-mysql-3.0.4-BETA-distribution.zip unzip amoeba-mysql-3.0.4-BETA-distribution.zip
安裝目錄是
/usr/local/amoeba/amoeba-mysql-3.0.4-BETA
配置兩個文件:conf/dbServers.xml和conf/amoeba.xml
amoeba.xml如下
<?xml version="1.0" encoding="gbk"?>
<!DOCTYPE amoeba:configuration SYSTEM "amoeba.dtd">
<amoeba:configuration xmlns:amoeba="http://amoeba.meidusa.com/">
<proxy>
<!-- service class must implements com.meidusa.amoeba.service.Service -->
<service name="Amoeba for Mysql" class="com.meidusa.amoeba.mysql.server.MySQLService">
<!-- port -->
<property name="port">8066</property>
<!-- bind ipAddress -->
<property name="ipAddress">172.16.6.146</property>
<property name="connectionFactory">
<bean class="com.meidusa.amoeba.mysql.net.MysqlClientConnectionFactory">
<property name="sendBufferSize">128</property>
@
"amoeba.xml" 88L, 3019C
<?xml version="1.0" encoding="gbk"?>
<!DOCTYPE amoeba:configuration SYSTEM "amoeba.dtd">
<amoeba:configuration xmlns:amoeba="http://amoeba.meidusa.com/">
<proxy>
<!-- service class must implements com.meidusa.amoeba.service.Service -->
<service name="Amoeba for Mysql" class="com.meidusa.amoeba.mysql.server.MySQLService">
<!-- port -->
<property name="port">8066</property>
<!-- bind ipAddress -->
<property name="ipAddress">172.16.6.146</property>
<property name="connectionFactory">
<bean class="com.meidusa.amoeba.mysql.net.MysqlClientConnectionFactory">
<property name="sendBufferSize">128</property>
<property name="receiveBufferSize">64</property>
</bean>
</property>
<property name="authenticateProvider">
<bean class="com.meidusa.amoeba.mysql.server.MysqlClientAuthenticator">
<property name="user">root</property>
<property name="password">111111</property>
<property name="filter">
<bean class="com.meidusa.toolkit.net.authenticate.server.IPAccessController">
<property name="ipFile">${amoeba.home}/conf/access_list.conf</property>
</bean>
</property>
</bean>
</property>
</service>
<runtime class="com.meidusa.amoeba.mysql.context.MysqlRuntimeContext">
<!-- proxy server client process thread size -->
<property name="executeThreadSize">128</property>
<!-- per connection cache prepared statement size -->
<property name="statementCacheSize">500</property>
<!-- query timeout( default: 60 second , TimeUnit:second) -->
<property name="queryTimeout">60</property>
</runtime>
</proxy>
<!--
Each ConnectionManager will start as thread
manager responsible for the Connection IO read , Death Detection
-->
<connectionManagerList>
<connectionManager name="defaultManager" class="com.meidusa.toolkit.net.MultiConnectionManagerWrapper">
<property name="subManagerClassName">com.meidusa.toolkit.net.AuthingableConnectionManager</property>
</connectionManager>
</connectionManagerList>
<!-- default using file loader -->
<dbServerLoader class="com.meidusa.amoeba.context.DBServerConfigFileLoader">
<property name="configFile">${amoeba.home}/conf/dbServers.xml</property>
</dbServerLoader>
<queryRouter class="com.meidusa.amoeba.mysql.parser.MysqlQueryRouter">
<property name="ruleLoader">
<bean class="com.meidusa.amoeba.route.TableRuleFileLoader">
<property name="ruleFile">${amoeba.home}/conf/rule.xml</property>
<property name="functionFile">${amoeba.home}/conf/ruleFunctionMap.xml</property>
</bean>
</property>
<property name="sqlFunctionFile">${amoeba.home}/conf/functionMap.xml</property>
<property name="LRUMapSize">1500</property>
<property name="defaultPool">master1</property>
<property name="writePool">master1</property> (寫庫)
<property name="readPool">slave1</property>(讀庫)
<property name="needParse">true</property>
</queryRouter>
</amoeba:configuration>
下面是dbServers.xml的配置
<?xml version="1.0" encoding="gbk"?>
<!DOCTYPE amoeba:dbServers SYSTEM "dbserver.dtd">
<amoeba:dbServers xmlns:amoeba="http://amoeba.meidusa.com/">
<!--
Each dbServer needs to be configured into a Pool,
If you need to configure multiple dbServer with load balancing that can be simplified by the following configuration:
add attribute with name virtual = "true" in dbServer, but the configuration does not allow the element with name factoryConfig
such as 'multiPool' dbServer
-->
<dbServer name="abstractServer" abstractive="true">
<factoryConfig class="com.meidusa.amoeba.mysql.net.MysqlServerConnectionFactory">
<property name="connectionManager">${defaultManager}</property>
<property name="sendBufferSize">64</property>
<property name="receiveBufferSize">128</property>
<!-- mysql port -->
<property name="port">3306</property>(主從庫允許的端口)
<!-- mysql schema -->
<property name="schema">db</property> (允許讀寫分離的數(shù)據(jù)庫)
<!-- mysql user -->
<property name="user">shmily</property> (主從庫分配給amoeba的賬戶)
<property name="password">shmily</property> (主從庫分配給amoeba的密碼)
</factoryConfig>
<poolConfig class="com.meidusa.toolkit.common.poolable.PoolableObjectPool">
<property name="maxActive">500</property>
<property name="maxIdle">500</property>
<property name="minIdle">1</property>
<property name="minEvictableIdleTimeMillis">600000</property>
<property name="timeBetweenEvictionRunsMillis">600000</property>
<property name="testOnBorrow">true</property>
<property name="testOnReturn">true</property>
<property name="testWhileIdle">true</property>
</poolConfig>
</dbServer>
(寫庫的配置)
<dbServer name="master1" parent="abstractServer">
<factoryConfig>
<!-- mysql ip -->
<property name="ipAddress">172.16.6.143</property>
</factoryConfig>
</dbServer>
(讀庫的配置 ,可以配置多個)
<dbServer name="slave1" parent="abstractServer">
<factoryConfig>
<!-- mysql ip -->
<property name="ipAddress">172.16.6.149</property>
</factoryConfig>
</dbServer>
(當(dāng)為多個讀庫時,同時還得設(shè)置這里)
<dbServer name="virtualbox" virtual="true">
<poolConfig class="com.meidusa.amoeba.server.MultipleServerPool">
<!-- Load balancing strategy: 1=ROUNDROBIN , 2=WEIGHTBASED , 3=HA-->
<property name="loadbalance">1</property>
<!-- Separated by commas,such as: server1,server2,server1 -->
<property name="poolNames">slave1</property>
</poolConfig>
</dbServer>
</amoeba:dbServers>
以上則是配置文件
啟動amoeba操作
切換到amoeba目錄
cd /usr/local/moeba/moeba-mysql-3.0.4-BETA/bin
啟動
./launcher
停止
./shutdown
從庫遠(yuǎn)程登錄調(diào)度服務(wù)器
mysql -uroot -p111111 -h172.16.6.146 -P8066
ps 查看進程
kill -9 pid 強制殺死進程
rpm ql 軟件 查看軟件路徑
rpm -ivh rpm包 安裝rpm包
rz windows復(fù)制文件
rpm -qa|grep XX 查看相關(guān)安裝的軟件
yum list| grep XX 列表
用戶組列表文件:/etc/group
查看系統(tǒng)中有哪些用戶:cut -d : -f 1 /etc/passwd查看可以登錄系統(tǒng)的用戶:cat /etc/passwd | grep -v /sbin/nologin | cut -d : -f 1查看用戶操作:w命令(需要root權(quán)限)查看某一用戶:w 用戶名查看登錄用戶:who查看用戶登錄歷史記錄:last