数据恢复篇
XtraBackup工具介绍:
现在我们给MySQL做备份的时候经常会考虑到数据量的增长,数据量较小的时候用mysqldump,随着数据量越来越大mysqldump也就不太合适了.第一不支持增量备份,第二恢复的时候也较慢.这里推荐使用Percona公司的XtraBackup.
简单介绍一下热备份工具XtraBackup,它是Percona公司推出的一款热备份工具,备份的时候不影响数据读写操作,是商业工具HotBackup的一个替代(现在应该叫MySQL Enterprise Backup这个工具不止可以备份InnoDB还可以备份MyISAM等等)
XtraBackup有两个工具:xtrabackup和innobackupex.
xtrabackup本身只能备份InnoDB和XtraDB,不能备份MyISAM.
innobackupex本身是Hot Backup脚本修改而来,同时可以备份MyISAM和InnoDB,但是备份MyISAM需要加读锁.
#安装percona-xtrabackup rpm软件包。
1 2 3 4 5 | [root@localhost data] # rpm -ivh percona-xtrabackup-2.3.2-1.el6.x86_64.rpm warning: percona-xtrabackup-2.3.2-1.el6.x86_64.rpm: Header V4 DSA /SHA1 Signature, key ID cd2efd2a: NOKEY error: Failed dependencies: libev.so.4()(64bit) is needed by percona-xtrabackup-2.3.2-1.el6.x86_64 perl(DBD::mysql) is needed by percona-xtrabackup-2.3.2-1.el6.x86_64 |
#安装依赖:
1 | yum install perl-DBD-MySQL |
#安装libev包
到http://rpmfind.net/linux/rpm2html/search.php?query=libev.so.4%28%29%2864bit%29&submit=Search+...&system=&arch= 下载操作系统对应的libev包,注意系统号与版本位数
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | [root@master ~] # wget ftp://rpmfind.net/linux/dag/redhat/el6/en/x86_64/dag/RPMS/libev-4.15-1.el6.rf.x86_64.rpm --2016-05-17 12:54:43-- ftp : //rpmfind .net /linux/dag/redhat/el6/en/x86_64/dag/RPMS/libev-4 .15-1.el6.rf.x86_64.rpm => “libev-4.15-1.el6.rf.x86_64.rpm” Resolving rpmfind.net... 195.220.108.108 Connecting to rpmfind.net|195.220.108.108|:21... connected. Logging in as anonymous ... Logged in ! ==> SYST ... done . ==> PWD ... done . ==> TYPE I ... done . ==> CWD (1) /linux/dag/redhat/el6/en/x86_64/dag/RPMS ... done . ==> SIZE libev-4.15-1.el6.rf.x86_64.rpm ... 74692 ==> PASV ... done . ==> RETR libev-4.15-1.el6.rf.x86_64.rpm ... done . Length: 74692 (73K) (unauthoritative) 100%[===============================================================================================================>] 74,692 187K /s in 0.4s 2016-05-17 12:54:46 (187 KB /s ) - “libev-4.15-1.el6.rf.x86_64.rpm” saved [74692] |
安装RHEL6对应的libev-4.15-1.el6.rf.x86_64.rpm包
1 2 3 4 | [root@master ~] # rpm -ivh libev-4.15-1.el6.rf.x86_64.rpm warning: libev-4.15-1.el6.rf.x86_64.rpm: Header V3 DSA /SHA1 Signature, key ID 6b8d79e6: NOKEY Preparing... ########################################### [100%] 1:libev ########################################### [100%] |
#安装percona-xtrabackup软件包:
1 2 3 4 | [root@localhost data] # rpm -ivh percona-xtrabackup-2.3.2-1.el6.x86_64.rpm warning: percona-xtrabackup-2.3.2-1.el6.x86_64.rpm: Header V4 DSA /SHA1 Signature, key ID cd2efd2a: NOKEY Preparing... ########################################### [100%] percona-xtrabackup ########################################### [100%] |
操作过程如图:
#备份恢复时数据库必须停止的。
1 2 | [root@localhost ~] # /etc/init.d/mysqld stop Shutting down MySQL.. [ OK ] |
#上篇fpm封装好的二进制mysql,纯净。
注意:备份时候MySQL必须是运行状态,而在做数据恢复时候服务必须是处于停滞状态。
重要参数:
--defaults-file=/etc/my.cnf:备份时,指定配置文件
--socket=/tmp/mysql.sock:备份时,连接到备份的mysql服务器所使用的 套接字文件
--user=root:连接备份mysqld所使用的用户
--password=hello:连接备份mysqld所使用的用户密码
--no-timestamp:默认情况下,备份目录会使用以当前时间为名称创建一个备份目录,加上此参数则不会做这个动作
--apply-log:创建ib_logfile文件,并恢复备份期间产生的redo日志到ib_logfile和表空间中。注意:若备份期间产生了提交的事务日志,则apply-log操作会将未提交的事务回滚
--copy-back:执行恢复数据到目标mysqld,但是目标mysqld的数据目录不 能用—datadir指定,这一点不同与meb,而是需要使用—defaults-file指 定一个配置文件,在配置文件中指定恢复的datadir。与meb不同,恢复前必 须清空目录数据目录
--use-memory=1G:备份时xtrabackup可用的内存大小,取决于操作系统可用的内存量,默认为100M,使用大的内存可加速备份的速度
--incremental:表示为增量备份,增量备份只对innodb表有效,对于non-innodb表,例如myisam则无效,因为每次增量备份都会拷贝myisam相关的所有文件,不管上次备份到这次备份有没有发生更改;而对于innodb表,增量备份则只会备份上次备份到这次备份所更改的表
--incremental-basedir=/lxw/full-back/:基于上一次的备份目录做增量 备份
--incremental-dir=/lxw/incr-backup/:用于将增量备份恢复到全备时指定增量备份的目录
--redo-only:增量备份恢复到之前的备份时,除了最后一个增量备份外,其他的增量备份最好指定该参数
--databases=’dba.t1 mysql performance_schema lxw.t1’:用于部分备份,可以用空格分开,指定备份哪些库的所有表,或者哪些库的某些表
--export:在执行apply-log操作时,将应用信息保存到文件中,在恢复部分备份时有用
--compact:采用压缩备份
--rebuild-indexes:解压缩
----记录备份信息(如备份命令、备份开始和完成时间、binlog position、备份开始和完成的lsn等
注意:备份使用的innobackupex是2.3.1 恢复可以使用innobackupex2.3.x等小版本,但是不能夸大版本,否则会有问题。
新建一个数据目录,和原数据目录同样的名字:
1 | mkdir /data/mysqldata |
#为了练习使用percona-xtrabackup全量备份了线上的mysql,scp到新机器上恢复练习。
备份完备份的时间目录下都有xtrabackup_info 文件,记录了你备份时的动作。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 | [root@localhost 2017-03-27_04-00-01] # cat xtrabackup_info uuid = c4c25074-1260-11e7-9a6a-069e3400093a name = tool_name = innobackupex tool_command = --defaults- file = /etc/my .cnf --socket= /usr/local/mysql56/data/mysqld .sock --host=localhost --user=backuser --password=... /usr/local/mysql56/mysql_data_backup/o2o/full_backup/ tool_version = 2.3.3 #工具的版本 ibbackup_version = 2.3.3 server_version = 5.6.25-log start_time = 2017-03-27 04:00:02 end_time = 2017-03-27 04:14:08 lock_time = 0 binlog_pos = filename 'mysql-bin.000285' , position '764217961' innodb_from_lsn = 0 innodb_to_lsn = 393358640191 partial = N incremental = N format = file compact = N compressed = N encrypted = N |
#首先恢复事务,也可以加--user=:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 | [root@localhost data] # innobackupex --apply-log --use-memory=10M 2017-03-27_04-00-01/ 170331 01:28:54 innobackupex: Starting the apply-log operation IMPORTANT: Please check that the apply-log run completes successfully. At the end of a successful apply-log run innobackupex prints "completed OK!" . innobackupex version 2.3.2 based on MySQL server 5.6.24 Linux (x86_64) (revision id : 306a2e0) xtrabackup: cd to /data/2017-03-27_04-00-01/ xtrabackup: This target seems to be not prepared yet. xtrabackup: xtrabackup_logfile detected: size=26689536, start_lsn=(393334934174) xtrabackup: using the following InnoDB configuration for recovery: xtrabackup: innodb_data_home_dir = ./ xtrabackup: innodb_data_file_path = ibdata1:10M:autoextend xtrabackup: innodb_log_group_home_dir = ./ xtrabackup: innodb_log_files_in_group = 1 xtrabackup: innodb_log_file_size = 26689536 xtrabackup: using the following InnoDB configuration for recovery: xtrabackup: innodb_data_home_dir = ./ xtrabackup: innodb_data_file_path = ibdata1:10M:autoextend xtrabackup: innodb_log_group_home_dir = ./ xtrabackup: innodb_log_files_in_group = 1 xtrabackup: innodb_log_file_size = 26689536 xtrabackup: Starting InnoDB instance for recovery. xtrabackup: Using 10485760 bytes for buffer pool ( set by --use-memory parameter) InnoDB: Using atomics to ref count buffer pool pages InnoDB: The InnoDB memory heap is disabled InnoDB: Mutexes and rw_locks use GCC atomic builtins InnoDB: Memory barrier is not used InnoDB: Compressed tables use zlib 1.2.3 InnoDB: Using CPU crc32 instructions InnoDB: Initializing buffer pool, size = 10.0M InnoDB: Completed initialization of buffer pool InnoDB: Highest supported file format is Barracuda. InnoDB: Log scan progressed past the checkpoint lsn 393334934174 InnoDB: Database was not shutdown normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages InnoDB: from the doublewrite buffer... InnoDB: Starting an apply batch of log records to the database... InnoDB: Progress in percent: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 InnoDB: Apply batch completed InnoDB: Starting an apply batch of log records to the database... InnoDB: Progress in percent: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 InnoDB: Apply batch completed InnoDB: Doing recovery: scanned up to log sequence number 393340176896 (22%) InnoDB: Starting an apply batch of log records to the database... InnoDB: Progress in percent: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 InnoDB: Apply batch completed InnoDB: Starting an apply batch of log records to the database... InnoDB: Progress in percent: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 InnoDB: Apply batch completed InnoDB: Doing recovery: scanned up to log sequence number 393345419776 (44%) InnoDB: Starting an apply batch of log records to the database... InnoDB: Progress in percent: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 InnoDB: Apply batch completed InnoDB: Starting an apply batch of log records to the database... InnoDB: Progress in percent: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 InnoDB: Apply batch completed InnoDB: Starting an apply batch of log records to the database... InnoDB: Progress in percent: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 InnoDB: Apply batch completed InnoDB: Doing recovery: scanned up to log sequence number 393350662656 (66%) InnoDB: Starting an apply batch of log records to the database... InnoDB: Progress in percent: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 InnoDB: Apply batch completed InnoDB: Starting an apply batch of log records to the database... InnoDB: Progress in percent: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 InnoDB: Apply batch completed InnoDB: Doing recovery: scanned up to log sequence number 393355905536 (88%) InnoDB: Starting an apply batch of log records to the database... InnoDB: Progress in percent: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 InnoDB: Apply batch completed InnoDB: Doing recovery: scanned up to log sequence number 393358640191 (99%) InnoDB: Starting an apply batch of log records to the database... InnoDB: Progress in percent: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 InnoDB: Apply batch completed InnoDB: 128 rollback segment(s) are active. InnoDB: Waiting for purge to start InnoDB: 5.6.24 started; log sequence number 393358640191 xtrabackup: Last MySQL binlog file position 764217961, file name mysql-bin.000285 xtrabackup: starting shutdown with innodb_fast_shutdown = 1 InnoDB: FTS optimize thread exiting. InnoDB: Starting shutdown ... InnoDB: Shutdown completed; log sequence number 393358641472 xtrabackup: using the following InnoDB configuration for recovery: xtrabackup: innodb_data_home_dir = ./ xtrabackup: innodb_data_file_path = ibdata1:10M:autoextend xtrabackup: innodb_log_group_home_dir = ./ xtrabackup: innodb_log_files_in_group = 2 xtrabackup: innodb_log_file_size = 50331648 InnoDB: Using atomics to ref count buffer pool pages InnoDB: The InnoDB memory heap is disabled InnoDB: Mutexes and rw_locks use GCC atomic builtins InnoDB: Memory barrier is not used InnoDB: Compressed tables use zlib 1.2.3 InnoDB: Using CPU crc32 instructions InnoDB: Initializing buffer pool, size = 10.0M InnoDB: Completed initialization of buffer pool InnoDB: Setting log file . /ib_logfile101 size to 48 MB InnoDB: Setting log file . /ib_logfile1 size to 48 MB InnoDB: Renaming log file . /ib_logfile101 to . /ib_logfile0 InnoDB: New log files created, LSN=393358641472 InnoDB: Highest supported file format is Barracuda. InnoDB: 128 rollback segment(s) are active. InnoDB: Waiting for purge to start InnoDB: 5.6.24 started; log sequence number 393358641676 xtrabackup: starting shutdown with innodb_fast_shutdown = 1 InnoDB: FTS optimize thread exiting. InnoDB: Starting shutdown ... InnoDB: Shutdown completed; log sequence number 393358644153 170331 01:29:33 completed OK! |
#恢复数据,也可以加--defaults-file=/etc/my.cnf --rsync 也可以去掉
1 2 3 4 5 6 7 8 9 10 11 12 13 | innobackupex --copy-back -- rsync 2017-03-27_04-00-01/ ........ ......... 170331 03:03:04 [01] ... done 170331 03:03:04 [01] Copying . /ecos_notice/tbl_notice_file .ibd to /data/mysqldata/ecos_notice/tbl_notice_file .ibd 170331 03:03:04 [01] ... done 170331 03:03:04 [01] Copying . /stest/tbl_test_user_info .ibd to /data/mysqldata/stest/tbl_test_user_info .ibd 170331 03:03:04 [01] ... done 170331 03:03:04 [01] Copying . /stest/tbl_test_user_info .frm to /data/mysqldata/stest/tbl_test_user_info .frm 170331 03:03:04 [01] ... done 170331 03:03:04 [01] Copying . /stest/db .opt to /data/mysqldata/stest/db .opt 170331 03:03:04 [01] ... done 170331 03:03:04 completed OK! |
授权恢复数据的目录为mysql用户和mysql组:
1 2 3 4 5 6 7 8 | [root@localhost data] # ll total 5624 drwx------ 14 root root 4096 Mar 31 02:33 2017-03-27_04-00-01 -rw-r--r-- 1 root root 74692 Mar 31 01:10 libev-4.15-1.el6.rf.x86_64.rpm drwxr-xr-x 13 mysql mysql 4096 Mar 31 02:42 mysql drwxr-xr-x 14 root root 4096 Mar 31 03:03 mysqldata drwxr-xr-x 2 mysql mysql 4096 Mar 31 02:47 mysqldata20170330 -rw-r--r-- 1 root root 5664452 Mar 31 01:09 percona-xtrabackup-2.3.2-1.el6.x86_64.rpm |
#授权mysql用户和mysql组
1 | [root@localhost data] # chown mysql:mysql mysqldata -R #授权mysql用户和mysql组 |
#mysql启动跳过授权用户表
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 | [root@localhost data] # cat /etc/my.cnf [mysqld] basedir= /data/mysql datadir= /data/mysqldata port=3306 socket= /tmp/mysql .sock user=mysql innodb_file_per_table=on skip_name_resolve=on symbolic-links=0 init-connect= 'SET NAMES utf8' character- set -server = utf8 max_connections = 5000 max_connect_errors = 100000 skip-grant-tables #添加启动mysql跳过用户授权表 [mysqld_safe] log-error= /data/mysql/mysqld .log pid- file = /data/mysql/mysqld .pid、 |
#启动mysql服务:
1 2 3 4 5 6 7 | [root@localhost data] # /etc/init.d/mysqld start Starting MySQL... [ OK ] [root@localhost data] # cd mysqldata [root@localhost mysqldata] # ls 2017-03-31_02-33-37 ecos ecoslog ibdata1 localhost.localdomain.pid stest xtrabackup_info appversion ecosantispam ecos_notice ib_logfile0 mysql test auto.cnf ecos_delivery ecoswallet ib_logfile1 performance_schema xtrabackup_binlog_pos_innodb |
#如果忘记了登录mysql的用户密码可以跳过用户认证表,当然也可以加在my.cnf文件中:
1 | /etc/rc .d /init .d /mysqld start --skip-grant-tables; #跳过权限表启动mysql不用密码; |