Redis Full Synchronization(전체 동기화) Intro

Redis Server 教育
Redis Server Course 		Redis 定期点検/技術支援
Redis Technical Support 		Redis エンタープライズサーバ
Redis Enterprise Server

Redis 전체 동기화 3가지 방식

이 문서는 버전 7.0.8을 기준으로 작성했습니다.

  • Disk-to-Disk: 마스터는 메모리에 있는 전체 데이터를 디스크에 파일로 저장후 전송한다.
    복제본은 데이터를 받아 디스크에 파일로 저장한 다음 메모리에 적재(로딩)한다.
    버전 6.x까지 기본 동작(Defaut로 사용 시): repl-diskless-sync no (버전 3.0부터 사용 가능)
  • Memory-to-Disk: 마스터는 메모리에 있는 전체 데이터를 소켓으로 복제본에 바로 전송한다.
    복제본은 데이터를 받아 디스크에 파일로 저장한 다음 메모리에 적재(로딩)한다.
    버전 7.0부터 기본 동작: repl-diskless-sync yes
  • Memory-to-Memory: 마스터는 메모리에 있는 전체 데이터를 소켓으로 복제본에 바로 전송한다.
    복제본은 데이터를 받아 바로 메모리에 적재(로딩)한다.
    repl-diskless-load swapdb : 6.0부터 사용 가능, 메모리를 2배로 사용하므로 사용상 주의가 필요하다.

Full Sync: Disk-to-Disk

Defaut로 사용 시 6.x까지 기본 동작

Full Sync Disk-Disk
  • rdb-del-sync-files no : Master/Replica의 rdb 파일을 지울지 여부를 정한다. 6.0에서 추가
    Yes면 삭제. 이 옵션은 AOF와 RDB(save)가 모두 비활성화된 상태에서만 작동합니다.
    규제(regulations) 또는 기타 보안 문제로 디스크에 데이터를 남기지 않을 경우 사용합니다.
Master Log

굵은 글씨로 표시된 로그는 Memory-to-Disk과 비교해서 추가되었거나 다르게 표시되는 로그이다. 

32221:M 2023-02-27 10:32:21.512204 * Replica 127.0.0.1:18505 asks for synchronization
32221:M 2023-02-27 10:32:21.512255 * Partial resynchronization not accepted: 
   Replication ID mismatch (Replica asked for '8527610751db040f683426b45f82d875d37f455b', 
   my replication IDs are 'ccac3ea676564a1bd64410095df68e801434681a' and 
   '0000000000000000000000000000000000000000')
32221:M 2023-02-27 10:32:21.512294 * Replication backlog created, 
   my new replication IDs are '6c51a9e46110e675f249ffb3b91ee82d825efa2b' and 
   '0000000000000000000000000000000000000000'
32221:M 2023-02-27 10:32:21.512321 * Starting BGSAVE for SYNC with target: disk
32221:M 2023-02-27 10:32:21.525410 * Background saving started by pid 32333
32333:C 2023-02-27 10:32:25.579347 * DB saved on disk
32333:C 2023-02-27 10:32:25.588843 * Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB
32221:M 2023-02-27 10:32:25.615219 * Background saving terminated with success
32221:M 2023-02-27 10:32:26.511645 * Synchronization with replica 127.0.0.1:18505 succeeded
32221:M 2023-02-27 10:32:31.699392 * (Received REPLCONF ACK 0)
Replica Log 
32279:S 2023-02-27 10:32:21.511306 * Before turning into a replica, using my own master parameters to 
   synthesize a cached master: I may be able to synchronize with the new master with just a partial transfer.
32279:S 2023-02-27 10:32:21.511364 * Connecting to MASTER 127.0.0.1:18504
32279:S 2023-02-27 10:32:21.511452 * MASTER <-> REPLICA sync started
32279:S 2023-02-27 10:32:21.511514 * REPLICAOF 127.0.0.1:18504 enabled (user request from 'id=4 
   addr=127.0.0.1:32994 laddr=127.0.0.1:18505 fd=11 name= age=0 idle=0 flags=N db=0 sub=0 
   psub=0 ssub=0 multi=-1 qbuf=45 qbuf-free=20429 argv-mem=23 multi-mem=0 rbs=16384 rbp=16384 
   obl=0 oll=0 omem=0 tot-mem=37679 events=r cmd=replicaof user=default redir=-1 resp=2')
32279:S 2023-02-27 10:32:21.511652 * Non blocking connect for SYNC fired the event.
32279:S 2023-02-27 10:32:21.511906 * Master replied to PING, replication can continue...
32279:S 2023-02-27 10:32:21.512100 * Trying a partial resynchronization 
   (request 8527610751db040f683426b45f82d875d37f455b:1).
32279:S 2023-02-27 10:32:21.525583 * Full resync from master: 
   6c51a9e46110e675f249ffb3b91ee82d825efa2b:0
32279:S 2023-02-27 10:32:25.615348 * MASTER <-> REPLICA sync: 
   receiving 388011570 bytes from master to disk
32279:S 2023-02-27 10:32:26.521962 * Discarding previously cached master state.
32279:S 2023-02-27 10:32:26.522016 * MASTER <-> REPLICA sync: Flushing old data
32279:S 2023-02-27 10:32:26.524074 * MASTER <-> REPLICA sync: Loading DB in memory
32279:S 2023-02-27 10:32:26.799299 * Loading RDB produced by version 7.0.8
32279:S 2023-02-27 10:32:26.799357 * RDB age 5 seconds
32279:S 2023-02-27 10:32:26.799410 * RDB memory usage when created 963.86 Mb
32279:S 2023-02-27 10:32:31.687766 * Done loading RDB, keys loaded: 9949010, keys expired: 0.
32279:S 2023-02-27 10:32:31.687885 * MASTER <-> REPLICA sync: Finished with success
32279:S 2023-02-27 10:32:31.699320 * (Send REPLCONF ACK 0)

Full Sync: Memory-to-Disk

Full Sync Memory-Disk
  • 다음 3개 파라미터는 마스터에 적용되는 옵션이다.
  • repl-diskless-sync no or yes : 3.0에 추가, 6.x까지 default no, 7.0부터 yes
  • repl-diskless-sync-delay 5 <seconds> 두번째 복제본의 요청을 기다린다.
    5초 내로 전체 동기화 요청이 오면 첫번째 복제본과 같이 데이터를 전송한다.
    파일 저장 전송(Disk-Disk)일 경우 파일을 저장하는 동안 요청이 들어오면 여러 복제본에 한번에 전송한다. 이러한 효과를 유지하려고 이 파라이터를 만들었다.
  • repl-diskless-sync-max-replicas 0 : 0은 delay 5초를 기다리고, 2로 설정하면 5초가 되지 않았어도 2개의 replica가 full sync를 요청하면 바로 전달한다. 7.0에서 추가
Master Log

굵은 글씨로 표시된 로그는 Disk-to-Disk과 비교해서 추가되었거나 다르게 표시되는 로그이다. 

15958:M 2023-02-27 10:10:58.005810 * Replica 127.0.0.1:18505 asks for synchronization
15958:M 2023-02-27 10:10:58.005863 * Partial resynchronization not accepted: 
   Replication ID mismatch (Replica asked for '05be3d5a2a4bf277b9308f259e3acdffa769b6d1', 
   my replication IDs are 'f94df0cb471c240c46b97db0b03f549ca3bed26f' and 
   '0000000000000000000000000000000000000000')
15958:M 2023-02-27 10:10:58.005915 * Replication backlog created, 
   my new replication IDs are '6d1b6c0382da7de05546d8e7c260d4a49187a74e' and 
   '0000000000000000000000000000000000000000'
15958:M 2023-02-27 10:10:58.005954 * Delay next BGSAVE for diskless SYNC
15958:M 2023-02-27 10:11:03.805858 * Starting BGSAVE for SYNC with target: replicas sockets
15958:M 2023-02-27 10:11:03.819060 * Background RDB transfer started by pid 16083
16083:C 2023-02-27 10:11:07.421776 * Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB
15958:M 2023-02-27 10:11:07.421832 # Diskless rdb transfer, done reading from pipe, 1 replicas still up.
15958:M 2023-02-27 10:11:07.527382 * Background RDB transfer terminated with success
15958:M 2023-02-27 10:11:07.527460 * Streamed RDB transfer with replica 127.0.0.1:18505 succeeded (socket). 
   Waiting for REPLCONF ACK from slave to enable streaming
15958:M 2023-02-27 10:11:07.527478 * Synchronization with replica 127.0.0.1:18505 succeeded
Replica Log 
15998:S 2023-02-27 10:10:58.005019 * Before turning into a replica, using my own master parameters to 
   synthesize a cached master: I may be able to synchronize with the new master with just a partial transfer.
15998:S 2023-02-27 10:10:58.005072 * Connecting to MASTER 127.0.0.1:18504
15998:S 2023-02-27 10:10:58.005174 * MASTER <-> REPLICA sync started
15998:S 2023-02-27 10:10:58.005226 * REPLICAOF 127.0.0.1:18504 enabled (user request from 'id=6 
   addr=127.0.0.1:46058 laddr=127.0.0.1:18505 fd=11 name= age=28 idle=1 flags=N db=0 sub=0 
   psub=0 ssub=0 multi=-1 qbuf=45 qbuf-free=20429 argv-mem=23 multi-mem=0 rbs=1024 rbp=0 
   obl=0 oll=0 omem=0 tot-mem=22319 events=r cmd=replicaof user=default redir=-1 resp=2')
15998:S 2023-02-27 10:10:58.005299 * Non blocking connect for SYNC fired the event.
15998:S 2023-02-27 10:10:58.005495 * Master replied to PING, replication can continue...
15998:S 2023-02-27 10:10:58.005678 * Trying a partial resynchronization 
   (request 05be3d5a2a4bf277b9308f259e3acdffa769b6d1:1).
15998:S 2023-02-27 10:11:03.806075 * Full resync from master: 
   6d1b6c0382da7de05546d8e7c260d4a49187a74e:14
15998:S 2023-02-27 10:11:03.829715 * MASTER <-> REPLICA sync: 
   receiving streamed RDB from master with EOF to disk
15998:S 2023-02-27 10:11:07.421924 * Discarding previously cached master state.
15998:S 2023-02-27 10:11:07.421949 * MASTER <-> REPLICA sync: Flushing old data
15998:S 2023-02-27 10:11:11.066119 * MASTER <-> REPLICA sync: Loading DB in memory
15998:S 2023-02-27 10:11:11.071316 * Loading RDB produced by version 7.0.8
15998:S 2023-02-27 10:11:11.071344 * RDB age 8 seconds
15998:S 2023-02-27 10:11:11.071363 * RDB memory usage when created 963.91 Mb
15998:S 2023-02-27 10:11:16.032246 * Done loading RDB, keys loaded: 9949010, keys expired: 0.
15998:S 2023-02-27 10:11:16.032321 * MASTER <-> REPLICA sync: Finished with success
15998:S 2023-02-27 10:11:16.032340 * (Send REPLCONF ACK 0)

Full Sync: Memory-to-Memory

Full Sync Memory-Memory
  • repl-diskless-load disabled/on-empty-db/swapdb 6.0에 추가
    복제본에 적용되는 옵션이다.
    • disabled: default 미적용
    • on-empty-db: 복제본에 기존 데이터가 없을 때 적용
    • swapdb: 복제본의 기존 데이터를 임시로 보관 -> 마스터의 데이터를 모두 받아 DB(메모리)에 저장 후, 임시로 보관한 기존 데이터를 삭제한다.
  • 적용 시 주의사항: 기존 데이터와 새로 받은 데이터를 일시적으로 둘 다 메모리에 보관하므로 일반적으로 2배 메모리가 필요하다.
Master Log

굵은 글씨로 표시된 로그는 Memory-to-Disk과 비교해서 추가되었거나 다르게 표시되는 로그이다. 

7492:M 2023-02-27 12:55:39.889362 * Replica 127.0.0.1:18505 asks for synchronization
7492:M 2023-02-27 12:55:39.889412 * Partial resynchronization not accepted: 
   Replication ID mismatch (Replica asked for 'bb412d2dd8fe6592b084a0921d92743af76bb408', 
   my replication IDs are '4727e5c85a163b80fd81b1d32a27bc9e1d70d2cd' and 
   '0000000000000000000000000000000000000000')
7492:M 2023-02-27 12:55:39.889450 * Replication backlog created, 
   my new replication IDs are '1c71f9cfa5cfd7a50ab7a76888d079dea71da179' and 
   '0000000000000000000000000000000000000000'
7492:M 2023-02-27 12:55:39.889472 * Delay next BGSAVE for diskless SYNC
7492:M 2023-02-27 12:55:44.928578 * Starting BGSAVE for SYNC with target: replicas sockets
7492:M 2023-02-27 12:55:44.942181 * Background RDB transfer started by pid 7526
7526:C 2023-02-27 12:55:44.942353 * (Send(Start) EOF:9183d8d1d2950c95e00ad572404a54c2dbfaaf0f)
7526:C 2023-02-27 12:55:49.842008 * (Send(End) EOF:9183d8d1d2950c95e00ad572404a54c2dbfaaf0f)
7526:C 2023-02-27 12:55:49.852363 * Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB
7492:M 2023-02-27 12:55:49.859206 # Diskless rdb transfer, done reading from pipe, 1 replicas still up.
7492:M 2023-02-27 12:55:49.955488 * Background RDB transfer terminated with success
7492:M 2023-02-27 12:55:49.955509 * Streamed RDB transfer with replica 127.0.0.1:18505 succeeded (socket). 
   Waiting for REPLCONF ACK from slave to enable streaming
7492:M 2023-02-27 12:55:49.955551 * Synchronization with replica 127.0.0.1:18505 succeeded
7492:M 2023-02-27 12:55:49.975804 * (Received REPLCONF ACK 14)
Replica Log 
7504:S 2023-02-27 12:55:39.888450 * Before turning into a replica, using my own master parameters 
   to synthesize a cached master: I may be able to synchronize with the new master with just a partial transfer.
7504:S 2023-02-27 12:55:39.888510 * Connecting to MASTER 127.0.0.1:18504
7504:S 2023-02-27 12:55:39.888625 * MASTER <-> REPLICA sync started
7504:S 2023-02-27 12:55:39.888727 * REPLICAOF 127.0.0.1:18504 enabled (user request from 'id=4 
   addr=127.0.0.1:36164 laddr=127.0.0.1:18505 fd=11 name= age=0 idle=0 flags=N db=0 sub=0 
   psub=0 ssub=0 multi=-1 qbuf=45 qbuf-free=20429 argv-mem=23 multi-mem=0 rbs=16384 rbp=16384 
   obl=0 oll=0 omem=0 tot-mem=37679 events=r cmd=replicaof user=default redir=-1 resp=2')
7504:S 2023-02-27 12:55:39.888884 * Non blocking connect for SYNC fired the event.
7504:S 2023-02-27 12:55:39.889106 * Master replied to PING, replication can continue...
7504:S 2023-02-27 12:55:39.889242 * Trying a partial resynchronization 
   (request bb412d2dd8fe6592b084a0921d92743af76bb408:1).
7504:S 2023-02-27 12:55:44.928807 * Full resync from master: 
   1c71f9cfa5cfd7a50ab7a76888d079dea71da179:14
7504:S 2023-02-27 12:55:44.952917 * MASTER <-> REPLICA sync: 
   receiving streamed RDB from master with EOF to parser
7504:S 2023-02-27 12:55:44.952955 * MASTER <-> REPLICA sync: Loading DB in memory
7504:S 2023-02-27 12:55:44.952984 * Loading RDB produced by version 7.0.8
7504:S 2023-02-27 12:55:44.952998 * RDB age 0 seconds
7504:S 2023-02-27 12:55:44.953012 * RDB memory usage when created 963.87 Mb
7504:S 2023-02-27 12:55:49.952858 * Done loading RDB, keys loaded: 9949010, keys expired: 0.
7504:S 2023-02-27 12:55:49.952911 * Discarding previously cached master state.
7504:S 2023-02-27 12:55:49.952929 * MASTER <-> REPLICA sync: Swapping active DB with loaded DB
7504:S 2023-02-27 12:55:49.952954 * MASTER <-> REPLICA sync: Discarding old DB in background
7504:S 2023-02-27 12:55:49.952981 * MASTER <-> REPLICA sync: Finished with success
7504:S 2023-02-27 12:55:49.953024 * (Send REPLCONF ACK 14)
<< Replication Performance Full Sync Disk-to-Disk >>

Email 返事がかかってなれば、メールでお知らせします。
Email redisgate@gmail.com Address Seoul Gangnam-gu Gangnam-daero 342 Yeoksam Bldg 5F Telephone 02.503.2235 Copyright © 2014-2024 redisGate
All right reserved