커뮤니티 > 질문과 답변(Cluster) > 두대의 서버에서 클러스터링운영

공지사항

뉴스

질문과 답변

최신글

질문과 답변 (Cluster)

> 커뮤니티 존 > 질문과 답변 (Cluster)

두대의 서버에서 클러스터링운영

글쓴이 : gostjayh 날짜 : 08-06-26 17:43 조회수 : 9707

현재 두대의 서버에 각각 mgmd+ndbd+mysqld 노드를 띄워놓고 운영중입니다.

추천하는 방법은 아니지만 이론상으로는 문제없이 돌아가야하는게 정상인것 같은데...

ndb노드가 master로 되어있는쪽 시스템을 리부팅을 하면(reboot명령어를 이용한 강제종료) 클러스터가 깨져버리고 맙니다.

Cluster Configuration
---------------------
[ndbd(NDB)] 2 node(s)
id=3 (not connected, accepting connect from 192.168.1.144)
id=4 (not connected, accepting connect from 192.168.1.145)

[ndb_mgmd(MGM)] 2 node(s)
id=1 (not connected, accepting connect from 192.168.1.144)
id=2 @192.168.1.145 (Version: 5.0.45)

[mysqld(API)] 2 node(s)
id=5 (not connected, accepting connect from 192.168.1.144)
id=6 (not connected, accepting connect from 192.168.1.145)

이렇게요...

마스터로 되어있지 않은 시스템을 리부팅하면 괜찮은데요...해결방법이 있을까요??

로그는 다음과 같습니다..

2008-06-26 14:20:54 [MgmSrvr] ALERT    -- Node 3: Node 5 Disconnected
2008-06-26 14:20:54 [MgmSrvr] INFO     -- Node 3: Communication to Node 5 closed
2008-06-26 14:20:55 [MgmSrvr] ALERT    -- Node 4: Node 5 Disconnected
2008-06-26 14:20:55 [MgmSrvr] INFO     -- Node 4: Communication to Node 5 closed
2008-06-26 14:20:58 [MgmSrvr] INFO     -- Node 4: Communication to Node 5 opened
2008-06-26 14:20:58 [MgmSrvr] INFO     -- Node 3: Communication to Node 5 opened
2008-06-26 14:21:06 [MgmSrvr] WARNING -- Node 4: Node 3 missed heartbeat 2
2008-06-26 14:21:06 [MgmSrvr] WARNING -- Node 4: Node 1 missed heartbeat 2
2008-06-26 14:21:07 [MgmSrvr] WARNING -- Node 4: Node 3 missed heartbeat 3
2008-06-26 14:21:07 [MgmSrvr] WARNING -- Node 4: Node 1 missed heartbeat 3
2008-06-26 14:21:08 [MgmSrvr] ALERT    -- Node 2: Node 3 Disconnected
2008-06-26 14:21:08 [MgmSrvr] ALERT    -- Node 2: Node 3 Disconnected
2008-06-26 14:21:10 [MgmSrvr] ALERT    -- Node 2: Node 4 Disconnected
2008-06-26 14:21:58 [MgmSrvr] ALERT    -- Node 4: Forced node shutdown completed. Caused by error 2305: 'Node lost connection to other nodes and can n
ot form a unpartitioned cluster, please investigate if there are error(s) on other node(s)(Arbitration error). Temporary error, restart node'.

정진삼

올려 놓으신 상태를 보니
Cluster Configuration
---------------------
[ndbd(NDB)] 2 node(s)
id=3 (not connected, accepting connect from 192.168.1.144)
id=4 (not connected, accepting connect from 192.168.1.145)
[ndb_mgmd(MGM)] 2 node(s)
id=1 (not connected, accepting connect from 192.168.1.144)
id=2 @192.168.1.145 (Version: 5.0.45)
[mysqld(API)] 2 node(s)
id=5 (not connected, accepting connect from 192.168.1.144)
id=6 (not connected, accepting connect from 192.168.1.145)

2번 서버에 관리노드 빼고 ndb 노드, mysql 노드 모두 다 죽어 있는 상태네요...
우선 모든 노드들을 띄워주시고요....

띄워주시기 전에... 네트워크도 체크해 주시면 될 것 같네요

gostjayh

네트워크 상태는 항상 정상입니다.

위에 적어놨지만 항상 master쪽 노드가 있는 시스템을 리부팅하면 이렇게 됩니다.
master가 아닌쪽의 노드가 있는 시스템을 리부팅하면 정상이란 말이죠...

Cluster Configuration
---------------------
[ndbd(NDB)] 2 node(s)
id=3 @192.168.1.144 (Version: 5.0.45, Nodegroup: 0, Master)
id=4 (not connected, accepting connect from 192.168.1.145)

[ndb_mgmd(MGM)] 2 node(s)
id=1 @192.168.1.144 (Version: 5.0.45)
id=2 (not connected, accepting connect from 192.168.1.145)

[mysqld(API)] 2 node(s)
id=5 @192.168.1.144 (Version: 5.0.45)
id=6 (not connected, accepting connect from 192.168.1.145)

이렇게요...

특별히 설정해줘야하는 것이 있는지 궁금할 뿐입니다.

아래는 설정파일 정보입니다. 도와주세요...ㅜㅜ

=====>> config.ini

[NDBD DEFAULT] # Options affecting ndbd processes on all data nodes
NoOfReplicas=2 # Number of replicas
DataMemory=500M
IndexMemory=100M
MaxNoOfConcurrentOperations=100000
MaxNoOfLocalOperations=100000
MaxNoOfAttributes=100000
MaxNoOfOrderedIndexes=5000
MaxNoOfUniqueHashIndexes=5000
MaxNoOfTables=1000

[TCP DEFAULT] # TCP/IP options:
portnumber=2202

[NDB_MGMD]
Id=1
HostName=192.168.1.144 # Hostname or IP address of MGM node
DataDir=/var/lib/mysql-cluster # Directory for MGM node logfiles

[NDB_MGMD]
Id=2
HostName=192.168.1.145 # Hostname or IP address of MGM node
DataDir=/var/lib/mysql-cluster # Directory for MGM node logfiles

[NDBD]
Id=3 # Node ID
HostName=192.168.1.144 # Hostname or IP address
DataDir=/usr/mysql/data # Directory for this data node's datafiles

[NDBD]
Id=4 # Node ID
HostName=192.168.1.145 # Hostname or IP address
DataDir=/usr/mysql/data # Directory for this data node's datafiles

[MYSQLD]
Id=5 # Node ID
HostName=192.168.1.144 # Hostname or IP address

[MYSQLD]
Id=6 # Node ID
HostName=192.168.1.145 # Hostname or IP address

=====>> my.cnf

[mysql_cluster] # Options for ndbd process
ndb-connectstring=192.168.1.145 # location of MGM node

[ndb_mgmd]
config-file=/var/lib/mysql-cluster/config.ini

# The following options will be passed to all MySQL clients
[client]
#password = your_password
#port = 3306
socket = /var/lib/mysql/mysql.sock

# Here follows entries for some specific programs

# The MySQL server
[mysqld]
#port = 3306
ndbcluster
default-character-set = euckr
language = /usr/mysql/share/mysql/korean/
skip-character-set-client-handshake
socket = /var/lib/mysql/mysql.sock
skip-locking
key_buffer = 256M
max_allowed_packet = 1M
table_cache = 256
sort_buffer_size = 1M
read_buffer_size = 1M
read_rnd_buffer_size = 4M
myisam_sort_buffer_size = 64M
thread_cache_size = 8
query_cache_size= 16M
# Try number of CPU's*2 for thread_concurrency
thread_concurrency = 8

# Don't listen on a TCP/IP port at all. This can be a security enhancement,
# if all processes that need to connect to mysqld run on the same host.
# All interaction with mysqld must be made via Unix sockets or named pipes.
# Note that using this option without enabling named pipes on Windows
# (via the "enable-named-pipe" option) will render mysqld useless!
#
#skip-networking

# Replication Master Server (default)
# binary logging is required for replication
log-bin=mysql-bin

# required unique id between 1 and 2^32 - 1
# defaults to 1 if master-host is not set
# but will not function as a master if omitted
server-id = 1

# Replication Slave (comment out master section to use this)
#
# To configure this host as a replication slave, you can choose between
# two methods :
#
# 1) Use the CHANGE MASTER TO command (fully described in our manual) -
# the syntax is:
#
# CHANGE MASTER TO MASTER_HOST=<host>, MASTER_PORT=<port>,
# MASTER_USER=<user>, MASTER_PASSWORD=<password> ;
#
# where you replace <host>, <user>, <password> by quoted strings and
# <port> by the master's port number (3306 by default).
#
# Example:
#
# CHANGE MASTER TO MASTER_HOST='125.564.12.1', MASTER_PORT=3306,
# MASTER_USER='joe', MASTER_PASSWORD='secret';
#
# OR
#
# 2) Set the variables below. However, in case you choose this method, then
# start replication for the first time (even unsuccessfully, for example
# if you mistyped the password in master-password and the slave fails to
# connect), the slave will create a master.info file, and any later
# change in this file to the variables' values below will be ignored and
# overridden by the content of the master.info file, unless you shutdown
# the slave server, delete master.info and restart the slaver server.
# For that reason, you may want to leave the lines below untouched
# (commented) and instead use CHANGE MASTER TO (see above)
#
# required unique id between 2 and 2^32 - 1
# (and different from the master)
# defaults to 2 if master-host is set
# but will not function as a slave if omitted
#server-id = 2
#
# The replication master for this slave - required
#master-host = <hostname>
#
# The username the slave will use for authentication when connecting
# to the master - required
#master-user = <username>
#
# The password the slave will authenticate with when connecting to
# the master - required
#master-password = <password>
#
# The port the master is listening on.
# optional - defaults to 3306
#master-port = <port>
#
# binary logging - not required for slaves, but recommended
#log-bin=mysql-bin

# Point the following paths to different dedicated disks
#tmpdir = /tmp/
#log-update = /path-to-dedicated-directory/hostname

# Uncomment the following if you are using BDB tables
#bdb_cache_size = 64M
#bdb_max_lock = 100000

# Uncomment the following if you are using InnoDB tables
#innodb_data_home_dir = /usr/local/mysql/data/
#innodb_data_file_path = ibdata1:10M:autoextend
#innodb_log_group_home_dir = /usr/local/mysql/data/
#innodb_log_arch_dir = /usr/local/mysql/data/
# You can set .._buffer_pool_size up to 50 - 80 %
# of RAM but beware of setting memory usage too high
#innodb_buffer_pool_size = 256M
#innodb_additional_mem_pool_size = 20M
# Set .._log_file_size to 25 % of buffer pool size
#innodb_log_file_size = 64M
#innodb_log_buffer_size = 8M
#innodb_flush_log_at_trx_commit = 1
#innodb_lock_wait_timeout = 50

[mysqldump]
quick
max_allowed_packet = 16M

[mysql]
no-auto-rehash
# Remove the next comment character if you are not familiar with SQL
#safe-updates

[isamchk]
key_buffer = 128M
sort_buffer_size = 128M
read_buffer = 2M
write_buffer = 2M

[myisamchk]
key_buffer = 128M
sort_buffer_size = 128M
read_buffer = 2M
write_buffer = 2M

[mysqlhotcopy]
interactive-timeout

네트워크 상태는 항상 정상입니다.

위에 적어놨지만 항상 master쪽 노드가 있는 시스템을 리부팅하면 이렇게 됩니다.
master가 아닌쪽의 노드가 있는 시스템을 리부팅하면 정상이란 말이죠...

Cluster Configuration
---------------------
[ndbd(NDB)]     2 node(s)
id=3    @192.168.1.144  (Version: 5.0.45, Nodegroup: 0, Master)
id=4    (not connected, accepting connect from 192.168.1.145)

[ndb_mgmd(MGM)] 2 node(s)
id=1    @192.168.1.144  (Version: 5.0.45)
id=2    (not connected, accepting connect from 192.168.1.145)

[mysqld(API)]   2 node(s)
id=5    @192.168.1.144  (Version: 5.0.45)
id=6    (not connected, accepting connect from 192.168.1.145)

이렇게요...

특별히 설정해줘야하는 것이 있는지 궁금할 뿐입니다.

아래는 설정파일 정보입니다. 도와주세요...ㅜㅜ

=====>> config.ini

[NDBD DEFAULT]                      # Options affecting ndbd processes on all data nodes
NoOfReplicas=2                      # Number of replicas
DataMemory=500M
IndexMemory=100M
MaxNoOfConcurrentOperations=100000
MaxNoOfLocalOperations=100000
MaxNoOfAttributes=100000
MaxNoOfOrderedIndexes=5000
MaxNoOfUniqueHashIndexes=5000
MaxNoOfTables=1000

[TCP DEFAULT]                       # TCP/IP options:
portnumber=2202

[NDB_MGMD]
Id=1
HostName=192.168.1.144              # Hostname or IP address of MGM node
DataDir=/var/lib/mysql-cluster      # Directory for MGM node logfiles

[NDB_MGMD]
Id=2
HostName=192.168.1.145              # Hostname or IP address of MGM node
DataDir=/var/lib/mysql-cluster      # Directory for MGM node logfiles

[NDBD]
Id=3                                # Node ID
HostName=192.168.1.144              # Hostname or IP address
DataDir=/usr/mysql/data             # Directory for this data node's datafiles

[NDBD]
Id=4                                # Node ID
HostName=192.168.1.145              # Hostname or IP address
DataDir=/usr/mysql/data             # Directory for this data node's datafiles

[MYSQLD]
Id=5                                # Node ID
HostName=192.168.1.144              # Hostname or IP address

[MYSQLD]
Id=6                                # Node ID
HostName=192.168.1.145              # Hostname or IP address

=====>> my.cnf

[mysql_cluster]                                 # Options for ndbd process
ndb-connectstring=192.168.1.145                 # location of MGM node

[ndb_mgmd]
config-file=/var/lib/mysql-cluster/config.ini

# The following options will be passed to all MySQL clients
[client]
#password       = your_password
#port            = 3306
socket          = /var/lib/mysql/mysql.sock

# Here follows entries for some specific programs

# The MySQL server
[mysqld]
#port            = 3306
ndbcluster
default-character-set = euckr
language = /usr/mysql/share/mysql/korean/
skip-character-set-client-handshake
socket          = /var/lib/mysql/mysql.sock
skip-locking
key_buffer = 256M
max_allowed_packet = 1M
table_cache = 256
sort_buffer_size = 1M
read_buffer_size = 1M
read_rnd_buffer_size = 4M
myisam_sort_buffer_size = 64M
thread_cache_size = 8
query_cache_size= 16M
# Try number of CPU's*2 for thread_concurrency
thread_concurrency = 8

# Don't listen on a TCP/IP port at all. This can be a security enhancement,
# if all processes that need to connect to mysqld run on the same host.
# All interaction with mysqld must be made via Unix sockets or named pipes.
# Note that using this option without enabling named pipes on Windows
# (via the "enable-named-pipe" option) will render mysqld useless!
# 
#skip-networking

# Replication Master Server (default)
# binary logging is required for replication
log-bin=mysql-bin

# required unique id between 1 and 2^32 - 1
# defaults to 1 if master-host is not set
# but will not function as a master if omitted
server-id       = 1

# Replication Slave (comment out master section to use this)
#
# To configure this host as a replication slave, you can choose between
# two methods :
#
# 1) Use the CHANGE MASTER TO command (fully described in our manual) -
#    the syntax is:
#
#    CHANGE MASTER TO MASTER_HOST=<host>, MASTER_PORT=<port>,
#    MASTER_USER=<user>, MASTER_PASSWORD=<password> ;
#
#    where you replace <host>, <user>, <password> by quoted strings and
#    <port> by the master's port number (3306 by default).
#
#    Example:
#
#    CHANGE MASTER TO MASTER_HOST='125.564.12.1', MASTER_PORT=3306,
#    MASTER_USER='joe', MASTER_PASSWORD='secret';
#
# OR
#
# 2) Set the variables below. However, in case you choose this method, then
#    start replication for the first time (even unsuccessfully, for example
#    if you mistyped the password in master-password and the slave fails to
#    connect), the slave will create a master.info file, and any later
#    change in this file to the variables' values below will be ignored and
#    overridden by the content of the master.info file, unless you shutdown
#    the slave server, delete master.info and restart the slaver server.
#    For that reason, you may want to leave the lines below untouched
#    (commented) and instead use CHANGE MASTER TO (see above)
#
# required unique id between 2 and 2^32 - 1
# (and different from the master)
# defaults to 2 if master-host is set
# but will not function as a slave if omitted
#server-id       = 2
#
# The replication master for this slave - required
#master-host     =   <hostname>
#
# The username the slave will use for authentication when connecting
# to the master - required
#master-user     =   <username>
#
# The password the slave will authenticate with when connecting to
# the master - required
#master-password =   <password>
#
# The port the master is listening on.
# optional - defaults to 3306
#master-port     =  <port>
#
# binary logging - not required for slaves, but recommended
#log-bin=mysql-bin

# Point the following paths to different dedicated disks
#tmpdir         = /tmp/
#log-update     = /path-to-dedicated-directory/hostname

# Uncomment the following if you are using BDB tables
#bdb_cache_size = 64M
#bdb_max_lock = 100000

# Uncomment the following if you are using InnoDB tables
#innodb_data_home_dir = /usr/local/mysql/data/
#innodb_data_file_path = ibdata1:10M:autoextend
#innodb_log_group_home_dir = /usr/local/mysql/data/
#innodb_log_arch_dir = /usr/local/mysql/data/
# You can set .._buffer_pool_size up to 50 - 80 %
# of RAM but beware of setting memory usage too high
#innodb_buffer_pool_size = 256M
#innodb_additional_mem_pool_size = 20M
# Set .._log_file_size to 25 % of buffer pool size
#innodb_log_file_size = 64M
#innodb_log_buffer_size = 8M
#innodb_flush_log_at_trx_commit = 1
#innodb_lock_wait_timeout = 50

[mysqldump]
quick
max_allowed_packet = 16M

[mysql]
no-auto-rehash
# Remove the next comment character if you are not familiar with SQL
#safe-updates

[isamchk]
key_buffer = 128M
sort_buffer_size = 128M
read_buffer = 2M
write_buffer = 2M

[myisamchk]
key_buffer = 128M
sort_buffer_size = 128M
read_buffer = 2M
write_buffer = 2M

[mysqlhotcopy]
interactive-timeout

아스라다

저도 님과 같은 구성을 했었는데, 클러스터가 불안정하게 동작했습니다.

구성을 보니 관리노드와 ndbd 노드가 같이 올라가 있는데, 이 것을 따로 분리하시는게 좋을 거 같습니다.

제가 예전에 테스트 했을 때 관리노드와 ndbd 노드가 함께 올라가있을 경우 님과 같은 상황이 발생했었거든요.

아마도 관리노드를 실행하실 때 경고메세지가 출력되었을거 같은데요, 관리노드와 데이터 노드를 분리하라고요.?

dbalee

ndb_mgmd와 ndbd를 한 서버에서 같이 기동해도,
mysql clusrer가 시작하는데는 문제가 없습니다만,
이 상황에서 서버가 리부팅되면, ndb_mgmd도 죽게 되므로, 전체 ndbd데몬도 죽게 됩니다.
ndb_mgmd는 반드시 ndbd데몬과 분리된 서버에서 실행해야 합니다.
ndb_mgmd는 평소에는 전혀 cpu를 사용하지 않고, mysql cluster의 설정 변경이나 문제 발생시에만 동작을 하게 되니,
web서버에 ndb_mgmd를 실행시켜도 웹서버 부하에 전혀 영향없이,
안정적으로 mysql cluster를 운영할수 있게 됩니다.

이전글	데이터정리시 메모리용량

다음글	SQL node의 부하 분산 방법