mir/ovs
2
0
Fork 0
mirror of https://github.com/openvswitch/ovs synced 2025-08-31 14:25:26 +00:00
Code Issues Releases Wiki Activity

OVN resource agent - make promotion synchronous

Browse Source 
Currently inside the ovsdb_server_promote() function we call 'promote_ovnnb'
and 'promote_ovnsb' and then just record the new master state in the
CIB.

This creates a race because those two promote commands are asynchronous
so when we exit the ovsdb_server_promote() function the underlying DBs
are not guaranteed to be in master state. That means that clients might
connect to an instance that is in read-only mode.

We add a simple sleep loop where we wait for the underlying DB state to
confirm the master state. We do not need to add a timeout loop because
in case of an issue the resource timeout set within pacemaker will kick
in and the resource agent script will be killed by pacemaker.

Tested this within an openstack environment using ovn with roughly ~20
reboots and was unable to trigger the issue (before the patch we would
trigger the issue after a couple of reboots tops).

Acked-by: Numan Siddique <nusiddiq@redhat.com>
Acked-By: Daniel Alvarez <dalvarez@redhat.com>
Signed-off-by: Michele Baldessari <michele@acksyn.org>
Signed-off-by: Ben Pfaff <blp@ovn.org>
This commit is contained in:
Michele Baldessari
2019-07-09 09:02:57 +02:00
committed by Ben Pfaff
parent 0746807054
commit 00dbfe5a3b
1 changed files with 11 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
ovn/utilities/ovndb-servers.ocf
View File

@@ -516,6 +516,8 @@ ovsdb_server_stop() {
} }
ovsdb_server_promote() { ovsdb_server_promote() {
local state
ovsdb_server_check_status ignore_northd ovsdb_server_check_status ignore_northd
rc=$? rc=$?
case $rc in case $rc in
@@ -540,7 +542,15 @@ ovsdb_server_promote() {
${OVN_CTL} --ovn-manage-ovsdb=no start_northd ${OVN_CTL} --ovn-manage-ovsdb=no start_northd
fi fi
ocf_log debug "ovndb_servers: Promoting $host_name as the master" ocf_log debug "ovndb_servers: Waiting for promotion $host_name as master to complete"
ovsdb_server_check_status
state=$?
while [ "$state" != "$OCF_RUNNING_MASTER" ]; do
sleep 1
ovsdb_server_check_status
state=$?
done
ocf_log debug "ovndb_servers: Promotion of $host_name as the master completed"
# Record ourselves so that the agent has a better chance of doing # Record ourselves so that the agent has a better chance of doing
# the right thing at startup # the right thing at startup
${CRM_ATTR_REPL_INFO} -v "$host_name" ${CRM_ATTR_REPL_INFO} -v "$host_name"