2
0
mirror of https://github.com/openvswitch/ovs synced 2025-08-22 09:58:01 +00:00
ovs/ovsdb/transaction-forward.c

184 lines
5.2 KiB
C
Raw Permalink Normal View History

ovsdb: relay: Add support for transaction forwarding. Current version of ovsdb relay allows to scale out read-only access to the primary database. However, many clients are not read-only but read-mostly. For example, ovn-controller. In order to scale out database access for this case ovsdb-server need to process transactions that are not read-only. Relay is not allowed to do that, i.e. not allowed to modify the database, but it can act like a proxy and forward transactions that includes database modifications to the primary server and forward replies back to a client. At the same time it may serve read-only transactions and monitor requests by itself greatly reducing the load on primary server. This configuration will slightly increase transaction latency, but it's not very important for read-mostly use cases. Implementation details: With this change instead of creating a trigger to commit the transaction, ovsdb-server will create a trigger for transaction forwarding. Later, ovsdb_relay_run() will send all new transactions to the relay source. Once transaction reply received from the relay source, ovsdb-relay module will update the state of the transaction forwarding with the reply. After that, trigger_run() will complete the trigger and jsonrpc_server_run() will send the reply back to the client. Since transaction reply from the relay source will be received after all the updates, client will receive all the updates before receiving the transaction reply as it is in a normal scenario with other database models. Acked-by: Mark D. Gray <mark.d.gray@redhat.com> Acked-by: Dumitru Ceara <dceara@redhat.com> Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2021-04-15 19:05:40 +02:00
/*
* Copyright (c) 2021, Red Hat, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at:
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
#include <config.h>
#include "transaction-forward.h"
#include "coverage.h"
#include "jsonrpc.h"
#include "openvswitch/hmap.h"
#include "openvswitch/json.h"
#include "openvswitch/list.h"
#include "openvswitch/poll-loop.h"
#include "openvswitch/vlog.h"
#include "ovsdb.h"
#include "ovsdb-cs.h"
#include "util.h"
VLOG_DEFINE_THIS_MODULE(transaction_forward);
COVERAGE_DEFINE(txn_forward_cancel);
COVERAGE_DEFINE(txn_forward_complete);
COVERAGE_DEFINE(txn_forward_create);
COVERAGE_DEFINE(txn_forward_sent);
struct ovsdb_txn_forward {
struct ovs_list new_node; /* In 'txn_forward_new' of struct ovsdb. */
struct hmap_node sent_node; /* In 'txn_forward_sent' of struct ovsdb. */
struct json *id; /* 'id' of the forwarded transaction. */
struct jsonrpc_msg *request; /* Original request. */
struct jsonrpc_msg *reply; /* Reply from the server. */
};
struct ovsdb_txn_forward *
ovsdb_txn_forward_create(struct ovsdb *db, const struct jsonrpc_msg *request)
{
struct ovsdb_txn_forward *txn_fwd = xzalloc(sizeof *txn_fwd);
COVERAGE_INC(txn_forward_create);
txn_fwd->request = jsonrpc_msg_clone(request);
ovs_list_push_back(&db->txn_forward_new, &txn_fwd->new_node);
ovsdb: transaction-forward: Fix initialization of the 'sent' hmap node. 'sent_node' is initialized to all zeroes by xzalloc(), but HMAP_NODE_NULL is not all zeroes. hmap_node_is_null() is used to detect if the node is valid, but it will fail and cause segmentation fault on attempt to remove the non-existent node from the hash map. This can happen if client disconnected while the transaction is not yet forwarded to the relay source: Program terminated with signal 11, Segmentation fault. 0 in hmap_remove at include/openvswitch/hmap.h:293 293 while (*bucket != node) { (gdb) bt 0 hmap_remove at include/openvswitch/hmap.h:293 1 ovsdb_txn_forward_unlist at ovsdb/transaction-forward.c:67 2 ovsdb_txn_forward_destroy at ovsdb/transaction-forward.c:79 3 ovsdb_trigger_destroy at ovsdb/trigger.c:70 4 ovsdb_jsonrpc_trigger_complete at ovsdb/jsonrpc-server.c:1192 5 ovsdb_jsonrpc_trigger_remove__ at ovsdb/jsonrpc-server.c:1204 6 ovsdb_jsonrpc_trigger_complete_all at ovsdb/jsonrpc-server.c:1223 7 ovsdb_jsonrpc_session_run at ovsdb/jsonrpc-server.c:546 8 ovsdb_jsonrpc_session_run_all at ovsdb/jsonrpc-server.c:591 9 ovsdb_jsonrpc_server_run at ovsdb/jsonrpc-server.c:406 10 main_loop (gdb) print db->txn_forward_sent $20 = {buckets = 0x..., one = 0x0, mask = 63, n = 0} (gdb) print txn_fwd->sent_node $24 = {hash = 0, next = 0x0} Fix that by correct initialization of the 'sent_node'. Reported-by: Wentao Jia <wentao.jia@easystack.cn> Reported-at: https://mail.openvswitch.org/pipermail/ovs-discuss/2021-August/051354.html Fixes: 7964ffe7d2bf ("ovsdb: relay: Add support for transaction forwarding.") Signed-off-by: Ilya Maximets <i.maximets@ovn.org> Acked-by: Mark D. Gray <mark.d.gray@redhat.com>
2021-08-05 14:57:46 +02:00
hmap_node_nullify(&txn_fwd->sent_node);
ovsdb: relay: Add support for transaction forwarding. Current version of ovsdb relay allows to scale out read-only access to the primary database. However, many clients are not read-only but read-mostly. For example, ovn-controller. In order to scale out database access for this case ovsdb-server need to process transactions that are not read-only. Relay is not allowed to do that, i.e. not allowed to modify the database, but it can act like a proxy and forward transactions that includes database modifications to the primary server and forward replies back to a client. At the same time it may serve read-only transactions and monitor requests by itself greatly reducing the load on primary server. This configuration will slightly increase transaction latency, but it's not very important for read-mostly use cases. Implementation details: With this change instead of creating a trigger to commit the transaction, ovsdb-server will create a trigger for transaction forwarding. Later, ovsdb_relay_run() will send all new transactions to the relay source. Once transaction reply received from the relay source, ovsdb-relay module will update the state of the transaction forwarding with the reply. After that, trigger_run() will complete the trigger and jsonrpc_server_run() will send the reply back to the client. Since transaction reply from the relay source will be received after all the updates, client will receive all the updates before receiving the transaction reply as it is in a normal scenario with other database models. Acked-by: Mark D. Gray <mark.d.gray@redhat.com> Acked-by: Dumitru Ceara <dceara@redhat.com> Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2021-04-15 19:05:40 +02:00
return txn_fwd;
}
static void
ovsdb_txn_forward_unlist(struct ovsdb *db, struct ovsdb_txn_forward *txn_fwd)
{
if (!ovs_list_is_empty(&txn_fwd->new_node)) {
ovs_list_remove(&txn_fwd->new_node);
ovs_list_init(&txn_fwd->new_node);
}
if (!hmap_node_is_null(&txn_fwd->sent_node)) {
hmap_remove(&db->txn_forward_sent, &txn_fwd->sent_node);
hmap_node_nullify(&txn_fwd->sent_node);
}
}
void
ovsdb_txn_forward_destroy(struct ovsdb *db, struct ovsdb_txn_forward *txn_fwd)
{
if (!txn_fwd) {
return;
}
ovsdb_txn_forward_unlist(db, txn_fwd);
json_destroy(txn_fwd->id);
jsonrpc_msg_destroy(txn_fwd->request);
jsonrpc_msg_destroy(txn_fwd->reply);
free(txn_fwd);
}
bool
ovsdb_txn_forward_is_complete(const struct ovsdb_txn_forward *txn_fwd)
{
return txn_fwd->reply != NULL;
}
void
ovsdb_txn_forward_complete(struct ovsdb *db, const struct jsonrpc_msg *reply)
{
struct ovsdb_txn_forward *t;
size_t hash = json_hash(reply->id, 0);
HMAP_FOR_EACH_WITH_HASH (t, sent_node, hash, &db->txn_forward_sent) {
if (json_equal(reply->id, t->id)) {
COVERAGE_INC(txn_forward_complete);
t->reply = jsonrpc_msg_clone(reply);
/* Replacing id with the id of the original request. */
json_destroy(t->reply->id);
t->reply->id = json_clone(t->request->id);
hmap_remove(&db->txn_forward_sent, &t->sent_node);
hmap_node_nullify(&t->sent_node);
db->run_triggers_now = db->run_triggers = true;
return;
}
}
}
struct jsonrpc_msg *
ovsdb_txn_forward_steal_reply(struct ovsdb_txn_forward *txn_fwd)
{
struct jsonrpc_msg *reply = txn_fwd->reply;
txn_fwd->reply = NULL;
return reply;
}
void
ovsdb_txn_forward_run(struct ovsdb *db, struct ovsdb_cs *cs)
{
struct ovsdb_txn_forward *t;
ovsdb: relay: Add support for transaction forwarding. Current version of ovsdb relay allows to scale out read-only access to the primary database. However, many clients are not read-only but read-mostly. For example, ovn-controller. In order to scale out database access for this case ovsdb-server need to process transactions that are not read-only. Relay is not allowed to do that, i.e. not allowed to modify the database, but it can act like a proxy and forward transactions that includes database modifications to the primary server and forward replies back to a client. At the same time it may serve read-only transactions and monitor requests by itself greatly reducing the load on primary server. This configuration will slightly increase transaction latency, but it's not very important for read-mostly use cases. Implementation details: With this change instead of creating a trigger to commit the transaction, ovsdb-server will create a trigger for transaction forwarding. Later, ovsdb_relay_run() will send all new transactions to the relay source. Once transaction reply received from the relay source, ovsdb-relay module will update the state of the transaction forwarding with the reply. After that, trigger_run() will complete the trigger and jsonrpc_server_run() will send the reply back to the client. Since transaction reply from the relay source will be received after all the updates, client will receive all the updates before receiving the transaction reply as it is in a normal scenario with other database models. Acked-by: Mark D. Gray <mark.d.gray@redhat.com> Acked-by: Dumitru Ceara <dceara@redhat.com> Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2021-04-15 19:05:40 +02:00
/* Send all transactions that needs to be forwarded. */
LIST_FOR_EACH_SAFE (t, new_node, &db->txn_forward_new) {
ovsdb: relay: Add support for transaction forwarding. Current version of ovsdb relay allows to scale out read-only access to the primary database. However, many clients are not read-only but read-mostly. For example, ovn-controller. In order to scale out database access for this case ovsdb-server need to process transactions that are not read-only. Relay is not allowed to do that, i.e. not allowed to modify the database, but it can act like a proxy and forward transactions that includes database modifications to the primary server and forward replies back to a client. At the same time it may serve read-only transactions and monitor requests by itself greatly reducing the load on primary server. This configuration will slightly increase transaction latency, but it's not very important for read-mostly use cases. Implementation details: With this change instead of creating a trigger to commit the transaction, ovsdb-server will create a trigger for transaction forwarding. Later, ovsdb_relay_run() will send all new transactions to the relay source. Once transaction reply received from the relay source, ovsdb-relay module will update the state of the transaction forwarding with the reply. After that, trigger_run() will complete the trigger and jsonrpc_server_run() will send the reply back to the client. Since transaction reply from the relay source will be received after all the updates, client will receive all the updates before receiving the transaction reply as it is in a normal scenario with other database models. Acked-by: Mark D. Gray <mark.d.gray@redhat.com> Acked-by: Dumitru Ceara <dceara@redhat.com> Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2021-04-15 19:05:40 +02:00
if (!ovsdb_cs_may_send_transaction(cs)) {
break;
}
ovs_assert(!strcmp(t->request->method, "transact"));
t->id = ovsdb_cs_send_transaction(cs, json_clone(t->request->params));
if (t->id) {
COVERAGE_INC(txn_forward_sent);
ovs_list_remove(&t->new_node);
ovs_list_init(&t->new_node);
hmap_insert(&db->txn_forward_sent, &t->sent_node,
json_hash(t->id, 0));
}
}
}
void
ovsdb_txn_forward_wait(struct ovsdb *db, struct ovsdb_cs *cs)
{
if (ovsdb_cs_may_send_transaction(cs)
&& !ovs_list_is_empty(&db->txn_forward_new)) {
poll_immediate_wake();
}
}
void
ovsdb_txn_forward_cancel(struct ovsdb *db, struct ovsdb_txn_forward *txn_fwd)
{
COVERAGE_INC(txn_forward_cancel);
jsonrpc_msg_destroy(txn_fwd->reply);
txn_fwd->reply = jsonrpc_create_error(json_string_create("canceled"),
txn_fwd->request->id);
ovsdb_txn_forward_unlist(db, txn_fwd);
}
void
ovsdb_txn_forward_cancel_all(struct ovsdb *db, bool sent_only)
{
struct ovsdb_txn_forward *t;
ovsdb: relay: Add support for transaction forwarding. Current version of ovsdb relay allows to scale out read-only access to the primary database. However, many clients are not read-only but read-mostly. For example, ovn-controller. In order to scale out database access for this case ovsdb-server need to process transactions that are not read-only. Relay is not allowed to do that, i.e. not allowed to modify the database, but it can act like a proxy and forward transactions that includes database modifications to the primary server and forward replies back to a client. At the same time it may serve read-only transactions and monitor requests by itself greatly reducing the load on primary server. This configuration will slightly increase transaction latency, but it's not very important for read-mostly use cases. Implementation details: With this change instead of creating a trigger to commit the transaction, ovsdb-server will create a trigger for transaction forwarding. Later, ovsdb_relay_run() will send all new transactions to the relay source. Once transaction reply received from the relay source, ovsdb-relay module will update the state of the transaction forwarding with the reply. After that, trigger_run() will complete the trigger and jsonrpc_server_run() will send the reply back to the client. Since transaction reply from the relay source will be received after all the updates, client will receive all the updates before receiving the transaction reply as it is in a normal scenario with other database models. Acked-by: Mark D. Gray <mark.d.gray@redhat.com> Acked-by: Dumitru Ceara <dceara@redhat.com> Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2021-04-15 19:05:40 +02:00
HMAP_FOR_EACH_SAFE (t, sent_node, &db->txn_forward_sent) {
ovsdb: relay: Add support for transaction forwarding. Current version of ovsdb relay allows to scale out read-only access to the primary database. However, many clients are not read-only but read-mostly. For example, ovn-controller. In order to scale out database access for this case ovsdb-server need to process transactions that are not read-only. Relay is not allowed to do that, i.e. not allowed to modify the database, but it can act like a proxy and forward transactions that includes database modifications to the primary server and forward replies back to a client. At the same time it may serve read-only transactions and monitor requests by itself greatly reducing the load on primary server. This configuration will slightly increase transaction latency, but it's not very important for read-mostly use cases. Implementation details: With this change instead of creating a trigger to commit the transaction, ovsdb-server will create a trigger for transaction forwarding. Later, ovsdb_relay_run() will send all new transactions to the relay source. Once transaction reply received from the relay source, ovsdb-relay module will update the state of the transaction forwarding with the reply. After that, trigger_run() will complete the trigger and jsonrpc_server_run() will send the reply back to the client. Since transaction reply from the relay source will be received after all the updates, client will receive all the updates before receiving the transaction reply as it is in a normal scenario with other database models. Acked-by: Mark D. Gray <mark.d.gray@redhat.com> Acked-by: Dumitru Ceara <dceara@redhat.com> Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2021-04-15 19:05:40 +02:00
ovsdb_txn_forward_cancel(db, t);
}
if (sent_only) {
return;
}
LIST_FOR_EACH_SAFE (t, new_node, &db->txn_forward_new) {
ovsdb: relay: Add support for transaction forwarding. Current version of ovsdb relay allows to scale out read-only access to the primary database. However, many clients are not read-only but read-mostly. For example, ovn-controller. In order to scale out database access for this case ovsdb-server need to process transactions that are not read-only. Relay is not allowed to do that, i.e. not allowed to modify the database, but it can act like a proxy and forward transactions that includes database modifications to the primary server and forward replies back to a client. At the same time it may serve read-only transactions and monitor requests by itself greatly reducing the load on primary server. This configuration will slightly increase transaction latency, but it's not very important for read-mostly use cases. Implementation details: With this change instead of creating a trigger to commit the transaction, ovsdb-server will create a trigger for transaction forwarding. Later, ovsdb_relay_run() will send all new transactions to the relay source. Once transaction reply received from the relay source, ovsdb-relay module will update the state of the transaction forwarding with the reply. After that, trigger_run() will complete the trigger and jsonrpc_server_run() will send the reply back to the client. Since transaction reply from the relay source will be received after all the updates, client will receive all the updates before receiving the transaction reply as it is in a normal scenario with other database models. Acked-by: Mark D. Gray <mark.d.gray@redhat.com> Acked-by: Dumitru Ceara <dceara@redhat.com> Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2021-04-15 19:05:40 +02:00
ovsdb_txn_forward_cancel(db, t);
}
}