2
0
mirror of https://github.com/openvswitch/ovs synced 2025-08-31 14:25:26 +00:00
Commit Graph

6 Commits

Author SHA1 Message Date
Eelco Chaudron
13dde11310 utilities: Add kernel_delay.py script to debug a busy Linux kernel.
This patch adds an utility that can be used to determine if
an issue is related to a lack of Linux kernel resources.

This tool is also featured in a Red Hat developers blog article:

  https://developers.redhat.com/articles/2023/07/24/troubleshooting-open-vswitch-kernel-blame

Reviewed-by: Adrian Moreno <amorenoz@redhat.com>
Acked-by: Aaron Conole <aconole@redhat.com>
Signed-off-by: Eelco Chaudron <echaudro@redhat.com>
2023-09-27 11:21:16 +02:00
Eelco Chaudron
6ad35dd80e utilities: Add revalidator measurement script and needed USDT probes.
This patch adds a Python script that can be used to analyze the
revalidator runs by providing statistics (including some real time
graphs).

The USDT events can also be captured to a file and used for
later offline analysis.

The following blog explains the Open vSwitch revalidator
implementation and how this tool can help you understand what is
happening in your system.

https://developers.redhat.com/articles/2022/10/19/open-vswitch-revalidator-process-explained

Signed-off-by: Eelco Chaudron <echaudro@redhat.com>
Acked-by: Adrian Moreno <amorenoz@redhat.com>
Acked-by: Simon Horman <simon.horman@corigine.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2023-01-27 16:57:27 +01:00
Eelco Chaudron
62e85106b4 utilities: Add USDT script to monitor dpif netlink execute message queuing.
This patch adds the dpif_nl_exec_monitor.py script that will used the
existing dpif_netlink_operate__:op_flow_execute USDT probe to show
all DPIF_OP_EXECUTE operations being queued for transmission over
the netlink interface.

Here is an example, truncated output:

Display DPIF_OP_EXECUTE operations being queued for transmission...
TIME               CPU  COMM             PID        NL_SIZE
3124.516679897     1    ovs-vswitchd     8219       180
    nlmsghdr  : len = 0, type = 36, flags = 1, seq = 0, pid = 0
    genlmsghdr: cmd = 3, version = 1, reserver = 0
    ovs_header: dp_ifindex = 21
      > Decode OVS_PACKET_ATTR_* TLVs:
      nla_len 46, nla_type OVS_PACKET_ATTR_PACKET[1], data: 00 00 00...
      nla_len 20, nla_type OVS_PACKET_ATTR_KEY[2], data: 08 00 02 00...
          > Decode OVS_KEY_ATTR_* TLVs:
          nla_len 8, nla_type OVS_KEY_ATTR_PRIORITY[2], data: 00 00...
          nla_len 8, nla_type OVS_KEY_ATTR_SKB_MARK[15], data: 00 00...
      nla_len 88, nla_type OVS_PACKET_ATTR_ACTIONS[3], data: 4c 00 03...
          > Decode OVS_ACTION_ATTR_* TLVs:
          nla_len 76, nla_type OVS_ACTION_ATTR_SET[3], data: 48 00...
                  > Decode OVS_TUNNEL_KEY_ATTR_* TLVs:
                  nla_len 12, nla_type OVS_TUNNEL_KEY_ATTR_ID[0], data:...
                  nla_len 20, nla_type OVS_TUNNEL_KEY_ATTR_IPV6_DST[13], ...
                  nla_len 5, nla_type OVS_TUNNEL_KEY_ATTR_TTL[4], data: 40
                  nla_len 4, nla_type OVS_TUNNEL_KEY_ATTR_DONT_FRAGMENT[5]...
                  nla_len 4, nla_type OVS_TUNNEL_KEY_ATTR_CSUM[6], data:
                  nla_len 6, nla_type OVS_TUNNEL_KEY_ATTR_TP_DST[10],...
                  nla_len 12, nla_type OVS_TUNNEL_KEY_ATTR_GENEVE_OPTS[8],...
          nla_len 8, nla_type OVS_ACTION_ATTR_OUTPUT[1], data: 02 00 00 00
      - Dumping OVS_PACKET_ATR_PACKET data:
      ###[ Ethernet ]###
        dst       = 00:00:00:00:ec:01
        src       = 04:f4:bc:28:57:00
        type      = IPv4
      ###[ IP ]###
           version   = 4
           ihl       = 5
           tos       = 0x0
           len       = 50
           id        = 0
           flags     =
           frag      = 0
           ttl       = 127
           proto     = icmp
           chksum    = 0x2767
           src       = 10.0.0.1
           dst       = 10.0.0.100
           \options   \
      ###[ ICMP ]###
              type      = echo-request
              code      = 0
              chksum    = 0xf7f3
              id        = 0x0
              seq       = 0xc

Acked-by: Adrian Moreno <amorenoz@redhat.com>
Signed-off-by: Eelco Chaudron <echaudro@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2023-01-06 14:42:14 +01:00
Eelco Chaudron
85d3785e6a utilities: Add netlink flow operation USDT probes and upcall_cost script.
This patch adds a series of NetLink flow operation USDT probes.
These probes are in turn used in the upcall_cost Python script,
which in addition of some kernel tracepoints, give an insight into
the time spent on processing upcall.

Signed-off-by: Eelco Chaudron <echaudro@redhat.com>
Acked-by: Paolo Valerio <pvalerio@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2022-01-18 00:46:30 +01:00
Eelco Chaudron
51ec98635e utilities: Add upcall USDT probe and associated script.
Added the dpif_recv:recv_upcall USDT probe, which is used by the
included upcall_monitor.py script. This script receives all upcall
packets sent by the kernel to ovs-vswitchd. By default, it will
show all  upcall events, which looks something like this:

 TIME               CPU  COMM      PID      DPIF_NAME          TYPE PKT_LEN FLOW_KEY_LEN
 5952147.003848809  2    handler4  1381158  system@ovs-system  0    98      132
 5952147.003879643  2    handler4  1381158  system@ovs-system  0    70      160
 5952147.003914924  2    handler4  1381158  system@ovs-system  0    98      152

It can also dump the packet and NetLink content, and if required,
the packets can also be written to a pcap file.

Signed-off-by: Eelco Chaudron <echaudro@redhat.com>
Acked-by: Paolo Valerio <pvalerio@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2022-01-18 00:46:30 +01:00
Eelco Chaudron
ff4c712d45 Documentation: Add USDT documentation and bpftrace example.
Add the USDT documentation and a bpftrace example using the
bridge run USDT probes.

Signed-off-by: Eelco Chaudron <echaudro@redhat.com>
Acked-by: Paolo Valerio <pvalerio@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2022-01-18 00:46:30 +01:00