Documentations: Analyze heavily used Networking related structs

Analyzed a few structs in the networking stack by looking at variables
within them that are used in the TCP/IP fast path.

Fast path is defined as TCP path where data is transferred from sender to
receiver unidirectionally. It doesn't include phases other than
TCP_ESTABLISHED, nor does it look at error paths.

We hope to re-organizing variables that span many cachelines whose fast
path variables are also spread out, and this document can help future
developers keep networking fast path cachelines small.

Optimized_cacheline field is computed as
(Fastpath_Bytes/L3_cacheline_size_x86), and not the actual organized
results (see patches to come for these).

Investigation is done on 6.5

Name	                Struct_Cachelines  Cur_fastpath_cache Fastpath_Bytes Optimized_cacheline
tcp_sock	        42 (2664 Bytes)	   12   		396		8
net_device	        39 (2240 bytes)	   12			234		4
inet_sock	        15 (960 bytes)	   14			922		14
Inet_connection_sock	22 (1368 bytes)	   18			1166		18
Netns_ipv4 (sysctls)	12 (768 bytes)     4			77		2
linux_mib	        16 (1060)	   6			104		2

Note how there isn't much improvement space for inet_sock and
Inet_connection_sock because sk and icsk_inet respectively takes up so
much of the struct that rest of the variables become a small portion of
the struct size.

So, we decided to reorganize tcp_sock, net_device, netns_ipv4

Suggested-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: Coco Li <lixiaoyan@google.com>
Reviewed-by: Eric Dumazet <edumazet@google.com>
Reviewed-by: Shakeel Butt <shakeelb@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

7 files changed, 731 insertions, 0 deletions

diff --git a/Documentation/networking/net_cachelines/index.rst b/Documentation/networking/net_cachelines/index.rst
new file mode 100644
index 000000000000..6f1a99989bbd
--- /dev/null
+++ b/Documentation/networking/net_cachelines/index.rst
@@ -0,0 +1,15 @@
+.. SPDX-License-Identifier: GPL-2.0
+.. Copyright (C) 2023 Google LLC
+===================================
+Common Networking Struct Cachelines
+===================================
+
+.. toctree::
+   :maxdepth: 1
+
+   inet_connection_sock
+   inet_sock
+   net_device
+   netns_ipv4_sysctl
+   snmp
+   tcp_sock
diff --git a/Documentation/networking/net_cachelines/inet_connection_sock.rst b/Documentation/networking/net_cachelines/inet_connection_sock.rst
new file mode 100644
index 000000000000..ad2b200147a6
--- /dev/null
+++ b/Documentation/networking/net_cachelines/inet_connection_sock.rst
@@ -0,0 +1,49 @@
+.. SPDX-License-Identifier: GPL-2.0
+.. Copyright (C) 2023 Google LLC
+=====================================================
+inet_connection_sock struct fast path usage breakdown
+=====================================================
+
+Type                                Name                   fastpath_tx_access  fastpath_rx_access  comment
+..struct                            ..inet_connection_sock                                         
+struct_inet_sock                    icsk_inet              read_mostly         read_mostly         tcp_init_buffer_space,tcp_init_transfer,tcp_finish_connect,tcp_connect,tcp_send_rcvq,tcp_send_syn_data
+struct_request_sock_queue           icsk_accept_queue      -                   -                   
+struct_inet_bind_bucket             icsk_bind_hash         read_mostly         -                   tcp_set_state
+struct_inet_bind2_bucket            icsk_bind2_hash        read_mostly         -                   tcp_set_state,inet_put_port
+unsigned_long                       icsk_timeout           read_mostly         -                   inet_csk_reset_xmit_timer,tcp_connect
+struct_timer_list                   icsk_retransmit_timer  read_mostly         -                   inet_csk_reset_xmit_timer,tcp_connect
+struct_timer_list                   icsk_delack_timer      read_mostly         -                   inet_csk_reset_xmit_timer,tcp_connect
+u32                                 icsk_rto               read_write          -                   tcp_cwnd_validate,tcp_schedule_loss_probe,tcp_connect_init,tcp_connect,tcp_write_xmit,tcp_push_one
+u32                                 icsk_rto_min           -                   -                   
+u32                                 icsk_delack_max        -                   -                   
+u32                                 icsk_pmtu_cookie       read_write          -                   tcp_sync_mss,tcp_current_mss,tcp_send_syn_data,tcp_connect_init,tcp_connect
+struct_tcp_congestion_ops           icsk_ca_ops            read_write          -                   tcp_cwnd_validate,tcp_tso_segs,tcp_ca_dst_init,tcp_connect_init,tcp_connect,tcp_write_xmit
+struct_inet_connection_sock_af_ops  icsk_af_ops            read_mostly         -                   tcp_finish_connect,tcp_send_syn_data,tcp_mtup_init,tcp_mtu_check_reprobe,tcp_mtu_probe,tcp_connect_init,tcp_connect,__tcp_transmit_skb
+struct_tcp_ulp_ops*                 icsk_ulp_ops           -                   -                   
+void*                               icsk_ulp_data          -                   -                   
+u8:5                                icsk_ca_state          read_write          -                   tcp_cwnd_application_limited,tcp_set_ca_state,tcp_enter_cwr,tcp_tso_should_defer,tcp_mtu_probe,tcp_schedule_loss_probe,tcp_write_xmit,__tcp_transmit_skb
+u8:1                                icsk_ca_initialized    read_write          -                   tcp_init_transfer,tcp_init_congestion_control,tcp_init_transfer,tcp_finish_connect,tcp_connect
+u8:1                                icsk_ca_setsockopt     -                   -                   
+u8:1                                icsk_ca_dst_locked     write_mostly        -                   tcp_ca_dst_init,tcp_connect_init,tcp_connect
+u8                                  icsk_retransmits       write_mostly        -                   tcp_connect_init,tcp_connect
+u8                                  icsk_pending           read_write          -                   inet_csk_reset_xmit_timer,tcp_connect,tcp_check_probe_timer,__tcp_push_pending_frames,tcp_rearm_rto,tcp_event_new_data_sent,tcp_event_new_data_sent
+u8                                  icsk_backoff           write_mostly        -                   tcp_write_queue_purge,tcp_connect_init
+u8                                  icsk_syn_retries       -                   -                   
+u8                                  icsk_probes_out        -                   -                   
+u16                                 icsk_ext_hdr_len       read_mostly         -                   __tcp_mtu_to_mss,tcp_mtu_to_rss,tcp_mtu_probe,tcp_write_xmit,tcp_mtu_to_mss,
+struct_icsk_ack_u8                  pending                read_write          read_write          inet_csk_ack_scheduled,__tcp_cleanup_rbuf,tcp_cleanup_rbuf,inet_csk_clear_xmit_timer,tcp_event_ack-sent,inet_csk_reset_xmit_timer
+struct_icsk_ack_u8                  quick                  read_write          write_mostly        tcp_dec_quickack_mode,tcp_event_ack_sent,__tcp_transmit_skb,__tcp_select_window,__tcp_cleanup_rbuf
+struct_icsk_ack_u8                  pingpong               -                   -                   
+struct_icsk_ack_u8                  retry                  write_mostly        read_write          inet_csk_clear_xmit_timer,tcp_rearm_rto,tcp_event_new_data_sent,tcp_write_xmit,__tcp_send_ack,tcp_send_ack,
+struct_icsk_ack_u8                  ato                    read_mostly         write_mostly        tcp_dec_quickack_mode,tcp_event_ack_sent,__tcp_transmit_skb,__tcp_send_ack,tcp_send_ack
+struct_icsk_ack_unsigned_long       timeout                read_write          read_write          inet_csk_reset_xmit_timer,tcp_connect
+struct_icsk_ack_u32                 lrcvtime               read_write          -                   tcp_finish_connect,tcp_connect,tcp_event_data_sent,__tcp_transmit_skb
+struct_icsk_ack_u16                 rcv_mss                write_mostly        read_mostly         __tcp_select_window,__tcp_cleanup_rbuf,tcp_initialize_rcv_mss,tcp_connect_init
+struct_icsk_mtup_int                search_high            read_write          -                   tcp_mtup_init,tcp_sync_mss,tcp_connect_init,tcp_mtu_check_reprobe,tcp_write_xmit
+struct_icsk_mtup_int                search_low             read_write          -                   tcp_mtu_probe,tcp_mtu_check_reprobe,tcp_write_xmit,tcp_sync_mss,tcp_connect_init,tcp_mtup_init
+struct_icsk_mtup_u32:31             probe_size             read_write          -                   tcp_mtup_init,tcp_connect_init,__tcp_transmit_skb
+struct_icsk_mtup_u32:1              enabled                read_write          -                   tcp_mtup_init,tcp_sync_mss,tcp_connect_init,tcp_mtu_probe,tcp_write_xmit
+struct_icsk_mtup_u32                probe_timestamp        read_write          -                   tcp_mtup_init,tcp_connect_init,tcp_mtu_check_reprobe,tcp_mtu_probe
+u32                                 icsk_probes_tstamp     -                   -                   
+u32                                 icsk_user_timeout      -                   -                   
+u64[104/sizeof(u64)]                icsk_ca_priv           -                   -                   
diff --git a/Documentation/networking/net_cachelines/inet_sock.rst b/Documentation/networking/net_cachelines/inet_sock.rst
new file mode 100644
index 000000000000..4077febdb995
--- /dev/null
+++ b/Documentation/networking/net_cachelines/inet_sock.rst
@@ -0,0 +1,43 @@
+.. SPDX-License-Identifier: GPL-2.0
+.. Copyright (C) 2023 Google LLC
+=====================================================
+inet_connection_sock struct fast path usage breakdown
+=====================================================
+
+Type                    Name                  fastpath_tx_access  fastpath_rx_access  comment
+..struct                ..inet_sock                                                     
+struct_sock             sk                    read_mostly         read_mostly         tcp_init_buffer_space,tcp_init_transfer,tcp_finish_connect,tcp_connect,tcp_send_rcvq,tcp_send_syn_data
+struct_ipv6_pinfo*      pinet6                -                   -                   
+be16                    inet_sport            read_mostly         -                   __tcp_transmit_skb
+be32                    inet_daddr            read_mostly         -                   ip_select_ident_segs
+be32                    inet_rcv_saddr        -                   -                   
+be16                    inet_dport            read_mostly         -                   __tcp_transmit_skb
+u16                     inet_num              -                   -                   
+be32                    inet_saddr            -                   -                   
+s16                     uc_ttl                read_mostly         -                   __ip_queue_xmit/ip_select_ttl
+u16                     cmsg_flags            -                   -                   
+struct_ip_options_rcu*  inet_opt              read_mostly         -                   __ip_queue_xmit
+u16                     inet_id               read_mostly         -                   ip_select_ident_segs
+u8                      tos                   read_mostly         -                   ip_queue_xmit
+u8                      min_ttl               -                   -                   
+u8                      mc_ttl                -                   -                   
+u8                      pmtudisc              -                   -                   
+u8:1                    recverr               -                   -                   
+u8:1                    is_icsk               -                   -                   
+u8:1                    freebind              -                   -                   
+u8:1                    hdrincl               -                   -                   
+u8:1                    mc_loop               -                   -                   
+u8:1                    transparent           -                   -                   
+u8:1                    mc_all                -                   -                   
+u8:1                    nodefrag              -                   -                   
+u8:1                    bind_address_no_port  -                   -                   
+u8:1                    recverr_rfc4884       -                   -                   
+u8:1                    defer_connect         read_mostly         -                   tcp_sendmsg_fastopen
+u8                      rcv_tos               -                   -                   
+u8                      convert_csum          -                   -                   
+int                     uc_index              -                   -                   
+int                     mc_index              -                   -                   
+be32                    mc_addr               -                   -                   
+struct_ip_mc_socklist*  mc_list               -                   -                   
+struct_inet_cork_full   cork                  read_mostly         -                   __tcp_transmit_skb
+struct                  local_port_range      -                   -                   
diff --git a/Documentation/networking/net_cachelines/net_device.rst b/Documentation/networking/net_cachelines/net_device.rst
new file mode 100644
index 000000000000..98508aa4f800
--- /dev/null
+++ b/Documentation/networking/net_cachelines/net_device.rst
@@ -0,0 +1,177 @@
+.. SPDX-License-Identifier: GPL-2.0
+.. Copyright (C) 2023 Google LLC
+===========================================
+net_device struct fast path usage breakdown
+===========================================
+
+Type                                Name                    fastpath_tx_access  fastpath_rx_access  Comments
+..struct                            ..net_device                                                    
+char                                name[16]                -                   -                   
+struct_netdev_name_node*            name_node                                                       
+struct_dev_ifalias*                 ifalias                                                         
+unsigned_long                       mem_end                                                         
+unsigned_long                       mem_start                                                       
+unsigned_long                       base_addr                                                       
+unsigned_long                       state                                                           
+struct_list_head                    dev_list                                                        
+struct_list_head                    napi_list                                                       
+struct_list_head                    unreg_list                                                      
+struct_list_head                    close_list                                                      
+struct_list_head                    ptype_all               read_mostly         -                   dev_nit_active(tx)
+struct_list_head                    ptype_specific                              read_mostly         deliver_ptype_list_skb/__netif_receive_skb_core(rx)
+struct                              adj_list                                                        
+unsigned_int                        flags                   read_mostly         read_mostly         __dev_queue_xmit,__dev_xmit_skb,ip6_output,__ip6_finish_output(tx);ip6_rcv_core(rx)
+xdp_features_t                      xdp_features                                                    
+unsigned_long_long                  priv_flags              read_mostly         -                   __dev_queue_xmit(tx)
+struct_net_device_ops*              netdev_ops              read_mostly         -                   netdev_core_pick_tx,netdev_start_xmit(tx)
+struct_xdp_metadata_ops*            xdp_metadata_ops                                                
+int                                 ifindex                 -                   read_mostly         ip6_rcv_core
+unsigned_short                      gflags                                                          
+unsigned_short                      hard_header_len         read_mostly         read_mostly         ip6_xmit(tx);gro_list_prepare(rx)
+unsigned_int                        mtu                     read_mostly         -                   ip_finish_output2
+unsigned_short                      needed_headroom         read_mostly         -                   LL_RESERVED_SPACE/ip_finish_output2
+unsigned_short                      needed_tailroom                                                 
+netdev_features_t                   features                read_mostly         read_mostly         HARD_TX_LOCK,netif_skb_features,sk_setup_caps(tx);netif_elide_gro(rx)
+netdev_features_t                   hw_features                                                     
+netdev_features_t                   wanted_features                                                 
+netdev_features_t                   vlan_features                                                   
+netdev_features_t                   hw_enc_features         -                   -                   netif_skb_features
+netdev_features_t                   mpls_features                                                   
+netdev_features_t                   gso_partial_features                                            
+unsigned_int                        min_mtu                                                         
+unsigned_int                        max_mtu                                                         
+unsigned_short                      type                                                            
+unsigned_char                       min_header_len                                                  
+unsigned_char                       name_assign_type                                                
+int                                 group                                                           
+struct_net_device_stats             stats                                                           
+struct_net_device_core_stats*       core_stats                                                      
+atomic_t                            carrier_up_count                                                
+atomic_t                            carrier_down_count                                              
+struct_iw_handler_def*              wireless_handlers                                               
+struct_iw_public_data*              wireless_data                                                   
+struct_ethtool_ops*                 ethtool_ops                                                     
+struct_l3mdev_ops*                  l3mdev_ops                                                      
+struct_ndisc_ops*                   ndisc_ops                                                       
+struct_xfrmdev_ops*                 xfrmdev_ops                                                     
+struct_tlsdev_ops*                  tlsdev_ops                                                      
+struct_header_ops*                  header_ops              read_mostly         -                   ip_finish_output2,ip6_finish_output2(tx)
+unsigned_char                       operstate                                                       
+unsigned_char                       link_mode                                                       
+unsigned_char                       if_port                                                         
+unsigned_char                       dma                                                             
+unsigned_char                       perm_addr[32]                                                   
+unsigned_char                       addr_assign_type                                                
+unsigned_char                       addr_len                                                        
+unsigned_char                       upper_level                                                     
+unsigned_char                       lower_level                                                     
+unsigned_short                      neigh_priv_len                                                  
+unsigned_short                      padded                                                          
+unsigned_short                      dev_id                                                          
+unsigned_short                      dev_port                                                        
+spinlock_t                          addr_list_lock                                                  
+int                                 irq                                                             
+struct_netdev_hw_addr_list          uc                                                              
+struct_netdev_hw_addr_list          mc                                                              
+struct_netdev_hw_addr_list          dev_addrs                                                       
+struct_kset*                        queues_kset                                                     
+struct_list_head                    unlink_list                                                     
+unsigned_int                        promiscuity                                                     
+unsigned_int                        allmulti                                                        
+bool                                uc_promisc                                                      
+unsigned_char                       nested_level                                                    
+struct_in_device*                   ip_ptr                  read_mostly         read_mostly         __in_dev_get
+struct_inet6_dev*                   ip6_ptr                 read_mostly         read_mostly         __in6_dev_get
+struct_vlan_info*                   vlan_info                                                       
+struct_dsa_port*                    dsa_ptr                                                         
+struct_tipc_bearer*                 tipc_ptr                                                        
+void*                               atalk_ptr                                                       
+void*                               ax25_ptr                                                        
+struct_wireless_dev*                ieee80211_ptr                                                   
+struct_wpan_dev*                    ieee802154_ptr                                                  
+struct_mpls_dev*                    mpls_ptr                                                        
+struct_mctp_dev*                    mctp_ptr                                                        
+unsigned_char*                      dev_addr                                                        
+struct_netdev_queue*                _rx                     read_mostly         -                   netdev_get_rx_queue(rx)
+unsigned_int                        num_rx_queues                                                   
+unsigned_int                        real_num_rx_queues      -                   read_mostly         get_rps_cpu
+struct_bpf_prog*                    xdp_prog                                                        
+unsigned_long                       gro_flush_timeout       -                   read_mostly         napi_complete_done
+int                                 napi_defer_hard_irqs    -                   read_mostly         napi_complete_done
+unsigned_int                        gro_max_size            -                   read_mostly         skb_gro_receive
+unsigned_int                        gro_ipv4_max_size       -                   read_mostly         skb_gro_receive
+rx_handler_func_t*                  rx_handler              read_mostly         -                   __netif_receive_skb_core
+void*                               rx_handler_data         read_mostly         -                   
+struct_netdev_queue*                ingress_queue           read_mostly         -                   
+struct_bpf_mprog_entry              tcx_ingress             -                   read_mostly         sch_handle_ingress
+struct_nf_hook_entries*             nf_hooks_ingress                                                
+unsigned_char                       broadcast[32]                                                   
+struct_cpu_rmap*                    rx_cpu_rmap                                                     
+struct_hlist_node                   index_hlist                                                     
+struct_netdev_queue*                _tx                     read_mostly         -                   netdev_get_tx_queue(tx)
+unsigned_int                        num_tx_queues           -                   -                   
+unsigned_int                        real_num_tx_queues      read_mostly         -                   skb_tx_hash,netdev_core_pick_tx(tx)
+unsigned_int                        tx_queue_len                                                    
+spinlock_t                          tx_global_lock                                                  
+struct_xdp_dev_bulk_queue__percpu*  xdp_bulkq                                                       
+struct_xps_dev_maps*                xps_maps[2]             read_mostly         -                   __netif_set_xps_queue
+struct_bpf_mprog_entry              tcx_egress              read_mostly         -                   sch_handle_egress
+struct_nf_hook_entries*             nf_hooks_egress         read_mostly         -                   
+struct_hlist_head                   qdisc_hash[16]                                                  
+struct_timer_list                   watchdog_timer                                                  
+int                                 watchdog_timeo                                                  
+u32                                 proto_down_reason                                               
+struct_list_head                    todo_list                                                       
+int__percpu*                        pcpu_refcnt                                                     
+refcount_t                          dev_refcnt                                                      
+struct_ref_tracker_dir              refcnt_tracker                                                  
+struct_list_head                    link_watch_list                                                 
+enum:8                              reg_state                                                       
+bool                                dismantle                                                       
+enum:16                             rtnl_link_state                                                 
+bool                                needs_free_netdev                                               
+void*priv_destructor                struct_net_device                                               
+struct_netpoll_info*                npinfo                  -                   read_mostly         napi_poll/napi_poll_lock
+possible_net_t                      nd_net                  -                   read_mostly         (dev_net)napi_busy_loop,tcp_v(4/6)_rcv,ip(v6)_rcv,ip(6)_input,ip(6)_input_finish
+void*                               ml_priv                                                         
+enum_netdev_ml_priv_type            ml_priv_type                                                    
+struct_pcpu_lstats__percpu*         lstats                                                          
+struct_pcpu_sw_netstats__percpu*    tstats                                                          
+struct_pcpu_dstats__percpu*         dstats                                                          
+struct_garp_port*                   garp_port                                                       
+struct_mrp_port*                    mrp_port                                                        
+struct_dm_hw_stat_delta*            dm_private                                                      
+struct_device                       dev                     -                   -                   
+struct_attribute_group*             sysfs_groups[4]                                                 
+struct_attribute_group*             sysfs_rx_queue_group                                            
+struct_rtnl_link_ops*               rtnl_link_ops                                                   
+unsigned_int                        gso_max_size            read_mostly         -                   sk_dst_gso_max_size
+unsigned_int                        tso_max_size                                                    
+u16                                 gso_max_segs            read_mostly         -                   gso_max_segs
+u16                                 tso_max_segs                                                    
+unsigned_int                        gso_ipv4_max_size       read_mostly         -                   sk_dst_gso_max_size
+struct_dcbnl_rtnl_ops*              dcbnl_ops                                                       
+s16                                 num_tc                  read_mostly         -                   skb_tx_hash
+struct_netdev_tc_txq                tc_to_txq[16]           read_mostly         -                   skb_tx_hash
+u8                                  prio_tc_map[16]                                                 
+unsigned_int                        fcoe_ddp_xid                                                    
+struct_netprio_map*                 priomap                                                         
+struct_phy_device*                  phydev                                                          
+struct_sfp_bus*                     sfp_bus                                                         
+struct_lock_class_key*              qdisc_tx_busylock                                               
+bool                                proto_down                                                      
+unsigned:1                          wol_enabled                                                     
+unsigned:1                          threaded                -                   -                   napi_poll(napi_enable,dev_set_threaded)
+struct_list_head                    net_notifier_list                                               
+struct_macsec_ops*                  macsec_ops                                                      
+struct_udp_tunnel_nic_info*         udp_tunnel_nic_info                                             
+struct_udp_tunnel_nic*              udp_tunnel_nic                                                  
+unsigned_int                        xdp_zc_max_segs                                                 
+struct_bpf_xdp_entity               xdp_state[3]                                                    
+u8                                  dev_addr_shadow[32]                                             
+netdevice_tracker                   linkwatch_dev_tracker                                           
+netdevice_tracker                   watchdog_dev_tracker                                            
+netdevice_tracker                   dev_registered_tracker                                          
+struct_rtnl_hw_stats64*             offload_xstats_l3                                               
+struct_devlink_port*                devlink_port                                                    
+struct_dpll_pin*                    dpll_pin                                                        
diff --git a/Documentation/networking/net_cachelines/netns_ipv4_sysctl.rst b/Documentation/networking/net_cachelines/netns_ipv4_sysctl.rst
new file mode 100644
index 000000000000..47e1a0105384
--- /dev/null
+++ b/Documentation/networking/net_cachelines/netns_ipv4_sysctl.rst
@@ -0,0 +1,157 @@
+.. SPDX-License-Identifier: GPL-2.0
+.. Copyright (C) 2023 Google LLC
+===========================================
+netns_ipv4 struct fast path usage breakdown
+===========================================
+
+Type                            Name                                         fastpath_tx_access  fastpath_rx_access  comment
+..struct                        ..netns_ipv4                                                                         
+struct_inet_timewait_death_row  tcp_death_row                                                                        
+struct_udp_table*               udp_table                                                                            
+struct_ctl_table_header*        forw_hdr                                                                             
+struct_ctl_table_header*        frags_hdr                                                                            
+struct_ctl_table_header*        ipv4_hdr                                                                             
+struct_ctl_table_header*        route_hdr                                                                            
+struct_ctl_table_header*        xfrm4_hdr                                                                            
+struct_ipv4_devconf*            devconf_all                                                                          
+struct_ipv4_devconf*            devconf_dflt                                                                         
+struct_ip_ra_chain              ra_chain                                                                             
+struct_mutex                    ra_mutex                                                                             
+struct_fib_rules_ops*           rules_ops                                                                            
+struct_fib_table                fib_main                                                                             
+struct_fib_table                fib_default                                                                          
+unsigned_int                    fib_rules_require_fldissect                                                          
+bool                            fib_has_custom_rules                                                                 
+bool                            fib_has_custom_local_routes                                                          
+bool                            fib_offload_disabled                                                                 
+atomic_t                        fib_num_tclassid_users                                                               
+struct_hlist_head*              fib_table_hash                                                                       
+struct_sock*                    fibnl                                                                                
+struct_sock*                    mc_autojoin_sk                                                                       
+struct_inet_peer_base*          peers                                                                                
+struct_fqdir*                   fqdir                                                                                
+u8                              sysctl_icmp_echo_ignore_all                                                          
+u8                              sysctl_icmp_echo_enable_probe                                                        
+u8                              sysctl_icmp_echo_ignore_broadcasts                                                   
+u8                              sysctl_icmp_ignore_bogus_error_responses                                             
+u8                              sysctl_icmp_errors_use_inbound_ifaddr                                                
+int                             sysctl_icmp_ratelimit                                                                
+int                             sysctl_icmp_ratemask                                                                 
+u32                             ip_rt_min_pmtu                               -                   -                   
+int                             ip_rt_mtu_expires                            -                   -                   
+int                             ip_rt_min_advmss                             -                   -                   
+struct_local_ports              ip_local_ports                               -                   -                   
+u8                              sysctl_tcp_ecn                               -                   -                   
+u8                              sysctl_tcp_ecn_fallback                      -                   -                   
+u8                              sysctl_ip_default_ttl                        -                   -                   ip4_dst_hoplimit/ip_select_ttl
+u8                              sysctl_ip_no_pmtu_disc                       -                   -                   
+u8                              sysctl_ip_fwd_use_pmtu                       read_mostly         -                   ip_dst_mtu_maybe_forward/ip_skb_dst_mtu
+u8                              sysctl_ip_fwd_update_priority                -                   -                   ip_forward
+u8                              sysctl_ip_nonlocal_bind                      -                   -                   
+u8                              sysctl_ip_autobind_reuse                     -                   -                   
+u8                              sysctl_ip_dynaddr                            -                   -                   
+u8                              sysctl_ip_early_demux                        -                   read_mostly         ip(6)_rcv_finish_core
+u8                              sysctl_raw_l3mdev_accept                     -                   -                   
+u8                              sysctl_tcp_early_demux                       -                   read_mostly         ip(6)_rcv_finish_core
+u8                              sysctl_udp_early_demux                                                               
+u8                              sysctl_nexthop_compat_mode                   -                   -                   
+u8                              sysctl_fwmark_reflect                        -                   -                   
+u8                              sysctl_tcp_fwmark_accept                     -                   -                   
+u8                              sysctl_tcp_l3mdev_accept                     -                   -                   
+u8                              sysctl_tcp_mtu_probing                       -                   -                   
+int                             sysctl_tcp_mtu_probe_floor                   -                   -                   
+int                             sysctl_tcp_base_mss                          -                   -                   
+int                             sysctl_tcp_min_snd_mss                       read_mostly         -                   __tcp_mtu_to_mss(tcp_write_xmit)
+int                             sysctl_tcp_probe_threshold                   -                   -                   tcp_mtu_probe(tcp_write_xmit)
+u32                             sysctl_tcp_probe_interval                    -                   -                   tcp_mtu_check_reprobe(tcp_write_xmit)
+int                             sysctl_tcp_keepalive_time                    -                   -                   
+int                             sysctl_tcp_keepalive_intvl                   -                   -                   
+u8                              sysctl_tcp_keepalive_probes                  -                   -                   
+u8                              sysctl_tcp_syn_retries                       -                   -                   
+u8                              sysctl_tcp_synack_retries                    -                   -                   
+u8                              sysctl_tcp_syncookies                        -                   -                   generated_on_syn
+u8                              sysctl_tcp_migrate_req                       -                   -                   reuseport
+u8                              sysctl_tcp_comp_sack_nr                      -                   -                   __tcp_ack_snd_check
+int                             sysctl_tcp_reordering                        -                   read_mostly         tcp_may_raise_cwnd/tcp_cong_control
+u8                              sysctl_tcp_retries1                          -                   -                   
+u8                              sysctl_tcp_retries2                          -                   -                   
+u8                              sysctl_tcp_orphan_retries                    -                   -                   
+u8                              sysctl_tcp_tw_reuse                          -                   -                   timewait_sock_ops
+int                             sysctl_tcp_fin_timeout                       -                   -                   TCP_LAST_ACK/tcp_rcv_state_process
+unsigned_int                    sysctl_tcp_notsent_lowat                     read_mostly         -                   tcp_notsent_lowat/tcp_stream_memory_free
+u8                              sysctl_tcp_sack                              -                   -                   tcp_syn_options
+u8                              sysctl_tcp_window_scaling                    -                   -                   tcp_syn_options,tcp_parse_options
+u8                              sysctl_tcp_timestamps                                                                
+u8                              sysctl_tcp_early_retrans                     read_mostly         -                   tcp_schedule_loss_probe(tcp_write_xmit)
+u8                              sysctl_tcp_recovery                          -                   -                   tcp_fastretrans_alert
+u8                              sysctl_tcp_thin_linear_timeouts              -                   -                   tcp_retrans_timer(on_thin_streams)
+u8                              sysctl_tcp_slow_start_after_idle             -                   -                   unlikely(tcp_cwnd_validate-network-not-starved)
+u8                              sysctl_tcp_retrans_collapse                  -                   -                   
+u8                              sysctl_tcp_stdurg                            -                   -                   unlikely(tcp_check_urg)
+u8                              sysctl_tcp_rfc1337                           -                   -                   
+u8                              sysctl_tcp_abort_on_overflow                 -                   -                   
+u8                              sysctl_tcp_fack                              -                   -                   
+int                             sysctl_tcp_max_reordering                    -                   -                   tcp_check_sack_reordering
+int                             sysctl_tcp_adv_win_scale                     -                   -                   tcp_init_buffer_space
+u8                              sysctl_tcp_dsack                             -                   -                   partial_packet_or_retrans_in_tcp_data_queue
+u8                              sysctl_tcp_app_win                           -                   -                   tcp_win_from_space
+u8                              sysctl_tcp_frto                              -                   -                   tcp_enter_loss
+u8                              sysctl_tcp_nometrics_save                    -                   -                   TCP_LAST_ACK/tcp_update_metrics
+u8                              sysctl_tcp_no_ssthresh_metrics_save