upstream/opnsense-src
1
0
Fork 0
mirror of https://github.com/opnsense/src.git synced 2026-07-13 11:11:11 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions
opnsense-src/sys/netinet
History
Hans Petter Selasky e31579b855 Add TCP LRO support for VLAN and VxLAN. 
This change makes the TCP LRO code more generic and flexible with regards
to supporting multiple different TCP encapsulation protocols and in general
lays the ground for broader TCP LRO support. The main job of the TCP LRO code is
to merge TCP packets for the same flow, to reduce the number of calls to upper
layers. This reduces CPU and increases performance, due to being able to send
larger TSO offloaded data chunks at a time. Basically the TCP LRO makes it
possible to avoid per-packet interaction by the host CPU.

Because the current TCP LRO code was tightly bound and optimized for TCP/IP
over ethernet only, several larger changes were needed. Also a minor bug was
fixed in the flushing mechanism for inactive entries, where the expire time,
"le->mtime" was not always properly set.

To avoid having to re-run time consuming regression tests for every change,
it was chosen to squash the following list of changes into a single commit:
- Refactor parsing of all address information into the "lro_parser" structure.
  This easily allows to reuse parsing code for inner headers.
- Speedup header data comparison. Don't compare field by field, but
  instead use an unsigned long array, where the fields get packed.
- Refactor the IPv4/TCP/UDP checksum computations, so that they may be computed
  recursivly, only applying deltas as the result of updating payload data.
- Make smaller inline functions doing one operation at a time instead of
  big functions having repeated code.
- Refactor the TCP ACK compression code to only execute once
  per TCP LRO flush. This gives a minor performance improvement and
  keeps the code simple.
- Use sbintime() for all time-keeping. This change also fixes flushing
  of inactive entries.
- Try to shrink the size of the LRO entry, because it is frequently zeroed.
- Removed unused TCP LRO macros.
- Cleanup unused TCP LRO statistics counters while at it.
- Try to use __predict_true() and predict_false() to optimise CPU branch
  predictions.

Bump the __FreeBSD_version due to adding new member to the "lro_ctrl" structure.

Tested by:	Netflix
Reviewed by:	rrs (transport)
Differential Revision:	https://reviews.freebsd.org/D29564
Sponsored by:	Mellanox Technologies // NVIDIA Networking

(cherry picked from commit 9ca874cf74)
2021-06-02 13:55:45 +02:00
..
cc tcp: Use local CC data only in the correct context 2021-05-31 08:56:34 +02:00
khelp
libalias libalias: replace placeholder with static constant 2021-05-30 15:49:14 +02:00
netdump Use zfree() instead of explicit_bzero() and free(). 2020-06-25 20:17:34 +00:00
tcp_stacks net: Introduce IPV6_DSCP(), IPV6_ECN() and IPV6_TRAFFIC_CLASS() macros 2021-05-10 16:30:44 +02:00
accf_data.c Define a module version for accept filter modules. 2020-05-19 18:35:08 +00:00
accf_dns.c Define a module version for accept filter modules. 2020-05-19 18:35:08 +00:00
accf_http.c net: clean up empty lines in .c and .h files 2020-09-01 21:19:14 +00:00
dccp.h Add header definition for RFC4340, Datagram Congestion Control Protocol 2020-06-17 13:27:13 +00:00
icmp6.h icmp6: Count packets dropped due to an invalid hop limit 2020-10-19 17:07:19 +00:00
icmp_var.h net: clean up empty lines in .c and .h files 2020-09-01 21:19:14 +00:00
if_ether.c arp/nd: Cope with late calls to iflladdr_event 2021-03-02 15:50:21 +01:00
if_ether.h
igmp.c igmp: Avoid an out-of-bounds access when zeroing counters 2021-05-12 09:31:39 -04:00
igmp.h
igmp_var.h igmp: convert igmpstat to use PCPU counters 2020-11-08 18:49:23 +00:00
in.c Make in_localip_more() fib-aware. 2021-03-10 21:47:39 +00:00
in.h Add IP(V6)_VLAN_PCP to set 802.1 priority per-flow. 2020-10-09 12:06:43 +00:00
in_cksum.c
in_debug.c
in_fib.c Add modular fib lookup framework. 2020-12-25 11:33:17 +00:00
in_fib.h Refactor fib4/fib6 functions. 2020-11-29 13:41:49 +00:00
in_fib_algo.c Fix IPv4 fib bsearch4() lookup array construction. 2021-01-17 20:32:26 +00:00
in_gif.c net: clean up empty lines in .c and .h files 2020-09-01 21:19:14 +00:00
in_jail.c
in_kdtrace.c Separate out SCTP related dtrace code. 2019-10-14 20:32:11 +00:00
in_kdtrace.h Separate out SCTP related dtrace code. 2019-10-14 20:32:11 +00:00
in_mcast.c Always use inp fib in the inp_lookup_mcast_ifp(). 2021-04-29 08:47:31 +00:00
in_pcb.c Add missing sockaddr length and family validation to various protocols 2021-05-17 13:43:07 -04:00
in_pcb.h Add TCP LRO support for VLAN and VxLAN. 2021-06-02 13:55:45 +02:00
in_pcbgroup.c
in_prot.c
in_proto.c Remove unused nhop_ref_any() function. 2020-09-20 21:32:52 +00:00
in_rmx.c Refactor rib iterator functions. 2020-11-22 20:21:10 +00:00
in_rss.c Implement flowid calculation for outbound connections to balance 2020-10-18 17:15:47 +00:00
in_rss.h Implement flowid calculation for outbound connections to balance 2020-10-18 17:15:47 +00:00
in_systm.h
in_var.h Further refactor IPv4 interface route creation. 2021-01-21 21:48:49 +00:00
ip.h
ip6.h net: Introduce IPV6_DSCP(), IPV6_ECN() and IPV6_TRAFFIC_CLASS() macros 2021-05-10 16:30:44 +02:00
ip_carp.c Further refactor IPv4 interface route creation. 2021-01-21 21:48:49 +00:00
ip_carp.h carp: replace caddr_t with char * 2019-12-06 16:35:48 +00:00
ip_divert.c Fix mbuf leaks in various pru_send implementations 2021-05-25 21:49:53 -04:00
ip_divert.h
ip_dummynet.h net: clean up empty lines in .c and .h files 2020-09-01 21:19:14 +00:00
ip_ecn.c
ip_ecn.h
ip_encap.c Widen NET_EPOCH coverage. 2019-10-07 22:40:05 +00:00
ip_encap.h
ip_fastfwd.c ip_fastfwd: style(9) tidy for r367628 2020-11-13 18:25:07 +00:00
ip_fw.h Allow setting alias port ranges in libalias and ipfw. 2021-02-17 10:13:54 -08:00
ip_gre.c Introduce NET_EPOCH_CALL() macro and use it everywhere where we free 2020-01-15 06:05:20 +00:00
ip_icmp.c net: clean up empty lines in .c and .h files 2020-09-01 21:19:14 +00:00
ip_icmp.h
ip_id.c Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many) 2020-02-26 14:26:36 +00:00
ip_input.c Flush remaining routes from the routing table during VNET shutdown. 2021-03-13 20:19:17 +00:00
ip_mroute.c ip_mroute: fix the viftable export sysctl 2020-10-11 00:01:00 +00:00
ip_mroute.h net: clean up empty lines in .c and .h files 2020-09-01 21:19:14 +00:00
ip_options.c net: clean up empty lines in .c and .h files 2020-09-01 21:19:14 +00:00
ip_options.h
ip_output.c Catch up with 6edfd179c8: mechanically rename IFCAP_NOMAP to IFCAP_MEXTPG. 2021-02-08 14:33:35 -08:00
ip_reass.c Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many) 2020-02-26 14:26:36 +00:00
ip_var.h An earlier commit effectively turned out the fast forwading path 2020-11-12 21:58:47 +00:00
pim.h
pim_var.h
raw_ip.c Fix mbuf leaks in various pru_send implementations 2021-05-25 21:49:53 -04:00
sctp.h Improve the handling of cookie life times. 2020-10-16 10:44:48 +00:00
sctp_asconf.c Improve input validation for parameters in ASCONF and ASCONF-ACK chunks 2020-12-23 18:03:47 +01:00
sctp_asconf.h net: clean up empty lines in .c and .h files 2020-09-01 21:19:14 +00:00
sctp_auth.c net: clean up empty lines in .c and .h files 2020-09-01 21:19:14 +00:00
sctp_auth.h net: clean up empty lines in .c and .h files 2020-09-01 21:19:14 +00:00
sctp_bsd_addr.c Don't pass RFPROC to kproc_create(), it is redundant. 2021-03-29 11:10:49 -07:00
sctp_bsd_addr.h net: clean up empty lines in .c and .h files 2020-09-01 21:19:14 +00:00
sctp_cc_functions.c Minor cleanups. 2020-10-07 15:22:48 +00:00
sctp_constants.h net: clean up empty lines in .c and .h files 2020-09-01 21:19:14 +00:00
sctp_crc32.c No need to include netinet/sctp_crc32.h twice. 2020-06-22 14:36:14 +00:00
sctp_crc32.h Add the SCTP_SUPPORT kernel option. 2020-06-18 19:32:34 +00:00
sctp_header.h Whitespace changes. 2020-09-24 12:26:06 +00:00
sctp_indata.c sctp: fix a locking issue for old unordered data 2021-03-02 13:28:09 +01:00
sctp_indata.h net: clean up empty lines in .c and .h files 2020-09-01 21:19:14 +00:00
sctp_input.c sctp: improve consistency 2021-03-02 13:22:40 +01:00
sctp_input.h net: clean up empty lines in .c and .h files 2020-09-01 21:19:14 +00:00
sctp_kdtrace.c Separate out SCTP related dtrace code. 2019-10-14 20:32:11 +00:00
sctp_kdtrace.h Separate out SCTP related dtrace code. 2019-10-14 20:32:11 +00:00
sctp_lock_bsd.h Whitespace changes. 2020-09-24 12:26:06 +00:00
sctp_module.c Provide support for building SCTP as a loadable module. 2020-07-10 14:56:05 +00:00
sctp_os.h net: clean up empty lines in .c and .h files 2020-09-01 21:19:14 +00:00
sctp_os_bsd.h net: clean up empty lines in .c and .h files 2020-09-01 21:19:14 +00:00
sctp_output.c sctp: improve input validation 2021-03-02 13:29:00 +01:00
sctp_output.h Whitespace changes. 2020-09-24 12:26:06 +00:00
sctp_pcb.c Don't pass RFPROC to kproc_create(), it is redundant. 2021-03-29 11:10:49 -07:00
sctp_pcb.h net: clean up empty lines in .c and .h files 2020-09-01 21:19:14 +00:00
sctp_peeloff.c Non-functional changes due to upstream cleanup. 2020-06-11 13:34:09 +00:00
sctp_peeloff.h
sctp_ss_functions.c Fix a few typos in comments 2021-03-20 11:13:01 +01:00
sctp_structs.h sctp: improve consistency 2021-03-02 13:22:40 +01:00
sctp_syscalls.c Convert remaining cap_rights_init users to cap_rights_init_one 2021-01-12 13:16:10 +00:00
sctp_sysctl.c net: clean up empty lines in .c and .h files 2020-09-01 21:19:14 +00:00
sctp_sysctl.h Improve the handling of cookie life times. 2020-10-16 10:44:48 +00:00
sctp_timer.c sctp: improve computation of an alternate net 2021-03-02 13:32:21 +01:00
sctp_timer.h net: clean up empty lines in .c and .h files 2020-09-01 21:19:14 +00:00
sctp_uio.h net: clean up empty lines in .c and .h files 2020-09-01 21:19:14 +00:00
sctp_usrreq.c Add missing sockaddr length and family validation to various protocols 2021-05-17 13:43:07 -04:00
sctp_var.h net: clean up empty lines in .c and .h files 2020-09-01 21:19:14 +00:00
sctputil.c sctp: avoid integer overflow when starting the HB timer 2021-03-02 13:33:01 +01:00
sctputil.h net: clean up empty lines in .c and .h files 2020-09-01 21:19:14 +00:00
siftr.c net: clean up empty lines in .c and .h files 2020-09-01 21:19:14 +00:00
tcp.h Filter TCP connections to SO_REUSEPORT_LB listen sockets by NUMA domain 2020-12-19 22:04:46 +00:00
tcp_debug.c
tcp_debug.h
tcp_fastopen.c net: clean up empty lines in .c and .h files 2020-09-01 21:19:14 +00:00
tcp_fastopen.h
tcp_fsm.h White space cleanup -- remove trailing tab's or spaces 2020-02-12 13:31:36 +00:00
tcp_hostcache.c tcp: Use jenkins_hash32() in hostcache 2021-04-16 22:44:03 +02:00
tcp_hostcache.h tcp: Use jenkins_hash32() in hostcache 2021-04-16 22:44:03 +02:00
tcp_hpts.c net: clean up empty lines in .c and .h files 2020-09-01 21:19:14 +00:00
tcp_hpts.h net: clean up empty lines in .c and .h files 2020-09-01 21:19:14 +00:00
tcp_input.c net: Introduce IPV6_DSCP(), IPV6_ECN() and IPV6_TRAFFIC_CLASS() macros 2021-05-10 16:30:44 +02:00
tcp_log_buf.c net: clean up empty lines in .c and .h files 2020-09-01 21:19:14 +00:00
tcp_log_buf.h net: clean up empty lines in .c and .h files 2020-09-01 21:19:14 +00:00
tcp_lro.c Add TCP LRO support for VLAN and VxLAN. 2021-06-02 13:55:45 +02:00
tcp_lro.h Add TCP LRO support for VLAN and VxLAN. 2021-06-02 13:55:45 +02:00
tcp_offload.c Initial support for kernel offload of TLS receive. 2020-04-27 23:17:19 +00:00
tcp_offload.h Initial support for kernel offload of TLS receive. 2020-04-27 23:17:19 +00:00
tcp_output.c [tcp] Fix ECN on finalizing sessions. 2021-04-11 11:18:35 +02:00
tcp_pcap.c Step 4.2: start divorce of M_EXT and M_EXTPG 2020-05-03 00:37:16 +00:00
tcp_pcap.h
tcp_ratelimit.c Add m_snd_tag_alloc() as a wrapper around if_snd_tag_alloc(). 2020-10-29 23:28:39 +00:00
tcp_ratelimit.h Fix copyright year and eliminate the obsolete all rights reserved line. 2020-04-08 17:55:45 +00:00
tcp_reass.c Prevent premature SACK block transmission during loss recovery 2020-11-08 18:47:05 +00:00
tcp_sack.c tcp: stop spurious rescue retransmissions and potential asserts 2021-05-01 10:26:53 +02:00
tcp_seq.h
tcp_stats.c Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many) 2020-02-26 14:26:36 +00:00
tcp_subr.c Add TCP LRO support for VLAN and VxLAN. 2021-06-02 13:55:45 +02:00
tcp_syncache.c TCP_NOOPT may prevent the negotiation of TCP features, which a server 2021-02-28 09:11:18 +01:00
tcp_syncache.h Fix the following issues related to the TCP SYN-cache: 2020-08-10 20:24:48 +00:00
tcp_timer.c Improve the TCP blackhole detection. The principle is to reduce the 2020-04-14 16:35:05 +00:00
tcp_timer.h 504ebd612e: kern: sonewconn: set so_options before pru_attach() 2021-02-23 22:48:42 -06:00
tcp_timewait.c tcp: add sysctl to tolerate TCP segments missing timestamps 2021-01-14 19:28:25 +01:00
tcp_usrreq.c Fix mbuf leaks in various pru_send implementations 2021-05-25 21:49:53 -04:00
tcp_var.h Add TCP LRO support for VLAN and VxLAN. 2021-06-02 13:55:45 +02:00
tcpip.h
toecore.c net: clean up empty lines in .c and .h files 2020-09-01 21:19:14 +00:00
toecore.h Initial support for kernel offload of TLS receive. 2020-04-27 23:17:19 +00:00
udp.h White space cleanup -- remove trailing tab's or spaces 2020-02-12 13:31:36 +00:00
udp_usrreq.c Fix mbuf leaks in various pru_send implementations 2021-05-25 21:49:53 -04:00
udp_var.h Add a knob to allow zero UDP checksums for UDP/IPv6 traffic on the given UDP port. 2020-09-18 02:21:15 +00:00
udplite.h White space cleanup -- remove trailing tab's or spaces 2020-02-12 13:31:36 +00:00