opnsense-src/sys/netinet
Hans Petter Selasky e936121d31 Add optimizing LRO wrapper:
- Add optimizing LRO wrapper which pre-sorts all incoming packets
  according to the hash type and flowid. This prevents exhaustion of
  the LRO entries due to too many connections at the same time.
  Testing using a larger number of higher bandwidth TCP connections
  showed that the incoming ACK packet aggregation rate increased from
  ~1.3:1 to almost 3:1. Another test showed that for a number of TCP
  connections greater than 16 per hardware receive ring, where 8 TCP
  connections was the LRO active entry limit, there was a significant
  improvement in throughput due to being able to fully aggregate more
  than 8 TCP stream. For very few very high bandwidth TCP streams, the
  optimizing LRO wrapper will add CPU usage instead of reducing CPU
  usage. This is expected. Network drivers which want to use the
  optimizing LRO wrapper needs to call "tcp_lro_queue_mbuf()" instead
  of "tcp_lro_rx()" and "tcp_lro_flush_all()" instead of
  "tcp_lro_flush()". Further the LRO control structure must be
  initialized using "tcp_lro_init_args()" passing a non-zero number
  into the "lro_mbufs" argument.

- Make LRO statistics 64-bit. Previously 32-bit integers were used for
  statistics which can be prone to wrap-around. Fix this while at it
  and update all SYSCTL's which expose LRO statistics.

- Ensure all data is freed when destroying a LRO control structures,
  especially leftover LRO entries.

- Reduce number of memory allocations needed when setting up a LRO
  control structure by precomputing the total amount of memory needed.

- Add own memory allocation counter for LRO.

- Bump the FreeBSD version to force recompilation of all KLDs due to
  change of the LRO control structure size.

Sponsored by:	Mellanox Technologies
Reviewed by:	gallatin, sbruno, rrs, gnn, transport
Tested by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D4914
2016-01-19 15:33:28 +00:00
..
cc Add an option to use rfc6675 based pipe/inflight bytes calculation in newreno. 2015-12-09 08:53:41 +00:00
khelp The TCP PAWS fix for kernels with fast tick rates (r231767) changed the TCP 2012-08-17 01:49:51 +00:00
libalias mdoc: fix rendering issues 2015-04-26 11:39:25 +00:00
tcp_stacks Apply the changes from r293284 to one additional file. 2016-01-07 11:54:20 +00:00
accf_data.c Rework socket upcalls to close some races with setup/teardown of upcalls. 2009-06-01 21:17:03 +00:00
accf_dns.c In preparation of merging projects/sendfile, transform bare access to 2014-11-12 09:57:15 +00:00
accf_http.c In preparation of merging projects/sendfile, transform bare access to 2014-11-12 09:57:15 +00:00
cc.h DCTCP (Data Center TCP) implementation. 2015-01-12 08:33:04 +00:00
icmp6.h Implement Enhanced DAD algorithm for IPv6 described in 2015-03-02 17:30:26 +00:00
icmp_var.h Remove more constants related to static sysctl nodes. The MAXID constants 2014-02-25 18:44:33 +00:00
if_atm.c The r48589 promised to remove implicit inclusion of if_var.h soon. Prepare 2013-10-26 17:58:36 +00:00
if_atm.h Add const qualifier to the dst parameter of the ifnet if_output method. 2013-04-26 12:50:32 +00:00
if_ether.c Remove second EVENTHANDLER_REGISTER slipped in r292978. 2016-01-01 10:15:06 +00:00
if_ether.h Implement interface link header precomputation API. 2015-12-31 05:03:27 +00:00
igmp.c In the same way fix the problem described in r291578 for IGMPv3. 2015-12-01 11:24:30 +00:00
igmp.h These are no longer referenced in the tree, so can be safely removed. 2009-06-10 18:12:15 +00:00
igmp_var.h - Rename 'struct igmp_ifinfo' into 'struct igmp_ifsoftc', since it really 2015-02-19 22:35:23 +00:00
in.c Add rib_lookup_info() to provide API for retrieving individual route 2016-01-04 15:03:20 +00:00
in.h Provide functions to determine presence of a given address 2015-04-17 11:57:06 +00:00
in_cksum.c nobody uses this file except the userspace ipfw code, but the cast 2012-07-31 08:04:49 +00:00
in_debug.c Remove last remnants of classful addressing: 2011-10-15 16:28:06 +00:00
in_fib.c Bring RADIX_MPATH support to new routing KPI to ease migration. 2016-01-11 08:45:28 +00:00
in_fib.h Merge helper fib* functions used for basic lookups. 2015-12-08 10:50:03 +00:00
in_gif.c Merge helper fib* functions used for basic lookups. 2015-12-08 10:50:03 +00:00
in_kdtrace.c Fix style issues around existing SDT probes. 2015-12-16 23:39:27 +00:00
in_kdtrace.h Fix style issues around existing SDT probes. 2015-12-16 23:39:27 +00:00
in_mcast.c Make in_arpinput(), inp_lookup_mcast_ifp(), icmp_reflect(), 2015-12-09 11:14:27 +00:00
in_pcb.c Remove sys/eventhandler.h from net/route.h 2016-01-09 09:34:39 +00:00
in_pcb.h Use Jenkins hash for TCP syncache. 2015-09-05 10:15:19 +00:00
in_pcbgroup.c Refactor / restructure the RSS code into generic, IPv4 and IPv6 specific 2015-01-18 18:06:40 +00:00
in_proto.c Finish r274118: remove useless fields from struct domain. 2014-11-06 14:39:04 +00:00
in_rmx.c Remove now-unused wrappers for various routing functions. 2016-01-14 08:54:44 +00:00
in_rss.c Rename rss_soft_m2cpuid() -> rss_soft_m2cpuid_v4() in preparation for 2015-08-29 06:58:30 +00:00
in_rss.h Rename rss_soft_m2cpuid() -> rss_soft_m2cpuid_v4() in preparation for 2015-08-29 06:58:30 +00:00
in_systm.h Where appropriate, use the modern terms for the one true time base 2014-12-21 05:07:11 +00:00
in_var.h Remove now-unused wrappers for various routing functions. 2016-01-14 08:54:44 +00:00
ip.h Since the IETF has redefined the meaning of the tos field to accommodate 2015-07-18 06:48:30 +00:00
ip6.h Eliminate use of M_EXT in IP6_EXTHDR_CHECK() by trimming a redundant 2014-10-05 06:28:53 +00:00
ip_carp.c Revert r292275 & r292379 2015-12-17 14:41:30 +00:00
ip_carp.h Merge 'struct ip6protosw' and 'struct protosw' into one. Now we have 2014-08-08 01:57:15 +00:00
ip_divert.c Convert netinet6/ to use new routing API. 2014-11-04 15:39:56 +00:00
ip_divert.h Various cleanup done in ipfw3-head branch including: 2010-01-04 19:01:22 +00:00
ip_dummynet.h ECN marking implenetation for dummynet. 2014-06-01 07:28:24 +00:00
ip_ecn.c Add FBSDID to all files in netinet so that people can more 2007-10-07 20:44:24 +00:00
ip_ecn.h Remove unneded #include "opt_inet.h". 2015-07-31 09:02:28 +00:00
ip_encap.c Remove sys/eventhandler.h from net/route.h 2016-01-09 09:34:39 +00:00
ip_encap.h Merge 'struct ip6protosw' and 'struct protosw' into one. Now we have 2014-08-08 01:57:15 +00:00
ip_fastfwd.c Replace the fastforward path with tryforward which does not require a 2015-11-05 07:26:32 +00:00
ip_fw.h Implement ipfw internal olist command to list named objects. 2015-11-03 10:21:53 +00:00
ip_gre.c o Use new function ip_fillid() in all places throughout the kernel, 2015-04-01 22:26:39 +00:00
ip_icmp.c Make in_arpinput(), inp_lookup_mcast_ifp(), icmp_reflect(), 2015-12-09 11:14:27 +00:00
ip_icmp.h Where appropriate, use the modern terms for the one true time base 2014-12-21 05:07:11 +00:00
ip_id.c Provide a comment explaining issues with the counter(9) trick, so that 2015-04-02 14:22:59 +00:00
ip_input.c Make in_arpinput(), inp_lookup_mcast_ifp(), icmp_reflect(), 2015-12-09 11:14:27 +00:00
ip_ipsec.c Set the proper direction to check for policies in this one case. 2015-10-29 21:26:32 +00:00
ip_ipsec.h Remove flag/flags argument from the following functions: 2014-12-11 18:35:34 +00:00
ip_mroute.c Remove now-unused wrappers for various routing functions. 2016-01-14 08:54:44 +00:00
ip_mroute.h Migrate structs arpstat, icmpstat, mrtstat, pimstat and udpstat to PCPU 2013-07-09 09:50:15 +00:00
ip_options.c Make in_arpinput(), inp_lookup_mcast_ifp(), icmp_reflect(), 2015-12-09 11:14:27 +00:00
ip_options.h Make net.inet.ip.sourceroute, net.inet.ip.accept_sourceroute, and 2014-09-15 07:20:40 +00:00
ip_output.c Finish r275196: do not dereference rtentry in if_output() routines. 2016-01-09 16:34:37 +00:00
ip_reass.c Fix RSS build - netisr input / NETISR_IP_DIRECT is used here. 2015-04-15 00:57:21 +00:00
ip_var.h Make in_arpinput(), inp_lookup_mcast_ifp(), icmp_reflect(), 2015-12-09 11:14:27 +00:00
pim.h
pim_var.h Merge 'struct ip6protosw' and 'struct protosw' into one. Now we have 2014-08-08 01:57:15 +00:00
raw_ip.c Remove sys/eventhandler.h from net/route.h 2016-01-09 09:34:39 +00:00
sctp.h Cleanup the handling of error causes for ERROR chunks. This fixes 2015-09-12 17:08:51 +00:00
sctp_asconf.c Provide consistent error causes whenever an ABORT chunk is sent. 2015-07-27 22:35:54 +00:00
sctp_asconf.h Pass the src and dst address of a received packet explicitly around. 2012-06-28 16:01:08 +00:00
sctp_auth.c Use __func__ instead of __FUNCTION__. 2015-10-19 11:17:54 +00:00
sctp_auth.h Use a consistent type for the number of HMAC algorithms. 2014-09-16 14:20:33 +00:00
sctp_bsd_addr.c Minimize the usage of SCTP_BUF_IS_EXTENDED. 2015-01-10 20:49:57 +00:00
sctp_bsd_addr.h Do packet logging in a consistent way. 2012-06-24 21:25:54 +00:00
sctp_cc_functions.c Fix style issues around existing SDT probes. 2015-12-16 23:39:27 +00:00
sctp_constants.h Ensure that ERROR chunks are always padded by implementing this 2015-09-11 13:54:33 +00:00
sctp_crc32.c Use consistent text at the begining of the files. 2012-05-23 11:26:28 +00:00
sctp_crc32.h Use consistent text at the begining of the files. 2012-05-23 11:26:28 +00:00
sctp_dtrace_declare.h - For kernel compiled only with KDTRACE_HOOKS and not any lock debugging 2013-11-25 07:38:45 +00:00
sctp_dtrace_define.h dtrace sdt: remove the ugly sname parameter of SDT_PROBE_DEFINE 2013-11-26 08:46:27 +00:00
sctp_header.h Cleanup the handling of error causes for ERROR chunks. This fixes 2015-09-12 17:08:51 +00:00
sctp_indata.c Stop processing of a SACK when the association has been aborted. 2015-12-21 18:52:02 +00:00
sctp_indata.h Don't take the port numbers for packets containing ABORT chunks from 2015-08-02 16:07:30 +00:00
sctp_input.c Retire sctp_validate_no_locks(). 2015-12-10 11:49:32 +00:00
sctp_input.h Add FIB support for SCTP. 2015-06-17 15:20:14 +00:00
sctp_lock_bsd.h Use consistent text at the begining of the files. 2012-05-23 11:26:28 +00:00
sctp_os.h Use consistent text at the begining of the files. 2012-05-23 11:26:28 +00:00
sctp_os_bsd.h Replace sys/crypto/sha2/sha2.c with lib/libmd/sha512c.c 2015-12-27 17:33:59 +00:00
sctp_output.c Don't implicitly terminate a user message when moving it to the 2015-12-25 18:11:40 +00:00
sctp_output.h Fix an issue with MAC OS locking and also optimize the case 2015-07-24 14:09:03 +00:00
sctp_pcb.c Fix a bug in INIT handling on accepted 1-to-1 style sockets when the 2016-01-15 00:26:15 +00:00
sctp_pcb.h Retire sctp_validate_no_locks(). 2015-12-10 11:49:32 +00:00
sctp_peeloff.c Add a SCTP socket option to limit the cwnd for each path. 2015-03-10 19:49:25 +00:00
sctp_peeloff.h Remove unused function. 2012-11-25 14:25:08 +00:00
sctp_ss_functions.c Use consistent text at the begining of the files. 2012-05-23 11:26:28 +00:00
sctp_structs.h Get struct sctp_net_route in sync with struct route again. 2016-01-04 20:34:40 +00:00
sctp_syscalls.c Fix two KTRACE related bugs. 2015-06-19 21:55:12 +00:00
sctp_sysctl.c Fix the exporting of SCTP association states to userland. Without this, 2015-08-29 09:14:32 +00:00
sctp_sysctl.h Remove redundant 'man page' 2015-09-15 21:16:45 +00:00
sctp_timer.c Ensure that outgoing streams get reset when they run dry. 2015-12-03 15:19:29 +00:00
sctp_timer.h Use consistent text at the begining of the files. 2012-05-23 11:26:28 +00:00
sctp_uio.h The field sinfo_timetolive should have been sinfo_pr_value. 2015-11-06 14:00:26 +00:00
sctp_usrreq.c Fail the SCTP_GET_ASSOC_NUMBER and SCTP_GET_ASSOC_ID_LIST 2016-01-14 11:25:28 +00:00
sctp_var.h Stop the heartbeat timer when removing a net. 2015-06-14 17:48:44 +00:00
sctputil.c Store the timer type for logging, because the timer can be freed 2016-01-13 14:28:12 +00:00
sctputil.h Fix the allocation of outgoing streams: 2015-12-06 16:17:57 +00:00
siftr.c Move the SIFTR DTrace probe out of the writing thread context 2015-04-30 17:43:40 +00:00
tcp.h Implementation of server-side TCP Fast Open (TFO) [RFC7413]. 2015-12-24 19:09:48 +00:00
tcp_debug.c Switch the entire IPv4 stack to keep the IP packet header 2012-10-22 21:09:03 +00:00
tcp_debug.h Use uint32_t instead of n_long and n_time, and uint16_t instead of n_short. 2009-02-13 15:14:43 +00:00
tcp_fastopen.c Implementation of server-side TCP Fast Open (TFO) [RFC7413]. 2015-12-24 19:09:48 +00:00
tcp_fastopen.h Implementation of server-side TCP Fast Open (TFO) [RFC7413]. 2015-12-24 19:09:48 +00:00
tcp_fsm.h
tcp_hostcache.c Clean up unused bandwidth entry in the TCP hostcache. 2015-12-11 06:22:58 +00:00
tcp_hostcache.h Clean up unused bandwidth entry in the TCP hostcache. 2015-12-11 06:22:58 +00:00
tcp_input.c Historically we have two fields in tcpcb to describe sender MSS: t_maxopd, 2016-01-07 00:14:42 +00:00
tcp_lro.c Add optimizing LRO wrapper: 2016-01-19 15:33:28 +00:00
tcp_lro.h Add optimizing LRO wrapper: 2016-01-19 15:33:28 +00:00
tcp_offload.c Convert netinet6/ to use new routing API. 2014-11-04 15:39:56 +00:00
tcp_offload.h - Updated TOE support in the kernel. 2012-06-19 07:34:13 +00:00
tcp_output.c There is a bug in tcp_output()'s implementation of the TCP_SIGNATURE 2016-01-14 10:22:45 +00:00
tcp_pcap.c Hopefully also unbreak VIMAGE kernels replacing the &V_... with 2015-10-15 01:44:32 +00:00
tcp_pcap.h There are times when it would be really nice to have a record of the last few 2015-10-14 00:35:37 +00:00
tcp_reass.c Remove sys/eventhandler.h from net/route.h 2016-01-09 09:34:39 +00:00
tcp_sack.c First cut of the modularization of our TCP stack. Still 2015-12-16 00:56:45 +00:00
tcp_seq.h Fix PAWS (Protect Against Wrapped Sequence numbers) in cases when 2012-02-15 16:09:56 +00:00
tcp_subr.c Remove sys/eventhandler.h from net/route.h 2016-01-09 09:34:39 +00:00
tcp_syncache.c Implementation of server-side TCP Fast Open (TFO) [RFC7413]. 2015-12-24 19:09:48 +00:00
tcp_syncache.h Implementation of server-side TCP Fast Open (TFO) [RFC7413]. 2015-12-24 19:09:48 +00:00
tcp_timer.c Historically we have two fields in tcpcb to describe sender MSS: t_maxopd, 2016-01-07 00:14:42 +00:00
tcp_timer.h Put r284245 back in place: If at first this fix was seen as a temporary 2015-08-30 13:44:39 +00:00
tcp_timewait.c Decompose TCP INP_INFO lock to increase short-lived TCP connections scalability: 2015-08-03 12:13:54 +00:00
tcp_usrreq.c Historically we have two fields in tcpcb to describe sender MSS: t_maxopd, 2016-01-07 00:14:42 +00:00
tcp_var.h Historically we have two fields in tcpcb to describe sender MSS: t_maxopd, 2016-01-07 00:14:42 +00:00
tcpip.h
toecore.c (Temporarily) remove route_redirect_event eventhandler. 2016-01-09 06:26:40 +00:00
toecore.h * Convert TOE framework to use new routing api. 2014-10-25 18:25:00 +00:00
udp.h Add placeholder constants to reserve a portion of the socket option 2013-02-01 15:32:20 +00:00
udp_usrreq.c Fixed comment placement. 2015-12-24 13:57:43 +00:00
udp_var.h get_inpcbinfo() and get_pcblist() are UDP local functions and 2015-08-27 15:27:41 +00:00
udplite.h Add support for UDP-Lite protocol (RFC 3828) to IPv4 and IPv6 stacks. 2014-04-07 01:53:03 +00:00