Commit graph

384 commits

Author SHA1 Message Date
Hans Petter Selasky
57a865f808 Implement the __sg_alloc_table_from_pages() function based on the existing
sg_alloc_table_from_pages() function in the LinuxKPI.

This basically allow segments to have a limit, max_segment.

Submitted by:	Johannes Lundberg <johalun0@gmail.com>
MFC after:	1 week
Sponsored by:	Mellanox Technologies
Sponsored by:	Limelight Networks
2018-06-01 12:09:07 +00:00
Hans Petter Selasky
6fad8d171a Implement radix_tree_iter_delete() in the LinuxKPI.
Submitted by:	Johannes Lundberg <johalun0@gmail.com>
MFC after:	1 week
Sponsored by:	Mellanox Technologies
Sponsored by:	Limelight Networks
2018-06-01 11:42:09 +00:00
Hans Petter Selasky
0a85496223 Improve high resolution timer support in the LinuxKPI.
Submitted by:	Johannes Lundberg <johalun0@gmail.com>
MFC after:	1 week
Sponsored by:	Mellanox Technologies
Sponsored by:	Limelight Networks
2018-06-01 11:33:14 +00:00
Hans Petter Selasky
f03ae7e802 Add more GFP macro definitions in the LinuxKPI.
Submitted by:	Johannes Lundberg <johalun0@gmail.com>
MFC after:	1 week
Sponsored by:	Mellanox Technologies
Sponsored by:	Limelight Networks
2018-06-01 11:14:59 +00:00
Hans Petter Selasky
13a5c70b91 Implement support for the PCI_BUS_NUM() function macro in the LinuxKPI.
Submitted by:	Johannes Lundberg <johalun0@gmail.com>
MFC after:	1 week
Sponsored by:	Mellanox Technologies
2018-05-31 13:17:34 +00:00
Hans Petter Selasky
f1aa567bfe Implement support for the kvmalloc_array() function in the LinuxKPI.
Submitted by:	Johannes Lundberg <johalun0@gmail.com>
MFC after:	1 week
Sponsored by:	Mellanox Technologies
2018-05-31 13:13:08 +00:00
Hans Petter Selasky
f6d4552417 Correct macroname in the LinuxKPI.
Submitted by:	Johannes Lundberg <johalun0@gmail.com>
MFC after:	1 week
Sponsored by:	Mellanox Technologies
2018-05-31 12:55:38 +00:00
Hans Petter Selasky
cbea4f294f Define __initconst in the LinuxKPI.
Submitted by:	Johannes Lundberg <johalun0@gmail.com>
MFC after:	1 week
Sponsored by:	Mellanox Technologies
2018-05-31 12:50:42 +00:00
Hans Petter Selasky
7ce7605ece Implement bitmap_complement() in the LinuxKPI.
Submitted by:	Johannes Lundberg <johalun0@gmail.com>
MFC after:	1 week
Sponsored by:	Mellanox Technologies
2018-05-31 12:48:24 +00:00
Hans Petter Selasky
69d6653ba4 Implement idr_is_empty() in the LinuxKPI and make idr_remove() API compatible
with upstream Linux by returning the pointer to the removed element.

Submitted by:	Johannes Lundberg <johalun0@gmail.com>
MFC after:	1 week
Sponsored by:	Mellanox Technologies
2018-05-31 12:35:21 +00:00
Hans Petter Selasky
666cda9e0c The schedule_timeout_killable() function should listen for signals
in the LinuxKPI.

Found by:	Johannes Lundberg <johalun0@gmail.com>
MFC after:	1 week
Sponsored by:	Mellanox Technologies
2018-05-28 11:26:40 +00:00
Hans Petter Selasky
bd40dea7e2 Implement wait_event_killable() in the LinuxKPI.
Requested by:	Johannes Lundberg <johalun0@gmail.com>
MFC after:	1 week
Sponsored by:	Mellanox Technologies
2018-05-28 10:54:24 +00:00
Hans Petter Selasky
2a3ec12831 Allow TASK_PARKED bit being set when going to sleep in the LinuxKPI.
Found by:	Johannes Lundberg <johalun0@gmail.com>
MFC after:	1 week
Sponsored by:	Mellanox Technologies
2018-05-28 10:51:39 +00:00
Matt Macy
4f6c66cc9c UDP: further performance improvements on tx
Cumulative throughput while running 64
  netperf -H $DUT -t UDP_STREAM -- -m 1
on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps

Single stream throughput increases from 910kpps to 1.18Mpps

Baseline:
https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg

- Protect read access to global ifnet list with epoch
https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg

- Protect short lived ifaddr references with epoch
https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg

- Convert if_afdata read lock path to epoch
https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg

A fix for the inpcbhash contention is pending sufficient time
on a canary at LLNW.

Reviewed by:	gallatin
Sponsored by:	Limelight Networks
Differential Revision:	https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
Mark Johnston
e3d5c4ade1 Remove "All rights reserved" from my files.
See r333391 for the rationale.

MFC after:	1 week
2018-05-09 20:57:18 +00:00
Matt Macy
cbd92ce62e Eliminate the overhead of gratuitous repeated reinitialization of cap_rights
- Add macros to allow preinitialization of cap_rights_t.

- Convert most commonly used code paths to use preinitialized cap_rights_t.
  A 3.6% speedup in fstat was measured with this change.

Reported by:	mjg
Reviewed by:	oshogbo
Approved by:	sbruno
MFC after:	1 month
2018-05-09 18:47:24 +00:00
Hans Petter Selasky
c20feee43b Add myself to copyright in the LinuxKPI RCU support layer.
Suggested by:	mmacy@
Sponsored by:	Mellanox Technologies
2018-05-09 08:50:42 +00:00
Hans Petter Selasky
45aca3b1a1 Define USEC_PER_MSEC and USEC_PER_SEC in the LinuxKPI.
MFC after:	1 week
Sponsored by:	Mellanox Technologies
2018-04-30 09:31:09 +00:00
Mark Johnston
e4ba1a50af Fix the definitions of get_cpu() and put_cpu().
They are supposed to disable preemption.

Reported by:	rstone
MFC after:	5 days
2018-04-05 17:26:03 +00:00
Mark Johnston
2a1067a9f0 Wrap long lines.
MFC after:	3 days
2018-04-03 18:41:27 +00:00
Hans Petter Selasky
4b70609941 Optimise use of Giant in the LinuxKPI.
- Make sure Giant is locked when calling PCI device methods.
Newbus currently requires this.

- Avoid unlocking Giant right before aquiring the sleepqueue lock.
This can save a task switch.

MFC after:	1 week
Sponsored by:	Mellanox Technologies
2018-03-30 20:11:12 +00:00
Hans Petter Selasky
fa0d4f31a7 Swap two instances of regular macros with function macros in the LinuxKPI,
to narrow down the substitution scope.

MFC after:	1 week
Sponsored by:	Mellanox Technologies
2018-03-28 17:54:34 +00:00
Ed Maste
8363051739 linuxkpi whitespace cleanup
Reviewed by:	hselasky, markj
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D14807
2018-03-23 15:50:01 +00:00
Hans Petter Selasky
555deb3c96 The pci_disable_device() function is also expected to clear the PCI
busmaster. This fixes LinuxKPI compliancy with Linux.

MFC after:	1 week
Sponsored by:	Mellanox Technologies
2018-03-22 13:30:35 +00:00
Hans Petter Selasky
e1992aa142 Clear old MSIX IRQ numbers in the LinuxKPI.
When disabling the MSIX IRQ vectors for a PCI device through the
LinuxKPI, make sure any old MSIX IRQ numbers are no longer visible to
the linux_pci_find_irq_dev() function else IRQs can be requested from
the wrong PCI device.

MFC after:	1 week
Sponsored by:	Mellanox Technologies
2018-03-22 12:26:27 +00:00
Hans Petter Selasky
cbfc3c73ce Fix compliancy of the kstrtoXXX() functions in the LinuxKPI, by skipping
one newline character at the end, if any.

Found by:	greg@unrelenting.technology
MFC after:	1 week
Sponsored by:	Mellanox Technologies
2018-03-14 19:51:28 +00:00
Hans Petter Selasky
be15e1332d Implement proper support for complete_all() in the LinuxKPI.
When complete_all() is called there might be multiple waiters. The
current implementation could only handle one waiter. Make sure the
completion is sticky when complete_all() is called to be compatible
with Linux.

Found by:	Johannes Lundberg <johalun0@gmail.com>
MFC after:	1 week
Sponsored by:	Mellanox Technologies
Sponsored by:	Limelight Networks
2018-03-09 12:16:55 +00:00
Andrey V. Elsukov
c2a5dc6cd7 Add mapping for several ethernet types used by Linux to FreeBSD
ethernet types.

Reviewed by:	hselasky
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D14594
2018-03-06 12:58:00 +00:00
Hans Petter Selasky
e9e4ec118f Properly wrap the BUILD_BUG() function macro in the LinuxKPI.
MFC after:	1 week
Sponsored by:	Mellanox Technologies
2018-03-04 19:42:50 +00:00
Hans Petter Selasky
20789a72e0 Stub kernel_param_lock() and kernel_param_unlock() in the LinuxKPI.
MFC after:	1 week
Submitted by:	Johannes Lundberg <johalun0@gmail.com>
Sponsored by:	Mellanox Technologies
Sponsored by:	Limelight Networks
2018-03-04 19:10:30 +00:00
Hans Petter Selasky
c3bfe0de4c Implement wait_event_lock_irq() macro function in the LinuxKPI.
MFC after:	1 week
Requested by:	Johannes Lundberg <johalun0@gmail.com>
Sponsored by:	Mellanox Technologies
2018-03-04 19:07:10 +00:00
Hans Petter Selasky
ce930365d1 Keep the old SLAB_DESTROY_BY_RCU macro definition around in the LinuxKPI
to avoid compilation breakage in external kernel modules.

MFC after:	1 week
Sponsored by:	Mellanox Technologies
2018-03-04 18:53:41 +00:00
Hans Petter Selasky
8f368d485d Implement DEFINE_WAIT_FUNC() function macro and default_wake_function()
in the LinuxKPI.

MFC after:	1 week
Submitted by:	Johannes Lundberg <johalun0@gmail.com>
Sponsored by:	Mellanox Technologies
Sponsored by:	Limelight Networks
2018-03-04 18:51:43 +00:00
Hans Petter Selasky
ab7da72090 Implement pr_err_ratelimited() function macro in the LinuxKPI.
MFC after:	1 week
Submitted by:	Johannes Lundberg <johalun0@gmail.com>
Sponsored by:	Mellanox Technologies
Sponsored by:	Limelight Networks
2018-03-04 18:27:50 +00:00
Hans Petter Selasky
8961c48323 Implement __MODULE_STRING() function macro in the LinuxKPI.
MFC after:	1 week
Submitted by:	Johannes Lundberg <johalun0@gmail.com>
Sponsored by:	Mellanox Technologies
Sponsored by:	Limelight Networks
2018-03-04 18:21:21 +00:00
Hans Petter Selasky
20c8d8270c Implement BUILD_BUG() function macro in the LinuxKPI.
MFC after:	1 week
Submitted by:	Johannes Lundberg <johalun0@gmail.com>
Sponsored by:	Mellanox Technologies
Sponsored by:	Limelight Networks
2018-03-04 18:19:44 +00:00
Hans Petter Selasky
6c51dfb060 Implement writel_relaxed() in the LinuxKPI.
MFC after:	1 week
Submitted by:	Johannes Lundberg <johalun0@gmail.com>
Sponsored by:	Mellanox Technologies
Sponsored by:	Limelight Networks
2018-03-04 18:17:54 +00:00
Hans Petter Selasky
dc354b1551 Define noinline and __maybe_unused macros in the LinuxKPI.
MFC after:	1 week
Submitted by:	Johannes Lundberg <johalun0@gmail.com>
Sponsored by:	Mellanox Technologies
Sponsored by:	Limelight Networks
2018-03-04 18:13:31 +00:00
Hans Petter Selasky
9bce524efa Implement for_each_clear_bit() function macro in the LinuxKPI.
MFC after:	1 week
Submitted by:	Johannes Lundberg <johalun0@gmail.com>
Sponsored by:	Mellanox Technologies
Sponsored by:	Limelight Networks
2018-03-04 18:10:18 +00:00
Hans Petter Selasky
5d503e30ad Implement GENMASK_ULL() function macro in the LinuxKPI.
MFC after:	1 week
Submitted by:	Johannes Lundberg <johalun0@gmail.com>
Sponsored by:	Mellanox Technologies
Sponsored by:	Limelight Networks
2018-03-04 18:08:21 +00:00
Hans Petter Selasky
782a90d16a Rename the SLAB_DESTROY_BY_RCU flag into SLAB_TYPESAFE_BY_RCU in the LinuxKPI
to be compatible with Linux.

MFC after:	1 week
Requested by:	Johannes Lundberg <johalun0@gmail.com>
Sponsored by:	Mellanox Technologies
2018-03-04 18:04:37 +00:00
Hans Petter Selasky
ccae7bb851 Use mstosbt() instead of SBT_1MS in the LinuxKPI to get the last few bits
of precision.

MFC after:	1 week
Suggested by:	ian@
Sponsored by:	Mellanox Technologies
2018-03-03 19:26:40 +00:00
Hans Petter Selasky
7cf1c51588 Implement msleep_interruptible() in the LinuxKPI. While at it use pause_sbt()
instead of pause() in the msleep() function to avoid rounding errors when
converting delay values forth and back. Add a guard for a delay value
of zero milliseconds which is undefined.

MFC after:	1 week
Requested by:	Johannes Lundberg <johalun0@gmail.com>
Sponsored by:	Mellanox Technologies
2018-03-03 18:54:16 +00:00
Hans Petter Selasky
86ba49a722 Implement more lockdep stubs in the LinuxKPI.
MFC after:	1 week
Submitted by:	Johannes Lundberg <johalun0@gmail.com>
Sponsored by:	Mellanox Technologies
Sponsored by:	Limelight Networks
2018-03-02 08:59:53 +00:00
Hans Petter Selasky
8554bc585b Implement ktime_get_raw() function in the LinuxKPI.
MFC after:	1 week
Submitted by:	Johannes Lundberg <johalun0@gmail.com>
Sponsored by:	Mellanox Technologies
Sponsored by:	Limelight Networks
2018-03-02 08:58:32 +00:00
Hans Petter Selasky
d901abf167 Implement wait_on_bit() function macro in the LinuxKPI.
MFC after:	1 week
Submitted by:	Johannes Lundberg <johalun0@gmail.com>
Sponsored by:	Mellanox Technologies
Sponsored by:	Limelight Networks
2018-03-02 08:56:15 +00:00
Hans Petter Selasky
9555cfd2b2 Rename callout member in struct timer_list to match the one in struct
delayed_work in the LinuxKPI. This allows the timer_pending() function
macro to be used with delayed work structures.

No functional nor structural change.

MFC after:	1 week
Submitted by:	Johannes Lundberg <johalun0@gmail.com>
Sponsored by:	Mellanox Technologies
Sponsored by:	Limelight Networks
2018-03-02 08:52:27 +00:00
Hans Petter Selasky
b44247b1a9 Correct the return value from flush_work() and flush_delayed_work() in the
LinuxKPI to comply more with Linux. This fixes an issue when these functions
are used in waiting loops.

MFC after:	1 week
Sponsored by:	Mellanox Technologies
2018-03-01 10:31:51 +00:00
Hans Petter Selasky
949440623b Return correct error code to user-space when a system call receives a
signal in the LinuxKPI.

The read(), write() and mmap() system calls can return either EINTR or
ERESTART upon receiving a signal. Add code to figure out the correct
return value by temporarily storing the return code from the relevant
FreeBSD kernel APIs in the Linux task structure.

MFC after:	3 days
Sponsored by:	Mellanox Technologies
2018-02-22 15:29:19 +00:00
Konstantin Belousov
2c0f13aa59 vm_wait() rework.
Make vm_wait() take the vm_object argument which specifies the domain
set to wait for the min condition pass.  If there is no object
associated with the wait, use curthread' policy domainset.  The
mechanics of the wait in vm_wait() and vm_wait_domain() is supplied by
the new helper vm_wait_doms(), which directly takes the bitmask of the
domains to wait for passing min condition.

Eliminate pagedaemon_wait().  vm_domain_clear() handles the same
operations.

Eliminate VM_WAIT and VM_WAITPFAULT macros, the direct functions calls
are enough.

Eliminate several control state variables from vm_domain, unneeded
after the vm_wait() conversion.

Scetched and reviewed by:	jeff
Tested by:	pho
Sponsored by:	The FreeBSD Foundation, Mellanox Technologies
Differential revision:	https://reviews.freebsd.org/D14384
2018-02-20 10:13:13 +00:00