Hans Petter Selasky
57a865f808
Implement the __sg_alloc_table_from_pages() function based on the existing
...
sg_alloc_table_from_pages() function in the LinuxKPI.
This basically allow segments to have a limit, max_segment.
Submitted by: Johannes Lundberg <johalun0@gmail.com>
MFC after: 1 week
Sponsored by: Mellanox Technologies
Sponsored by: Limelight Networks
2018-06-01 12:09:07 +00:00
Hans Petter Selasky
6fad8d171a
Implement radix_tree_iter_delete() in the LinuxKPI.
...
Submitted by: Johannes Lundberg <johalun0@gmail.com>
MFC after: 1 week
Sponsored by: Mellanox Technologies
Sponsored by: Limelight Networks
2018-06-01 11:42:09 +00:00
Hans Petter Selasky
0a85496223
Improve high resolution timer support in the LinuxKPI.
...
Submitted by: Johannes Lundberg <johalun0@gmail.com>
MFC after: 1 week
Sponsored by: Mellanox Technologies
Sponsored by: Limelight Networks
2018-06-01 11:33:14 +00:00
Hans Petter Selasky
f03ae7e802
Add more GFP macro definitions in the LinuxKPI.
...
Submitted by: Johannes Lundberg <johalun0@gmail.com>
MFC after: 1 week
Sponsored by: Mellanox Technologies
Sponsored by: Limelight Networks
2018-06-01 11:14:59 +00:00
Hans Petter Selasky
13a5c70b91
Implement support for the PCI_BUS_NUM() function macro in the LinuxKPI.
...
Submitted by: Johannes Lundberg <johalun0@gmail.com>
MFC after: 1 week
Sponsored by: Mellanox Technologies
2018-05-31 13:17:34 +00:00
Hans Petter Selasky
f1aa567bfe
Implement support for the kvmalloc_array() function in the LinuxKPI.
...
Submitted by: Johannes Lundberg <johalun0@gmail.com>
MFC after: 1 week
Sponsored by: Mellanox Technologies
2018-05-31 13:13:08 +00:00
Hans Petter Selasky
f6d4552417
Correct macroname in the LinuxKPI.
...
Submitted by: Johannes Lundberg <johalun0@gmail.com>
MFC after: 1 week
Sponsored by: Mellanox Technologies
2018-05-31 12:55:38 +00:00
Hans Petter Selasky
cbea4f294f
Define __initconst in the LinuxKPI.
...
Submitted by: Johannes Lundberg <johalun0@gmail.com>
MFC after: 1 week
Sponsored by: Mellanox Technologies
2018-05-31 12:50:42 +00:00
Hans Petter Selasky
7ce7605ece
Implement bitmap_complement() in the LinuxKPI.
...
Submitted by: Johannes Lundberg <johalun0@gmail.com>
MFC after: 1 week
Sponsored by: Mellanox Technologies
2018-05-31 12:48:24 +00:00
Hans Petter Selasky
69d6653ba4
Implement idr_is_empty() in the LinuxKPI and make idr_remove() API compatible
...
with upstream Linux by returning the pointer to the removed element.
Submitted by: Johannes Lundberg <johalun0@gmail.com>
MFC after: 1 week
Sponsored by: Mellanox Technologies
2018-05-31 12:35:21 +00:00
Hans Petter Selasky
666cda9e0c
The schedule_timeout_killable() function should listen for signals
...
in the LinuxKPI.
Found by: Johannes Lundberg <johalun0@gmail.com>
MFC after: 1 week
Sponsored by: Mellanox Technologies
2018-05-28 11:26:40 +00:00
Hans Petter Selasky
bd40dea7e2
Implement wait_event_killable() in the LinuxKPI.
...
Requested by: Johannes Lundberg <johalun0@gmail.com>
MFC after: 1 week
Sponsored by: Mellanox Technologies
2018-05-28 10:54:24 +00:00
Hans Petter Selasky
2a3ec12831
Allow TASK_PARKED bit being set when going to sleep in the LinuxKPI.
...
Found by: Johannes Lundberg <johalun0@gmail.com>
MFC after: 1 week
Sponsored by: Mellanox Technologies
2018-05-28 10:51:39 +00:00
Matt Macy
4f6c66cc9c
UDP: further performance improvements on tx
...
Cumulative throughput while running 64
netperf -H $DUT -t UDP_STREAM -- -m 1
on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps
Single stream throughput increases from 910kpps to 1.18Mpps
Baseline:
https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg
- Protect read access to global ifnet list with epoch
https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg
- Protect short lived ifaddr references with epoch
https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg
- Convert if_afdata read lock path to epoch
https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg
A fix for the inpcbhash contention is pending sufficient time
on a canary at LLNW.
Reviewed by: gallatin
Sponsored by: Limelight Networks
Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
Mark Johnston
e3d5c4ade1
Remove "All rights reserved" from my files.
...
See r333391 for the rationale.
MFC after: 1 week
2018-05-09 20:57:18 +00:00
Matt Macy
cbd92ce62e
Eliminate the overhead of gratuitous repeated reinitialization of cap_rights
...
- Add macros to allow preinitialization of cap_rights_t.
- Convert most commonly used code paths to use preinitialized cap_rights_t.
A 3.6% speedup in fstat was measured with this change.
Reported by: mjg
Reviewed by: oshogbo
Approved by: sbruno
MFC after: 1 month
2018-05-09 18:47:24 +00:00
Hans Petter Selasky
c20feee43b
Add myself to copyright in the LinuxKPI RCU support layer.
...
Suggested by: mmacy@
Sponsored by: Mellanox Technologies
2018-05-09 08:50:42 +00:00
Hans Petter Selasky
45aca3b1a1
Define USEC_PER_MSEC and USEC_PER_SEC in the LinuxKPI.
...
MFC after: 1 week
Sponsored by: Mellanox Technologies
2018-04-30 09:31:09 +00:00
Mark Johnston
e4ba1a50af
Fix the definitions of get_cpu() and put_cpu().
...
They are supposed to disable preemption.
Reported by: rstone
MFC after: 5 days
2018-04-05 17:26:03 +00:00
Mark Johnston
2a1067a9f0
Wrap long lines.
...
MFC after: 3 days
2018-04-03 18:41:27 +00:00
Hans Petter Selasky
4b70609941
Optimise use of Giant in the LinuxKPI.
...
- Make sure Giant is locked when calling PCI device methods.
Newbus currently requires this.
- Avoid unlocking Giant right before aquiring the sleepqueue lock.
This can save a task switch.
MFC after: 1 week
Sponsored by: Mellanox Technologies
2018-03-30 20:11:12 +00:00
Hans Petter Selasky
fa0d4f31a7
Swap two instances of regular macros with function macros in the LinuxKPI,
...
to narrow down the substitution scope.
MFC after: 1 week
Sponsored by: Mellanox Technologies
2018-03-28 17:54:34 +00:00
Ed Maste
8363051739
linuxkpi whitespace cleanup
...
Reviewed by: hselasky, markj
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D14807
2018-03-23 15:50:01 +00:00
Hans Petter Selasky
555deb3c96
The pci_disable_device() function is also expected to clear the PCI
...
busmaster. This fixes LinuxKPI compliancy with Linux.
MFC after: 1 week
Sponsored by: Mellanox Technologies
2018-03-22 13:30:35 +00:00
Hans Petter Selasky
e1992aa142
Clear old MSIX IRQ numbers in the LinuxKPI.
...
When disabling the MSIX IRQ vectors for a PCI device through the
LinuxKPI, make sure any old MSIX IRQ numbers are no longer visible to
the linux_pci_find_irq_dev() function else IRQs can be requested from
the wrong PCI device.
MFC after: 1 week
Sponsored by: Mellanox Technologies
2018-03-22 12:26:27 +00:00
Hans Petter Selasky
cbfc3c73ce
Fix compliancy of the kstrtoXXX() functions in the LinuxKPI, by skipping
...
one newline character at the end, if any.
Found by: greg@unrelenting.technology
MFC after: 1 week
Sponsored by: Mellanox Technologies
2018-03-14 19:51:28 +00:00
Hans Petter Selasky
be15e1332d
Implement proper support for complete_all() in the LinuxKPI.
...
When complete_all() is called there might be multiple waiters. The
current implementation could only handle one waiter. Make sure the
completion is sticky when complete_all() is called to be compatible
with Linux.
Found by: Johannes Lundberg <johalun0@gmail.com>
MFC after: 1 week
Sponsored by: Mellanox Technologies
Sponsored by: Limelight Networks
2018-03-09 12:16:55 +00:00
Andrey V. Elsukov
c2a5dc6cd7
Add mapping for several ethernet types used by Linux to FreeBSD
...
ethernet types.
Reviewed by: hselasky
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D14594
2018-03-06 12:58:00 +00:00
Hans Petter Selasky
e9e4ec118f
Properly wrap the BUILD_BUG() function macro in the LinuxKPI.
...
MFC after: 1 week
Sponsored by: Mellanox Technologies
2018-03-04 19:42:50 +00:00
Hans Petter Selasky
20789a72e0
Stub kernel_param_lock() and kernel_param_unlock() in the LinuxKPI.
...
MFC after: 1 week
Submitted by: Johannes Lundberg <johalun0@gmail.com>
Sponsored by: Mellanox Technologies
Sponsored by: Limelight Networks
2018-03-04 19:10:30 +00:00
Hans Petter Selasky
c3bfe0de4c
Implement wait_event_lock_irq() macro function in the LinuxKPI.
...
MFC after: 1 week
Requested by: Johannes Lundberg <johalun0@gmail.com>
Sponsored by: Mellanox Technologies
2018-03-04 19:07:10 +00:00
Hans Petter Selasky
ce930365d1
Keep the old SLAB_DESTROY_BY_RCU macro definition around in the LinuxKPI
...
to avoid compilation breakage in external kernel modules.
MFC after: 1 week
Sponsored by: Mellanox Technologies
2018-03-04 18:53:41 +00:00
Hans Petter Selasky
8f368d485d
Implement DEFINE_WAIT_FUNC() function macro and default_wake_function()
...
in the LinuxKPI.
MFC after: 1 week
Submitted by: Johannes Lundberg <johalun0@gmail.com>
Sponsored by: Mellanox Technologies
Sponsored by: Limelight Networks
2018-03-04 18:51:43 +00:00
Hans Petter Selasky
ab7da72090
Implement pr_err_ratelimited() function macro in the LinuxKPI.
...
MFC after: 1 week
Submitted by: Johannes Lundberg <johalun0@gmail.com>
Sponsored by: Mellanox Technologies
Sponsored by: Limelight Networks
2018-03-04 18:27:50 +00:00
Hans Petter Selasky
8961c48323
Implement __MODULE_STRING() function macro in the LinuxKPI.
...
MFC after: 1 week
Submitted by: Johannes Lundberg <johalun0@gmail.com>
Sponsored by: Mellanox Technologies
Sponsored by: Limelight Networks
2018-03-04 18:21:21 +00:00
Hans Petter Selasky
20c8d8270c
Implement BUILD_BUG() function macro in the LinuxKPI.
...
MFC after: 1 week
Submitted by: Johannes Lundberg <johalun0@gmail.com>
Sponsored by: Mellanox Technologies
Sponsored by: Limelight Networks
2018-03-04 18:19:44 +00:00
Hans Petter Selasky
6c51dfb060
Implement writel_relaxed() in the LinuxKPI.
...
MFC after: 1 week
Submitted by: Johannes Lundberg <johalun0@gmail.com>
Sponsored by: Mellanox Technologies
Sponsored by: Limelight Networks
2018-03-04 18:17:54 +00:00
Hans Petter Selasky
dc354b1551
Define noinline and __maybe_unused macros in the LinuxKPI.
...
MFC after: 1 week
Submitted by: Johannes Lundberg <johalun0@gmail.com>
Sponsored by: Mellanox Technologies
Sponsored by: Limelight Networks
2018-03-04 18:13:31 +00:00
Hans Petter Selasky
9bce524efa
Implement for_each_clear_bit() function macro in the LinuxKPI.
...
MFC after: 1 week
Submitted by: Johannes Lundberg <johalun0@gmail.com>
Sponsored by: Mellanox Technologies
Sponsored by: Limelight Networks
2018-03-04 18:10:18 +00:00
Hans Petter Selasky
5d503e30ad
Implement GENMASK_ULL() function macro in the LinuxKPI.
...
MFC after: 1 week
Submitted by: Johannes Lundberg <johalun0@gmail.com>
Sponsored by: Mellanox Technologies
Sponsored by: Limelight Networks
2018-03-04 18:08:21 +00:00
Hans Petter Selasky
782a90d16a
Rename the SLAB_DESTROY_BY_RCU flag into SLAB_TYPESAFE_BY_RCU in the LinuxKPI
...
to be compatible with Linux.
MFC after: 1 week
Requested by: Johannes Lundberg <johalun0@gmail.com>
Sponsored by: Mellanox Technologies
2018-03-04 18:04:37 +00:00
Hans Petter Selasky
ccae7bb851
Use mstosbt() instead of SBT_1MS in the LinuxKPI to get the last few bits
...
of precision.
MFC after: 1 week
Suggested by: ian@
Sponsored by: Mellanox Technologies
2018-03-03 19:26:40 +00:00
Hans Petter Selasky
7cf1c51588
Implement msleep_interruptible() in the LinuxKPI. While at it use pause_sbt()
...
instead of pause() in the msleep() function to avoid rounding errors when
converting delay values forth and back. Add a guard for a delay value
of zero milliseconds which is undefined.
MFC after: 1 week
Requested by: Johannes Lundberg <johalun0@gmail.com>
Sponsored by: Mellanox Technologies
2018-03-03 18:54:16 +00:00
Hans Petter Selasky
86ba49a722
Implement more lockdep stubs in the LinuxKPI.
...
MFC after: 1 week
Submitted by: Johannes Lundberg <johalun0@gmail.com>
Sponsored by: Mellanox Technologies
Sponsored by: Limelight Networks
2018-03-02 08:59:53 +00:00
Hans Petter Selasky
8554bc585b
Implement ktime_get_raw() function in the LinuxKPI.
...
MFC after: 1 week
Submitted by: Johannes Lundberg <johalun0@gmail.com>
Sponsored by: Mellanox Technologies
Sponsored by: Limelight Networks
2018-03-02 08:58:32 +00:00
Hans Petter Selasky
d901abf167
Implement wait_on_bit() function macro in the LinuxKPI.
...
MFC after: 1 week
Submitted by: Johannes Lundberg <johalun0@gmail.com>
Sponsored by: Mellanox Technologies
Sponsored by: Limelight Networks
2018-03-02 08:56:15 +00:00
Hans Petter Selasky
9555cfd2b2
Rename callout member in struct timer_list to match the one in struct
...
delayed_work in the LinuxKPI. This allows the timer_pending() function
macro to be used with delayed work structures.
No functional nor structural change.
MFC after: 1 week
Submitted by: Johannes Lundberg <johalun0@gmail.com>
Sponsored by: Mellanox Technologies
Sponsored by: Limelight Networks
2018-03-02 08:52:27 +00:00
Hans Petter Selasky
b44247b1a9
Correct the return value from flush_work() and flush_delayed_work() in the
...
LinuxKPI to comply more with Linux. This fixes an issue when these functions
are used in waiting loops.
MFC after: 1 week
Sponsored by: Mellanox Technologies
2018-03-01 10:31:51 +00:00
Hans Petter Selasky
949440623b
Return correct error code to user-space when a system call receives a
...
signal in the LinuxKPI.
The read(), write() and mmap() system calls can return either EINTR or
ERESTART upon receiving a signal. Add code to figure out the correct
return value by temporarily storing the return code from the relevant
FreeBSD kernel APIs in the Linux task structure.
MFC after: 3 days
Sponsored by: Mellanox Technologies
2018-02-22 15:29:19 +00:00
Konstantin Belousov
2c0f13aa59
vm_wait() rework.
...
Make vm_wait() take the vm_object argument which specifies the domain
set to wait for the min condition pass. If there is no object
associated with the wait, use curthread' policy domainset. The
mechanics of the wait in vm_wait() and vm_wait_domain() is supplied by
the new helper vm_wait_doms(), which directly takes the bitmask of the
domains to wait for passing min condition.
Eliminate pagedaemon_wait(). vm_domain_clear() handles the same
operations.
Eliminate VM_WAIT and VM_WAITPFAULT macros, the direct functions calls
are enough.
Eliminate several control state variables from vm_domain, unneeded
after the vm_wait() conversion.
Scetched and reviewed by: jeff
Tested by: pho
Sponsored by: The FreeBSD Foundation, Mellanox Technologies
Differential revision: https://reviews.freebsd.org/D14384
2018-02-20 10:13:13 +00:00