MIPS: Fix bitrot in __get_user_unaligned() Signed-off-by: Al Viro <viro@zeniv.linux.org.uk> Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
MIPS: uaccess: Take EVA into account in [__]clear_user __clear_user() (and clear_user() which uses it), always access the user mode address space, which results in EVA store instructions when EVA is enabled even if the current user address limit is KERNEL_DS. Fix this by adding a new symbol __bzero_kernel for the normal kernel address space bzero in EVA mode, and call that from __clear_user() if eva_kernel_access(). Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Markos Chandras <markos.chandras@imgtec.com> Cc: Paul Burton <paul.burton@imgtec.com> Cc: Leonid Yegoshin <leonid.yegoshin@imgtec.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/10844/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
MIPS: uaccess: Take EVA into account in __copy_from_user() When EVA is in use, __copy_from_user() was unconditionally using the EVA instructions to read the user address space, however this can also be used for kernel access. If the address isn't a valid user address it will cause an address error or TLB exception, and if it is then user memory may be read instead of kernel memory. For example in the following stack trace from Linux v3.10 (changes since then will prevent this particular one still happening) kernel_sendmsg() set the user address limit to KERNEL_DS, and tcp_sendmsg() goes on to use __copy_from_user() with a kernel address in KSeg0. [<8002d434>] __copy_fromuser_common+0x10c/0x254 [<805710e0>] tcp_sendmsg+0x5f4/0xf00 [<804e8e3c>] sock_sendmsg+0x78/0xa0 [<804e8f28>] kernel_sendmsg+0x24/0x38 [<804ee0f8>] sock_no_sendpage+0x70/0x7c [<8017c820>] pipe_to_sendpage+0x80/0x98 [<8017c6b0>] splice_from_pipe_feed+0xa8/0x198 [<8017cc54>] __splice_from_pipe+0x4c/0x8c [<8017e844>] splice_from_pipe+0x58/0x78 [<8017e884>] generic_splice_sendpage+0x20/0x2c [<8017d690>] do_splice_from+0xb4/0x110 [<8017d710>] direct_splice_actor+0x24/0x30 [<8017d394>] splice_direct_to_actor+0xd8/0x208 [<8017d51c>] do_splice_direct+0x58/0x7c [<8014eaf4>] do_sendfile+0x1dc/0x39c [<8014f82c>] SyS_sendfile+0x90/0xf8 Add the eva_kernel_access() check in __copy_from_user() like the one in copy_from_user(). Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Markos Chandras <markos.chandras@imgtec.com> Cc: Paul Burton <paul.burton@imgtec.com> Cc: Leonid Yegoshin <leonid.yegoshin@imgtec.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/10843/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
MIPS: uaccess: Fix strlen_user with EVA The strlen_user() function calls __strlen_kernel_asm in both branches of the eva_kernel_access() conditional. For EVA it should be calling __strlen_user_eva for user accesses, otherwise it will load from the kernel address space instead of the user address space, and the access checking will likely be ineffective at preventing it due to EVA's overlapping user and kernel address spaces. This was found after extending the test_user_copy module to cover user string access functions, which gave the following error with EVA: test_user_copy: illegal strlen_user passed Fortunately the use of strlen_user() has been all but eradicated from the mainline kernel, so only out of tree modules could be affected. Fixes: e3a9b07a9caf ("MIPS: asm: uaccess: Add EVA support for str*_user operations") Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Markos Chandras <markos.chandras@imgtec.com> Cc: Paul Burton <paul.burton@imgtec.com> Cc: Leonid Yegoshin <leonid.yegoshin@imgtec.com> Cc: linux-mips@linux-mips.org Cc: <stable@vger.kernel.org> # 3.15.x- Patchwork: https://patchwork.linux-mips.org/patch/10842/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Merge branch 'upstream' of git://git.linux-mips.org/ralf/upstream-linus Pull MIPS updates from Ralf Baechle: - Improvements to the tlb_dump code - KVM fixes - Add support for appended DTB - Minor improvements to the R12000 support - Minor improvements to the R12000 support - Various platform improvments for BCM47xx - The usual pile of minor cleanups - A number of BPF fixes and improvments - Some improvments to the support for R3000 and DECstations - Some improvments to the ATH79 platform support - A major patchset for the JZ4740 SOC adding support for the CI20 platform - Add support for the Pistachio SOC - Minor BMIPS/BCM63xx platform support improvments. - Avoid "SYNC 0" as memory barrier when unlocking spinlocks - Add support for the XWR-1750 board. - Paul's __cpuinit/__cpuinitdata cleanups. - New Malta CPU board support large memory so enable ZONE_DMA32. * 'upstream' of git://git.linux-mips.org/pub/scm/ralf/upstream-linus: (131 commits) MIPS: spinlock: Adjust arch_spin_lock back-off time MIPS: asmmacro: Ensure 64-bit FP registers are used with MSA MIPS: BCM47xx: Simplify handling SPROM revisions MIPS: Cobalt Don't use module_init in non-modular MTD registration. MIPS: BCM47xx: Move NVRAM driver to the drivers/firmware/ MIPS: use for_each_sg() MIPS: BCM47xx: Don't select BCMA_HOST_PCI MIPS: BCM47xx: Add helper variable for storing NVRAM length MIPS: IRQ/IP27: Move IRQ allocation API to platform code. MIPS: Replace smp_mb with release barrier function in unlocks. MIPS: i8259: DT support MIPS: Malta: Basic DT plumbing MIPS: include errno.h for ENODEV in mips-cm.h MIPS: Define GCR_GIC_STATUS register fields MIPS: BPF: Introduce BPF ASM helpers MIPS: BPF: Use BPF register names to describe the ABI MIPS: BPF: Move register definition to the BPF header MIPS: net: BPF: Replace RSIZE with SZREG MIPS: BPF: Free up some callee-saved registers MIPS: Xtalk: Update xwidget.h with known Xtalk device numbers ...
MIPS: Optimise non-EVA kernel user memory accesses Commits ac1d8590d3ae (MIPS: asm: uaccess: Use EVA instructions wrappers), 05c6516005c4 (MIPS: asm: uaccess: Add EVA support to copy_{in, to,from}_user) & e3a9b07a9caf (MIPS: asm: uaccess: Add EVA support for str*_user operations) added checks to various user memory access functions & macros in order to determine whether to perform standard memory accesses or their EVA userspace equivalents. In kernels built without support for EVA these checks are entirely redundant. Avoid emitting them & allow the compiler to optimise out the EVA userspace code in such kernels by checking config_enabled(CONFIG_EVA). This reduces the size of a malta_defconfig kernel built using GCC 4.9.2 by approximately 33KB (from 5995072 to 5962304 bytes). Signed-off-by: Paul Burton <paul.burton@imgtec.com> Cc: Markos Chandras <markos.chandras@imgtec.com> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: linux-kernel@vger.kernel.org Cc: linux-mips@linux-mips.org Patchwork: http://patchwork.linux-mips.org/patch/10165/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
mm/uaccess, mm/fault: Clarify that uaccess may only sleep if pagefaults are enabled In general, non-atomic variants of user access functions must not sleep if pagefaults are disabled. Let's update all relevant comments in uaccess code. This also reflects the might_sleep() checks in might_fault(). Reviewed-and-tested-by: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: David Hildenbrand <dahi@linux.vnet.ibm.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Cc: David.Laight@ACULAB.COM Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: airlied@linux.ie Cc: akpm@linux-foundation.org Cc: benh@kernel.crashing.org Cc: bigeasy@linutronix.de Cc: borntraeger@de.ibm.com Cc: daniel.vetter@intel.com Cc: heiko.carstens@de.ibm.com Cc: herbert@gondor.apana.org.au Cc: hocko@suse.cz Cc: hughd@google.com Cc: mst@redhat.com Cc: paulus@samba.org Cc: ralf@linux-mips.org Cc: schwidefsky@de.ibm.com Cc: yang.shi@windriver.com Link: http://lkml.kernel.org/r/1431359540-32227-4-git-send-email-dahi@linux.vnet.ibm.com Signed-off-by: Ingo Molnar <mingo@kernel.org>
MIPS: uaccess.h: Fix strnlen_user comment. Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
MIPS: Remove __strlen_user(). Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
MIPS: Zero variable read by get_user / __get_user in case of an error. This wasn't happening in all cases. Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
MIPS: asm: uaccess: Add v1 register to clobber list on EVA When EVA is turned on and prefetching is being used in memcpy.S, the v1 register is being used as a helper register to the PREFE instruction. However, v1 ($3) was not in the clobber list, which means that the compiler did not preserve it across function calls, and that could corrupt the value of the register leading to all sorts of userland crashes. We fix this problem by using the DADDI_SCRATCH macro to define the clobbered register when CONFIG_EVA && CONFIG_CPU_HAS_PREFETCH are enabled. Signed-off-by: Markos Chandras <markos.chandras@imgtec.com> Cc: <stable@vger.kernel.org> # v3.15+ Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/8510/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
MIPS: asm: uaccess: Add EVA support for str*_user operations The str*_user functions are used to securely access NULL terminated strings from userland. Therefore, it's necessary to use the appropriate EVA function. However, if the string is in kernel space, then the normal instructions are being used to access it. The __str*_kernel_asm and __str*_user_asm symbols are the same for non-EVA mode so there is no functional change for the non-EVA kernels. Signed-off-by: Markos Chandras <markos.chandras@imgtec.com>
MIPS: asm: uaccess: Add EVA support to copy_{in, to,from}_user Use the EVA specific functions from memcpy.S to perform userspace operations. When get_fs() == get_ds() the usual load/store instructions are used because the destination address is located in the kernel address space region. Otherwise, the EVA specifc load/store instructions are used which will go through th TLB to perform the virtual to physical translation for the userspace address. Signed-off-by: Markos Chandras <markos.chandras@imgtec.com>
MIPS: asm: uaccess: Rename {get,put}_user_asm macros The {get,put}_user_asm functions can be used to load data from kernel or the user address space so rename them to avoid confusion. Signed-off-by: Markos Chandras <markos.chandras@imgtec.com>
MIPS: asm: uaccess: Use EVA instructions wrappers Use the EVA instruction wrappers from asm.h to perform read/write operations from userland. Signed-off-by: Markos Chandras <markos.chandras@imgtec.com>
MIPS: asm: uaccess: Disable unaligned access macros for EVA ulb, ulh, ulw are macros which emulate unaligned access for MIPS. However, no such macros exist for EVA mode, so the only way to do EVA unaligned accesses is in the ADE exception handler. As a result of which, disable these macros for EVA. Signed-off-by: Leonid Yegoshin <Leonid.Yegoshin@imgtec.com> Signed-off-by: Markos Chandras <markos.chandras@imgtec.com>
MIPS: asm: uaccess: Move duplicated code to common function Similar to __get_user_* functions, move common code to __put_user_*_common so it can be shared among similar users. Signed-off-by: Markos Chandras <markos.chandras@imgtec.com>
MIPS: asm: uaccess: Add instruction argument to __{put,get}_user_asm In preparation for EVA support, an instruction argument is needed for the __get_user_asm{,_ll32} functions to allow instruction overrides in EVA mode. Even though EVA only works for MIPS 32-bit, both codepaths are changed (32-bit and 64-bit) for consistency reasons. Signed-off-by: Markos Chandras <markos.chandras@imgtec.com>
Merge branch 'mti-next' of git://git.linux-mips.org/sjhill/linux-sjhill into mips-for-linux-next
MIPS: microMIPS: Disable LL/SC and fix linker bug. Partially revert commit e0c14a260d66ba35935600d6435940a566fe806b and turn off LL/SC when building a pure microMIPS kernel. This is a temporary fix until the cmpxchg assembly macro functions are re-written to not use the HI/LO registers in address calculations. Also add .insn in selected user access functions which would otherwise produce ISA mode jump incompatibilities. This is also a temporary fix. Signed-off-by: Steven J. Hill <Steven.Hill@imgtec.com>