[media] mx2-camera: potential negative underflow bug

My static checker complains:

	drivers/media/platform/soc_camera/mx2_camera.c:1070
	mx2_emmaprp_resize() warn: no lower bound on 'num'

The heuristic is that it's looking for values which the user can
influence and we put an upper bound on them but we (perhaps
accidentally) allow negative numbers.

I am not very familiar with this code but I have looked at it and think
there might be a bug.  Making the variable unsigned seems like a safe
option either way and this silences the static checker warning.

The call tree is:
  -> subdev_do_ioctl()
     -> mx2_camera_set_fmt()
        -> mx2_emmaprp_resize()
The check:
	if (num > RESIZE_NUM_MAX)
can underflow and then we use "num" on the else path.

Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>

diff --git a/drivers/media/platform/soc_camera/mx2_camera.c b/drivers/media/platform/soc_camera/mx2_camera.c
index 2d57c1d272b828691fec0a688b36e074cb1466f8..2347612a4cc167a08af0bf831020d7b7dc932ebd 100644 (file)
--- a/drivers/media/platform/soc_camera/mx2_camera.c
+++ b/drivers/media/platform/soc_camera/mx2_camera.c
@@ -1002,7 +1002,7 @@ static int mx2_emmaprp_resize(struct mx2_camera_dev *pcdev,
                              struct v4l2_mbus_framefmt *mf_in,
                              struct v4l2_pix_format *pix_out, bool apply)
 {
-       int num, den;
+       unsigned int num, den;
        unsigned long m;
        int i, dir;