Revert "cpufreq: Fix governor module removal race"

Revert "cpufreq: Fix governor module removal race"

This reverts commit 3f7dfb7fcf98a7e73dee018c4a68537ce7fec646 which was
commit a8b149d32b663c1a4105273295184b78f53d33cf upstream.

The backport was not correct, so just drop it entirely.

Reported-by: Ben Hutchings <ben.hutchings@codethink.co.uk>
Cc: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Cc: Viresh Kumar <viresh.kumar@linaro.org>
Cc: Sasha Levin <alexander.levin@microsoft.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>