How to use sysprep without removing drivers:
Configure your answer file to persist the drivers by specifying the PersistAllDeviceInstalls setting in the Microsoft-Windows-PnPSysprep.
Another way would be to configure the registry key:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Setup\Sysprep\Settings\sppnp set PersistAllDeviceInstalls to 1.