My first thought was tge clutch release bearing, as they are a know failure point. However... they normally sound like an old roller skate rather than a woosh and in my case at least, the noise happens is when the clutch pedal is up.

Is the noise constant when the pedal is depressed?

 

Thanks Kaviar. It's only when actually depressing the pedal. Weirdly I tried it today with the car off and a cold engine and it sounded more like a squeak than a whoosh, but it was definitely outside the car (not part of the linkage at the pedal).