In VirtualDub you need to be in capture mode before you select which device to capture from. To do so, under file select Capture Avi... then do the Video > Source.
As for whether the cable is in the right jack, I couldn't say. I own the ATI RADEON AIW base and the connectors are completely different. Check in the manual, it has pictures with everything labeled.