Here's Wikipedia's description of device drivers
- not specific to GUI.
Here's a Wikipedia description of RT-11
, with a main section called "device drivers". It was released in 1970 and happens to be one of the earliest OS's I am familiar with, so that's what I looked up.