Unless each of the classes are a Connection class, then no you wouldn't extend it as that doesn't make any sense (ie: a User is not a type of Connection).
You need to determine if your classes are aggregates or composites using the connection. If they are aggregates, then you simply factory a connection within the constructor (or whatever method you want), particularly if they allow alternate connections. If they are composites then you pass an existing Connection object to the construction of the new class. Given that the singleton is specific to only a single instance regardless of credentials, this to me would be governed by a composite design. If each class were capable of choosing its own storage, I'd still use a factory but leave those details up to the class by letting it aggregate a connection itself based on its configurations.
Since you're already using OOP here, drop the use of mysql library and use either PDO or MySQLi. Don't forget to add a protected or private constructor override, otherwise you can indefinitely create Connection objects, even though there are no specific properties here unique to each connection, its probably not wise to have a ton of unnecessary class object's hanging around doing nothing in particular.