We are stuck for too long on the official definition of e-marketing permission. Just the fact that someone is not unsubscribing doesn't mean they like receiving our emails and could still be considered as 'warm', let alone 'hot', prospects.
So, whatever email marketing database you have, cut the third of it out. Feel free to think about that third as 'silent unsubscribes', people who simply can't be arsed to go and unsubscribe from your lovingly crafted emails.
That could explain a lot of response figures and could bring the real picture about what is happening with email today closer to the surface. Namely, that it is losing even more of its initial power than we thought.
How to find out who the 'silent unsubscribes' are? Ask them. Do a survey among your subscribers and you will find out, give or take a few percents. Do an in-depth, face-to-face interview and you will find out that even those that told you in the survey they are 'OK' to receive your emails actually don't read them.
Maybe it is time for us to change the definition of permission and to start grading it: 'basic permission' (only a legal minimum), 'active permission' (i.e. people who really read your emails) and 'request-level permission' (people who would actively go and ask you to send them the email again in case they've missed the last issue). How many of each do you have in your database, do you know? Any strategies to migrate your recipients through the tiers?
I know you have permission to talk to them, but think again.