"The important thing is that we make our users aware of social engineering threats and at the very least teach them to not just blindly give out information. If they are unsure then they need to refer the person to management. Teach them to stop and think before acting."
This is an incredibly normal line of thinking, and it is the traditional way of dealing with social engineering. The main issue with it is two-fold: first, an even half-way prepared social engineer will have prepared a strong enough frame to verify most of the simple checks that a normal user is going to have.
But the bigger issue is that, when we talk about things like "not blindly giving out information", what we're really saying is that we need to teach our users not to trust each other.
The problem with this is simple: an agile, responsive and successful business is built on a lack of boundaries and a healthy set of organizational trust. The kind of mistrust that most infosec people would engender intentionally in their users would cause significant inefficiencies within most organizations.
So, if we're not teaching our users to not blindly give out information, or to verify everything, what do I think we should be teaching them?
Instinct. Most who are in infosec have developed an instinct for when things "don't smell right". When an email just seems a little bit "phishy" (pun intended).
I believe that can be taught (well, indoctrinated) into our users, with about the same effort as it takes to teach them not to trust each other.