I am really getting tired of the people who interrupt you when you begin a sentence with "To be honest". They stop you mid sentence and say "What do you mean, 'To be honest'? Does that mean you aren't honest the rest of the time? That you were lying?"
You get the same criticism for "Honestly," or "To be frank," - any of these phrases that connote that you're about to get serious with the other person. My last boss was particularly strict about this. He absolutely did not want to hear "To be honest" because he thought it spoke badly of your character. And because he was the boss, it filtered through the organization to the point where nobody could comfortably use the phrase.
Well I'm here to tell you that's bullshit. "To be honest" is an enormously useful, perfectly respectful phrase that conveys a definite message and says absolutely nothing about the character of the speaker.
Human conversation is full of artful deception. All of it. All the time. Most of it is benign. When someone says "How are you?", and you say "Doin' just fine Dave." it is understood that you elected not to give a full, detailed reply. Both parties are fine with that. The question was merely a greeting, and the response was just an acknowledgement. That's it. When the wife says "How do I look in these pants?" and you say "I'm not sure I like that style on you", you are choosing not to say "You look like a cow."
Every interaction we participate in has a certain amount of this. We sugarcoat. We spare each other's feelings. We hype up. We put a spin on our message. We highball. We lowball. We use euphamism. We temper, we boast, we use modesty or arrogance - depending on circumstance. We deflect, we downplay, we shift the focus, we exaggerate, we flatter, we console - all of it is to some various degree an acting job. We do not, absolutely do not use pure honesty all the time. Anyone who did try to offer total honesty all the time would be some kind of social outcast.
This is why the phrase "To be honest" is so useful. It alerts the listener that a bullshit-free opinion is about to follow, in a situation where bullshit would normally be employed. It doesn't mean that all bullshit is necessarily bad, or that in generally I am an untrustworthy person - it simply means that I am about to give you a fully truthful, unvarnished opinion. Many people do not even want to hear a fully truthful opinion, and the phrase "To be honest" gives them a second or two to tense up for the blow. It's a phrase that encourages you to pay close attention, to listen in. It advertises the arrival of a meaningful message. It does all of this in three simple words.
And it's particularly useful in a business environment where massaged and sugarcoated opinions are the norm. In my job, I'm expected to be optimistic in marketing, pessimistic to the top brass, and realistic to operations. The boss doesn't want to hear "to be honest"? He doesn't think that these aren't useful distinctions to make? If he asks me how sales are doing for the quarter, I can use A) "We'll get there, boss.", or B) "With a little luck, we can make quota." or C) "To be honest, I'm worried about making quota if that last contract falls through." All three responses are actually quite truthful. It depends on context. If he comes into my office with his shirt sleeves rolled up, closes my door and sits down, I'm going to use C. If he asks my opinion during a meeting where he's going around the table, it might be B. If he's walking through the office giving a tour to the board of directors, stops at my desk and introduces me, then asks, you better believe it will be A.
This is life. This is about the delicacy of using the right words in the right situations. So when I use C, how do you think I feel when I get rebuked for the implied dishonesty?
To be honest, the people who correct you when you start a sentence this way and rebuke you for the lies are deeply confused jackasses. Anyone who gives you a hard time for "To be honest" is saying that every conversation should be bullshit free. That is delusional. It takes a special pair of balls to correct people mid-sentence on their choice of words, and so if you're going to do it you should probably make sure you have a valid point, and are not simply advertising your lack of intelligence.