Good point - you DO have to worry about what people are thinking - and not just the people observing the friendly lunches, but the actual other person(s) involved. I've been on the innocent end of thinking all of this was friendly and haramless, and then I realized that the other luncher was on a TOTALLY different page, and was was thrown when he realized that I wasn't - by then it was awkward and embarassing, and hurtful for him.

Just because your intentions are innocent, don't assume that the other person's match up with your own. And even if they start out that way, things can change.

I see nothing wrong with going out to lunch or after work with a small group (even DH and 2 or 3 women), but I wouldn't like the idea of one-on-one co-worker lunches happening too frequently. Could be a slippery slope. Probably wouldn't be, but who wants to find out for sure?