Is it right to segregate?

It's fine to surround yourself to people you can relate to as long as that relation or the surrounding doesn't hinder you from making it in society.