The conductor has absolute authority and can remove you from the train. The OBS crew does not have any, but can get the conductor. Then it depends on who they believe, how you handle yourself, and how they are feeling. You really do not want to irritate a conductor.
Almost all conductors are quite professional. On a couple of occasions over many years and many miles I encountered a very few with little tin god complexes. Chances are if you are polite and professional and calmly tell your side of the story, they'll be fine with you. If nothing else, most do not want to fill out the paperwork required when they remove a passenger if they don't have to.
If they find you in an active shouting match with an OBS crew member, though, chances of getting booted off at the next stop are pretty high, no matter how egregious the act of the OBS crew member was that started it. If you lose it, you look like a loose cannon and they'll want you off their train, pronto. And all conductors legitimately consider it to be their train.