No. If you do not have an agreement with the affinity group or relocation company, there is no legal obligation to pay the referral fee. However, this may put your client in a difficult position because in most of these relationships the employee will lose her benefits if she does not follow the rules. If you insist on not paying a referral fee, the client will have to pay more to work with you because of the loss of benefits. Many real estate agents have adopted a policy to ask potential clients up front if they have any agreements with other service providers to avoid surprises later.
Worldwide ERC offers resources outlining procedures to facilitate dispute resolution relating to after-the-fact referral fees. Find these resources at worldwideerc.org/interchange.