If someone you swings with starts to have feelings for you, first you certainly would tell your spouse/partner! You must always maintain an open, honest communication within your relationship. I would not only tell this "too attached" swinging partner that it's inappropriate, I would also tell them that you do not share those feelings, that you have told your spouse/partner and that you will no longer be swinging with them. You may want to stay friends but I would advise you to place distance between you and them. This kind of situation happens much more often than many of us may realize. Those not into polyamory do not accept this while swinging but there are many polyamorous swingers. For you in this situation, simply move on.
There is no one like you. When filling out an adult dating profile, say something about your non-sexual passionsyour job, your volunteerism, your hobby, your art. Its good to have something to talk about when your just recovering from marathon sex. A tree-hugging-dirt-worshiper probably doesn't have anything pleasant to say to paper mill owner!