If you have people in your group literate in sign, I would venture to say that would be a HUGE benefit for your team. That may not answer your question directly but if the other team also knows sign than you are not winning any points, either way.
Sometimes, the movement of your body reveals your intention better than taking the time to sign... Just a thought
They still need to team with another who could understand sign language. How do you say 'what' in Sign language would be for me. And somebody I know takes sign language just so they can win Charades. Some people. :)