http://www.WestwoodASL.com This video demonstrates who is asking who based on the direction and the sign.
[TRANSLATION]
1 – I ask you.
2 – You ask me.
3 – You ask him/her.
4a – I ask each of you (individually).
4b – I ask all of you (as a group).
5 – Each of you ask me.
6 – Hey! I have a question for you. Would you give me some paper?
7 – What is his name? Don't ask me, ask him!
8 – A question for all of you. On Monday we're going to have a quiz. Are you ready?

http://www.WestwoodASL.com/moodle