When the battery of your Apple Remote is low, the Apple Remote will 
send a "battery low" event instead of the actual button press 
information. Therefore, you may have to press a button twice before 
the actual code is being sent, and holding a button may not work. A 
fresh battery for the AR should solve the problem.

Please note that even a new Apple Remote is not guaranteed to have a 
full battery. I've received a number of reports in the past from users 
who found they bought an Apple Remote with empty or almost empty 
battery.

Best regards, 
Felix Schwarz