First of all, Remote Buddy is designed to be as efficient as possible 
with system resources, i.e. avoiding AppleScript where possible and 
is generally event driven, so unless something of significance 
happens (i.e. active application changes, button presses, animating 
prefs imagery, receiving data from the hardware, f.ex. interpreting 
Wiimote ir sensor input), it will just sit idle, needing 0% of your 
CPU resources.

Second, it does use low level graphics functions. It uses Quartz to 
render the entire interface (even the selection bar is drawn entirely 
in Quartz - no pre-rendered graphics, here). It is, however, often of 
advantage to cache commonly used images (i.e. Behaviour logos, base 
image for the remote's button legend) to avoid loading and decoding 
time, so Remote Buddy will do this at some point. Still, no problem 
with that. Remote Buddy even closes and frees the window it uses for 
its menu entirely when you hide it to minimize graphics memory usage.

What *may* cause a problem, though, is the way OS X graphics 
subsystem works to speed up drawing: it caches images and usually 
also complete windows in graphics card memory. An application has no 
direct influence on whether OS X will do this or not. Of course, OS X 
is smart, so it will eventually drop older graphics (still keeping it 
in main memory, of course) to replace it with newer or more recently 
used graphics.

Windows are composed by the window server, Quartz Compositor (not 
Quartz Composer - those are two totally different things) and, since 
Jaguar (10.2) they are rendered as an OpenGL scene (it was, btw, this 
very fact that finally made me purchase my first OS X system ;-). 
That has the advantage, that only little CPU time is spent on 
composing the different buffers, rendering alpha effects and shadows 
- it's all done on the graphics chip. The downside of this is 
increased graphics memory consumption through the system.

Now, if you run on a machine with only 32 MB of graphics memory 
(which is around the amount of memory needed to hold a single 8 
megapixel image), it's easy to run out of it - especially with data 
intense, high bandwidth fullscreen video. And that's probably what is 
happening here.

An example from my own experience: I have noticed "jumpy" and smooth 
graphics performance of OS X myself on two systems when using Expose:

- iMac Core Duo, external 24" display, 128 MB graphics memory => 
Exposé animation will be "jumpy" 
- MacBook Pro, same 24" display, 256 MB graphics memory => Exposé is 
absolutely smooth

It is worth noticing, that both systems have virtually the same 
specs: same graphics chip, same chipset and same amount of installed 
main memory. They mainly differ in the amount of installed graphics 
memory.

So, yes, you may just be asking too much from the graphics chip / 
memory of your system. The only scenario that I can think of, where 
Remote Buddy could have a direct (= something, I might be able to do 
anything about) influence on the graphics performance of your system 
would be when it is performing CPU intense tasks in the background. 
You should be able to spot this with Activity Monitor, though. Unless 
one of the events of interest listed above occurs, its CPU usage 
should generally be at 0%. If it isn't, I'd be very much interested 
in the log output of Activity Monitor's "Analyze" function.

Something you may want to try out, though, is to run at a lower 
screen resolution. This will reduce both the amount of memory needed 
for rendering your desktop and fullscreen video.

Best regards, 
Felix