This posting is older than 6 months and can contain outdated information.
I may have clarified the scope of the problem and just wanted to pass along an update in case anyone had any other ideas. If I delete the com.apple.Bluetooth.plist file found in "/Library/Preferences/" (not ~/Library/Preferences/ as there is no com.apple.Bluetooth.plist file in that dir) and then reboot my machine, I can successfully pair both PS3 gamepads.
They both initially show up in the Bluetooth OSX settings and I also receive the pairing code request popup; however, if I ignore the popup, it will go away after 30 seconds or so and the controllers work great in MAME. When I turn the controllers off and back on again, they pair almost instantly without any pairing code request popup appearing.
However if I reboot my machine, I'm back to the same problem of getting the pairing code request popup and the controllers won't pair.
At the end of the day, it's a technically a working solution but it requires me to delete a plist file and reboot if I ever have to restart my machine. Not ideal by any means, but working nonetheless.
The last post I could find from Felix on adding PS3 gamepad support via bluetooth was back in 2007/2008 when it was on his to-do list, but I'd gladly be willing to pay an upgrade fee if PS3 gamepad support were added to Remote Buddy so I had a more functional solution to this problem :)