The current load for the 4-pin JST expansion hub is pretty high, with only 3 XM-540 actuators you should not have any problem. What is the Current capacity of your 12V Power Supply?
From my experiences, I always provide my RPi with its own Power Supply instead of sharing into the Power Supply that also drives the actuators - more for power stability for the RPi than anything else. Both PS should have the same ground of course.