The X3P hub is just a PCB with 5 DXL connectors hooked up in parallel.
The combo U2D2 + power hub is a good choice. The U2D2 itself is powered via the RPI USB port. But you need to have a separate 7.5 V power supply for your XL-320s as the RPI runs at 5V anyway. Just remember no daisy chain longer more than 4 DXLs.
All new DXLs come with the same ID=1 so you’ll need to rename their IDs, one at a time via the combo U2D2 + Power Hub and the Robotis tool called Dynamixel Wizard 2. It is free.
By the way RPi5 seemed to work fine with DXLs