Wow this was way more painful than it should've been. SSL is hard!
So I followed the instructions from Gandi and ended up with a private key, a certificate signing request, a certificate and an intermediate certificate. To load them into the Load Balancer, you can't do it directly from the Elastic Beanstalk console, instead you go to the EC2 console and look at your Load Balancer. Go to the Listeners tab and add an HTTPS listener, click "Change" on the certificate and upload a new certificate. Here's where I got really stuck. Turns out the private key is in the wrong format.
So....
openssl rsa -in <private key> -out server.key
Use THAT file for the private key, and all is hunky dory.
Don't forget to include the intermediate key too, which you download from Gandi.