What's the advantage of multi-GPU training in Alex-Net?

Question

I was reading ImageNet Classification with Deep Convolutional Neural Networks(Alex et al) and they trained their model on two GPUs following fine-grained structure. Can you tell me why they chose that structure of multi-GPU training and what is the advantage?