You need to set up two loadbalance groups for this.
LB Group_1 --> BT Primary / Colt Failover
LB Group_2 --> Colt Primary / BT Failover
then set VLANs 2 & 50 to use LB Group_1 and VLANs 1 & 10 to use LB Group_2 via the firewall modify rules -- they'll end up looking something like this
rule 1 - don't LB local nets
rule 10 - source VLAN2 / use group 1
rule 11 - source VLAN50 / use group 1
rule 20 - source VLAN1 / use group 2
rule 21 - source VLAN10 / use group 2