Hey, I wonder how you can get the AI to use custom abilitys, the Wizards I have on this map can devestate armys if used correctly but the AI just stand there throwing off the weakest of attacks, meh.
The AI uses custom abilities as it would use the abilities they are based on. That means if the base ability did not fit, then it will likely not use it in the right situation. If you used Channel, that one does not have any AI attached to it.
I figured that much, the problem is how to do it? The closest I can come is to get them to use one of the standard abilitys, if I use one of those as base ability will they use the custom ability?
Yes. Abilities have so called "Order Strings," which are used for issuing orders on them. Pick one for your custom ability (or leave the original one) and you can use the custom ability by issuing an order for that Order String / Base Ability.
You know the TFT nightelf-map where you should stop Illidan channeling a spell on the eye of sargeras (the one with blood-elf-allys)? There the 4 naga-summoners use the channel-ability. It's not a real ability, can be used for some scripts.
The unit control from Illidan is removed so you can't see if they simply added the channel-ability to him. But I don't think so because it's not needed there. In chapter7, they use channel so the fairy dragons are able to attack with "mana flare".