Yeah you might be right, but I just have a feeling that Xemnas could use a keyblade when Nomura is saying things like that in an interview, but we just don't know the details, but your guess is actually not bad
There was a theory here that the only keyblade AX could summon was MX's keyblade (as a note, it WOULD be nice if we knew the name of it) and terra was so disgusted by it since its MX's that he intentionally never used it, even though it fit in perfectly with Xemnas' goals. It would make sense as AX cast off his name of Xehanort and took on Ansem's name. that too could have been terra-controlled.
And maybe he couldn't wield Aqua's keyblade.
Out pf all the keyblades from BBS, only MX's keyblade as been a no show for 10 years (until this trailer).