Yes indeed, and if the port itself is on the switch, I'm not sure adding a vif to it will do anything.
set interfaces switch switch0 vif 10 address 192.168.1.1/24
and so on.
An easy way to see what you can and can't do is to start typing, then hit Tab to see available completions.