All I know is Lubavitch, but I would like to explore a different Nusach. What is the Halacha on doing this.
If all you know is Lubavitch because this happens to be the shul you daven at, there is nothing wrong with exploring different customs and prayers.
However, if this is your custom because this is the heritage of your ancestors, then you should generally not change your custom. However, some (Lithuanian) poskim write that one may change from Sephard to Ashkenaz. If a parent is makpid that a son should not change his nusach, the son should heed his father’s kepeidah.
Magen Avraham (68) writes that a person should not change his nusach. Avodas Hakodesh (Chida, Kesher Godal 12:9) also writes that one should not change his father’s customs, for each nusach of prayer has a source (a “gate” in heaven); concerning customs in general, see Shaul U’meishiv, vol. 3, 1:247, who writes that one should not change customs.
Concerning nusach ashkenaz and nusach sephard, the Chasam Sofer (no. 15) famously writes that all nuschaos are equal , whereas Divrei Chaim (2:8, also quoting from Mishnas Chassidim) writes that the nusach of Arizal has a special virtue, though he only writes that somebody unclear about his own nusach should adopt it. In spite of the above mentioned Magen Avraham, Maharam Shik (CM 24)writes that somebody whose soul desires nusach sephard may change over; on the other hand, Iggros Moshe writes that somebody may change back to the original nusach ashkenaz, but not vice versa. See also Minchas Elazar (1:11); Keren David(19); Maharam of Brisk (2:28); Zichron Yehudah (15).
These sources all refer to an ‘ancestral custom.’ For a nusach that one has become accustomed to due to lack of alternative, or for some other reason, the restriction on changing nusach will not apply, and a baal teshuva without a solid ancestral custom can check which nusach and style of davening he most connects with.