I have my doubts that it's your torsion arms. They might be set differently to take account of driver position, battery etc. Lots of vehicles have handed coils for the same reason.
It looks like most people have covered what you should check, but certainly don't underestimate the possibility of "rear wheel steer" if there's anything wrong at the back.
Dodgy caster angle from a lift will make your steering feel light,twitchy, wandery or a combination of those things.
You can get the toe settings overall correct but out side to side if you understand what I mean. Some cars even have the toe settings deliberately unequal to account for the camber on our roads.
Although I've never found it necessary on a 4x4, a proper 4 wheel align like on a Hunter machine makes an enormous difference to driving. I've had a couple of motors done in the past and it's amazing the difference, but it will cost you more to have done.
Over time and lots of miles a steering box/idler can get a sort of wear "notch" at the centre position and if you adjust the steering so that it is in a new position and no longer sitting in that "notch" then the steering can become wandery as the box floats in and out of the notch or worn area, though I doubt that's your problem from what you say and I don't know if it particularly occurs on Terranos.
The last thing I can think of is tyres. I've seen the blocks wear on the rear edge on some tyres and it's just seemed to be the tyre design/construction because the camber would be ok on the vehicle, it's called heel and toe wear or scalloping I think and we could never stop it, just swap tyres end to end. Some people will also say swap tyres side to side but I've always been reluctant to do this because I think that tyres may bed in in a particular direction of rotation and from experience, I've seen tyres that were swapped side to side suffer from accelerated wear. I'm not saying any tyre would suffer from this, just that I've seen it happen.
You could also experiment a little with tyre pressures. I've run tyres on 4x4s 4 or 5psi higher than might be suggested with good results, you just have to be wary initially of handling changes after upping the pressure, as well as doing careful monitoring of the tyres.