I'm in the process of changing the closet doors in our Montana (it's a rear bedroom and the closet is along the entire back wall). I'm removing the two stationary panels on each end of the closet, which is extremely hard to reach around and modifying them to make them slide doors, like the middle is.
My middle doors are mounted with these (24 inch)...
. These mount to the sides.
But, they also have these, that would mount on the top and bottom.
On each door, one is mounted to the top and the other is mounted on the bottom. Removing the panels, I'm attempting to duplicate them also by mounting them at the top and bottom also. I am in the process of fixing the panels to length right now and I'll need to add some lumber inside the closet for the slide rails to attach too also. Then finish out the trim.
But, rather than using hanging rollers, why not figure out a way to use drawer slides and then you'll never have issues again. You sound like the type of person who can easily do a switch over like this, not afraid to tackle a wood crafting project yourself.
The slides cost me about $23 for the set (2 in a package). I got two sets (for both doors), and I also got the military discount at Lowe's in that price.
So, rather than attempt to fix a problem that will just continue to be a problem if you do it like it's always been done, why not switch it up a little and never have issues again?