With nothing left to lose but their lives, these men will follow any leader willing to provide for them and have the potential to become Pendor's most powerful units.
Refugees can be recruited from rescued prisoner stacks or via intermediaries in taverns and can upgrade into Pendor Troops. Finding refugees to build an early pendor army is fairly easy, but only if one is confident in their weapons and horses to take down various outlaw and red brotherhood bands who often spawn with at least one or two in tow. They can also be easily added to the party if you find a group of Armed Refugees. A sure way for this to work is for the refugees to be from your faction.
Should you choose to employ them, they can upgrade into Pendor Recruits. Which makes them kind of ironic, in a sad way. Soon they'll be forcing villagers to the life of a refugee. Unless you choose to be honourable.