I’ve got to say, Javert showing up at Fantine’s hospital bed to arrest Valjean is one of the most infuriating things he does. The casual cruelty he dishes out, not even caring that it kills her, because he’s so happy that he was right about “M. Madeleine.”
Javert cares about order, hierarchy, authority. He doesn’t care about people. Anyone on top of the heap he figures deserves to be there, and vice versa. That’s why “Madeleine” throws him for such a loop, why he’s so vengeful in arresting him, why he becomes determined to find him.
Until the mayor pulls rank on him over the arrest of some prostitute, Valjean is just some side project to Javert. From that point on, Valjean/Madeleine is priority #1 until he’s identified and finally arrested.
Then he’s forgotten. A finished project.
Until he escapes again, and he becomes The One That Got Away.