Even if his mom doesn't care much about him, he's still her son. She wouldn't mistreat him, so it's better for him to stay with his mom.
Leng Yan was silent for a moment and then said, "I already predicted that she'd want to get married. She told me when she came back that she'd probably be getting married soon, and she also said she'd take me with her..."
So he was afraid that following his mom and getting married would make him feel sad about leaving Xia Nuo, so he deliberately distanced himself from her, not wanting to crave her kindness towards him too much.
But he overestimated himself; he couldn't refuse her kindness at all, and eventually fell step by step.
