Should I clean my society finches' nest out while they have new babies?

if yes, how do we go about doing it without disturbing or hurting the babies, or stressing out their parents?