improving small face generation on SDXL without using any face-fixers

Is there a way to finetune or align the sdxl model such that it can generate small faces with good quality and without any distortion and deformation. I don't want to use any kind of face fixers. Currently we have the models which can generate very good and crisp face portraits but for generating small faces is challenging for the model it operates in Latent space whose spatial resolution is very small (by factor of 8). But is the way to fix this during fine-tuning itself