...in order to get the handwritten text readable...
I would check to see if you have added the antialising tag...
<page
aa="true" blah blah blah><img src= "blah blah blah"></page>
Hand written text embeded inside a graphic image is much easier to read with it.
As for artifacts, your post work technique using PS may need to be addressed. I work on the largest possible file at the highest possible quality, filter for noise et cetera and then reduce image dimensions using the "bicubic sharper" option to match the dimension of your desired page for your book. I have been able to have file sizes for photographic images with text overlays on them at around 200kb each page.
Also use Image Ready (or similar) to check how much quality you can sacrifice before artifacts appear.