Giants are believed to come from the Monflora islands, where everything is larger, from the plants to the beasts. I have not traveled there myself, but I have read firsthand accounts of it.

[Much of what he had learned before he began his travels was from precious tomes brought into the village by outsiders.]

Yet your question is sound, if something I doubt they will be willing to answer in any meaningful way. They tell us what they wish us to know, for now.