To loop over table records you have to retrieve a list in memory and iterate over that list. With large quantities you get memory problems, which can be avoided by using batch parameters (number of records with offset) in a retrieve. But that needs an outer loop. It is a trial and error to know at how much records it will occur.
Can you create a retrieve in combination with an iterating loop based on a database cursor (or Java rowset), instead of loading all data in the list first. This would eliminate the outer loop and would eliminate trial and error.