Have the model you want as the only model to load. Swapping models does not free up memory. I noticed my RAM usage was increasing after every swap and only closing HONK would free up memory. Hope this helps in some way.
