Hmm. I'd need to look at the problem more deeply, but perhaps the Array Interface would be helpful- it provides a true mutable array of numbers, whereas Lil lists and dictionaries are all copy-on-write.
