List filtering and JList indexes trouble.

Hi. I'm trying to create a image JList, and using a disk based cache to remove the possibility of outofmemory exceptions.
To do this, i have a CellRenderer that displays the images and for any image that is not visible then, disposes it from the memory cache (to force to get from the disk cache). The memory cache is a Map<Integer listindex, Image image> mapping.

The mapping is something like this:
int firstIndex = list.getFirstVisibleIndex();
                int lastIndex = list.getLastVisibleIndex();
                Iterator<Integer> mapKeyIterator = memoryCache.keySet().iterator();
                while(mapKeyIterator.hasNext()){
                    Integer key = mapKeyIterator.next();
                    if(key < firstIndex || key > lastIndex){
                        Image img = memoryCache.get(key);
                        if(img != null){
                            mapKeyIterator.remove();
                            img.getGraphics().dispose();
                        }
                    }
                }

Trouble is this mapping won't work from a EventListModel from a (filtered) FilterList, since then the mappings are wrong - and not consecutive even if they were right. Anyone can think of a alternative (i'd still like to accept any ListModel). Can you think of any alternative mapping or a way that i can bend the passed list model?