List Pipelines, Concurrent Events, listChanged... and ObservableList

[Sam Berlin]

Hi Folks,

I'm trying to optimize away an ObservableList, to simplify the list pipeline, increase performance, and reduce overall memory.  The issues I'm seeing with ObservableList are:

1) The "easy-to-use" BeanConnector is a little heavy-weight, memory-wise.  I don't want to require each Bean to have its own PropertyChangeSupport & listeners installed.
 2) ObservableList keeps a cache of all observed elements, to disconnect & reconnect the connector.
 3) To update an element, ObservableList does a linear search through the list (and doesn't break when it finds it).

I can solve #1 easy enough -- substitute my own Connector that listens to a single 'parent' listener that is notified of all change, and return false on installListener.  However, #2 is still left around, despite not actually needing the cache (since no Connector will ever be installed).  And, #3 is a minor performance concern, given that the underlying list is in sorted order and I can do a binary search to find the right element (or, in some cases, I know the index already).

My first thought was to just ditch ObservableList entirely from the pipeline & use separate listeners and directly call source.set(idx, currentItem) when I detect a change from the listener, expecting that to trigger a list update on the downstream lists.  I ran into problems with this, because I want to trigger a change in response to a current listChanged event.

Some background on where I get confused:  If ObservableList calls elementChanged during an existing listChanged event, then the ObservableList's listeners will see the update *if* the item wasn't in the items changed in this event.  That is, consider I have items 'foo', 'bar', and 'baz'.  If Foo already exists in the list, and I receive a listChanged event that contains the additions of Bar & Baz and call ObservableList.elementChanged(Foo), then listeners to the ObservableList will see the Foo change.  (No new event will be generated if bar or baz get an elementChange, for good reason.. because they already have pending changes.)

Here's where I get confused:  If I call set(Foo) instead of elementChanged(Foo), I don't see any new event!  Ideally I wouldn't be using ObservableList at all, and I'd just have a single source list & call set on it in order to notify that the item changed, but it doesn't work.  (Although the issue isn't with using the source list directly, it's with using set instead of elementChanged.)

The reason it fails seems to be because elementChanged calls beginEvent/commitEvent on a separate ListEventAssembler, whereas set calls it on the ListEventAssembler that's already in the process of publishing a change.  'set' would want to trigger changes to all listeners of the source, whereas elementChanged only triggers changes on listeners of the ObservableList.

I can understand the reason it doesn't work... but I can't for the life of me figure out any way to work around it without creating a separate list just for the sake of triggering an extra update during an existing set of changes.

For some background, here's the basic pipeline I'm currenty using:

  BasicEventList -> ListenerA, ObservableList -> [lots & lots & lots of other lists]
  ListenerA: Looks at the new item & existing items, changes some bound properties in existing items, triggers additional changes in the ObservableList

I want to avoid the ObservableList and just tell ListenerA to directly trigger changes on the BasicEventList.  Is this an impossible goal?

Sorry for the incredibly long-winded question!  I had a lot of trouble figuring out what the issue was, and (apparently) an equally hard time explaining it in words.

Thanks,
Sam

[Kevin Day-7]

Sam-

 

Any chance of getting a unit test that demonstrates what you are going for?  It sounds like the primary issue is handling of a re-entrant change event, which should work fine.  The publisher definitely handles cases where one update causes another update, so I suspect the current behavior is related to dependencies between listeners.

 

Here's some info that may or may not be useful:

 

Calling set() on a transform list ultimately has the effect of calling set() on the source list of the pipeline.  This would then propogate the change downward through the entire pipeline.

 

Calling elementChanged() sends an event downstream only.

 

 

Do you have the following?

 

OE List depends on Source List

Transform List A depends on OE List

Listener B depends on Transform List A

 

Listener B makes changes to the objects stored in OEL

 

If this is the case, then there is an implied dependency that doesn't get automatically configured:

 

OE List depends on Listener B

 

 

 

You will probably need to explictitly register the additional dependancy that OE List is dependant on the listener using setDependantSubject() (I think that's the right one - I'm still a bit unclear on which call does what without looking at how it effects the publisher dependencies).

 

If I am on the right track here, then the code I posted last month for instrumenting the publisher dependencies may be of interest.  Let me know if you need help using it.

 

 

- K

 

----------------------- Original Message -----------------------

  

From: Sam Berlin sberlin@...

To: Glazed Lists users@...

Cc: 

Date: Fri, 10 Jul 2009 01:10:28 -0400

Subject: List Pipelines, Concurrent Events, listChanged... and ObservableList

  

Hi Folks,

I'm trying to optimize away an ObservableList, to simplify the list pipeline, increase performance, and reduce overall memory.  The issues I'm seeing with ObservableList are:

1) The "easy-to-use" BeanConnector is a little heavy-weight, memory-wise.  I don't want to require each Bean to have its own PropertyChangeSupport & listeners installed.
 2) ObservableList keeps a cache of all observed elements, to disconnect & reconnect the connector.
 3) To update an element, ObservableList does a linear search through the list (and doesn't break when it finds it).

I can solve #1 easy enough -- substitute my own Connector that listens to a single 'parent' listener that is notified of all change, and return false on installListener.  However, #2 is still left around, despite not actually needing the cache (since no Connector will ever be installed).  And, #3 is a minor performance concern, given that the underlying list is in sorted order and I can do a binary search to find the right element (or, in some cases, I know the index already).

My first thought was to just ditch ObservableList entirely from the pipeline & use separate listeners and directly call source.set(idx, currentItem) when I detect a change from the listener, expecting that to trigger a list update on the downstream lists.  I ran into problems with this, because I want to trigger a change in response to a current listChanged event.

Some background on where I get confused:  If ObservableList calls elementChanged during an existing listChanged event, then the ObservableList's listeners will see the update *if* the item wasn't in the items changed in this event.  That is, consider I have items 'foo', 'bar', and 'baz'.  If Foo already exists in the list, and I receive a listChanged event that contains the additions of Bar & Baz and call ObservableList.elementChanged(Foo), then listeners to the ObservableList will see the Foo change.  (No new event will be generated if bar or baz get an elementChange, for good reason.. because they already have pending changes.)

Here's where I get confused:  If I call set(Foo) instead of elementChanged(Foo), I don't see any new event!  Ideally I wouldn't be using ObservableList at all, and I'd just have a single source list & call set on it in order to notify that the item changed, but it doesn't work.  (Although the issue isn't with using the source list directly, it's with using set instead of elementChanged.)

The reason it fails seems to be because elementChanged calls beginEvent/commitEvent on a separate ListEventAssembler, whereas set calls it on the ListEventAssembler that's already in the process of publishing a change.  'set' would want to trigger changes to all listeners of the source, whereas elementChanged only triggers changes on listeners of the ObservableList.

I can understand the reason it doesn't work... but I can't for the life of me figure out any way to work around it without creating a separate list just for the sake of triggering an extra update during an existing set of changes.

For some background, here's the basic pipeline I'm currenty using:

  BasicEventList -> ListenerA, ObservableList -> [lots & lots & lots of other lists]
  ListenerA: Looks at the new item & existing items, changes some bound properties in existing items, triggers additional changes in the ObservableList

I want to avoid the ObservableList and just tell ListenerA to directly trigger changes on the BasicEventList.  Is this an impossible goal?

Sorry for the incredibly long-winded question!  I had a lot of trouble figuring out what the issue was, and (apparently) an equally hard time explaining it in words.

Thanks,
Sam
--------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscribe@... For additional commands, e-mail: users-help@...

[Sam Berlin]

Test is attached, please let me know if the tests aren't self-described enough!

The particular pipeline in question s:

OE List depends on SourceList
Listener A depends on SourceList
<other listeners> depend on OE List

Listener A makes changes to Objects in SourceList/OE List that trigger elementChanged events in OE List.

That setup works perfectly.  Where it doesn't work is when I remove OE List & try to manually handle the updates (by using re-entrant changes into the same assembler)... all subsequent updates get lost to the ether.  (Test #2 in the testcases also demonstrates that the events get lost into the ether if Listener A listeners on OE list & all else is the same.)

Sam

On Fri, Jul 10, 2009 at 9:21 AM, Kevin Day <kevin@...> wrote:

Sam-

 

Any chance of getting a unit test that demonstrates what you are going for?  It sounds like the primary issue is handling of a re-entrant change event, which should work fine.  The publisher definitely handles cases where one update causes another update, so I suspect the current behavior is related to dependencies between listeners.

 

Here's some info that may or may not be useful:

 

Calling set() on a transform list ultimately has the effect of calling set() on the source list of the pipeline.  This would then propogate the change downward through the entire pipeline.

 

Calling elementChanged() sends an event downstream only.

 

 

Do you have the following?

 

OE List depends on Source List

Transform List A depends on OE List

Listener B depends on Transform List A

 

Listener B makes changes to the objects stored in OEL

 

If this is the case, then there is an implied dependency that doesn't get automatically configured:

 

OE List depends on Listener B

 

 

 

You will probably need to explictitly register the additional dependancy that OE List is dependant on the listener using setDependantSubject() (I think that's the right one - I'm still a bit unclear on which call does what without looking at how it effects the publisher dependencies).

 

If I am on the right track here, then the code I posted last month for instrumenting the publisher dependencies may be of interest.  Let me know if you need help using it.

 

 

- K

 

----------------------- Original Message -----------------------

  

From: Sam Berlin sberlin@...

To: Glazed Lists users@...

Cc: 

Date: Fri, 10 Jul 2009 01:10:28 -0400

Subject: List Pipelines, Concurrent Events, listChanged... and ObservableList

  

Hi Folks,

I'm trying to optimize away an ObservableList, to simplify the list pipeline, increase performance, and reduce overall memory.  The issues I'm seeing with ObservableList are:

1) The "easy-to-use" BeanConnector is a little heavy-weight, memory-wise.  I don't want to require each Bean to have its own PropertyChangeSupport & listeners installed.
 2) ObservableList keeps a cache of all observed elements, to disconnect & reconnect the connector.
 3) To update an element, ObservableList does a linear search through the list (and doesn't break when it finds it).

I can solve #1 easy enough -- substitute my own Connector that listens to a single 'parent' listener that is notified of all change, and return false on installListener.  However, #2 is still left around, despite not actually needing the cache (since no Connector will ever be installed).  And, #3 is a minor performance concern, given that the underlying list is in sorted order and I can do a binary search to find the right element (or, in some cases, I know the index already).

My first thought was to just ditch ObservableList entirely from the pipeline & use separate listeners and directly call source.set(idx, currentItem) when I detect a change from the listener, expecting that to trigger a list update on the downstream lists.  I ran into problems with this, because I want to trigger a change in response to a current listChanged event.

Some background on where I get confused:  If ObservableList calls elementChanged during an existing listChanged event, then the ObservableList's listeners will see the update *if* the item wasn't in the items changed in this event.  That is, consider I have items 'foo', 'bar', and 'baz'.  If Foo already exists in the list, and I receive a listChanged event that contains the additions of Bar & Baz and call ObservableList.elementChanged(Foo), then listeners to the ObservableList will see the Foo change.  (No new event will be generated if bar or baz get an elementChange, for good reason.. because they already have pending changes.)

Here's where I get confused:  If I call set(Foo) instead of elementChanged(Foo), I don't see any new event!  Ideally I wouldn't be using ObservableList at all, and I'd just have a single source list & call set on it in order to notify that the item changed, but it doesn't work.  (Although the issue isn't with using the source list directly, it's with using set instead of elementChanged.)

The reason it fails seems to be because elementChanged calls beginEvent/commitEvent on a separate ListEventAssembler, whereas set calls it on the ListEventAssembler that's already in the process of publishing a change.  'set' would want to trigger changes to all listeners of the source, whereas elementChanged only triggers changes on listeners of the ObservableList.

I can understand the reason it doesn't work... but I can't for the life of me figure out any way to work around it without creating a separate list just for the sake of triggering an extra update during an existing set of changes.

For some background, here's the basic pipeline I'm currenty using:

  BasicEventList -> ListenerA, ObservableList -> [lots & lots & lots of other lists]
  ListenerA: Looks at the new item & existing items, changes some bound properties in existing items, triggers additional changes in the ObservableList

I want to avoid the ObservableList and just tell ListenerA to directly trigger changes on the BasicEventList.  Is this an impossible goal?

Sorry for the incredibly long-winded question!  I had a lot of trouble figuring out what the issue was, and (apparently) an equally hard time explaining it in words.

Thanks,
Sam

--------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscribe@... For additional commands, e-mail: users-help@...

[EventTest.java]

---

import java.beans.PropertyChangeListener;
import java.util.ArrayList;
import java.util.List;

import junit.framework.TestCase;
import ca.odell.glazedlists.BasicEventList;
import ca.odell.glazedlists.GlazedLists;
import ca.odell.glazedlists.ObservableElementList;
import ca.odell.glazedlists.event.ListEvent;
import ca.odell.glazedlists.event.ListEventListener;

public class EventTest extends TestCase {
   
    public void testElementChangedOtherIndexTriggersDownstreamEventsWhenListeningToSource() throws Exception {
        final BasicEventList<Bean> list = new BasicEventList<Bean>();        
        final ObservableElementList<Bean> olist = new ObservableElementList<Bean>(list, GlazedLists.beanConnector(Bean.class));
       
        list.add(new Bean());
       
        Listener oListener = new Listener();
        Listener sListener = new Listener();
       
        olist.addListEventListener(oListener);
        list.addListEventListener(sListener);
       
        // FAILS if listening to olist
        list.addListEventListener(new ListEventListener<Bean>() {
            @Override
            public void listChanged(ListEvent<Bean> listChanges) {
                // FAILS if calling set(..) instead of elementChanged
                olist.elementChanged(list.get(0));
            }
        });        
        list.add(new Bean());
       
        assertEquals(1, sListener.changes.size());
        assertEquals(new Change(ListEvent.INSERT, 1), sListener.changes.get(0));
       
        assertEquals(2, oListener.changes.size());
        assertEquals(new Change(ListEvent.UPDATE, 0), oListener.changes.get(0));
        assertEquals(new Change(ListEvent.INSERT, 1), oListener.changes.get(1));
    }
   
    public void testElementChangedOtherIndexTriggersDownstreamEventsWhenListeningToOList() throws Exception {
        final BasicEventList<Bean> list = new BasicEventList<Bean>();        
        final ObservableElementList<Bean> olist = new ObservableElementList<Bean>(list, GlazedLists.beanConnector(Bean.class));
       
        list.add(new Bean());
       
        Listener oListener = new Listener();
        Listener sListener = new Listener();
       
        olist.addListEventListener(oListener);
        list.addListEventListener(sListener);
       
        // SUCCEEDs if listening to source
        olist.addListEventListener(new ListEventListener<Bean>() {
            @Override
            public void listChanged(ListEvent<Bean> listChanges) {
                olist.elementChanged(list.get(0));
            }
        });        
        list.add(new Bean());
       
        assertEquals(1, sListener.changes.size());
        assertEquals(new Change(ListEvent.INSERT, 1), sListener.changes.get(0));
       
        assertEquals(2, oListener.changes.size());
        assertEquals(new Change(ListEvent.INSERT, 1), oListener.changes.get(0));
        assertEquals(new Change(ListEvent.UPDATE, 0), oListener.changes.get(1));
    }
   
    public void testSetTriggersEventsJustLikeElementChanged() throws Exception {
        final BasicEventList<Bean> list = new BasicEventList<Bean>();        
        final ObservableElementList<Bean> olist = new ObservableElementList<Bean>(list, GlazedLists.beanConnector(Bean.class));
       
        list.add(new Bean());
       
        Listener oListener = new Listener();
        Listener sListener = new Listener();
       
        olist.addListEventListener(oListener);
        list.addListEventListener(sListener);
       
        // FAILS if listening to olist
        list.addListEventListener(new ListEventListener<Bean>() {
            @Override
            public void listChanged(ListEvent<Bean> listChanges) {
                // FAILS if calling set(..) instead of elementChanged
                list.set(0, list.get(0));
            }
        });        
        list.add(new Bean());
       
        assertEquals(2, sListener.changes.size());
        assertEquals(new Change(ListEvent.INSERT, 1), sListener.changes.get(0));
        assertEquals(new Change(ListEvent.UPDATE, 0), sListener.changes.get(1));
       
        assertEquals(2, oListener.changes.size());
        assertEquals(new Change(ListEvent.INSERT, 1), oListener.changes.get(0));
        assertEquals(new Change(ListEvent.UPDATE, 0), oListener.changes.get(1));
    }
   
    public void testSetWithoutPipeline() throws Exception {
        final BasicEventList<Bean> list = new BasicEventList<Bean>();        
        list.add(new Bean());
       
        Listener sListener = new Listener();
        list.addListEventListener(sListener);
       
        // FAILS if listening to olist
        list.addListEventListener(new ListEventListener<Bean>() {
            @Override
            public void listChanged(ListEvent<Bean> listChanges) {
                // FAILS if calling set(..) instead of elementChanged
                list.set(0, list.get(0));
            }
        });        
        list.add(new Bean());
       
        assertEquals(2, sListener.changes.size());
        assertEquals(new Change(ListEvent.INSERT, 1), sListener.changes.get(0));
        assertEquals(new Change(ListEvent.UPDATE, 0), sListener.changes.get(1));
    }
   
    public static class Bean {
        public void addPropertyChangeListener(PropertyChangeListener pcl) {}
        public void removePropertyChangeListener(PropertyChangeListener plc) {}
    }
   
    private static class Listener implements ListEventListener<Bean> {
        private List<Change> changes = new ArrayList<Change>();
       
        @Override
        public void listChanged(ListEvent<Bean> listChanges) {
            while(listChanges.next()) {
                changes.add(new Change(listChanges.getType(), listChanges.getIndex()));
            }
        }  
    }
   
    private static class Change {
        final int idx;
        final int kind;
       
        Change(int kind, int idx) {
            this.idx = idx;
            this.kind = kind;
        }
       
        @Override
        public boolean equals(Object obj) {
            Change o2 = (Change)obj;
            return o2.idx == idx && o2.kind == kind;
        }
       
        @Override
        public String toString() {
            return "idx: " + idx + ", kind: " + kind;
        }
    }

}

---

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...