« Return to Thread: List construction for 'pivoting' list data

re[2]: List construction for 'pivoting' list data

by Kevin Day-7 :: Rate this Message:

Yeah, I hear you - the type of thing I'm going for isn't exactly a pivot table, it's more of a collectionlist inversion, I guess - or maybe a multi-valued grouping list where a single element in the source can wind up in multiple groups?? Not sure exactly what to call it.

As for API design for something like this, I envision something very similar to GroupingList, but instead of passing in a Comparator, the user would pass in an object that would provide the keys for a given list element (similar to CollectionList.Model, I guess). That's easy enough. But I'm struggling a bit with what the internal data structures of such a beast would look like.

I suppose creating another list with objects that hold the key and the original element is an obvious first step. Then grouping that other list by key.

This seems to open another avenue of solution here using existing GL constructs (I know you recommended against, but I can't help it)... I could use a CollectionList model to create FunctionLists that tranform each key list into a list of intermediate objects (holding the key and the original element). Then put that into a GroupingList with a comparitor that pulls the key value out of the intermediate object.

I think my pipeline would be:

EventList<DataWithKeys> --> CollectionList<DataWithKeys, KeyedData> -> GroupingList<KeyedData>

and the model for the CollectionList would be:

public List<KeyedData> getChildren(DataWithKeys parent){

KeyedDataFunction f = new KeyedDataFunction(parent);

return new FunctionList<DataWithKeys, KeyedData>(parent.getKeys(), f);

}

and KeyedDataFunction would be:

public KeyedData evaluate(Key sourceValue){

return new KeyedData(data, sourceValue);

}

Any comments on that type of approach? (And if you still think it would be better to construct something from scratch, please definitely let me know).

- K

----------------------- Original Message -----------------------

From: James Lemieux jplemieux@...

To: users@...

Cc:

Date: Wed, 20 May 2009 12:27:31 -0700

Subject: Re: List construction for 'pivoting' list data

If I was going to design a Pivot Solution to work with GL, I don't think I'd use too many GL existing transformations to accomplish it. It may seem tempting, but the potential to have a flood of ListEvents running everywhere is great.

It might be best to simply start with a ListEventListener and write your own logic for generating a pivot table from a raw list. If you want to make it generic, design an interface that you expect API users to implement which provides the tools you need to make decisions about the pivot data. e.g.: how many rows / columns will my pivot table have? what are the criteria by which data is judged for membership in each of those columns / rows?... these are questions you may want to consult a user implementation for....

Just a thought...

James

On Wed, May 20, 2009 at 8:50 AM, Kevin Day <kevin@...> wrote:

Hi all-

If anyone has any thoughts on this challenge, please share.

We have a list of data objects that themselves contain lists of keys. We need to transform this list into another list of lists, but grouped by each key value (note that a single Data object is associated with a given key only once, but could be associated with multiple keys

To make this concrete, here's the source list:

EventList<Data> allData = {

    OBJ1+{ABC, DEF, GHI},

    OBJ2+{ABC, GHI},

    OBJ3+{DEF}

    }

and here's the desired result list:

EventList<Data> dataForKeys={

    ABC+{OBJ1, OBJ2},

    DEF+{OBJ1, OBJ3},

    GHI+{OBJ1, OBJ2}

    }

We will probably have 500-2000 data objects, and 500-1500 different keys. Each data object will probably have between 1 and 3 keys associated with it, although it could be as high as 500 in some instances.

So far, I've come up with two approaches:

Approach 1:

Use a CollectionList against allData to get the list of keys, then apply UniqueList to get individual keys. Then apply a FunctionList against the list of keys that creates a FilterList for each key (filtering allData by the key passed into the Function). This approach has two issues (that I see, anyway) - first, there is an odd dependency in here that will probably require a related subject or listener to be registered with the publisher. I haven't tracked this down yet (if anyone has any quick pointers, I'd appreciate it)  The test harness I put together yeterday evening definitely has problems with the order that list updates occur. The bigger problem that I see with this approach is that we could wind up with a whole lot of FilterLists that are filtering down to a very small sub-set of the original data set. Many of the FilterLists will be returning just a single element. The thought of having 500 filters lists all running at the same time gives me an uncomfortable feeling (but maybe I'm concerned over nothing?)

Approach 2:

Some sort of map based approach. For this to work, I'd need to have a multi-map implementation that would allow for a list of keys to be specified (instead of a single key). And that list would have to be an EventList. Then the list returned from the multi-map would also have to be an eventlist. The issue with this approach is that it's not even remotely close to practical given the current GL implementations. I could see trying to exapand the GL map wrapper so that it returns live lists (I think the obstacle here is that delete operations from the source list do not contain the deleted element, so it could be a right bugger to figure out how to remove elements from the mapped lists). Support for multi-keys would also require information about the removed element. If this approach were possible, I think that it would certainly be performant.

Are there other approaches that may be better (or just different) than these two?

Thanks in advance,

- Kevin
--------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscribe@... For additional commands, e-mail: users-help@...

--------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscribe@... For additional commands, e-mail: users-help@...

« Return to Thread: List construction for 'pivoting' list data