Search against a large, rapidly changing data set?
|
View:
New views
4 Messages
—
Rating Filter:
Alert me
|
 |
Search against a large, rapidly changing data set?
by Alan Wexelblat-2
::
Rate this Message:
I'm going to guess I'm delving into sufficiently esoteric areas that
nobody will have an answer, but we are smarter than me, so here goes: I'm trying to improve one of our key search interfaces. The use cases involve people making searches against a large (hundreds of thousands of records) data set. To make matters more complicated, the data set changes very rapidly, to the point where any set of search results we can return may well be inaccurate or incomplete by the time it's returned. (*) Right now we allow unbounded searches, but truncate the result set at an arbitrary size. Result sets are timestamped so that people know the data were accurate as of the timestamp. My intuition and informal user research tells me that people don't really want these large data sets. They want more focused results. The typical interaction patterns I know to accomplish this are search-within-search, and faceted search. However, both patterns are confounded by the rapid pace of data change: - Search-within-search would result in increasingly inaccurate results as searches were performed against outdated information or might confuse people if we re-ran the search against the updated data, since the second result wouldn't be a true subset of the first result, but rather an updated subset. - faceted search interfaces typically give people a size for their too-large query, and then give actual results when the query parameters have been narrowed down to the point where the result set size is "reasonable." (for whatever definition of reasonable fits the problem domain). In my domain, the rapid change in the data confounds this process because the sizeof() queries are only accurate at the time they're performed and while we might tell the person that he'd get back 100 records based on the data now by the time the query has run he might get back 1000 records. Or 10,000. So it's not inherently clear to me that faceted search would help either. Has anyone tried anything like this or have any thoughts/insights to share about this problem? Best regards, --Alan (*) There's a different problem here of people wanting to monitor the changes, rather than perform static searches, but that's not what this song is about. ________________________________________________________________ Welcome to the Interaction Design Association (IxDA)! To post to this list ....... discuss@... Unsubscribe ................ http://www.ixda.org/unsubscribe List Guidelines ............ http://www.ixda.org/guidelines List Help .................. http://www.ixda.org/help |
|
|
Re: Search against a large, rapidly changing data set?
by jennifer.r.vignone
::
Rate this Message:
What do you mean when you use the term "accurate" to describe the results? I think I can see what you're getting at, but I would venture to say that any result set that is a true reflection of the criteria entered is "accurate". It is just that the data may have updated due to the rapidly changing addition/subtractions/and so on. That wouldn't make the results incorrect, as they were performed against the data available at that time.
That being said, it seems that you want to timestamp the results from the moment the user clicks "search" in order to give that results set a definition against the changing data. From that point on, any further filtering would be against that result set. If the user wanted to re-run it against what is "current" (in other words, may have changed since the running of the initial search) then the user could: -- re-run the search against the most recent data --"refresh" the results against the most recent data I would allow the user to save: -- search criteria -- any results set, which would automatically be dated (and that date perhaps un-editable so it wasn't lost). You could open each new/updated search in a new window or tab. If they were to refresh, you may want to highlight where data had changed against the earlier result(s). It depends on what the users want, need, is useful, have tolerance for (too much data isn't always a good thing). In regard to the size of the results set, you could, as part of the criteria, ask the user to bring back results in order of the percentage of matching to the criteria, or in chunks ("show me 25/50/75/ALL results at a time"). I hope this helps or furthers the ideas along. I love 'search-related' topics Jennifer ========================================================== -----Original Message----- From: discuss-bounces@... [mailto:discuss-bounces@...] On Behalf Of Alan Wexelblat Sent: Monday, November 02, 2009 5:16 PM To: list IXDA Subject: [IxDA Discuss] Search against a large, rapidly changing data set? I'm going to guess I'm delving into sufficiently esoteric areas that nobody will have an answer, but we are smarter than me, so here goes: Best regards, --Alan (*) There's a different problem here of people wanting to monitor the changes, rather than perform static searches, but that's not what this song is about. ________________________________________________________________ Welcome to the Interaction Design Association (IxDA)! To post to this list ....... discuss@... Unsubscribe ................ http://www.ixda.org/unsubscribe List Guidelines ............ http://www.ixda.org/guidelines List Help .................. http://www.ixda.org/help This email is confidential and subject to important disclaimers and conditions including on offers for the purchase or sale of securities, accuracy and completeness of information, viruses, confidentiality, legal privilege, and legal entity disclaimers, available at http://www.jpmorgan.com/pages/disclosures/email. ________________________________________________________________ Welcome to the Interaction Design Association (IxDA)! To post to this list ....... discuss@... Unsubscribe ................ http://www.ixda.org/unsubscribe List Guidelines ............ http://www.ixda.org/guidelines List Help .................. http://www.ixda.org/help |
|
|
|
|
|
|
| Free embeddable forum powered by Nabble | Forum Help |
