|
View:
New views
15 Messages
—
Rating Filter:
Alert me
|
|
|
stackoverflow errorI am getting a stackoverflow error while I am iterating through a table scan
http://pastebin.com/mc940416 is there any maintiance type task I should run to make my regionservers are all copacetic? Ananth T Sarathy |
|
|
Re: stackoverflow errorYou using a filter of some kind? What kinda scan is it? Are you on latest
hbase? Easy way to check healthyness is scan of the table. Use count in shell or rowcounter mr job. If you are just doing straight scan, then something is up. Looks at the 0.20.1 bug fixes to see if any address a stackoverflow issue. Otherwise, try to figure why its happening for you and not for the rest of us. St.Ack On Wed, Oct 28, 2009 at 1:51 PM, Ananth T. Sarathy < ananth.t.sarathy@...> wrote: > I am getting a stackoverflow error while I am iterating through a table > scan > > http://pastebin.com/mc940416 > > is there any maintiance type task I should run to make my regionservers are > all copacetic? > > Ananth T Sarathy > |
|
|
Re: stackoverflow errorok... I when I try to do i straight countin the shell
I get 09/10/28 18:08:56 DEBUG client.HTable$ClientScanner: Advancing internal scanner to startKey at 'FileZilla\x203.2.33.2.3http:\x2F\x2Ffilehippo.com\x2Fdownload_filezilla\x2F5408\x2 which has been repeating for a while... Ananth T Sarathy On Wed, Oct 28, 2009 at 5:08 PM, stack <stack@...> wrote: > You using a filter of some kind? What kinda scan is it? Are you on > latest > hbase? Easy way to check healthyness is scan of the table. Use count in > shell or rowcounter mr job. If you are just doing straight scan, then > something is up. Looks at the 0.20.1 bug fixes to see if any address a > stackoverflow issue. Otherwise, try to figure why its happening for you > and > not for the rest of us. > St.Ack > > On Wed, Oct 28, 2009 at 1:51 PM, Ananth T. Sarathy < > ananth.t.sarathy@...> wrote: > > > I am getting a stackoverflow error while I am iterating through a table > > scan > > > > http://pastebin.com/mc940416 > > > > is there any maintiance type task I should run to make my regionservers > are > > all copacetic? > > > > Ananth T Sarathy > > > |
|
|
Re: stackoverflow errorSame region over and over? Then it ain't advancing? Or does it vary?
St.Ack On Wed, Oct 28, 2009 at 3:09 PM, Ananth T. Sarathy < ananth.t.sarathy@...> wrote: > ok... I when I try to do i straight countin the shell > I get > > > 09/10/28 18:08:56 DEBUG client.HTable$ClientScanner: Advancing internal > scanner to startKey at > > 'FileZilla\x203.2.33.2.3http:\x2F\x2Ffilehippo.com\x2Fdownload_filezilla\x2F5408\x2 > > which has been repeating for a while... > Ananth T Sarathy > > > On Wed, Oct 28, 2009 at 5:08 PM, stack <stack@...> wrote: > > > You using a filter of some kind? What kinda scan is it? Are you on > > latest > > hbase? Easy way to check healthyness is scan of the table. Use count in > > shell or rowcounter mr job. If you are just doing straight scan, then > > something is up. Looks at the 0.20.1 bug fixes to see if any address a > > stackoverflow issue. Otherwise, try to figure why its happening for you > > and > > not for the rest of us. > > St.Ack > > > > On Wed, Oct 28, 2009 at 1:51 PM, Ananth T. Sarathy < > > ananth.t.sarathy@...> wrote: > > > > > I am getting a stackoverflow error while I am iterating through a table > > > scan > > > > > > http://pastebin.com/mc940416 > > > > > > is there any maintiance type task I should run to make my regionservers > > are > > > all copacetic? > > > > > > Ananth T Sarathy > > > > > > |
|
|
Re: stackoverflow errorIt's not advancing. I am not sure how you tell which region...
here is some of the output from the shell http://pastebin.org/49134 Where can I look for that info Ananth T Sarathy On Wed, Oct 28, 2009 at 6:24 PM, stack <stack@...> wrote: > Same region over and over? Then it ain't advancing? Or does it vary? > St.Ack > > On Wed, Oct 28, 2009 at 3:09 PM, Ananth T. Sarathy < > ananth.t.sarathy@...> wrote: > > > ok... I when I try to do i straight countin the shell > > I get > > > > > > 09/10/28 18:08:56 DEBUG client.HTable$ClientScanner: Advancing internal > > scanner to startKey at > > > > > 'FileZilla\x203.2.33.2.3http:\x2F\x2Ffilehippo.com\x2Fdownload_filezilla\x2F5408\x2 > > > > which has been repeating for a while... > > Ananth T Sarathy > > > > > > On Wed, Oct 28, 2009 at 5:08 PM, stack <stack@...> wrote: > > > > > You using a filter of some kind? What kinda scan is it? Are you on > > > latest > > > hbase? Easy way to check healthyness is scan of the table. Use count > in > > > shell or rowcounter mr job. If you are just doing straight scan, then > > > something is up. Looks at the 0.20.1 bug fixes to see if any address a > > > stackoverflow issue. Otherwise, try to figure why its happening for > you > > > and > > > not for the rest of us. > > > St.Ack > > > > > > On Wed, Oct 28, 2009 at 1:51 PM, Ananth T. Sarathy < > > > ananth.t.sarathy@...> wrote: > > > > > > > I am getting a stackoverflow error while I am iterating through a > table > > > > scan > > > > > > > > http://pastebin.com/mc940416 > > > > > > > > is there any maintiance type task I should run to make my > regionservers > > > are > > > > all copacetic? > > > > > > > > Ananth T Sarathy > > > > > > > > > > |
|
|
Re: stackoverflow errorBTW this is https://issues.apache.org/jira/browse/HBASE-1781
J-D On Wed, Oct 28, 2009 at 6:24 PM, Ananth T. Sarathy <ananth.t.sarathy@...> wrote: > It's not advancing. I am not sure how you tell which region... > here is some of the output from the shell > http://pastebin.org/49134 > > > Where can I look for that info > Ananth T Sarathy > > > On Wed, Oct 28, 2009 at 6:24 PM, stack <stack@...> wrote: > >> Same region over and over? Then it ain't advancing? Or does it vary? >> St.Ack >> >> On Wed, Oct 28, 2009 at 3:09 PM, Ananth T. Sarathy < >> ananth.t.sarathy@...> wrote: >> >> > ok... I when I try to do i straight countin the shell >> > I get >> > >> > >> > 09/10/28 18:08:56 DEBUG client.HTable$ClientScanner: Advancing internal >> > scanner to startKey at >> > >> > >> 'FileZilla\x203.2.33.2.3http:\x2F\x2Ffilehippo.com\x2Fdownload_filezilla\x2F5408\x2 >> > >> > which has been repeating for a while... >> > Ananth T Sarathy >> > >> > >> > On Wed, Oct 28, 2009 at 5:08 PM, stack <stack@...> wrote: >> > >> > > You using a filter of some kind? What kinda scan is it? Are you on >> > > latest >> > > hbase? Easy way to check healthyness is scan of the table. Use count >> in >> > > shell or rowcounter mr job. If you are just doing straight scan, then >> > > something is up. Looks at the 0.20.1 bug fixes to see if any address a >> > > stackoverflow issue. Otherwise, try to figure why its happening for >> you >> > > and >> > > not for the rest of us. >> > > St.Ack >> > > >> > > On Wed, Oct 28, 2009 at 1:51 PM, Ananth T. Sarathy < >> > > ananth.t.sarathy@...> wrote: >> > > >> > > > I am getting a stackoverflow error while I am iterating through a >> table >> > > > scan >> > > > >> > > > http://pastebin.com/mc940416 >> > > > >> > > > is there any maintiance type task I should run to make my >> regionservers >> > > are >> > > > all copacetic? >> > > > >> > > > Ananth T Sarathy >> > > > >> > > >> > >> > |
|
|
Re: stackoverflow errorI just updated to 0.20.1 and think that I'm seeing this as well.
Here's a sample of the client log: http://pastebin.com/m549fbcfa and a sample of the regionservers: http://pastebin.com/m5a8e89c3 Best I can tell, it seems to be data related in that it's reproducible with gets on specific rows. Other rows in the same region work normally. On Wed, Oct 28, 2009 at 9:25 PM, Jean-Daniel Cryans <jdcryans@...>wrote: > BTW this is https://issues.apache.org/jira/browse/HBASE-1781 > > J-D > > On Wed, Oct 28, 2009 at 6:24 PM, Ananth T. Sarathy > <ananth.t.sarathy@...> wrote: > > It's not advancing. I am not sure how you tell which region... > > here is some of the output from the shell > > http://pastebin.org/49134 > > > > > > Where can I look for that info > > Ananth T Sarathy > > > > > > On Wed, Oct 28, 2009 at 6:24 PM, stack <stack@...> wrote: > > > >> Same region over and over? Then it ain't advancing? Or does it vary? > >> St.Ack > >> > >> On Wed, Oct 28, 2009 at 3:09 PM, Ananth T. Sarathy < > >> ananth.t.sarathy@...> wrote: > >> > >> > ok... I when I try to do i straight countin the shell > >> > I get > >> > > >> > > >> > 09/10/28 18:08:56 DEBUG client.HTable$ClientScanner: Advancing > internal > >> > scanner to startKey at > >> > > >> > > >> > 'FileZilla\x203.2.33.2.3http:\x2F\x2Ffilehippo.com\x2Fdownload_filezilla\x2F5408\x2 > >> > > >> > which has been repeating for a while... > >> > Ananth T Sarathy > >> > > >> > > >> > On Wed, Oct 28, 2009 at 5:08 PM, stack <stack@...> wrote: > >> > > >> > > You using a filter of some kind? What kinda scan is it? Are you > on > >> > > latest > >> > > hbase? Easy way to check healthyness is scan of the table. Use > count > >> in > >> > > shell or rowcounter mr job. If you are just doing straight scan, > then > >> > > something is up. Looks at the 0.20.1 bug fixes to see if any > address a > >> > > stackoverflow issue. Otherwise, try to figure why its happening for > >> you > >> > > and > >> > > not for the rest of us. > >> > > St.Ack > >> > > > >> > > On Wed, Oct 28, 2009 at 1:51 PM, Ananth T. Sarathy < > >> > > ananth.t.sarathy@...> wrote: > >> > > > >> > > > I am getting a stackoverflow error while I am iterating through a > >> table > >> > > > scan > >> > > > > >> > > > http://pastebin.com/mc940416 > >> > > > > >> > > > is there any maintiance type task I should run to make my > >> regionservers > >> > > are > >> > > > all copacetic? > >> > > > > >> > > > Ananth T Sarathy > >> > > > > >> > > > >> > > >> > > > |
|
|
Re: stackoverflow errorWhoever wrote that checkColumn code was obviously trying to be a
smarty-pants but missed a vital check. Any of you fellas have an idea what the columns are that are causing this infinite recursion? Otherwise will poke around tomorrow to see if can create it artificially. Thanks, St.Ack On Wed, Oct 28, 2009 at 6:38 PM, Gary Helmling <ghelmling@...> wrote: > I just updated to 0.20.1 and think that I'm seeing this as well. > > Here's a sample of the client log: http://pastebin.com/m549fbcfa > and a sample of the regionservers: http://pastebin.com/m5a8e89c3 > > Best I can tell, it seems to be data related in that it's reproducible with > gets on specific rows. Other rows in the same region work normally. > > > > On Wed, Oct 28, 2009 at 9:25 PM, Jean-Daniel Cryans <jdcryans@... > >wrote: > > > BTW this is https://issues.apache.org/jira/browse/HBASE-1781 > > > > J-D > > > > On Wed, Oct 28, 2009 at 6:24 PM, Ananth T. Sarathy > > <ananth.t.sarathy@...> wrote: > > > It's not advancing. I am not sure how you tell which region... > > > here is some of the output from the shell > > > http://pastebin.org/49134 > > > > > > > > > Where can I look for that info > > > Ananth T Sarathy > > > > > > > > > On Wed, Oct 28, 2009 at 6:24 PM, stack <stack@...> wrote: > > > > > >> Same region over and over? Then it ain't advancing? Or does it vary? > > >> St.Ack > > >> > > >> On Wed, Oct 28, 2009 at 3:09 PM, Ananth T. Sarathy < > > >> ananth.t.sarathy@...> wrote: > > >> > > >> > ok... I when I try to do i straight countin the shell > > >> > I get > > >> > > > >> > > > >> > 09/10/28 18:08:56 DEBUG client.HTable$ClientScanner: Advancing > > internal > > >> > scanner to startKey at > > >> > > > >> > > > >> > > > 'FileZilla\x203.2.33.2.3http:\x2F\x2Ffilehippo.com\x2Fdownload_filezilla\x2F5408\x2 > > >> > > > >> > which has been repeating for a while... > > >> > Ananth T Sarathy > > >> > > > >> > > > >> > On Wed, Oct 28, 2009 at 5:08 PM, stack <stack@...> wrote: > > >> > > > >> > > You using a filter of some kind? What kinda scan is it? Are you > > on > > >> > > latest > > >> > > hbase? Easy way to check healthyness is scan of the table. Use > > count > > >> in > > >> > > shell or rowcounter mr job. If you are just doing straight scan, > > then > > >> > > something is up. Looks at the 0.20.1 bug fixes to see if any > > address a > > >> > > stackoverflow issue. Otherwise, try to figure why its happening > for > > >> you > > >> > > and > > >> > > not for the rest of us. > > >> > > St.Ack > > >> > > > > >> > > On Wed, Oct 28, 2009 at 1:51 PM, Ananth T. Sarathy < > > >> > > ananth.t.sarathy@...> wrote: > > >> > > > > >> > > > I am getting a stackoverflow error while I am iterating through > a > > >> table > > >> > > > scan > > >> > > > > > >> > > > http://pastebin.com/mc940416 > > >> > > > > > >> > > > is there any maintiance type task I should run to make my > > >> regionservers > > >> > > are > > >> > > > all copacetic? > > >> > > > > > >> > > > Ananth T Sarathy > > >> > > > > > >> > > > > >> > > > >> > > > > > > |
|
|
Re: stackoverflow errorThe FindBugs static code checker flagged WildcardColumnTracker.checkColumn also, null dereference possible on some code paths. I checked in a fix which did not hurt this issue but would not help either. I added null tests and flow control to prevent the issue seen by the checker but did not change (if I was succesful in my intent) any semantics. That code should be refactored/rewritten for easier comprehension and cleaner flow control.
- Andy ________________________________ From: stack <stack@...> To: hbase-user@... Sent: Thu, October 29, 2009 1:35:27 PM Subject: Re: stackoverflow error Whoever wrote that checkColumn code was obviously trying to be a smarty-pants but missed a vital check. Any of you fellas have an idea what the columns are that are causing this infinite recursion? Otherwise will poke around tomorrow to see if can create it artificially. Thanks, St.Ack On Wed, Oct 28, 2009 at 6:38 PM, Gary Helmling <ghelmling@...> wrote: > I just updated to 0.20.1 and think that I'm seeing this as well. > > Here's a sample of the client log: http://pastebin.com/m549fbcfa > and a sample of the regionservers: http://pastebin.com/m5a8e89c3 > > Best I can tell, it seems to be data related in that it's reproducible with > gets on specific rows. Other rows in the same region work normally. > > > > On Wed, Oct 28, 2009 at 9:25 PM, Jean-Daniel Cryans <jdcryans@... > >wrote: > > > BTW this is https://issues.apache.org/jira/browse/HBASE-1781 > > > > J-D > > > > On Wed, Oct 28, 2009 at 6:24 PM, Ananth T. Sarathy > > <ananth.t.sarathy@...> wrote: > > > It's not advancing. I am not sure how you tell which region... > > > here is some of the output from the shell > > > http://pastebin.org/49134 > > > > > > > > > Where can I look for that info > > > Ananth T Sarathy > > > > > > > > > On Wed, Oct 28, 2009 at 6:24 PM, stack <stack@...> wrote: > > > > > >> Same region over and over? Then it ain't advancing? Or does it vary? > > >> St.Ack > > >> > > >> On Wed, Oct 28, 2009 at 3:09 PM, Ananth T. Sarathy < > > >> ananth.t.sarathy@...> wrote: > > >> > > >> > ok... I when I try to do i straight countin the shell > > >> > I get > > >> > > > >> > > > >> > 09/10/28 18:08:56 DEBUG client.HTable$ClientScanner: Advancing > > internal > > >> > scanner to startKey at > > >> > > > >> > > > >> > > > 'FileZilla\x203.2.33.2.3http:\x2F\x2Ffilehippo.com\x2Fdownload_filezilla\x2F5408\x2 > > >> > > > >> > which has been repeating for a while... > > >> > Ananth T Sarathy > > >> > > > >> > > > >> > On Wed, Oct 28, 2009 at 5:08 PM, stack <stack@...> wrote: > > >> > > > >> > > You using a filter of some kind? What kinda scan is it? Are you > > on > > >> > > latest > > >> > > hbase? Easy way to check healthyness is scan of the table. Use > > count > > >> in > > >> > > shell or rowcounter mr job. If you are just doing straight scan, > > then > > >> > > something is up. Looks at the 0.20.1 bug fixes to see if any > > address a > > >> > > stackoverflow issue. Otherwise, try to figure why its happening > for > > >> you > > >> > > and > > >> > > not for the rest of us. > > >> > > St.Ack > > >> > > > > >> > > On Wed, Oct 28, 2009 at 1:51 PM, Ananth T. Sarathy < > > >> > > ananth.t.sarathy@...> wrote: > > >> > > > > >> > > > I am getting a stackoverflow error while I am iterating through > a > > >> table > > >> > > > scan > > >> > > > > > >> > > > http://pastebin.com/mc940416 > > >> > > > > > >> > > > is there any maintiance type task I should run to make my > > >> regionservers > > >> > > are > > >> > > > all copacetic? > > >> > > > > > >> > > > Ananth T Sarathy > > >> > > > > > >> > > > > >> > > > >> > > > > > > |
|
|
Re: stackoverflow errorBy the way that's not an admission of anything. The recursion issue predates the FindBugs warning fix. :-)
________________________________ From: Andrew Purtell <apurtell@...> To: hbase-user@... Sent: Thu, October 29, 2009 5:59:39 PM Subject: Re: stackoverflow error The FindBugs static code checker flagged WildcardColumnTracker.checkColumn also, null dereference possible on some code paths. I checked in a fix which did not hurt this issue but would not help either. I added null tests and flow control to prevent the issue seen by the checker but did not change (if I was succesful in my intent) any semantics. That code should be refactored/rewritten for easier comprehension and cleaner flow control. - Andy ________________________________ From: stack <stack@...> To: hbase-user@... Sent: Thu, October 29, 2009 1:35:27 PM Subject: Re: stackoverflow error Whoever wrote that checkColumn code was obviously trying to be a smarty-pants but missed a vital check. Any of you fellas have an idea what the columns are that are causing this infinite recursion? Otherwise will poke around tomorrow to see if can create it artificially. Thanks, St.Ack On Wed, Oct 28, 2009 at 6:38 PM, Gary Helmling <ghelmling@...> wrote: > I just updated to 0.20.1 and think that I'm seeing this as well. > > Here's a sample of the client log: http://pastebin.com/m549fbcfa > and a sample of the regionservers: http://pastebin.com/m5a8e89c3 > > Best I can tell, it seems to be data related in that it's reproducible with > gets on specific rows. Other rows in the same region work normally. > > > > On Wed, Oct 28, 2009 at 9:25 PM, Jean-Daniel Cryans <jdcryans@... > >wrote: > > > BTW this is https://issues.apache.org/jira/browse/HBASE-1781 > > > > J-D > > > > On Wed, Oct 28, 2009 at 6:24 PM, Ananth T. Sarathy > > <ananth.t.sarathy@...> wrote: > > > It's not advancing. I am not sure how you tell which region... > > > here is some of the output from the shell > > > http://pastebin.org/49134 > > > > > > > > > Where can I look for that info > > > Ananth T Sarathy > > > > > > > > > On Wed, Oct 28, 2009 at 6:24 PM, stack <stack@...> wrote: > > > > > >> Same region over and over? Then it ain't advancing? Or does it vary? > > >> St.Ack > > >> > > >> On Wed, Oct 28, 2009 at 3:09 PM, Ananth T. Sarathy < > > >> ananth.t.sarathy@...> wrote: > > >> > > >> > ok... I when I try to do i straight countin the shell > > >> > I get > > >> > > > >> > > > >> > 09/10/28 18:08:56 DEBUG client.HTable$ClientScanner: Advancing > > internal > > >> > scanner to startKey at > > >> > > > >> > > > >> > > > 'FileZilla\x203.2.33.2.3http:\x2F\x2Ffilehippo.com\x2Fdownload_filezilla\x2F5408\x2 > > >> > > > >> > which has been repeating for a while... > > >> > Ananth T Sarathy > > >> > > > >> > > > >> > On Wed, Oct 28, 2009 at 5:08 PM, stack <stack@...> wrote: > > >> > > > >> > > You using a filter of some kind? What kinda scan is it? Are you > > on > > >> > > latest > > >> > > hbase? Easy way to check healthyness is scan of the table. Use > > count > > >> in > > >> > > shell or rowcounter mr job. If you are just doing straight scan, > > then > > >> > > something is up. Looks at the 0.20.1 bug fixes to see if any > > address a > > >> > > stackoverflow issue. Otherwise, try to figure why its happening > for > > >> you > > >> > > and > > >> > > not for the rest of us. > > >> > > St.Ack > > >> > > > > >> > > On Wed, Oct 28, 2009 at 1:51 PM, Ananth T. Sarathy < > > >> > > ananth.t.sarathy@...> wrote: > > >> > > > > >> > > > I am getting a stackoverflow error while I am iterating through > a > > >> table > > >> > > > scan > > >> > > > > > >> > > > http://pastebin.com/mc940416 > > >> > > > > > >> > > > is there any maintiance type task I should run to make my > > >> regionservers > > >> > > are > > >> > > > all copacetic? > > >> > > > > > >> > > > Ananth T Sarathy > > >> > > > > > >> > > > > >> > > > >> > > > > > > |
|
|
Re: stackoverflow errorNo idea.
Ananth T Sarathy On Thu, Oct 29, 2009 at 5:59 AM, Andrew Purtell <apurtell@...> wrote: > The FindBugs static code checker flagged WildcardColumnTracker.checkColumn > also, null dereference possible on some code paths. I checked in a fix which > did not hurt this issue but would not help either. I added null tests and > flow control to prevent the issue seen by the checker but did not change (if > I was succesful in my intent) any semantics. That code should be > refactored/rewritten for easier comprehension and cleaner flow control. > > - Andy > > > > > ________________________________ > From: stack <stack@...> > To: hbase-user@... > Sent: Thu, October 29, 2009 1:35:27 PM > Subject: Re: stackoverflow error > > Whoever wrote that checkColumn code was obviously trying to be a > smarty-pants but missed a vital check. Any of you fellas have an idea what > the columns are that are causing this infinite recursion? Otherwise will > poke around tomorrow to see if can create it artificially. > > Thanks, > St.Ack > > On Wed, Oct 28, 2009 at 6:38 PM, Gary Helmling <ghelmling@...> > wrote: > > > I just updated to 0.20.1 and think that I'm seeing this as well. > > > > Here's a sample of the client log: http://pastebin.com/m549fbcfa > > and a sample of the regionservers: http://pastebin.com/m5a8e89c3 > > > > Best I can tell, it seems to be data related in that it's reproducible > with > > gets on specific rows. Other rows in the same region work normally. > > > > > > > > On Wed, Oct 28, 2009 at 9:25 PM, Jean-Daniel Cryans <jdcryans@... > > >wrote: > > > > > BTW this is https://issues.apache.org/jira/browse/HBASE-1781 > > > > > > J-D > > > > > > On Wed, Oct 28, 2009 at 6:24 PM, Ananth T. Sarathy > > > <ananth.t.sarathy@...> wrote: > > > > It's not advancing. I am not sure how you tell which region... > > > > here is some of the output from the shell > > > > http://pastebin.org/49134 > > > > > > > > > > > > Where can I look for that info > > > > Ananth T Sarathy > > > > > > > > > > > > On Wed, Oct 28, 2009 at 6:24 PM, stack <stack@...> wrote: > > > > > > > >> Same region over and over? Then it ain't advancing? Or does it > vary? > > > >> St.Ack > > > >> > > > >> On Wed, Oct 28, 2009 at 3:09 PM, Ananth T. Sarathy < > > > >> ananth.t.sarathy@...> wrote: > > > >> > > > >> > ok... I when I try to do i straight countin the shell > > > >> > I get > > > >> > > > > >> > > > > >> > 09/10/28 18:08:56 DEBUG client.HTable$ClientScanner: Advancing > > > internal > > > >> > scanner to startKey at > > > >> > > > > >> > > > > >> > > > > > > 'FileZilla\x203.2.33.2.3http:\x2F\x2Ffilehippo.com\x2Fdownload_filezilla\x2F5408\x2 > > > >> > > > > >> > which has been repeating for a while... > > > >> > Ananth T Sarathy > > > >> > > > > >> > > > > >> > On Wed, Oct 28, 2009 at 5:08 PM, stack <stack@...> wrote: > > > >> > > > > >> > > You using a filter of some kind? What kinda scan is it? Are > you > > > on > > > >> > > latest > > > >> > > hbase? Easy way to check healthyness is scan of the table. Use > > > count > > > >> in > > > >> > > shell or rowcounter mr job. If you are just doing straight > scan, > > > then > > > >> > > something is up. Looks at the 0.20.1 bug fixes to see if any > > > address a > > > >> > > stackoverflow issue. Otherwise, try to figure why its happening > > for > > > >> you > > > >> > > and > > > >> > > not for the rest of us. > > > >> > > St.Ack > > > >> > > > > > >> > > On Wed, Oct 28, 2009 at 1:51 PM, Ananth T. Sarathy < > > > >> > > ananth.t.sarathy@...> wrote: > > > >> > > > > > >> > > > I am getting a stackoverflow error while I am iterating > through > > a > > > >> table > > > >> > > > scan > > > >> > > > > > > >> > > > http://pastebin.com/mc940416 > > > >> > > > > > > >> > > > is there any maintiance type task I should run to make my > > > >> regionservers > > > >> > > are > > > >> > > > all copacetic? > > > >> > > > > > > >> > > > Ananth T Sarathy > > > >> > > > > > > >> > > > > > >> > > > > >> > > > > > > > > > > > > > > |
|
|
Re: stackoverflow errorAnanth,
If you can, please try the patch included in HBASE-1781 (https://issues.apache.org/jira/browse/HBASE-1781) Thx, J-D On Thu, Oct 29, 2009 at 6:41 AM, Ananth T. Sarathy <ananth.t.sarathy@...> wrote: > No idea. > Ananth T Sarathy > > > On Thu, Oct 29, 2009 at 5:59 AM, Andrew Purtell <apurtell@...> wrote: > >> The FindBugs static code checker flagged WildcardColumnTracker.checkColumn >> also, null dereference possible on some code paths. I checked in a fix which >> did not hurt this issue but would not help either. I added null tests and >> flow control to prevent the issue seen by the checker but did not change (if >> I was succesful in my intent) any semantics. That code should be >> refactored/rewritten for easier comprehension and cleaner flow control. >> >> - Andy >> >> >> >> >> ________________________________ >> From: stack <stack@...> >> To: hbase-user@... >> Sent: Thu, October 29, 2009 1:35:27 PM >> Subject: Re: stackoverflow error >> >> Whoever wrote that checkColumn code was obviously trying to be a >> smarty-pants but missed a vital check. Any of you fellas have an idea what >> the columns are that are causing this infinite recursion? Otherwise will >> poke around tomorrow to see if can create it artificially. >> >> Thanks, >> St.Ack >> >> On Wed, Oct 28, 2009 at 6:38 PM, Gary Helmling <ghelmling@...> >> wrote: >> >> > I just updated to 0.20.1 and think that I'm seeing this as well. >> > >> > Here's a sample of the client log: http://pastebin.com/m549fbcfa >> > and a sample of the regionservers: http://pastebin.com/m5a8e89c3 >> > >> > Best I can tell, it seems to be data related in that it's reproducible >> with >> > gets on specific rows. Other rows in the same region work normally. >> > >> > >> > >> > On Wed, Oct 28, 2009 at 9:25 PM, Jean-Daniel Cryans <jdcryans@... >> > >wrote: >> > >> > > BTW this is https://issues.apache.org/jira/browse/HBASE-1781 >> > > >> > > J-D >> > > >> > > On Wed, Oct 28, 2009 at 6:24 PM, Ananth T. Sarathy >> > > <ananth.t.sarathy@...> wrote: >> > > > It's not advancing. I am not sure how you tell which region... >> > > > here is some of the output from the shell >> > > > http://pastebin.org/49134 >> > > > >> > > > >> > > > Where can I look for that info >> > > > Ananth T Sarathy >> > > > >> > > > >> > > > On Wed, Oct 28, 2009 at 6:24 PM, stack <stack@...> wrote: >> > > > >> > > >> Same region over and over? Then it ain't advancing? Or does it >> vary? >> > > >> St.Ack >> > > >> >> > > >> On Wed, Oct 28, 2009 at 3:09 PM, Ananth T. Sarathy < >> > > >> ananth.t.sarathy@...> wrote: >> > > >> >> > > >> > ok... I when I try to do i straight countin the shell >> > > >> > I get >> > > >> > >> > > >> > >> > > >> > 09/10/28 18:08:56 DEBUG client.HTable$ClientScanner: Advancing >> > > internal >> > > >> > scanner to startKey at >> > > >> > >> > > >> > >> > > >> >> > > >> > >> 'FileZilla\x203.2.33.2.3http:\x2F\x2Ffilehippo.com\x2Fdownload_filezilla\x2F5408\x2 >> > > >> > >> > > >> > which has been repeating for a while... >> > > >> > Ananth T Sarathy >> > > >> > >> > > >> > >> > > >> > On Wed, Oct 28, 2009 at 5:08 PM, stack <stack@...> wrote: >> > > >> > >> > > >> > > You using a filter of some kind? What kinda scan is it? Are >> you >> > > on >> > > >> > > latest >> > > >> > > hbase? Easy way to check healthyness is scan of the table. Use >> > > count >> > > >> in >> > > >> > > shell or rowcounter mr job. If you are just doing straight >> scan, >> > > then >> > > >> > > something is up. Looks at the 0.20.1 bug fixes to see if any >> > > address a >> > > >> > > stackoverflow issue. Otherwise, try to figure why its happening >> > for >> > > >> you >> > > >> > > and >> > > >> > > not for the rest of us. >> > > >> > > St.Ack >> > > >> > > >> > > >> > > On Wed, Oct 28, 2009 at 1:51 PM, Ananth T. Sarathy < >> > > >> > > ananth.t.sarathy@...> wrote: >> > > >> > > >> > > >> > > > I am getting a stackoverflow error while I am iterating >> through >> > a >> > > >> table >> > > >> > > > scan >> > > >> > > > >> > > >> > > > http://pastebin.com/mc940416 >> > > >> > > > >> > > >> > > > is there any maintiance type task I should run to make my >> > > >> regionservers >> > > >> > > are >> > > >> > > > all copacetic? >> > > >> > > > >> > > >> > > > Ananth T Sarathy >> > > >> > > > >> > > >> > > >> > > >> > >> > > >> >> > > > >> > > >> > >> >> >> >> >> > |
|
|
Re: stackoverflow errorWell I found a fix that works, currently testing.
J-D On Tue, Nov 3, 2009 at 2:15 PM, Jean-Daniel Cryans <jdcryans@...> wrote: > Ananth, > > If you can, please try the patch included in HBASE-1781 > (https://issues.apache.org/jira/browse/HBASE-1781) > > Thx, > > J-D > > On Thu, Oct 29, 2009 at 6:41 AM, Ananth T. Sarathy > <ananth.t.sarathy@...> wrote: >> No idea. >> Ananth T Sarathy >> >> >> On Thu, Oct 29, 2009 at 5:59 AM, Andrew Purtell <apurtell@...> wrote: >> >>> The FindBugs static code checker flagged WildcardColumnTracker.checkColumn >>> also, null dereference possible on some code paths. I checked in a fix which >>> did not hurt this issue but would not help either. I added null tests and >>> flow control to prevent the issue seen by the checker but did not change (if >>> I was succesful in my intent) any semantics. That code should be >>> refactored/rewritten for easier comprehension and cleaner flow control. >>> >>> - Andy >>> >>> >>> >>> >>> ________________________________ >>> From: stack <stack@...> >>> To: hbase-user@... >>> Sent: Thu, October 29, 2009 1:35:27 PM >>> Subject: Re: stackoverflow error >>> >>> Whoever wrote that checkColumn code was obviously trying to be a >>> smarty-pants but missed a vital check. Any of you fellas have an idea what >>> the columns are that are causing this infinite recursion? Otherwise will >>> poke around tomorrow to see if can create it artificially. >>> >>> Thanks, >>> St.Ack >>> >>> On Wed, Oct 28, 2009 at 6:38 PM, Gary Helmling <ghelmling@...> >>> wrote: >>> >>> > I just updated to 0.20.1 and think that I'm seeing this as well. >>> > >>> > Here's a sample of the client log: http://pastebin.com/m549fbcfa >>> > and a sample of the regionservers: http://pastebin.com/m5a8e89c3 >>> > >>> > Best I can tell, it seems to be data related in that it's reproducible >>> with >>> > gets on specific rows. Other rows in the same region work normally. >>> > >>> > >>> > >>> > On Wed, Oct 28, 2009 at 9:25 PM, Jean-Daniel Cryans <jdcryans@... >>> > >wrote: >>> > >>> > > BTW this is https://issues.apache.org/jira/browse/HBASE-1781 >>> > > >>> > > J-D >>> > > >>> > > On Wed, Oct 28, 2009 at 6:24 PM, Ananth T. Sarathy >>> > > <ananth.t.sarathy@...> wrote: >>> > > > It's not advancing. I am not sure how you tell which region... >>> > > > here is some of the output from the shell >>> > > > http://pastebin.org/49134 >>> > > > >>> > > > >>> > > > Where can I look for that info >>> > > > Ananth T Sarathy >>> > > > >>> > > > >>> > > > On Wed, Oct 28, 2009 at 6:24 PM, stack <stack@...> wrote: >>> > > > >>> > > >> Same region over and over? Then it ain't advancing? Or does it >>> vary? >>> > > >> St.Ack >>> > > >> >>> > > >> On Wed, Oct 28, 2009 at 3:09 PM, Ananth T. Sarathy < >>> > > >> ananth.t.sarathy@...> wrote: >>> > > >> >>> > > >> > ok... I when I try to do i straight countin the shell >>> > > >> > I get >>> > > >> > >>> > > >> > >>> > > >> > 09/10/28 18:08:56 DEBUG client.HTable$ClientScanner: Advancing >>> > > internal >>> > > >> > scanner to startKey at >>> > > >> > >>> > > >> > >>> > > >> >>> > > >>> > >>> 'FileZilla\x203.2.33.2.3http:\x2F\x2Ffilehippo.com\x2Fdownload_filezilla\x2F5408\x2 >>> > > >> > >>> > > >> > which has been repeating for a while... >>> > > >> > Ananth T Sarathy >>> > > >> > >>> > > >> > >>> > > >> > On Wed, Oct 28, 2009 at 5:08 PM, stack <stack@...> wrote: >>> > > >> > >>> > > >> > > You using a filter of some kind? What kinda scan is it? Are >>> you >>> > > on >>> > > >> > > latest >>> > > >> > > hbase? Easy way to check healthyness is scan of the table. Use >>> > > count >>> > > >> in >>> > > >> > > shell or rowcounter mr job. If you are just doing straight >>> scan, >>> > > then >>> > > >> > > something is up. Looks at the 0.20.1 bug fixes to see if any >>> > > address a >>> > > >> > > stackoverflow issue. Otherwise, try to figure why its happening >>> > for >>> > > >> you >>> > > >> > > and >>> > > >> > > not for the rest of us. >>> > > >> > > St.Ack >>> > > >> > > >>> > > >> > > On Wed, Oct 28, 2009 at 1:51 PM, Ananth T. Sarathy < >>> > > >> > > ananth.t.sarathy@...> wrote: >>> > > >> > > >>> > > >> > > > I am getting a stackoverflow error while I am iterating >>> through >>> > a >>> > > >> table >>> > > >> > > > scan >>> > > >> > > > >>> > > >> > > > http://pastebin.com/mc940416 >>> > > >> > > > >>> > > >> > > > is there any maintiance type task I should run to make my >>> > > >> regionservers >>> > > >> > > are >>> > > >> > > > all copacetic? >>> > > >> > > > >>> > > >> > > > Ananth T Sarathy >>> > > >> > > > >>> > > >> > > >>> > > >> > >>> > > >> >>> > > > >>> > > >>> > >>> >>> >>> >>> >>> >> > |
|
|
Re: stackoverflow errorSorry, you still want to me test it? I have been sick.
Ananth T Sarathy On Tue, Nov 3, 2009 at 11:11 PM, Jean-Daniel Cryans <jdcryans@...>wrote: > Well I found a fix that works, currently testing. > > J-D > > On Tue, Nov 3, 2009 at 2:15 PM, Jean-Daniel Cryans <jdcryans@...> > wrote: > > Ananth, > > > > If you can, please try the patch included in HBASE-1781 > > (https://issues.apache.org/jira/browse/HBASE-1781) > > > > Thx, > > > > J-D > > > > On Thu, Oct 29, 2009 at 6:41 AM, Ananth T. Sarathy > > <ananth.t.sarathy@...> wrote: > >> No idea. > >> Ananth T Sarathy > >> > >> > >> On Thu, Oct 29, 2009 at 5:59 AM, Andrew Purtell <apurtell@...> > wrote: > >> > >>> The FindBugs static code checker flagged > WildcardColumnTracker.checkColumn > >>> also, null dereference possible on some code paths. I checked in a fix > which > >>> did not hurt this issue but would not help either. I added null tests > and > >>> flow control to prevent the issue seen by the checker but did not > change (if > >>> I was succesful in my intent) any semantics. That code should be > >>> refactored/rewritten for easier comprehension and cleaner flow control. > >>> > >>> - Andy > >>> > >>> > >>> > >>> > >>> ________________________________ > >>> From: stack <stack@...> > >>> To: hbase-user@... > >>> Sent: Thu, October 29, 2009 1:35:27 PM > >>> Subject: Re: stackoverflow error > >>> > >>> Whoever wrote that checkColumn code was obviously trying to be a > >>> smarty-pants but missed a vital check. Any of you fellas have an idea > what > >>> the columns are that are causing this infinite recursion? Otherwise > will > >>> poke around tomorrow to see if can create it artificially. > >>> > >>> Thanks, > >>> St.Ack > >>> > >>> On Wed, Oct 28, 2009 at 6:38 PM, Gary Helmling <ghelmling@...> > >>> wrote: > >>> > >>> > I just updated to 0.20.1 and think that I'm seeing this as well. > >>> > > >>> > Here's a sample of the client log: http://pastebin.com/m549fbcfa > >>> > and a sample of the regionservers: http://pastebin.com/m5a8e89c3 > >>> > > >>> > Best I can tell, it seems to be data related in that it's > reproducible > >>> with > >>> > gets on specific rows. Other rows in the same region work normally. > >>> > > >>> > > >>> > > >>> > On Wed, Oct 28, 2009 at 9:25 PM, Jean-Daniel Cryans < > jdcryans@... > >>> > >wrote: > >>> > > >>> > > BTW this is https://issues.apache.org/jira/browse/HBASE-1781 > >>> > > > >>> > > J-D > >>> > > > >>> > > On Wed, Oct 28, 2009 at 6:24 PM, Ananth T. Sarathy > >>> > > <ananth.t.sarathy@...> wrote: > >>> > > > It's not advancing. I am not sure how you tell which region... > >>> > > > here is some of the output from the shell > >>> > > > http://pastebin.org/49134 > >>> > > > > >>> > > > > >>> > > > Where can I look for that info > >>> > > > Ananth T Sarathy > >>> > > > > >>> > > > > >>> > > > On Wed, Oct 28, 2009 at 6:24 PM, stack <stack@...> wrote: > >>> > > > > >>> > > >> Same region over and over? Then it ain't advancing? Or does it > >>> vary? > >>> > > >> St.Ack > >>> > > >> > >>> > > >> On Wed, Oct 28, 2009 at 3:09 PM, Ananth T. Sarathy < > >>> > > >> ananth.t.sarathy@...> wrote: > >>> > > >> > >>> > > >> > ok... I when I try to do i straight countin the shell > >>> > > >> > I get > >>> > > >> > > >>> > > >> > > >>> > > >> > 09/10/28 18:08:56 DEBUG client.HTable$ClientScanner: Advancing > >>> > > internal > >>> > > >> > scanner to startKey at > >>> > > >> > > >>> > > >> > > >>> > > >> > >>> > > > >>> > > >>> > 'FileZilla\x203.2.33.2.3http:\x2F\x2Ffilehippo.com\x2Fdownload_filezilla\x2F5408\x2 > >>> > > >> > > >>> > > >> > which has been repeating for a while... > >>> > > >> > Ananth T Sarathy > >>> > > >> > > >>> > > >> > > >>> > > >> > On Wed, Oct 28, 2009 at 5:08 PM, stack <stack@...> > wrote: > >>> > > >> > > >>> > > >> > > You using a filter of some kind? What kinda scan is it? > Are > >>> you > >>> > > on > >>> > > >> > > latest > >>> > > >> > > hbase? Easy way to check healthyness is scan of the table. > Use > >>> > > count > >>> > > >> in > >>> > > >> > > shell or rowcounter mr job. If you are just doing straight > >>> scan, > >>> > > then > >>> > > >> > > something is up. Looks at the 0.20.1 bug fixes to see if > any > >>> > > address a > >>> > > >> > > stackoverflow issue. Otherwise, try to figure why its > happening > >>> > for > >>> > > >> you > >>> > > >> > > and > >>> > > >> > > not for the rest of us. > >>> > > >> > > St.Ack > >>> > > >> > > > >>> > > >> > > On Wed, Oct 28, 2009 at 1:51 PM, Ananth T. Sarathy < > >>> > > >> > > ananth.t.sarathy@...> wrote: > >>> > > >> > > > >>> > > >> > > > I am getting a stackoverflow error while I am iterating > >>> through > >>> > a > >>> > > >> table > >>> > > >> > > > scan > >>> > > >> > > > > >>> > > >> > > > http://pastebin.com/mc940416 > >>> > > >> > > > > >>> > > >> > > > is there any maintiance type task I should run to make my > >>> > > >> regionservers > >>> > > >> > > are > >>> > > >> > > > all copacetic? > >>> > > >> > > > > >>> > > >> > > > Ananth T Sarathy > >>> > > >> > > > > >>> > > >> > > > >>> > > >> > > >>> > > >> > >>> > > > > >>> > > > >>> > > >>> > >>> > >>> > >>> > >>> > >> > > > |
|
|
Re: stackoverflow errorWell it's fixed and committed in the 0.20 branch. You can do a svn
checkout or wait for 0.20.2 (very soon). J-D On Wed, Nov 4, 2009 at 9:45 AM, Ananth T. Sarathy <ananth.t.sarathy@...> wrote: > Sorry, you still want to me test it? I have been sick. > Ananth T Sarathy > > > On Tue, Nov 3, 2009 at 11:11 PM, Jean-Daniel Cryans <jdcryans@...>wrote: > >> Well I found a fix that works, currently testing. >> >> J-D >> >> On Tue, Nov 3, 2009 at 2:15 PM, Jean-Daniel Cryans <jdcryans@...> >> wrote: >> > Ananth, >> > >> > If you can, please try the patch included in HBASE-1781 >> > (https://issues.apache.org/jira/browse/HBASE-1781) >> > >> > Thx, >> > >> > J-D >> > >> > On Thu, Oct 29, 2009 at 6:41 AM, Ananth T. Sarathy >> > <ananth.t.sarathy@...> wrote: >> >> No idea. >> >> Ananth T Sarathy >> >> >> >> >> >> On Thu, Oct 29, 2009 at 5:59 AM, Andrew Purtell <apurtell@...> >> wrote: >> >> >> >>> The FindBugs static code checker flagged >> WildcardColumnTracker.checkColumn >> >>> also, null dereference possible on some code paths. I checked in a fix >> which >> >>> did not hurt this issue but would not help either. I added null tests >> and >> >>> flow control to prevent the issue seen by the checker but did not >> change (if >> >>> I was succesful in my intent) any semantics. That code should be >> >>> refactored/rewritten for easier comprehension and cleaner flow control. >> >>> >> >>> - Andy >> >>> >> >>> >> >>> >> >>> >> >>> ________________________________ >> >>> From: stack <stack@...> >> >>> To: hbase-user@... >> >>> Sent: Thu, October 29, 2009 1:35:27 PM >> >>> Subject: Re: stackoverflow error >> >>> >> >>> Whoever wrote that checkColumn code was obviously trying to be a >> >>> smarty-pants but missed a vital check. Any of you fellas have an idea >> what >> >>> the columns are that are causing this infinite recursion? Otherwise >> will >> >>> poke around tomorrow to see if can create it artificially. >> >>> >> >>> Thanks, >> >>> St.Ack >> >>> >> >>> On Wed, Oct 28, 2009 at 6:38 PM, Gary Helmling <ghelmling@...> >> >>> wrote: >> >>> >> >>> > I just updated to 0.20.1 and think that I'm seeing this as well. >> >>> > >> >>> > Here's a sample of the client log: http://pastebin.com/m549fbcfa >> >>> > and a sample of the regionservers: http://pastebin.com/m5a8e89c3 >> >>> > >> >>> > Best I can tell, it seems to be data related in that it's >> reproducible >> >>> with >> >>> > gets on specific rows. Other rows in the same region work normally. >> >>> > >> >>> > >> >>> > >> >>> > On Wed, Oct 28, 2009 at 9:25 PM, Jean-Daniel Cryans < >> jdcryans@... >> >>> > >wrote: >> >>> > >> >>> > > BTW this is https://issues.apache.org/jira/browse/HBASE-1781 >> >>> > > >> >>> > > J-D >> >>> > > >> >>> > > On Wed, Oct 28, 2009 at 6:24 PM, Ananth T. Sarathy >> >>> > > <ananth.t.sarathy@...> wrote: >> >>> > > > It's not advancing. I am not sure how you tell which region... >> >>> > > > here is some of the output from the shell >> >>> > > > http://pastebin.org/49134 >> >>> > > > >> >>> > > > >> >>> > > > Where can I look for that info >> >>> > > > Ananth T Sarathy >> >>> > > > >> >>> > > > >> >>> > > > On Wed, Oct 28, 2009 at 6:24 PM, stack <stack@...> wrote: >> >>> > > > >> >>> > > >> Same region over and over? Then it ain't advancing? Or does it >> >>> vary? >> >>> > > >> St.Ack >> >>> > > >> >> >>> > > >> On Wed, Oct 28, 2009 at 3:09 PM, Ananth T. Sarathy < >> >>> > > >> ananth.t.sarathy@...> wrote: >> >>> > > >> >> >>> > > >> > ok... I when I try to do i straight countin the shell >> >>> > > >> > I get >> >>> > > >> > >> >>> > > >> > >> >>> > > >> > 09/10/28 18:08:56 DEBUG client.HTable$ClientScanner: Advancing >> >>> > > internal >> >>> > > >> > scanner to startKey at >> >>> > > >> > >> >>> > > >> > >> >>> > > >> >> >>> > > >> >>> > >> >>> >> 'FileZilla\x203.2.33.2.3http:\x2F\x2Ffilehippo.com\x2Fdownload_filezilla\x2F5408\x2 >> >>> > > >> > >> >>> > > >> > which has been repeating for a while... >> >>> > > >> > Ananth T Sarathy >> >>> > > >> > >> >>> > > >> > >> >>> > > >> > On Wed, Oct 28, 2009 at 5:08 PM, stack <stack@...> >> wrote: >> >>> > > >> > >> >>> > > >> > > You using a filter of some kind? What kinda scan is it? >> Are >> >>> you >> >>> > > on >> >>> > > >> > > latest >> >>> > > >> > > hbase? Easy way to check healthyness is scan of the table. >> Use >> >>> > > count >> >>> > > >> in >> >>> > > >> > > shell or rowcounter mr job. If you are just doing straight >> >>> scan, >> >>> > > then >> >>> > > >> > > something is up. Looks at the 0.20.1 bug fixes to see if >> any >> >>> > > address a >> >>> > > >> > > stackoverflow issue. Otherwise, try to figure why its >> happening >> >>> > for >> >>> > > >> you >> >>> > > >> > > and >> >>> > > >> > > not for the rest of us. >> >>> > > >> > > St.Ack >> >>> > > >> > > >> >>> > > >> > > On Wed, Oct 28, 2009 at 1:51 PM, Ananth T. Sarathy < >> >>> > > >> > > ananth.t.sarathy@...> wrote: >> >>> > > >> > > >> >>> > > >> > > > I am getting a stackoverflow error while I am iterating >> >>> through >> >>> > a >> >>> > > >> table >> >>> > > >> > > > scan >> >>> > > >> > > > >> >>> > > >> > > > http://pastebin.com/mc940416 >> >>> > > >> > > > >> >>> > > >> > > > is there any maintiance type task I should run to make my >> >>> > > >> regionservers >> >>> > > >> > > are >> >>> > > >> > > > all copacetic? >> >>> > > >> > > > >> >>> > > >> > > > Ananth T Sarathy >> >>> > > >> > > > >> >>> > > >> > > >> >>> > > >> > >> >>> > > >> >> >>> > > > >> >>> > > >> >>> > >> >>> >> >>> >> >>> >> >>> >> >>> >> >> >> > >> > |
| Free embeddable forum powered by Nabble | Forum Help |