|
»
»
reindexed data on master not replicated to slave
|
View:
New views
15 Messages
—
Rating Filter:
Alert me
|
 |
reindexed data on master not replicated to slave
by solr jay
::
Rate this Message:
Hi,
When index data were corrupted on master instance, I wanted to wipe out all the index data and re-index everything. I was hoping the newly created index data would be replicated to slaves, but it wasn't. Here are the steps I performed: 1. stop master 2. delete the directory 'index' 3. start master 4. disable replication on master 5. index all data from scratch 6. enable replication on master It seemed from log file that the slave instances discovered that new index are available and claimed that new index installed, and then trying to update index properties, but looking into the index directory on slaves, you will find that no index data files were updated or added, plus slaves keep trying to get new index. Here are some from slave's log file: Jul 1, 2009 3:59:33 PM org.apache.solr.handler.SnapPuller fetchLatestIndex INFO: Starting replication process Jul 1, 2009 3:59:33 PM org.apache.solr.handler.SnapPuller fetchLatestIndex INFO: Number of files in latest snapshot in master: 69 Jul 1, 2009 3:59:33 PM org.apache.solr.handler.SnapPuller fetchLatestIndex INFO: Total time taken for download : 0 secs Jul 1, 2009 3:59:33 PM org.apache.solr.handler.SnapPuller fetchLatestIndex INFO: Conf files are not downloaded or are in sync Jul 1, 2009 3:59:33 PM org.apache.solr.handler.SnapPuller modifyIndexProps INFO: New index installed. Updating index properties... Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller fetchLatestIndex INFO: Master's version: 1246488421310, generation: 9 Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller fetchLatestIndex INFO: Slave's version: 1246385166228, generation: 56 Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller fetchLatestIndex INFO: Starting replication process Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller fetchLatestIndex INFO: Number of files in latest snapshot in master: 69 Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller fetchLatestIndex INFO: Total time taken for download : 0 secs Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller fetchLatestIndex INFO: Conf files are not downloaded or are in sync Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller modifyIndexProps INFO: New index installed. Updating index properties... Is this process incorrect, or it is a bug? If the process is incorrect, what is the right process? Thanks, J |
|
|
Re: reindexed data on master not replicated to slave
by Otis Gospodnetic
::
Rate this Message:
Jay, You didn't mention which version of Solr you are using. It looks like some trunk or nightly version. Maybe you can try the latest nightly? Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch ----- Original Message ---- > From: solr jay <solrjay@...> > To: solr-user@... > Sent: Thursday, July 2, 2009 9:14:48 PM > Subject: reindexed data on master not replicated to slave > > Hi, > > When index data were corrupted on master instance, I wanted to wipe out all > the index data and re-index everything. I was hoping the newly created index > data would be replicated to slaves, but it wasn't. > > Here are the steps I performed: > > 1. stop master > 2. delete the directory 'index' > 3. start master > 4. disable replication on master > 5. index all data from scratch > 6. enable replication on master > > It seemed from log file that the slave instances discovered that new index > are available and claimed that new index installed, and then trying to > update index properties, but looking into the index directory on slaves, you > will find that no index data files were updated or added, plus slaves keep > trying to get new index. Here are some from slave's log file: > > Jul 1, 2009 3:59:33 PM org.apache.solr.handler.SnapPuller fetchLatestIndex > INFO: Starting replication process > Jul 1, 2009 3:59:33 PM org.apache.solr.handler.SnapPuller fetchLatestIndex > INFO: Number of files in latest snapshot in master: 69 > Jul 1, 2009 3:59:33 PM org.apache.solr.handler.SnapPuller fetchLatestIndex > INFO: Total time taken for download : 0 secs > Jul 1, 2009 3:59:33 PM org.apache.solr.handler.SnapPuller fetchLatestIndex > INFO: Conf files are not downloaded or are in sync > Jul 1, 2009 3:59:33 PM org.apache.solr.handler.SnapPuller modifyIndexProps > INFO: New index installed. Updating index properties... > Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller fetchLatestIndex > INFO: Master's version: 1246488421310, generation: 9 > Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller fetchLatestIndex > INFO: Slave's version: 1246385166228, generation: 56 > Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller fetchLatestIndex > INFO: Starting replication process > Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller fetchLatestIndex > INFO: Number of files in latest snapshot in master: 69 > Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller fetchLatestIndex > INFO: Total time taken for download : 0 secs > Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller fetchLatestIndex > INFO: Conf files are not downloaded or are in sync > Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller modifyIndexProps > INFO: New index installed. Updating index properties... > > > Is this process incorrect, or it is a bug? If the process is incorrect, what > is the right process? > > Thanks, > > J |
|
|
Re: reindexed data on master not replicated to slave
by solr jay
::
Rate this Message:
it's nightly build of May 10. I'll try the latest.
Thanks, J On Thu, Jul 2, 2009 at 8:09 PM, Otis Gospodnetic <otis_gospodnetic@... > wrote: > > Jay, > > You didn't mention which version of Solr you are using. It looks like some > trunk or nightly version. Maybe you can try the latest nightly? > > Otis > -- > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch > > > > ----- Original Message ---- > > From: solr jay <solrjay@...> > > To: solr-user@... > > Sent: Thursday, July 2, 2009 9:14:48 PM > > Subject: reindexed data on master not replicated to slave > > > > Hi, > > > > When index data were corrupted on master instance, I wanted to wipe out > all > > the index data and re-index everything. I was hoping the newly created > index > > data would be replicated to slaves, but it wasn't. > > > > Here are the steps I performed: > > > > 1. stop master > > 2. delete the directory 'index' > > 3. start master > > 4. disable replication on master > > 5. index all data from scratch > > 6. enable replication on master > > > > It seemed from log file that the slave instances discovered that new > index > > are available and claimed that new index installed, and then trying to > > update index properties, but looking into the index directory on slaves, > you > > will find that no index data files were updated or added, plus slaves > keep > > trying to get new index. Here are some from slave's log file: > > > > Jul 1, 2009 3:59:33 PM org.apache.solr.handler.SnapPuller > fetchLatestIndex > > INFO: Starting replication process > > Jul 1, 2009 3:59:33 PM org.apache.solr.handler.SnapPuller > fetchLatestIndex > > INFO: Number of files in latest snapshot in master: 69 > > Jul 1, 2009 3:59:33 PM org.apache.solr.handler.SnapPuller > fetchLatestIndex > > INFO: Total time taken for download : 0 secs > > Jul 1, 2009 3:59:33 PM org.apache.solr.handler.SnapPuller > fetchLatestIndex > > INFO: Conf files are not downloaded or are in sync > > Jul 1, 2009 3:59:33 PM org.apache.solr.handler.SnapPuller > modifyIndexProps > > INFO: New index installed. Updating index properties... > > Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller > fetchLatestIndex > > INFO: Master's version: 1246488421310, generation: 9 > > Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller > fetchLatestIndex > > INFO: Slave's version: 1246385166228, generation: 56 > > Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller > fetchLatestIndex > > INFO: Starting replication process > > Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller > fetchLatestIndex > > INFO: Number of files in latest snapshot in master: 69 > > Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller > fetchLatestIndex > > INFO: Total time taken for download : 0 secs > > Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller > fetchLatestIndex > > INFO: Conf files are not downloaded or are in sync > > Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller > modifyIndexProps > > INFO: New index installed. Updating index properties... > > > > > > Is this process incorrect, or it is a bug? If the process is incorrect, > what > > is the right process? > > > > Thanks, > > > > J > > |
|
|
Re: reindexed data on master not replicated to slave
by Noble Paul നോബിള് नोब्ळ्-2
::
Rate this Message:
jay , I see updating index properties... twice
this should happen rarely. in your case it should have happened only once. because you cleaned up the master only once On Fri, Jul 3, 2009 at 6:09 AM, Otis Gospodnetic<otis_gospodnetic@...> wrote: > > Jay, > > You didn't mention which version of Solr you are using. It looks like some trunk or nightly version. Maybe you can try the latest nightly? > > Otis > -- > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch > > > > ----- Original Message ---- >> From: solr jay <solrjay@...> >> To: solr-user@... >> Sent: Thursday, July 2, 2009 9:14:48 PM >> Subject: reindexed data on master not replicated to slave >> >> Hi, >> >> When index data were corrupted on master instance, I wanted to wipe out all >> the index data and re-index everything. I was hoping the newly created index >> data would be replicated to slaves, but it wasn't. >> >> Here are the steps I performed: >> >> 1. stop master >> 2. delete the directory 'index' >> 3. start master >> 4. disable replication on master >> 5. index all data from scratch >> 6. enable replication on master >> >> It seemed from log file that the slave instances discovered that new index >> are available and claimed that new index installed, and then trying to >> update index properties, but looking into the index directory on slaves, you >> will find that no index data files were updated or added, plus slaves keep >> trying to get new index. Here are some from slave's log file: >> >> Jul 1, 2009 3:59:33 PM org.apache.solr.handler.SnapPuller fetchLatestIndex >> INFO: Starting replication process >> Jul 1, 2009 3:59:33 PM org.apache.solr.handler.SnapPuller fetchLatestIndex >> INFO: Number of files in latest snapshot in master: 69 >> Jul 1, 2009 3:59:33 PM org.apache.solr.handler.SnapPuller fetchLatestIndex >> INFO: Total time taken for download : 0 secs >> Jul 1, 2009 3:59:33 PM org.apache.solr.handler.SnapPuller fetchLatestIndex >> INFO: Conf files are not downloaded or are in sync >> Jul 1, 2009 3:59:33 PM org.apache.solr.handler.SnapPuller modifyIndexProps >> INFO: New index installed. Updating index properties... >> Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller fetchLatestIndex >> INFO: Master's version: 1246488421310, generation: 9 >> Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller fetchLatestIndex >> INFO: Slave's version: 1246385166228, generation: 56 >> Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller fetchLatestIndex >> INFO: Starting replication process >> Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller fetchLatestIndex >> INFO: Number of files in latest snapshot in master: 69 >> Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller fetchLatestIndex >> INFO: Total time taken for download : 0 secs >> Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller fetchLatestIndex >> INFO: Conf files are not downloaded or are in sync >> Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller modifyIndexProps >> INFO: New index installed. Updating index properties... >> >> >> Is this process incorrect, or it is a bug? If the process is incorrect, what >> is the right process? >> >> Thanks, >> >> J > > -- ----------------------------------------------------- Noble Paul | Principal Engineer| AOL | http://aol.com |
|
|
Re: reindexed data on master not replicated to slave
by solr jay
::
Rate this Message:
I tried it with the latest nightly build and got the same result.
Actually that was the symptom and it made me looking at the index directory. The same log messages repeated again and again, never end. 2009/7/2 Noble Paul നോബിള് नोब्ळ् <noble.paul@...> > jay , I see updating index properties... twice > > > > this should happen rarely. in your case it should have happened only > once. because you cleaned up the master only once > > > On Fri, Jul 3, 2009 at 6:09 AM, Otis > Gospodnetic<otis_gospodnetic@...> wrote: > > > > Jay, > > > > You didn't mention which version of Solr you are using. It looks like > some trunk or nightly version. Maybe you can try the latest nightly? > > > > Otis > > -- > > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch > > > > > > > > ----- Original Message ---- > >> From: solr jay <solrjay@...> > >> To: solr-user@... > >> Sent: Thursday, July 2, 2009 9:14:48 PM > >> Subject: reindexed data on master not replicated to slave > >> > >> Hi, > >> > >> When index data were corrupted on master instance, I wanted to wipe out > all > >> the index data and re-index everything. I was hoping the newly created > index > >> data would be replicated to slaves, but it wasn't. > >> > >> Here are the steps I performed: > >> > >> 1. stop master > >> 2. delete the directory 'index' > >> 3. start master > >> 4. disable replication on master > >> 5. index all data from scratch > >> 6. enable replication on master > >> > >> It seemed from log file that the slave instances discovered that new > index > >> are available and claimed that new index installed, and then trying to > >> update index properties, but looking into the index directory on slaves, > you > >> will find that no index data files were updated or added, plus slaves > keep > >> trying to get new index. Here are some from slave's log file: > >> > >> Jul 1, 2009 3:59:33 PM org.apache.solr.handler.SnapPuller > fetchLatestIndex > >> INFO: Starting replication process > >> Jul 1, 2009 3:59:33 PM org.apache.solr.handler.SnapPuller > fetchLatestIndex > >> INFO: Number of files in latest snapshot in master: 69 > >> Jul 1, 2009 3:59:33 PM org.apache.solr.handler.SnapPuller > fetchLatestIndex > >> INFO: Total time taken for download : 0 secs > >> Jul 1, 2009 3:59:33 PM org.apache.solr.handler.SnapPuller > fetchLatestIndex > >> INFO: Conf files are not downloaded or are in sync > >> Jul 1, 2009 3:59:33 PM org.apache.solr.handler.SnapPuller > modifyIndexProps > >> INFO: New index installed. Updating index properties... > >> Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller > fetchLatestIndex > >> INFO: Master's version: 1246488421310, generation: 9 > >> Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller > fetchLatestIndex > >> INFO: Slave's version: 1246385166228, generation: 56 > >> Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller > fetchLatestIndex > >> INFO: Starting replication process > >> Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller > fetchLatestIndex > >> INFO: Number of files in latest snapshot in master: 69 > >> Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller > fetchLatestIndex > >> INFO: Total time taken for download : 0 secs > >> Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller > fetchLatestIndex > >> INFO: Conf files are not downloaded or are in sync > >> Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller > modifyIndexProps > >> INFO: New index installed. Updating index properties... > >> > >> > >> Is this process incorrect, or it is a bug? If the process is incorrect, > what > >> is the right process? > >> > >> Thanks, > >> > >> J > > > > > > > > -- > ----------------------------------------------------- > Noble Paul | Principal Engineer| AOL | http://aol.com > |
|
|
|
|
|
Re: reindexed data on master not replicated to slave
by Noble Paul നോബിള് नोब्ळ्-2
::
Rate this Message:
Jay ,
I am opening an issue SOLR-1264 https://issues.apache.org/jira/browse/SOLR-1264 I have attached a patch as well . I guess that is the fix. could you please confirm that. On Tue, Jul 7, 2009 at 12:59 AM, solr jay<solrjay@...> wrote: > It looks that the problem is here or before that in > SnapPuller.fetchLatestIndex(): > > > terminateAndWaitFsyncService(); > LOG.info("Conf files are not downloaded or are in sync"); > if (isSnapNeeded) { > modifyIndexProps(tmpIndexDir.getName()); > } else { > successfulInstall = copyIndexFiles(tmpIndexDir, indexDir); > } > if (successfulInstall) { > logReplicationTimeAndConfFiles(modifiedConfFiles); > doCommit(); > } > > > Debugged into the place, and noticed that isSnapNeeded is true and therefore > > modifyIndexProps(tmpIndexDir.getName()); > > executed, but from the function name it looks that installing index actually > happens in > > successfulInstall = copyIndexFiles(tmpIndexDir, indexDir); > > > The function returns false, but the caller (doSnapPull) never checked the > return value. > > > Thanks, > > J > > > On Mon, Jul 6, 2009 at 8:02 AM, solr jay <solrjay@...> wrote: >> >> There is only one index directory: index/ >> >> Here is the content of index.properties >> >> #index properties >> #Fri Jul 03 14:17:12 PDT 2009 >> index=index.20090703021705 >> >> >> Thanks, >> >> J >> >> 2009/7/5 Noble Paul നോബിള് नोब्ळ् <noble.paul@...> >>> >>> BTW , how many index dirs are there in the data dir ? what is there in >>> the <datadir>/index.properties ? >>> >>> On Sat, Jul 4, 2009 at 12:15 AM, solr jay<solrjay@...> wrote: >>> > >>> > >>> > I tried it with the latest nightly build and got the same result. >>> > >>> > Actually that was the symptom and it made me looking at the index >>> > directory. >>> > The same log messages repeated again and again, never end. >>> > >>> > >>> > >>> > 2009/7/2 Noble Paul നോബിള് नोब्ळ् <noble.paul@...> >>> >> >>> >> jay , I see updating index properties... twice >>> >> >>> >> >>> >> >>> >> this should happen rarely. in your case it should have happened only >>> >> once. because you cleaned up the master only once >>> >> >>> >> >>> >> On Fri, Jul 3, 2009 at 6:09 AM, Otis >>> >> Gospodnetic<otis_gospodnetic@...> wrote: >>> >> > >>> >> > Jay, >>> >> > >>> >> > You didn't mention which version of Solr you are using. It looks >>> >> > like >>> >> > some trunk or nightly version. Maybe you can try the latest >>> >> > nightly? >>> >> > >>> >> > Otis >>> >> > -- >>> >> > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch >>> >> > >>> >> > >>> >> > >>> >> > ----- Original Message ---- >>> >> >> From: solr jay <solrjay@...> >>> >> >> To: solr-user@... >>> >> >> Sent: Thursday, July 2, 2009 9:14:48 PM >>> >> >> Subject: reindexed data on master not replicated to slave >>> >> >> >>> >> >> Hi, >>> >> >> >>> >> >> When index data were corrupted on master instance, I wanted to wipe >>> >> >> out >>> >> >> all >>> >> >> the index data and re-index everything. I was hoping the newly >>> >> >> created >>> >> >> index >>> >> >> data would be replicated to slaves, but it wasn't. >>> >> >> >>> >> >> Here are the steps I performed: >>> >> >> >>> >> >> 1. stop master >>> >> >> 2. delete the directory 'index' >>> >> >> 3. start master >>> >> >> 4. disable replication on master >>> >> >> 5. index all data from scratch >>> >> >> 6. enable replication on master >>> >> >> >>> >> >> It seemed from log file that the slave instances discovered that >>> >> >> new >>> >> >> index >>> >> >> are available and claimed that new index installed, and then trying >>> >> >> to >>> >> >> update index properties, but looking into the index directory on >>> >> >> slaves, you >>> >> >> will find that no index data files were updated or added, plus >>> >> >> slaves >>> >> >> keep >>> >> >> trying to get new index. Here are some from slave's log file: >>> >> >> >>> >> >> Jul 1, 2009 3:59:33 PM org.apache.solr.handler.SnapPuller >>> >> >> fetchLatestIndex >>> >> >> INFO: Starting replication process >>> >> >> Jul 1, 2009 3:59:33 PM org.apache.solr.handler.SnapPuller >>> >> >> fetchLatestIndex >>> >> >> INFO: Number of files in latest snapshot in master: 69 >>> >> >> Jul 1, 2009 3:59:33 PM org.apache.solr.handler.SnapPuller >>> >> >> fetchLatestIndex >>> >> >> INFO: Total time taken for download : 0 secs >>> >> >> Jul 1, 2009 3:59:33 PM org.apache.solr.handler.SnapPuller >>> >> >> fetchLatestIndex >>> >> >> INFO: Conf files are not downloaded or are in sync >>> >> >> Jul 1, 2009 3:59:33 PM org.apache.solr.handler.SnapPuller >>> >> >> modifyIndexProps >>> >> >> INFO: New index installed. Updating index properties... >>> >> >> Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller >>> >> >> fetchLatestIndex >>> >> >> INFO: Master's version: 1246488421310, generation: 9 >>> >> >> Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller >>> >> >> fetchLatestIndex >>> >> >> INFO: Slave's version: 1246385166228, generation: 56 >>> >> >> Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller >>> >> >> fetchLatestIndex >>> >> >> INFO: Starting replication process >>> >> >> Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller >>> >> >> fetchLatestIndex >>> >> >> INFO: Number of files in latest snapshot in master: 69 >>> >> >> Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller >>> >> >> fetchLatestIndex >>> >> >> INFO: Total time taken for download : 0 secs >>> >> >> Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller >>> >> >> fetchLatestIndex >>> >> >> INFO: Conf files are not downloaded or are in sync >>> >> >> Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller >>> >> >> modifyIndexProps >>> >> >> INFO: New index installed. Updating index properties... >>> >> >> >>> >> >> >>> >> >> Is this process incorrect, or it is a bug? If the process is >>> >> >> incorrect, >>> >> >> what >>> >> >> is the right process? >>> >> >> >>> >> >> Thanks, >>> >> >> >>> >> >> J >>> >> > >>> >> > >>> >> >>> >> >>> >> >>> >> -- >>> >> ----------------------------------------------------- >>> >> Noble Paul | Principal Engineer| AOL | http://aol.com >>> > >>> > >>> >>> >>> >>> -- >>> ----------------------------------------------------- >>> Noble Paul | Principal Engineer| AOL | http://aol.com >> > > -- ----------------------------------------------------- Noble Paul | Principal Engineer| AOL | http://aol.com |
|
|
Re: reindexed data on master not replicated to slave
by solr jay
::
Rate this Message:
It seemed that the patch fixed the symptom, but not the problem itself.
Now the log messages looks good. After one download and installed the index, it printed out *Jul 7, 2009 10:35:10 AM org.apache.solr.handler.SnapPuller fetchLatestIndex INFO: Slave in sync with master.* but the files inside index directory did not change. Both index.properties and replication.properties were updated though. Just a couple of files: from master instance: -rw-r--r-- 1 worun wheel 181 Jul 7 09:28 _6.fdt -rw-r--r-- 1 worun wheel 12 Jul 7 09:28 _6.fdx -rw-r--r-- 1 worun wheel 131 Jul 7 09:28 _6.fnm -rw-r--r-- 1 worun wheel 27 Jul 7 09:28 _6.frq -rw-r--r-- 1 worun wheel 11 Jul 7 09:28 _6.nrm from slave instance: -rw-r--r-- 1 jianhanguo admin 70 Jul 6 18:55 _14_5.del -rw-r--r-- 1 jianhanguo admin 4016 Jul 6 18:55 _15.fdt -rw-r--r-- 1 jianhanguo admin 268 Jul 6 18:55 _15.fdx -rw-r--r-- 1 jianhanguo admin 131 Jul 6 18:55 _15.fnm -rw-r--r-- 1 jianhanguo admin 726 Jul 6 18:55 _15.frq Thanks, J 2009/7/7 Noble Paul നോബിള് नोब्ळ् <noble.paul@...> > Jay , > I am opening an issue SOLR-1264 > https://issues.apache.org/jira/browse/SOLR-1264 > > I have attached a patch as well . I guess that is the fix. could you > please confirm that. > > > On Tue, Jul 7, 2009 at 12:59 AM, solr jay<solrjay@...> wrote: > > It looks that the problem is here or before that in > > SnapPuller.fetchLatestIndex(): > > > > > > terminateAndWaitFsyncService(); > > LOG.info("Conf files are not downloaded or are in sync"); > > if (isSnapNeeded) { > > modifyIndexProps(tmpIndexDir.getName()); > > } else { > > successfulInstall = copyIndexFiles(tmpIndexDir, indexDir); > > } > > if (successfulInstall) { > > logReplicationTimeAndConfFiles(modifiedConfFiles); > > doCommit(); > > } > > > > > > Debugged into the place, and noticed that isSnapNeeded is true and > therefore > > > > modifyIndexProps(tmpIndexDir.getName()); > > > > executed, but from the function name it looks that installing index > actually > > happens in > > > > successfulInstall = copyIndexFiles(tmpIndexDir, indexDir); > > > > > > The function returns false, but the caller (doSnapPull) never checked the > > return value. > > > > > > Thanks, > > > > J > > > > > > On Mon, Jul 6, 2009 at 8:02 AM, solr jay <solrjay@...> wrote: > >> > >> There is only one index directory: index/ > >> > >> Here is the content of index.properties > >> > >> #index properties > >> #Fri Jul 03 14:17:12 PDT 2009 > >> index=index.20090703021705 > >> > >> > >> Thanks, > >> > >> J > >> > >> 2009/7/5 Noble Paul നോബിള് नोब्ळ् <noble.paul@...> > >>> > >>> BTW , how many index dirs are there in the data dir ? what is there in > >>> the <datadir>/index.properties ? > >>> > >>> On Sat, Jul 4, 2009 at 12:15 AM, solr jay<solrjay@...> wrote: > >>> > > >>> > > >>> > I tried it with the latest nightly build and got the same result. > >>> > > >>> > Actually that was the symptom and it made me looking at the index > >>> > directory. > >>> > The same log messages repeated again and again, never end. > >>> > > >>> > > >>> > > >>> > 2009/7/2 Noble Paul നോബിള് नोब्ळ् <noble.paul@...> > >>> >> > >>> >> jay , I see updating index properties... twice > >>> >> > >>> >> > >>> >> > >>> >> this should happen rarely. in your case it should have happened only > >>> >> once. because you cleaned up the master only once > >>> >> > >>> >> > >>> >> On Fri, Jul 3, 2009 at 6:09 AM, Otis > >>> >> Gospodnetic<otis_gospodnetic@...> wrote: > >>> >> > > >>> >> > Jay, > >>> >> > > >>> >> > You didn't mention which version of Solr you are using. It looks > >>> >> > like > >>> >> > some trunk or nightly version. Maybe you can try the latest > >>> >> > nightly? > >>> >> > > >>> >> > Otis > >>> >> > -- > >>> >> > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch > >>> >> > > >>> >> > > >>> >> > > >>> >> > ----- Original Message ---- > >>> >> >> From: solr jay <solrjay@...> > >>> >> >> To: solr-user@... > >>> >> >> Sent: Thursday, July 2, 2009 9:14:48 PM > >>> >> >> Subject: reindexed data on master not replicated to slave > >>> >> >> > >>> >> >> Hi, > >>> >> >> > >>> >> >> When index data were corrupted on master instance, I wanted to > wipe > >>> >> >> out > >>> >> >> all > >>> >> >> the index data and re-index everything. I was hoping the newly > >>> >> >> created > >>> >> >> index > >>> >> >> data would be replicated to slaves, but it wasn't. > >>> >> >> > >>> >> >> Here are the steps I performed: > >>> >> >> > >>> >> >> 1. stop master > >>> >> >> 2. delete the directory 'index' > >>> >> >> 3. start master > >>> >> >> 4. disable replication on master > >>> >> >> 5. index all data from scratch > >>> >> >> 6. enable replication on master > >>> >> >> > >>> >> >> It seemed from log file that the slave instances discovered that > >>> >> >> new > >>> >> >> index > >>> >> >> are available and claimed that new index installed, and then > trying > >>> >> >> to > >>> >> >> update index properties, but looking into the index directory on > >>> >> >> slaves, you > >>> >> >> will find that no index data files were updated or added, plus > >>> >> >> slaves > >>> >> >> keep > >>> >> >> trying to get new index. Here are some from slave's log file: > >>> >> >> > >>> >> >> Jul 1, 2009 3:59:33 PM org.apache.solr.handler.SnapPuller > >>> >> >> fetchLatestIndex > >>> >> >> INFO: Starting replication process > >>> >> >> Jul 1, 2009 3:59:33 PM org.apache.solr.handler.SnapPuller > >>> >> >> fetchLatestIndex > >>> >> >> INFO: Number of files in latest snapshot in master: 69 > >>> >> >> Jul 1, 2009 3:59:33 PM org.apache.solr.handler.SnapPuller > >>> >> >> fetchLatestIndex > >>> >> >> INFO: Total time taken for download : 0 secs > >>> >> >> Jul 1, 2009 3:59:33 PM org.apache.solr.handler.SnapPuller > >>> >> >> fetchLatestIndex > >>> >> >> INFO: Conf files are not downloaded or are in sync > >>> >> >> Jul 1, 2009 3:59:33 PM org.apache.solr.handler.SnapPuller > >>> >> >> modifyIndexProps > >>> >> >> INFO: New index installed. Updating index properties... > >>> >> >> Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller > >>> >> >> fetchLatestIndex > >>> >> >> INFO: Master's version: 1246488421310, generation: 9 > >>> >> >> Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller > >>> >> >> fetchLatestIndex > >>> >> >> INFO: Slave's version: 1246385166228, generation: 56 > >>> >> >> Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller > >>> >> >> fetchLatestIndex > >>> >> >> INFO: Starting replication process > >>> >> >> Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller > >>> >> >> fetchLatestIndex > >>> >> >> INFO: Number of files in latest snapshot in master: 69 > >>> >> >> Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller > >>> >> >> fetchLatestIndex > >>> >> >> INFO: Total time taken for download : 0 secs > >>> >> >> Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller > >>> >> >> fetchLatestIndex > >>> >> >> INFO: Conf files are not downloaded or are in sync > >>> >> >> Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller > >>> >> >> modifyIndexProps > >>> >> >> INFO: New index installed. Updating index properties... > >>> >> >> > >>> >> >> > >>> >> >> Is this process incorrect, or it is a bug? If the process is > >>> >> >> incorrect, > >>> >> >> what > >>> >> >> is the right process? > >>> >> >> > >>> >> >> Thanks, > >>> >> >> > >>> >> >> J > >>> >> > > >>> >> > > >>> >> > >>> >> > >>> >> > >>> >> -- > >>> >> ----------------------------------------------------- > >>> >> Noble Paul | Principal Engineer| AOL | http://aol.com > >>> > > >>> > > >>> > >>> > >>> > >>> -- > >>> ----------------------------------------------------- > >>> Noble Paul | Principal Engineer| AOL | http://aol.com > >> > > > > > > > > -- > ----------------------------------------------------- > Noble Paul | Principal Engineer| AOL | http://aol.com > |
|
|
Re: reindexed data on master not replicated to slave
by Shalin Shekhar Mangar
::
Rate this Message:
On Tue, Jul 7, 2009 at 11:50 PM, solr jay <solrjay@...> wrote:
> It seemed that the patch fixed the symptom, but not the problem itself. > > Now the log messages looks good. After one download and installed the > index, > it printed out > > *Jul 7, 2009 10:35:10 AM org.apache.solr.handler.SnapPuller > fetchLatestIndex > INFO: Slave in sync with master.* > > but the files inside index directory did not change. Both index.properties > and replication.properties were updated though. > Note that in this case, Solr would have created a new index directory. Are you comparing the files on the slave in the new index directory? You can get the new index directory's name from index.properties. -- Regards, Shalin Shekhar Mangar. |
|
|
Re: reindexed data on master not replicated to slave
by solr jay
::
Rate this Message:
I see. So I tried it again. Now index.properties has
#index properties #Tue Jul 07 12:13:49 PDT 2009 index=index.20090707121349 but there is no such directory index.20090707121349 under the data directory. Thanks, J On Tue, Jul 7, 2009 at 11:50 AM, Shalin Shekhar Mangar < shalinmangar@...> wrote: > On Tue, Jul 7, 2009 at 11:50 PM, solr jay <solrjay@...> wrote: > > > It seemed that the patch fixed the symptom, but not the problem itself. > > > > Now the log messages looks good. After one download and installed the > > index, > > it printed out > > > > *Jul 7, 2009 10:35:10 AM org.apache.solr.handler.SnapPuller > > fetchLatestIndex > > INFO: Slave in sync with master.* > > > > but the files inside index directory did not change. Both > index.properties > > and replication.properties were updated though. > > > > Note that in this case, Solr would have created a new index directory. Are > you comparing the files on the slave in the new index directory? You can > get > the new index directory's name from index.properties. > > -- > Regards, > Shalin Shekhar Mangar. > |
|
|
Re: reindexed data on master not replicated to slave
by solr jay
::
Rate this Message:
Ok, Here is the problem. In the function, the two directories tmpIndexDir
and indexDir are the same (in this case only?), and then at the end of the function, the directory tmpIndexDir is deleted, which deletes the new index directory. } finally { delTree(tmpIndexDir); } On Tue, Jul 7, 2009 at 12:17 PM, solr jay <solrjay@...> wrote: > I see. So I tried it again. Now index.properties has > > #index properties > #Tue Jul 07 12:13:49 PDT 2009 > index=index.20090707121349 > > but there is no such directory index.20090707121349 under the data > directory. > > Thanks, > > J > > > On Tue, Jul 7, 2009 at 11:50 AM, Shalin Shekhar Mangar < > shalinmangar@...> wrote: > >> On Tue, Jul 7, 2009 at 11:50 PM, solr jay <solrjay@...> wrote: >> >> > It seemed that the patch fixed the symptom, but not the problem itself. >> > >> > Now the log messages looks good. After one download and installed the >> > index, >> > it printed out >> > >> > *Jul 7, 2009 10:35:10 AM org.apache.solr.handler.SnapPuller >> > fetchLatestIndex >> > INFO: Slave in sync with master.* >> > >> > but the files inside index directory did not change. Both >> index.properties >> > and replication.properties were updated though. >> > >> >> Note that in this case, Solr would have created a new index directory. Are >> you comparing the files on the slave in the new index directory? You can >> get >> the new index directory's name from index.properties. >> >> -- >> Regards, >> Shalin Shekhar Mangar. >> > > |
|
|
Re: reindexed data on master not replicated to slave
by solr jay
::
Rate this Message:
In fact, I saw the directory was created and then deleted.
On Tue, Jul 7, 2009 at 12:29 PM, solr jay <solrjay@...> wrote: > Ok, Here is the problem. In the function, the two directories tmpIndexDir > and indexDir are the same (in this case only?), and then at the end of the > function, the directory tmpIndexDir is deleted, which deletes the new index > directory. > > > } finally { > delTree(tmpIndexDir); > > } > > > On Tue, Jul 7, 2009 at 12:17 PM, solr jay <solrjay@...> wrote: > >> I see. So I tried it again. Now index.properties has >> >> #index properties >> #Tue Jul 07 12:13:49 PDT 2009 >> index=index.20090707121349 >> >> but there is no such directory index.20090707121349 under the data >> directory. >> >> Thanks, >> >> J >> >> >> On Tue, Jul 7, 2009 at 11:50 AM, Shalin Shekhar Mangar < >> shalinmangar@...> wrote: >> >>> On Tue, Jul 7, 2009 at 11:50 PM, solr jay <solrjay@...> wrote: >>> >>> > It seemed that the patch fixed the symptom, but not the problem itself. >>> > >>> > Now the log messages looks good. After one download and installed the >>> > index, >>> > it printed out >>> > >>> > *Jul 7, 2009 10:35:10 AM org.apache.solr.handler.SnapPuller >>> > fetchLatestIndex >>> > INFO: Slave in sync with master.* >>> > >>> > but the files inside index directory did not change. Both >>> index.properties >>> > and replication.properties were updated though. >>> > >>> >>> Note that in this case, Solr would have created a new index directory. >>> Are >>> you comparing the files on the slave in the new index directory? You can >>> get >>> the new index directory's name from index.properties. >>> >>> -- >>> Regards, >>> Shalin Shekhar Mangar. >>> >> >> > |
|
|
Re: reindexed data on master not replicated to slave
by solr jay
::
Rate this Message:
I guess in this case it doesn't matter whether the two directories
tmpIndexDir and indexDir are the same or not. It looks that the index directory is switched to tmpIndexDir and then it is deleted inside "finally". On Tue, Jul 7, 2009 at 12:31 PM, solr jay <solrjay@...> wrote: > In fact, I saw the directory was created and then deleted. > > > On Tue, Jul 7, 2009 at 12:29 PM, solr jay <solrjay@...> wrote: > >> Ok, Here is the problem. In the function, the two directories tmpIndexDir >> and indexDir are the same (in this case only?), and then at the end of the >> function, the directory tmpIndexDir is deleted, which deletes the new index >> directory. >> >> >> } finally { >> delTree(tmpIndexDir); >> >> } >> >> >> On Tue, Jul 7, 2009 at 12:17 PM, solr jay <solrjay@...> wrote: >> >>> I see. So I tried it again. Now index.properties has >>> >>> #index properties >>> #Tue Jul 07 12:13:49 PDT 2009 >>> index=index.20090707121349 >>> >>> but there is no such directory index.20090707121349 under the data >>> directory. >>> >>> Thanks, >>> >>> J >>> >>> >>> On Tue, Jul 7, 2009 at 11:50 AM, Shalin Shekhar Mangar < >>> shalinmangar@...> wrote: >>> >>>> On Tue, Jul 7, 2009 at 11:50 PM, solr jay <solrjay@...> wrote: >>>> >>>> > It seemed that the patch fixed the symptom, but not the problem >>>> itself. >>>> > >>>> > Now the log messages looks good. After one download and installed the >>>> > index, >>>> > it printed out >>>> > >>>> > *Jul 7, 2009 10:35:10 AM org.apache.solr.handler.SnapPuller >>>> > fetchLatestIndex >>>> > INFO: Slave in sync with master.* >>>> > >>>> > but the files inside index directory did not change. Both >>>> index.properties >>>> > and replication.properties were updated though. >>>> > >>>> >>>> Note that in this case, Solr would have created a new index directory. >>>> Are >>>> you comparing the files on the slave in the new index directory? You can >>>> get >>>> the new index directory's name from index.properties. >>>> >>>> -- >>>> Regards, >>>> Shalin Shekhar Mangar. >>>> >>> >>> >> > -- J |
|
|
Re: reindexed data on master not replicated to slave
by Noble Paul നോബിള് नोब्ळ्-2
::
Rate this Message:
jay,
Thanks. The testcase was not enough. I have given a new patch . I guess that should solve this On Wed, Jul 8, 2009 at 3:48 AM, solr jay<solrjay@...> wrote: > I guess in this case it doesn't matter whether the two directories > tmpIndexDir and indexDir are the same or not. It looks that the index > directory is switched to tmpIndexDir and then it is deleted inside > "finally". > > On Tue, Jul 7, 2009 at 12:31 PM, solr jay <solrjay@...> wrote: > >> In fact, I saw the directory was created and then deleted. >> >> >> On Tue, Jul 7, 2009 at 12:29 PM, solr jay <solrjay@...> wrote: >> >>> Ok, Here is the problem. In the function, the two directories tmpIndexDir >>> and indexDir are the same (in this case only?), and then at the end of the >>> function, the directory tmpIndexDir is deleted, which deletes the new index >>> directory. >>> >>> >>> } finally { >>> delTree(tmpIndexDir); >>> >>> } >>> >>> >>> On Tue, Jul 7, 2009 at 12:17 PM, solr jay <solrjay@...> wrote: >>> >>>> I see. So I tried it again. Now index.properties has >>>> >>>> #index properties >>>> #Tue Jul 07 12:13:49 PDT 2009 >>>> index=index.20090707121349 >>>> >>>> but there is no such directory index.20090707121349 under the data >>>> directory. >>>> >>>> Thanks, >>>> >>>> J >>>> >>>> >>>> On Tue, Jul 7, 2009 at 11:50 AM, Shalin Shekhar Mangar < >>>> shalinmangar@...> wrote: >>>> >>>>> On Tue, Jul 7, 2009 at 11:50 PM, solr jay <solrjay@...> wrote: >>>>> >>>>> > It seemed that the patch fixed the symptom, but not the problem >>>>> itself. >>>>> > >>>>> > Now the log messages looks good. After one download and installed the >>>>> > index, >>>>> > it printed out >>>>> > >>>>> > *Jul 7, 2009 10:35:10 AM org.apache.solr.handler.SnapPuller >>>>> > fetchLatestIndex >>>>> > INFO: Slave in sync with master.* >>>>> > >>>>> > but the files inside index directory did not change. Both >>>>> index.properties >>>>> > and replication.properties were updated though. >>>>> > >>>>> >>>>> Note that in this case, Solr would have created a new index directory. >>>>> Are >>>>> you comparing the files on the slave in the new index directory? You can >>>>> get >>>>> the new index directory's name from index.properties. >>>>> >>>>> -- >>>>> Regards, >>>>> Shalin Shekhar Mangar. >>>>> >>>> >>>> >>> >> > > > -- > J > -- ----------------------------------------------------- Noble Paul | Principal Engineer| AOL | http://aol.com |
|
|
|
| Free embeddable forum powered by Nabble | Forum Help |
