Re: reindexed data on master not replicated to slave

> Jay ,
> I am opening an issue SOLR-1264
> https://issues.apache.org/jira/browse/SOLR-1264
>
> I have attached a patch as well . I guess that is the fix. could you
> please confirm that.
>
>
> On Tue, Jul 7, 2009 at 12:59 AM, solr jay<solrjay@...> wrote:
> > It looks that the problem is here or before that in
> > SnapPuller.fetchLatestIndex():
> >
> >
> >           terminateAndWaitFsyncService();
> >           LOG.info("Conf files are not downloaded or are in sync");
> >           if (isSnapNeeded) {
> >             modifyIndexProps(tmpIndexDir.getName());
> >           } else {
> >             successfulInstall = copyIndexFiles(tmpIndexDir, indexDir);
> >           }
> >           if (successfulInstall) {
> >             logReplicationTimeAndConfFiles(modifiedConfFiles);
> >             doCommit();
> >           }
> >
> >
> > Debugged into the place, and noticed that isSnapNeeded is true and
> therefore
> >
> > modifyIndexProps(tmpIndexDir.getName());
> >
> > executed, but from the function name it looks that installing index
> actually
> > happens in
> >
> > successfulInstall = copyIndexFiles(tmpIndexDir, indexDir);
> >
> >
> > The function returns false, but the caller (doSnapPull) never checked the
> > return value.
> >
> >
> > Thanks,
> >
> > J
> >
> >
> > On Mon, Jul 6, 2009 at 8:02 AM, solr jay <solrjay@...> wrote:
> >>
> >> There is only one index directory: index/
> >>
> >> Here is the content of index.properties
> >>
> >> #index properties
> >> #Fri Jul 03 14:17:12 PDT 2009
> >> index=index.20090703021705
> >>
> >>
> >> Thanks,
> >>
> >> J
> >>
> >> 2009/7/5 Noble Paul നോബിള്‍ नोब्ळ् <noble.paul@...>
> >>>
> >>> BTW , how many index dirs are there in the data dir ? what is there in
> >>> the <datadir>/index.properties ?
> >>>
> >>> On Sat, Jul 4, 2009 at 12:15 AM, solr jay<solrjay@...> wrote:
> >>> >
> >>> >
> >>> > I tried it with the latest nightly build and got the same result.
> >>> >
> >>> > Actually that was the symptom and it made me looking at the index
> >>> > directory.
> >>> > The same log messages repeated again and again, never end.
> >>> >
> >>> >
> >>> >
> >>> > 2009/7/2 Noble Paul നോബിള്‍ नोब्ळ् <noble.paul@...>
> >>> >>
> >>> >> jay , I see updating index properties... twice
> >>> >>
> >>> >>
> >>> >>
> >>> >> this should happen rarely. in your case it should have happened only
> >>> >> once. because you cleaned up the master only once
> >>> >>
> >>> >>
> >>> >> On Fri, Jul 3, 2009 at 6:09 AM, Otis
> >>> >> Gospodnetic<otis_gospodnetic@...> wrote:
> >>> >> >
> >>> >> > Jay,
> >>> >> >
> >>> >> > You didn't mention which version of Solr you are using.  It looks
> >>> >> > like
> >>> >> > some trunk or nightly version.  Maybe you can try the latest
> >>> >> > nightly?
> >>> >> >
> >>> >> >  Otis
> >>> >> > --
> >>> >> > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch
> >>> >> >
> >>> >> >
> >>> >> >
> >>> >> > ----- Original Message ----
> >>> >> >> From: solr jay <solrjay@...>
> >>> >> >> To: solr-user@...
> >>> >> >> Sent: Thursday, July 2, 2009 9:14:48 PM
> >>> >> >> Subject: reindexed data on master not replicated to slave
> >>> >> >>
> >>> >> >> Hi,
> >>> >> >>
> >>> >> >> When index data were corrupted on master instance, I wanted to
> wipe
> >>> >> >> out
> >>> >> >> all
> >>> >> >> the index data and re-index everything. I was hoping the newly
> >>> >> >> created
> >>> >> >> index
> >>> >> >> data would be replicated to slaves, but it wasn't.
> >>> >> >>
> >>> >> >> Here are the steps I performed:
> >>> >> >>
> >>> >> >> 1. stop master
> >>> >> >> 2. delete the directory 'index'
> >>> >> >> 3. start master
> >>> >> >> 4. disable replication on master
> >>> >> >> 5. index all data from scratch
> >>> >> >> 6. enable replication on master
> >>> >> >>
> >>> >> >> It seemed from log file that the slave instances discovered that
> >>> >> >> new
> >>> >> >> index
> >>> >> >> are available and claimed that new index installed, and then
> trying
> >>> >> >> to
> >>> >> >> update index properties, but looking into the index directory on
> >>> >> >> slaves, you
> >>> >> >> will find that no index data files were updated or added, plus
> >>> >> >> slaves
> >>> >> >> keep
> >>> >> >> trying to get new index. Here are some from slave's log file:
> >>> >> >>
> >>> >> >> Jul 1, 2009 3:59:33 PM org.apache.solr.handler.SnapPuller
> >>> >> >> fetchLatestIndex
> >>> >> >> INFO: Starting replication process
> >>> >> >> Jul 1, 2009 3:59:33 PM org.apache.solr.handler.SnapPuller
> >>> >> >> fetchLatestIndex
> >>> >> >> INFO: Number of files in latest snapshot in master: 69
> >>> >> >> Jul 1, 2009 3:59:33 PM org.apache.solr.handler.SnapPuller
> >>> >> >> fetchLatestIndex
> >>> >> >> INFO: Total time taken for download : 0 secs
> >>> >> >> Jul 1, 2009 3:59:33 PM org.apache.solr.handler.SnapPuller
> >>> >> >> fetchLatestIndex
> >>> >> >> INFO: Conf files are not downloaded or are in sync
> >>> >> >> Jul 1, 2009 3:59:33 PM org.apache.solr.handler.SnapPuller
> >>> >> >> modifyIndexProps
> >>> >> >> INFO: New index installed. Updating index properties...
> >>> >> >> Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller
> >>> >> >> fetchLatestIndex
> >>> >> >> INFO: Master's version: 1246488421310, generation: 9
> >>> >> >> Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller
> >>> >> >> fetchLatestIndex
> >>> >> >> INFO: Slave's version: 1246385166228, generation: 56
> >>> >> >> Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller
> >>> >> >> fetchLatestIndex
> >>> >> >> INFO: Starting replication process
> >>> >> >> Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller
> >>> >> >> fetchLatestIndex
> >>> >> >> INFO: Number of files in latest snapshot in master: 69
> >>> >> >> Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller
> >>> >> >> fetchLatestIndex
> >>> >> >> INFO: Total time taken for download : 0 secs
> >>> >> >> Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller
> >>> >> >> fetchLatestIndex
> >>> >> >> INFO: Conf files are not downloaded or are in sync
> >>> >> >> Jul 1, 2009 4:00:33 PM org.apache.solr.handler.SnapPuller
> >>> >> >> modifyIndexProps
> >>> >> >> INFO: New index installed. Updating index properties...
> >>> >> >>
> >>> >> >>
> >>> >> >> Is this process incorrect, or it is a bug? If the process is
> >>> >> >> incorrect,
> >>> >> >> what
> >>> >> >> is the right process?
> >>> >> >>
> >>> >> >> Thanks,
> >>> >> >>
> >>> >> >> J
> >>> >> >
> >>> >> >
> >>> >>
> >>> >>
> >>> >>
> >>> >> --
> >>> >> -----------------------------------------------------
> >>> >> Noble Paul | Principal Engineer| AOL | http://aol.com
> >>> >
> >>> >
> >>>
> >>>
> >>>
> >>> --
> >>> -----------------------------------------------------
> >>> Noble Paul | Principal Engineer| AOL | http://aol.com
> >>
> >
> >
>
>
>
> --
> -----------------------------------------------------
> Noble Paul | Principal Engineer| AOL | http://aol.com
>